OpenFX: remove reachable panics across the C ABI, null-check host GPU pointers, declare render InstanceSafe - #61
Open
Mldphotohraphie wants to merge 4 commits into
Open
Conversation
These code paths run inside OpenFX actions dispatched across the C ABI,
so a panic here would abort the host process (or worse, with unwinding).
- frame_from_timetype: log and fall back to frame 0 instead of
`panic!("Shouldn't happen")`. It is called from the parameter
get/set-at-time callbacks.
- Render / GetRegionOfDefinition: `get_bool_at_time(DontDrawOutside)`
now falls back to `false` (the parameter default) instead of
unwrapping.
- check_pending_file_info: ignore a failed `set_string` instead of
unwrapping, matching how set_string failures are handled elsewhere.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnShHBDs8UYxcRKSAaZzEF
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NnShHBDs8UYxcRKSAaZzEF
In the Render action the plugin reads raw pointers from the host image properties (Metal/OpenCL command queue and image buffers) and passes them to gyroflow-core, which retains the command queue and commits command buffers on it. A null pointer there is undefined behaviour. The downstream Metal path only null-checks the command queue, never the buffers, and the OpenCL/CUDA paths check nothing. Validate all of them up front and return a clean FAILED (with a log line) instead of feeding a null pointer into the GPU backend. Also guard the wgpu list_devices worker thread join against a panic instead of unwrapping it during the Describe action. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NnShHBDs8UYxcRKSAaZzEF Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NnShHBDs8UYxcRKSAaZzEF
CurrentFileInfo::query unwrapped get_fuscript() inside the worker thread. The LoadCurrent button that triggers it is only defined when the executable is available, but a race (file removed after describe) would panic the worker. Return early with a log line instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NnShHBDs8UYxcRKSAaZzEF Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NnShHBDs8UYxcRKSAaZzEF
The Render action receives `&mut InstanceData` and mutates per-instance state: it takes `file_path`, updates the stabilization-manager cache, timeline size and several parameters. Advertising FullySafe tells the host it may call Render concurrently on the *same* instance, which would hand out two `&mut InstanceData` (aliasing, undefined behaviour) and race those parameter updates. InstanceSafe serialises Render per instance while still allowing distinct instances to render in parallel. This also reduces concurrent commits on the host's Metal command queue through a shared stabilization manager. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NnShHBDs8UYxcRKSAaZzEF Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NnShHBDs8UYxcRKSAaZzEF
Mldphotohraphie
force-pushed
the
upstream-pr
branch
from
July 23, 2026 10:51
e44bf06 to
d977fe3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
DaVinci Resolve Studio 21.0.2 (macOS arm64, Apple M-series) crashed with SIGSEGV while using Gyroflow OFX v2.1.1 during Color page playback, right after an Edit -> Color page switch. Crash dump excerpt (from Resolve's ResolveDebug.txt):
While auditing the OpenFX plugin code around this crash, I found several robustness issues that this PR addresses. It does not claim to fix the root cause of the crash above (which likely involves the lifetime of the host's Metal command queue retained by the cached wgpu wrapper in gyroflow-core), but it removes several ways the plugin can take the host down and reduces concurrent access to the shared render state.
Changes
Remove reachable panics/unwraps from render and param paths. The plugin is loaded across the OpenFX C ABI;
main_entryhas nocatch_unwind, so any Rust panic unwinding out of an action is undefined behavior in the host process. Replaced reachablepanic!/unwrap()inframe_from_timetype,get_bool_at_time,set_stringand threadjoinwith logged fallbacks or error returns.Null-check host GPU pointers before handing them to the backend. The OpenCL/Metal/CUDA render paths passed raw host pointers (command queue, source/output buffers) straight to the GPU backend, where a null would be UB (
Retained::retain(...).unwrap()on the Metal side). The render action now fails cleanly instead.Avoid unwrap on missing fuscript executable.
Declare render as
InstanceSafeinstead ofFullySafe. The Render action takes&mut InstanceDataand mutates per-instance state (stabilization manager cache, timeline size, parameters).FullySafeallows the host to call Render concurrently on the same instance, which aliases that&mut(UB) and races those mutations.InstanceSafestill allows concurrent rendering of different instances.Notes
Two further issues were identified but need changes in gyroflow-core rather than this repo, so they are not part of this PR:
std::env::set_var(NO_OPENCLindisable_opencl) is called on the render path while gyroflow-core reads these variables from render threads;setenv/getenvracing is UB on Unix. A shared atomic flag would be safer.MTLCommandQueueacross renders; if the host tears down its GPU pipeline (e.g. Resolve page switches), a later commit on that queue can crash. Happy to file a separate issue with the full crash dump if useful.Tested:
cargo check/cargo build --releaseclean on macOS arm64 (rustc 1.94.1); the resulting build loads and renders correctly in Resolve Studio 21.0.2.🤖 Generated with Claude Code
https://claude.ai/code/session_01NnShHBDs8UYxcRKSAaZzEF