Repository navigation
Support use of Python subprocesses in bundled apps #3078
Description
Describe the bug
A desktop app bundled with Briefcase is unable to spawn a subprocess that uses sys.executable. This prohibits some notable use cases, including multiprocessing.
How to reproduce the bug
Run a program that includes an button on_press handler that invokes subprocess.run([sys.executable, ...])
When the subprocess is executed, a second copy of the application will start.
Minimum example code
import subprocess
import sys
def on_press(widget, **kwargs):
result = subprocess.run([sys.executable, "-c", "print(f'hello subprocess')"])Screenshots
No response
Environment details
- Operating system and version: All desktop OS
- Python version: All (tested recently on 3.13)
- Software versions:
- Briefcase: 0.3+ (tested recently on 0.4.5)
- Toga: 0.5.7
Logs
Additional context
This issue has been long standing; see:
- Stub binary doesn't support
sys.executablespawn/multiprocessing briefcase-linux-system-template#4 - Stub binary doesn't support
sys.executablespawn/multiprocessing briefcase-linux-flatpak-template#10 - Stub binary doesn't support
sys.executablespawn/multiprocessing briefcase-macOS-app-template#7 - Stub binary doesn't support
sys.executablespawn/multiprocessing briefcase-macOS-Xcode-template#20 - Stub binary doesn't support
sys.executablespawn/multiprocessing briefcase-windows-VisualStudio-template#8
There has been some recent movement on this issue:
- Document the
allow_subprocessesoption #3077 - Support multiprocessing in spawn mode briefcase-linux-system-template#61
- Support multiprocessing in spawn mode briefcase-linux-flatpak-template#86
- Remove Python binaries by default briefcase-macOS-app-template#111
- Support multiprocessing in spawn mode briefcase-macOS-Xcode-template#140
- Do not remove python*.exe if allow_subprocesses is set briefcase-windows-app-template#122
- Support multiprocessing in spawn mode briefcase-windows-VisualStudio-template#128
Activity
freakboy3742 commentedon Oct 2, 2026
I've tested the in-progress PRs (many thanks to @abdnh for that) - and they work great... with one side effect.
The PRs are based on re-writing
sys.executableto point at an actual Python binary. Historically, we have strippedpython.exefrom the packaged app; the proposed fix is to add anallow_subprocessesoption that will not strip the binary, and modifiessys.executablein the app's interpreter. We shouldn't includepython.exein all apps, as it is a full Python interpreter (which represents a very attractive attack surface); but there are enough uses formultiprocessinget al that it's worth including as an option.The complication is that Toga currently uses the name of
sys.executableto determine:If we rewrite
sys.executable, those techniques break.We can modify how Toga operates - see beeware/toga#4742 for a PR that modifies Toga to use
sys.orig_argv[0]instead ofsys.executable. However, we can't fix that retroactively.So - we have a decision to make. The options, as I see them:
0. Do nothing
Status quo; and we declare multiprocessing as officially unsupported. Not a very satisfying option, but it's an option.
1. Accept short-term breakage in subprocess-using Toga apps
Merge #3077 and the template changes; release Briefcase 0.4.6. Merge beeware/toga#4742.
Existing Briefcase apps won't change. However, it won't be possible to use the new feature until there's a Toga release that includes the beeware/toga#4742 change - based on normal schedule, that would be in December.
It's unclear how this will affect non-Toga apps. The default Qt and PySide CE apps both start without any problem, but it's impossible to know what usage of
sys.executableexists in the wild.2. Release a fixed Toga
Same as 1, but we also push out an early Toga release as well so there's a Toga release that will work with the new Briefcase feature.
Again, existing apps won't change; but Toga apps will have a path to using the new feature as long as they adopt Toga 0.5.8 as a minimum. This would be the default for any freshly generated app, but adding subprocess support to an existing app might need a bump to Toga version pins. However, that's fully opt in, so it's a safe option.
3. Add a safety catch
Same as 2, but we plan to also add a safety catch for
allow_subprocesses = Falseapps that try to invokesys.executable. If we setsys.executable = Noneif the app hasallow_subprocesses = False, the failure mode is a lot more pleasant - you don't get a spawned second copy of the app - you get a TypeError becausesubprocess.run()doesn't accept None as the first argument.This would be a great safety mechanism for any user who tries to call
subprocess.run([sys.executable, ...]) *without* usingallow_subprocesses` in their Briefcase configuration. I managed to fork-bomb myself more than once in testing this subprocess handling; that strikes me as something we should protect against if we can.However, we can't do it it immediately, because any existing Toga app would break with the new Briefcase version. There would be a fix - update your Toga version - but that's not something we can easily enforce without making a special case of Toga in Briefcase's code. So - we defer making this change for 6-12 months to give people a chance to update their Toga apps "organically" before they're forced to update.
4. Add a safety catch now
Same as 3, but we add the safety catch now, and document the requirement. Users either don't upgrade Briefcase, or must upgrade Toga.
I'm not sure which option is best - 4 is the best outcome and easiest to implement, but I'm unsure if it leans on people reading release notes a little too much.
Any thoughts?
abdnh commentedon Oct 2, 2026
What if we implement a safety catch in the form of detecting if the app is trying to run the stub binary as a Python subprocess?
Similar to my previous approach in beeware/briefcase-windows-VisualStudio-template#128 but with the difference that we fail with an error instead of running Py_RunMain().
mhsmith commentedon Oct 3, 2026
I agree; there's a lot of code in the wild that's at risk of doing that.
The
sys.executabledocumentation also mentions an empty string as a possible value. That should allow old Toga versions to give the correct result fromis_bundled, though I don't know what would happen with the icon code.If the only effect is that the combination of an old Toga and a new Briefcase would lose the app icon, I think that's acceptable, as long as it appears in the Deprecations and Removals section of the release notes.
If it would be more convenient, we could also delay merging this until after the upcoming Briefcase release, so both Briefcase and Toga would release their changes at the same time in December.
freakboy3742 commentedon Oct 5, 2026
Ah - that's actually much better, for two reasons:
is_bundledon historical Toga installssubprocess.run([""])raises aPermissionError, which is a more meaningful error response thanTypeError(which is what asubprocess.run([None])generates).We'd still need to update
toga.App.is_bundledbecause in new code,sys.executablewill point atpython.exein bundled code - but at least it won't break historical installs.As far as I can make out, this is the only impact - and even then, only on Windows and Linux. macOS uses a native bundle API, so changing sys.executable doesn't impact things.
I guess that's the easy approach - defer landing this for a week or so, make it the first new feature in the next Briefcase release, and then release both Toga and Briefcase in December.
freakboy3742 commentedon Oct 5, 2026
I've updated the stub apps to use
""forsys.executable; I've added aremovalnote to #3077 indicating the change tosys.executablehandling; and I've moved beeware/toga#4742 out of draft mode.At this point, we can wait until the Briefcase release next week, land the Briefcase and template PRs, and re-tag the stub binaries on Windows and macOS. Toga was released last week, so beeware/toga#4742 can be landed at any time.