Finds the sandbox escape hiding in the AST-walking Python interpreters that AI-agent frameworks trust to run untrusted code.
A small, read-only static detector for a dispatch asymmetry in AST-walking
restricted Python interpreters: attribute reads go through a dunder guard, but
the callable for a method call is fetched with a raw getattr that skips it.
Where this owner's llm-safety-auditor
red-teams a model at runtime, guard-gap-detector stays entirely static: it reads
one narrow structural pattern out of source with the standard library and runs
nothing.
These interpreters are the kind of in-process "sandbox" that some AI-agent frameworks use to run model-written code without a container or VM. On the shipped synthetic corpus the detector separates all 3 vulnerable fixtures from all 5 clean ones (precision, recall and F1 1.00; see Reproduce the evaluation). Standard library only.
In a vulnerable interpreter, a bare attribute read (obj.attr) is routed
through a guard that rejects dunder names (__globals__, __dict__, …), but the
callable fetched for a method call (obj.method(...)) is fetched with a raw
built-in getattr that skips that guard. The read path is protected and the
call path is not — an instance of what the accompanying paper calls a failure of
guard completeness: a guard is only as good as the set of dispatch paths it
actually covers.
This tool answers one narrow, auditable question:
Is the callable for a method dispatch assigned from an unguarded built-in
getattr(obj, name)and then invoked as a callable?
It keys on that call-path shape alone; it does not inspect or verify the
attribute-read path, so it is not an exploit, a general sandbox-robustness
prover, or a guard-completeness oracle. It parses source with Python's ast
module and never imports or executes the file under test. It emits no payloads.
No dependencies beyond the Python standard library (Python 3.8+).
git clone https://github.com/shaikn6/guard-gap-detector
cd guard-gap-detector
python3 -I detector.py corpus_synthetic/vuln_minimal.py # scan one or more files
python3 -I detector.py --json corpus_synthetic/vuln_minimal.py-I runs Python in isolated mode (no implicit sys.path entries), which is the
right habit when pointing an analysis tool at files you do not control. Output is
VULNERABLE/clean per file, with the line of the offending dispatch when found.
The repository ships a synthetic, author-written corpus (corpus_synthetic/)
of eight fixture files that contain no third-party source and no exploit payload,
only the structural cases the detector must separate:
- three toy AST-walkers carrying the unguarded method-call dispatch (a minimal one, one with renamed identifiers, and one in an independent non-lineage style);
- five that must stay clean: a
safe_getattrwrapper patch, a guard-complete recursive evaluator, a hard negative that calls built-ingetattronly for a read whose result is never invoked, and two non-interpreter modules.
Run the harness against the shipped manifest:
python3 -I evaluate_detector.py corpus_synthetic/manifest_synthetic.tsvExpected result: precision 1.00, recall 1.00, F1 1.00 (3 true positives,
5 true negatives) — reproduced 2026-11-15 on CPython 3.13, and pinned by the
test suite on 3.8/3.11/3.13 in CI. These numbers are a regression pin on
eight author-written fixtures, not an accuracy estimate against real
interpreters in the wild. The harness also runs against any other
<label>\t<path> manifest you supply, and exits non-zero if a manifest row
points at a file it cannot read or parse.
It flags an interpreter when both hold: (i) the file branches on
ast.Attribute (the gate that keeps it from flagging ordinary
f = getattr(o, n); f() helper code), and (ii) it assigns
name = getattr(obj, attr) to a plain name with the two-argument built-in
getattr and then invokes name as a callable. A dispatch routed through a
guarded wrapper (safe_getattr, guarded_getattr, …) or through a recursive
evaluator is, by construction, not flagged.
Misses (false negatives). It will miss a copy that invokes the callable
without a plain-name assignment (getattr(o, n)(...)), that assigns to an
attribute or subscript target or with an annotation (self.f = …, f: C = …),
that wraps the fetch in a helper function, that passes a third default
argument to getattr, that aliases or otherwise obtains getattr indirectly,
or that uses a non-ast parser.
Scope and read-path blindness. The name = getattr(...) / name(...) match
is file-wide, not scope-aware, so it over-flags when one function reads
x = getattr(o, n) and an unrelated function calls a different x(). Because it
never inspects the read path, it also flags an interpreter whose call path
carries its own inline (non-wrapper) dunder check, and it reports the unguarded
call-path shape even when reads have no dunder guard at all (there the shape is
real; the read/call asymmetry the tool is named for simply isn't present).
It says nothing about whether a flagged interpreter is actually reachable by
attacker-controlled input in a given deployment. A file it cannot read or parse
is printed as clean with exit 0 by the CLI (the evaluate_detector harness,
by contrast, fails loudly on such a row). Treat a hit as a prompt to read the
dispatch path by hand, not as a verdict.
Route the method-call callable through the same dunder-blocking wrapper the
interpreter already uses for attribute reads, instead of a bare getattr. That
is a one-call-site change, not new logic.
This is the released artifact for the measurement paper "When Vendoring Defeats the Patch: A Cross-Repository Measurement of a Sandbox-Guard Bypass in AI-Agent Code Interpreters" (Shaik, 2026; preprint forthcoming). The paper describes the mechanism, a cross-repository measurement, and a responsible-disclosure process. Consistent with that disclosure process, this repository names no affected project and ships no third-party source or exploit code.
MIT — see LICENSE.
Part of Nagizaaz Shaik's open-source work · all repositories