Skip to content

fix(taint): scope taint to functions and bind call-site arguments - #799

Open
rng1995 wants to merge 20 commits into
mainfrom
naren/fix-tt3-function-scope
Open

rng1995 wants to merge 20 commits into
mainfrom
naren/fix-tt3-function-scope

Conversation

@rng1995

@rng1995 rng1995 commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

behavioral_taint_tracking keys taint by bare variable name across the whole file. Since 26bc7d6 (#611) made propagation independent of AST visit order, a tainted local taints every same-named variable or parameter in the file. Example: a headers dict built from an environment token in one function taints an unrelated headers parameter in another function, and that is reported as TT3 (CRITICAL) at a request that never sees the credential. Before #611 such clashes fired only when the breadth-first visit order happened to line up, so 2.12.0 reported fewer of them. This regression is unreleased on main.

This PR keys taint by lexical scope and follows calls whose callee is known from syntax alone.

Revision note. The first two revisions also inferred receiver types, bound unknown receivers into every same-named method, followed callbacks, and kept one fact per source. Two review rounds showed that this kept producing denial-of-service paths and recall or precision regressions, so the third revision narrowed interprocedural flow to a bounded design. The third review round fixed two quadratic paths (one lambda bound to many names, nested lambda defaults), unpacked arguments filling self/cls, keywords double-bound into **kwargs, class-body reads, and messages that changed while the rule stayed the same. The fourth round removed per-name **kwargs forwarding (a fan-out and a false-positive source), stopped a source call's arguments from raising its rule, made sinks cite main's variables before newly bound parameters, and fixed several smaller precision gaps. The dropped features, and why, are listed under Not resolved.

Design

The analyzer changes are all in src/skillspector/nodes/analyzers/behavioral_taint_tracking.py, with tests in tests/nodes/analyzers/test_behavioral_taint_tracking.py and one Unreleased line in CHANGELOG.md.

  • Scopes follow Python's rules. Function locals and parameters are per function; module globals keep their bare keys. global/nonlocal and closures are honored. Class bodies are invisible to methods. Lambdas and comprehensions are scopes; a walrus binds in the enclosing function (PEP 572). A class-body read of a name the class also binds sees the class's value or, before the class binds it, the module global (CPython's LOAD_NAME), so it reads both, through a per-(class, name) view; the class's own value, and so self.x, never takes the global's taint. An annotation without a value binds nothing in a class body. Names resolve in one depth-first pass over the scope tree with a stack per identifier, so each name costs O(1) however deep the nesting.

  • Keys are small integers built from a scope id and a reference to the identifier string already in the tree. No key embeds a qualified name.

  • Direct calls by name to functions defined in the file, including nested functions visible lexically and lambdas assigned to a name:

    • positional and keyword arguments, defaults, and *args/**kwargs forwarding bind to parameters through per-callee argument slots: 16 positional slots, then one shared overflow slot; an *args unpacking fills a slot keyed by the parameter it starts at;
    • a keyword that names a parameter binds only that parameter, never also **kwargs (positional-only names do reach **kwargs);
    • a forwarded **kwargs (a function's own ** parameter passed on, as in super().__init__(**kwargs)) binds only into the callee's own ** parameter: which names it holds is not tracked, so it never fills named or keyword-only parameters. Any other **mapping fills every keyword parameter;
    • unpacked and overflow arguments of a bound call never fill the implicit self/cls; every argument costs one slot per callee definition group, whatever its shape;
    • every definition bound to one name is one callee, and a lambda bound to several names (a = b = lambda ...) is one callee they share, so binding work is linear in calls and parameters, not names × parameters;
    • a call reads its callee's return summary.
  • Returns are per call site. A callee's return summary carries only facts that do not depend on its parameters: facts bound from a call site never enter it. What a call returns from its own arguments is read at that call site, from those arguments (a call's value includes its arguments, as before). A source call is the exception, as on main: its value is what it reads (a file, a response, an environment value), never its arguments, so json.load(open(os.getenv(...))) is a file read. One caller's secret therefore cannot taint another caller's result, also through closures and lambdas. Summaries are computed by the same monotone worklist as everything else, so recursion terminates.

  • Methods, only where the callee is syntactic:

    • self.m(...) / cls.m(...) bind to the class's own m or that of its nearest in-file base. The lookup follows C3 (method resolution) order over all bases and is precomputed per class, keeping the first 8 classes (the class included); each base's own order is already cut at 8;
    • ClassName.m(obj, ...) binds with the unbound offset (obj is the first parameter), and super().m(...) starts after the class;
    • ClassName(...) / cls(...) bind to __init__ and __new__, and ClassName().m(...) / cls().m(...) call m on that class (arguments bind, the return value is read);
    • aliases of a class (B = Base; B.m(...), self.__class__.m(...)) and instances held in variables or attributes (v = Vault(); v.load(), self.vault.load()) bind nothing, as on main. See Residual for the return values this loses.
  • Attributes. self.x / cls.x stores and class-body names are attributes of the class. A read sees self.x stores in its class and its in-file bases, and the class-body x of the nearest of those classes whose body binds x in any way: an assignment, annotated assignment, method or import in a subclass shadows its base's value (only an assignment's value is modelled). Never siblings or subclasses. Attributes are class state, so a value stored from a parameter is read back by getters.

  • Rule and message per sink. Each key keeps three facts: the most severe source for each of two sink lanes, and the first source to arrive:

    • network output: credential > file read > external input (TT3 > TT4 > TT2);
    • code execution and deserialization: external input > file read > credential (TT5/TT6 > TT2);
    • first to arrive, unranked, in main's propagation order.

    The sink's ranked lane decides the rule, so a helper's network or file source can never hide a credential at a network sink, and a helper's credential can never hide user or network input reaching exec or pickle.loads. The message cites the first-arriving source whenever it gives the same rule, so a more severe source changes the message only where it changes the rule. When the three agree (almost always) they share one fact that propagates once.

  • Main's flows keep main's messages. Propagation runs in two phases. Phase A replays main's assignment flows in main's order, plus a class body's read of the module global under a name the class also binds (one name on main). Phase B adds argument binding, returns, attributes and defaults; it only fills keys phase A left clean or raises a ranked lane to a more severe source, and never replaces the first-arriving fact. At a sink, variables phase A tainted are cited before those only phase B taints (bound parameters, helper results). A finding both versions report with the same rule therefore has main's message, except in the cases listed under Messages below.

  • Messages name what the sink spells: the variable, self.x, helper(), self.m(), Class().m() or super().m(). Plain variables come first. Identifiers longer than 120 characters are shortened to prefix... in every spelling, so message bytes stay bounded per finding; real identifiers are unaffected.

  • Bounds. Indexing, name resolution, flow building, callee linking and propagation are linear in the file, and each of their loops calls check_runtime. Each flow fires at most once per (lane, source rank, slot): at most 14 times, whatever the number of sources. A default expression is scanned once, for its own function: a nested lambda in it is not walked again (only the body of an immediately called lambda, whose value the default holds). One key read by many flows checks the deadline every 1,024 flows, and binding checks it per argument.

Not resolved

These were in earlier revisions and are dropped. Each produced denial-of-service paths or recall/precision regressions.

  • Receiver type inference (per-scope type maps, annotations, with ... as, cast and builtin results) and @property reads. obj.m(...) on any receiver other than self, cls, a class name, super(), ClassName() or cls() binds nothing and reads no return value (the lost return values are listed under Residual).
  • The untyped-receiver fallback that bound x.m(...) into every same-named method in the file.
  • Callback rules (Thread(target=...), executor.submit(f, ...), run_in_executor): bind nothing, as on main (see Residual).
  • Class hierarchy families and the shared-descendant attribute rule.
  • Per-source fact sets, replaced by the lanes above.
  • Comprehension iteration flows. main does not model loop or comprehension targets, and these flows changed which variable the message cites at a sink.

Bounds (measured)

Every denial-of-service shape from the three review rounds, at the 1,000,000-character file cap, run with btt.node() on main (8f4ca7d, whose taint analyzer matches 3c8e4b9) and this branch at fcd8620. The later main merge brings in the urllib opener sink (#721) and source classification (#579); neither touches the flow analysis. Time is the better of two runs; memory is the tracemalloc peak of a separate run, parse included. Each run also analyzes a second file with a real TT3 under a 60 s workflow budget (30 s for some rows). The machine was shared with other jobs, so treat ratios as approximate.

Repro KB main s this s main MB this MB second file's TT3
one lambda bound to 72,300 names, 72,300 parameters (new) 990 0.77 1.24 139 139 reported on both
same, 39,358 names each called with **q (new) 990 1.50 2.52 255 255 reported on both
lambda defaults nested 600 deep × 149 (new) 984 2.40 3.51 428 428 reported on both
9,539 subclasses forwarding super().__init__(*a, **k) (new) 990 1.44 3.03 223 223 reported on both
10,000-char callee, 485,000 positional args 990 2.01 4.73 469 469 reported on both
10,000-char function with 66,000 locals 990 1.40 1.83 179 179 reported on both
50,000 class C + 123,000 C() 990 2.27 4.13 497 497 reported on both
23,600 redefined C with m(), 23,600 c.m(1) 990 1.53 3.52 291 291 reported on both
10,000 class C + def C, C( nested 190 deep × 1,500 990 3.17 7.36 542 587 reported on both
10,000-char callee, 108,000 keyword args into **kw 990 0.99 1.86 188 188 reported on both
63,000 keyword-passed params 990 0.77 1.62 175 175 reported on both
long class/method, 20,700 self.a{i} stores and loads 990 1.26 1.42 141 141 reported on both
90 nested defs with 11,000-char names 990 0.01 0.05 4 4 reported on both
Thread(target=LONG, args=...), submit, kwargs= 990 1.20 1.36 254 254 reported on both
10,000-char lambda, 485,000 args 990 2.54 4.60 498 498 reported on both
30 classes with 15,000-char m(**kw), untyped o.m(...) 977 0.01 0.05 3 3 reported on both
150,000-char receiver, 9,700 self.a sinks 990 0.54 0.74 72 72 reported on both
150,000-char receivers ×3, self.a stored in the class 990 0.13 0.23 18 18 reported on both
long function, 31,600 sinks 990 1.40 1.57 212 212 not reached (finding cap), as on main
15,700 Outer each with class C + C(1) 990 1.54 2.59 208 208 reported on both
same with c = C(); c.m(1) 990 2.00 3.05 264 264 reported on both
14,900 subclasses each with class C + C(1) 990 1.58 2.47 216 216 reported on both
c = K{i}() ×16,900, c.m(1) ×16,900 990 2.14 3.13 273 273 reported on both
self.x = K{i}() ×13,700, self.x.m(v) ×13,700 990 1.88 3.03 242 242 reported on both
41,000 def f + 41,000 f(1) 990 1.74 4.16 374 374 reported on both
20,900 classes with m, 20,900 untyped o.m(1) 990 1.22 2.98 259 259 reported on both
14,500 global C; class C + 14,500 C(1) 990 0.96 2.38 215 215 reported on both
19,800 classes on self.v, print(self.v.m, ...) ×39,700 990 1.92 2.44 232 232 reported on both
20,800 classes on x, 78,400 y = x.a 990 4.02 4.68 392 392 reported on both
23-source global, 201 functions × 1,000 locals 989 0.96 2.12 158 216 reported on both
23-source global, 100,000 locals 990 2.45 3.26 278 278 reported on both
40 nested functions, 493,000-target assignment 990 2.23 2.98 339 339 reported on both
print(f, print(f, ...)) 190 deep × 578 990 2.03 2.43 318 318 reported on both
443 × 32-class multiple-inheritance chains 988 1.33 2.08 233 233 reported on both
200 nested lambdas, 137,000 free names 990 1.30 1.03 171 171 reported on both
400 nested lambdas, 137,000 free names 990 1.36 1.01 171 171 reported on both
in-file f(f(...)) 190 deep × 1,730 990 3.79 9.61 594 712 reported on both
496 × 32-class chains, self.m0() in each 988 1.15 2.87 231 231 reported on both
one 14,000-class inheritance chain, self.m0() in each (new) 990 1.24 2.67 214 214 reported on both
5,000-char tainted name, 30-deep nested sinks 988 1.31 1.42 81 33 not reached (finding cap), as on main

Every file completes (or stops at the finding cap, as on main), and every second-file TT3 is reported except where both versions stop at the finding cap. Time is 0.7-2.5x main (at most 9.6 s, on the nested in-file calls shape). Peak memory matches main except for the 23-source shape (+37%), nested in-file calls (+20%) and nested constructor calls (+8%). At the reviewers' smaller sizes, the shapes this round fixed were: one lambda bound to 8,000 names, 3c12146 6.9 s / 4.2 GB, now 0.14 s / 15 MB (main 0.08 s); defaults nested 400 deep (220 KB), 3c12146 25.0 s, now 0.97 s (main 0.79 s). Each shape also runs as a test at 1x and 4x size. The test counts the analyzer's executed lines and loop iterations, and fails above 5x growth (a quadratic path grows about 16x).

Paths changed in the fourth round, measured again at this head (same machine and method; main 8f4ca7d, previous head 78b571c):

Repro KB main s 78b571c s this s main MB this MB
forwarded **k × 100,000 into a 16-parameter, multi-definition callee 401 0.82 6.89 1.76 146 146
same, × 247,288 (at the cap) 990 1.99 16.68 4.82 362 362
9,539 subclasses forwarding super().__init__(*a, **k) 990 1.32 2.86 2.73 223 223
lambda defaults nested 600 deep × 149 984 2.27 3.19 3.45 428 428
lambda defaults nested 640 deep through y if y else ... × 64 983 1.90 2.43 2.20 293 293
13,300 classes reading a class-bound name in the body 990 1.61 2.47 2.59 200 200
22,000 json.load(open(os.getenv(...))) 990 2.10 2.70 2.58 208 208
one lambda bound to 39,358 names, each called 990 1.39 2.67 2.71 255 255
in-file f(f(...)) 190 deep × 1,730 990 3.65 10.03 10.04 594 712
23,600 redefined C with m(), 23,600 c.m(1) 990 1.55 2.94 3.34 291 291

The time ratio to main stays at or below 2.8x (nested in-file calls), and memory is unchanged except nested in-file calls (+20%, as before). With four 1 MB forwarded-**k files followed by a file holding a real TT3, under a 60 s budget:

  • main takes 9.7 s and reports the TT3;
  • 78b571c ran out of budget at 61 s and lost the TT3;
  • this head takes 22.0 s and reports it.

Deadline adherence on one such file: with a 10 s budget, 78b571c returned after 17.7 s. This head completes in 6.2 s, and with 3, 4 and 5 s budgets it returns within 0.25 s of the deadline.

Impact (measured)

The corpus: every Python file in anthropics/skills @683bc88 and NVIDIA/skills @2c188f5 (738 files), the repo's test fixtures (13), and 1,032 deduplicated repro files from the review rounds. TT findings were compared per file against main (5edc347; its TT output on this corpus is identical to 3c8e4b9's), at this head. Each finding main reports that this head drops or reports with a different message was checked by replaying main's name-keyed propagation from the sink to the source, then checking the scope of every link with this head's resolver.

Public skills main this head
TT findings 64 81
TT2 / TT3 / TT4 / TT5 47 / 16 / 0 / 1 61 / 19 / 1 / 0
  • Same rule and line, same message: 26. Same rule and line, different message: 27, every one from a main chain that crosses into a same-named variable in another scope (a name clash). None is a same-rule source swap.
  • Dropped: 11, all name clashes, including the 3 TT3s and the TT5 main reports that this head does not (e.g. a parameter requested tainted through a same-named local in another function).
  • Added: 28 (21 TT2, 6 TT3, 1 TT4): call-site arguments, helper returns, constructors and self. attributes.
  • Review repros:
    • 38 changed messages: 37 name clashes and 1 identifier over 120 characters;
    • 73 dropped findings: 54 name clashes and 19 raised to a more severe rule.
    • The fixtures are unchanged.
  • Lost real flows against main: none in this corpus. The Residual list names the flow shapes main catches only through a name coincidence, and the keyword forwarding this revision no longer follows.
  • Messages: a finding both versions report with the same rule has main's message, except:
    • where main's fact came from a same-named variable in another scope, including a same-named parameter. This head then cites the variable, source and line that actually reach the sink, often the binding call;
    • where a more severe source raises the rule at that sink (TT2 to TT3 or TT4, TT4 to TT3, TT2 to TT5 or TT6). The raised finding names the more severe source, and a weaker finding at the same sink is either subsumed or now cites another variable;
    • where an identifier is longer than 120 characters (shortened to prefix...).
  • Baselines:
    • Message-glob rules keep matching every TT finding whose rule and message are unchanged, and stop matching exactly in the three cases above.
    • Exact fingerprints are bound to the scanner version and suppress nothing after an upgrade (docs/SUPPRESSION.md), so a release that ships this must regenerate them in any case. Within one scanner version, they too match exactly the unchanged findings.
    • match_fingerprint changes only when the rule does.
    • Round trip, with an exact baseline generated by main's CLI at the same scanner version and a scan by this head with --baseline, over 10 skills: the 4 public skills with clash corrections keep 2-9 TT findings each active. Those are findings whose message or rule differs because main's fact came from a same-named variable in another function, plus 2 new flows. The converter probe skill from the review keeps 3 active: 1 raised rule and 2 new flows. The thread-17 and lane-ranking probe skills, the malicious fixture, bionemo-evo2-nim and cuopt-server-api-python keep none active. No non-TT finding is active anywhere. The message-glob baseline from the review (*from os.getenv*, *file read*, *pathlib.Path.read_bytes*) still suppresses all of its targets.

Tests

  • main's 127 taint tests (including the 50 fix(taint): detect inline urllib opener network sinks #721 added) pass unchanged; the file has 301 taint tests.
  • New tests cover:
    • name-clash precision for each class of clash found in real skills, and for function-local import ... as, except ... as, match captures and later comprehension iterables;
    • direct calls:
      • helper returns;
      • argument binding: positional, keyword, *seq at any position, **mapping into keyword-only and **kwargs parameters, and the overflow slot at exactly index 16, also into *rest;
      • keywords that never double-bind into **kwargs, and forwarded **kwargs reaching only the callee's **kwargs;
      • defaults, including an immediately called lambda in a default;
      • forwarding wrappers, recursion, nested helpers, closures, lambdas bound to many names;
    • methods:
      • self./cls. methods and attributes without sibling leakage, unbound Class.m(obj, ...);
      • super(), also in comprehensions and lambdas, and when shadowed;
      • constructors through __init__ and __new__, Class().m() / cls().m(), and self() / a plain cls() not constructing;
      • C3 order (diamond, local precedence, more than 8 bases) and the ancestor bound through single and multiple inheritance;
      • class-body shadowing by any binding but a bare annotation;
      • unpacking that never fills self/cls, and method kinds;
    • class-body reads under LOAD_NAME, without tainting the class's own value;
    • source lanes in both orders, late upgrades in both ranked lanes, phase A variables reaching helpers in every lane;
    • message compatibility:
      • same-rule findings keep main's exact message;
      • a source call's value is what it reads;
      • main's variables are cited before bound parameters;
      • the first-arriving source through parameters;
      • every sink spelling capped at 120 characters, and super().m() spelled as written;
    • the deadline inside each stage: 19 cases, one loop each, counting only that stage's checks;
    • every DoS repro above, as an executed-work scaling test, a fact count, a byte count or an absolute bound (forwarded **k costs at most 2x an opaque mapping).
      • The scaling tests count lines, jumps and branches, so one-line comprehension loops count.
      • Each new shape grows its expensive dimension and fails on the design it guards: 3c12146 grows 14.8x and 13.4x where the limit is 5x, and a root-only nested-lambda revert grows 13.7x.
  • TestNotResolved pins main-like results for the dropped features, and that return values through instance variables are not read.
  • Mutation check of the changed code, at 9a938f3: 126 one-edit mutants, 121 killed.
    • 102 are semantic edits across scopes, binding, forwarded **kwargs, return summaries, propagation lanes and citation order, attributes, C3 and bounds, DoS guards and message spellings. The other 24 each remove one deadline check.
    • The 5 survivors are equivalent: comprehension-target visibility, comprehension and value-key locality, and the default-scan root (those keys never carry facts, or feed only binding flows), plus recording a read view for class-body stores, which no scan reads.
    • The slot-cap >=/> mutant, wrongly counted as equivalent before, is now killed by the *rest boundary test.
  • make lint and make format-check pass. The full suite (pytest tests -n 8) gives 13,032 passed, 0 failed.

Residual

  • Lost true positives main caught only by coincidence (its caller's variable and the callee's local or parameter share a name):
    • return values and arguments through instances held in variables, attributes or annotated names (v = Vault(); v.load(), self.vault.load(), v: Vault = ...), factory results, class aliases;
    • callbacks, @property reads, ClassName.attr reads;
    • subclass overrides reached from a base, and a base reading attributes only a subclass sets;
    • cross-file helpers, and dynamic dispatch through dicts or getattr;
    • a class nested in a function reading the function's local under a name the class binds (CPython raises NameError there).
  • Not followed: keyword forwarding. A forwarded **kwargs reaches only the callee's own **kwargs. Sub(api_key=secret) → super().__init__(**kwargs) → Base.__init__(self, api_key=None) is not followed into api_key; main binds nothing here either. Positional forwarding (*args) is followed.
  • Reporting trade-off: one fact per key and lane. A weaker finding on a variable that also carries a more severe source is subsumed, so finding counts at a sink can drop while the sink's maximum severity never does.
  • Over-approximations, as on main:
    • flow-insensitivity;
    • a call's value includes its arguments (except a source call's);
    • attributes are per class, not per instance, and class redefinitions merge;
    • a subclass's self.x = None does not hide a base's class-body x;
    • parameters merge their call sites (context-insensitive inside the callee).
  • Over-approximations of the new binding:
    • a **mapping built elsewhere can fill any keyword parameter of the callee except an implicit self/cls, and an *args unpacking can fill any parameter from where it starts;
    • a source used as a parameter default (def f(token=os.getenv(...)): return token) enters the return summary, so f("public") is tainted too;
    • redefinitions merge into one callee, including every name a lambda is bound to. When two merged definitions take different named parameters, a keyword one of them names can reach the other's **kwargs;
    • each base's C3 order is cut at 8 classes, so a hierarchy deeper than that can differ from Python's order past the classes kept.
  • Still not modeled, as on main: AugAssign, AnnAssign values, for/with/walrus targets, container stores and .update()/.append() mutation, yield.

🤖 Generated with Claude Code

Taint was keyed by bare variable name across the whole file. Since
26bc7d6 (#611) made propagation independent of AST visit order, a
tainted local taints every same-named variable or parameter in the file:
a `headers` dict built from an environment token in one function taints
an unrelated `headers` parameter in another, which is reported as TT3
(CRITICAL) at a request that never sees the credential. Before #611 the
same clash fired only when breadth-first visit order happened to line
up, so 2.12.0 reported fewer of these.

Key taint by lexical scope instead, and model the flows that bare names
only approximated by coincidence:

- Per-function namespaces following Python's rules: parameters and names
  bound in a body are local, global/nonlocal are honored, free names
  resolve through enclosing functions (closures) to the module, and
  class bodies are not visible to methods. Module globals keep bare-name
  keys.
- Call-site binding: arguments of calls to functions defined in the file
  flow positionally, by keyword, or into *args/**kwargs to the callee's
  parameters through per-callee argument slots, so binding stays linear
  even for many same-named methods. Parameter defaults flow to their
  parameter, and a function passed as an argument
  (Thread(target=f, args=...), executor.submit(f, ...)) receives the
  arguments after it.
- Return values: `return expr` taints the function's result, which every
  call to it reads, so `key = get_key()` and `post(data=get_key())` keep
  flowing.
- self.<attr>, cls.<attr> and class-body names share one attribute
  namespace per in-module inheritance family.
- Messages cite the line of the original source call rather than the
  last assignment that copied it.

Lambdas and comprehensions stay folded into the enclosing scope, which
can only over-approximate. The fixpoint remains a monotone worklist that
fires each flow at most once, with runtime checks per visited node and
per dequeue.

On a 141-skill corpus (including NVIDIA/skills and anthropics/skills),
the 10 TT findings that main reports only through unrelated same-named
variables (3 TT3, 1 TT5, 6 TT2) are gone. Every other TT finding on main
is still reported, and 28 more flows through helper returns and call
arguments are found.

This composes with the open #754 (retain sources across reassignment).
Both change the same four places: _mark_targets, the worklist item, the
fired set and the sink lookup. A mechanical port of #754 onto this
change passes both test suites.

Tests (the 77 existing taint tests pass unchanged):
- name clashes stay clean: an unrelated parameter in either definition
  order, the same local in two functions, a method parameter, a local
  shadowing a tainted global, the sibling of the bound parameter, and
  instance attributes of an unrelated class;
- recall: helper returns (same name, other name, inside sink arguments,
  chained), positional, keyword, unpacked and direct-source arguments,
  *args/**kwargs, defaults, callbacks, a global set inside a function,
  closures, nested-function arguments, nonlocal rebinding, attributes
  set in __init__ or from constructor arguments, class attributes,
  inherited attributes, calls on instances, self and staticmethods,
  input reaching exec through a helper, recursion, and a token passed
  through a header builder into a request helper;
- messages cite the source call line through reassignments, multi-line
  assignments and calls;
- binding stays linear for 400 same-named methods and call sites, and
  arguments beyond the positional slot cap still bind.

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review (6 independent lenses: Python scoping, security recall/evasion, DoS bounds, output compatibility, precision of the new findings, tests; every finding re-run on main and on this branch, and kept only if at least 2 of 3 skeptics who tried to refute it reproduced it). 26 findings raised, 25 survived; deduplicated to 18 comments: 2 P1, 12 P2, 4 P3.

Not ready to merge. The two P1s are resource-exhaustion paths on adversarial input that contradict the "linear on adversarial inputs" claim:

  • Memory: qualname-prefixed taint keys make memory grow as name length × references. A 220 KB file peaks at about 2 GB versus 98 MB on main.
  • Time: a per-call loop over same-named classes runs with no check_runtime in the statements loop. A 1 MB file overruns a 30 s deadline by about 350 s, and later files go unanalyzed.

Blocking test gap: no test makes the deadline expire inside _build_scope_index or _collect_tainted. Removing either check_runtime leaves the suite green.

Most P2s cluster in receiver resolution in callee_groups and are best fixed by one refactor. The others:

  • per-helper return and parameter keys are shared across call sites;
  • sibling-subclass family merging;
  • helper return edges that flip a TT3 to TT2/TT4 (composition with #754 matters here);
  • TT1+TT2 double reporting;
  • walrus-in-lambda shadowing.

Verified OK, no action needed:

  • deterministic output under hash randomization;
  • the 16-slot cap still binds overflow arguments;
  • global/nonlocal/closures/async def, positional-only/keyword-only defaults and PEP 695 generics don't crash;
  • the constructs listed as "not modeled" are also silent on main, so they are not regressions.
  • One finding was refuted: nested-call argument re-walks cost O(nodes × depth), but main has the same per-byte cost, so it is at most a memoization nit.

Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread tests/nodes/analyzers/test_behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread tests/nodes/analyzers/test_behavioral_taint_tracking.py Outdated
rng1995 and others added 2 commits October 7, 2026 22:12
…l site

Review follow-up for the function-scoped taint keys.

Resource bounds:
- Keys are small integers built from a scope or class id plus the
  identifier. No key copies a qualified name per reference, so memory
  no longer grows as name length times references. A 10,000-character
  function name called with 100,000 arguments peaks at 98 MB, the same
  as main.
- Redefinitions of a class merge into one class, and method lookups
  are cached, so same-named classes no longer cost a scan per call.
  1 MB of `class C:` plus `C()` calls takes 4.9 s, where it overran a
  30 s deadline by about 350 s.
- check_runtime now also runs per statement, per function and per
  post-indexing loop, not only per visited node and per dequeue.

Receivers. `obj.m(...)` resolves through obj's class, typed in obj's
own scope:
- self/cls and class objects; `Base.m(self, x)` passes self explicitly;
- `x = C()`, `x: C = C()`, `with C() as x`, `self.attr = C()`;
- `C().m()` and `cls().m()`;
- super() and super(C, self), along the in-file MRO.

`cls(...)` binds to `__init__`, and @Property reads return the getter's
value. Return values are read for every resolved receiver. Literal,
library and builtin receivers never bind into module methods. Unknown
receivers still bind arguments, but never for dunder methods.

Classes. Attributes are per class. A store in C reaches a read in E
when the two classes share a descendant (subclass, base, or mixin), so
sibling subclasses no longer share attributes or methods. Hierarchies
of more than 32 classes keep one shared namespace.

Call-site context. Taint that reached a function only through its own
parameters no longer enters its return value. Each call site reads its
own arguments, so one caller's secret no longer taints every other call
of a shared helper.

Sources:
- Each key keeps one fact per source (#754's per-variable source sets),
  and a sink reports the most severe rule. A helper's network or file
  source can no longer turn a TT3 into TT2 or TT4.
- A source call nested in a sink is reported once, as a direct flow,
  rather than again as a tainted flow through a helper in the same
  sink.

Scopes:
- Lambdas and comprehensions are scopes. A walrus binds in the lambda,
  or in the function enclosing a comprehension.
- Lambdas assigned to names are callable.
- A comprehension's iterable flows into its targets.
- Callback receivers may be attribute chains, constructors or
  parameters.

Messages keep main's wording: the variable read at the sink and the
line of the statement that last tainted it, with plain variables cited
first. A baseline generated on main suppresses the same TT findings on
this branch, both its exact fingerprints and its message-glob rules.
On the corpus, 49 of the 78 findings both versions report keep
identical text; 27 of the other 29 cited a same-named variable in
another function.

Corpus (141 Python skills): main 92 TT findings, this branch 109.
- Removed: 11 name-clash findings.
- Upgraded: 3 TT2 now TT3 on the same sink.
- Added: 31 flows main missed.
- Compared with the previous revision: no real flow is lost; 2 more
  flows are found through comprehension targets; 2 helper-return false
  positives are gone.
- Of the 28 targets with changed TT findings, 1 changes recommendation
  and exit code, from a new TT3 where an environment API key is passed
  into an Authorization header.

Tests: 144 taint tests, including #754's regression test. New tests
cover every review finding, and the deadline expiring inside scope
indexing, the statement loop and the worklist. 37 mutants of the new
paths each fail a test.

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two review rounds of the broad interprocedural design showed it kept
producing denial-of-service paths (per-call work proportional to the
number of candidate classes, unbounded receiver class lists, per-source
fact sets, messages copying long identifiers) and recall or precision
regressions (receiver typing that hid real callees, a fallback that bound
into every same-named method, callback rules that bound the wrong
arguments). This narrows interprocedural flow to calls whose callee is
known from syntax alone, in O(1) per call.

Kept:
- Lexical scopes. Function locals and parameters are per function and
  module globals keep bare keys; global/nonlocal and closures follow
  Python's rules; class bodies are invisible to methods; lambdas and
  comprehensions are scopes, with PEP 572 walrus binding. Names resolve
  in one depth-first pass over the scope tree with a stack per
  identifier, O(1) per name however deep the nesting. Keys are small ints
  built from a scope id and a reference to the identifier, never from a
  qualified name.
- Direct calls by name to functions defined in the file (including
  nested ones visible lexically and lambdas assigned to a name):
  positional and keyword arguments, defaults, *args/**kwargs forwarding,
  16 positional slots. Every definition bound to one name is one callee,
  so binding work is linear in calls, not calls x redefinitions.
- Per-call-site returns. A callee's return summary only carries facts
  that do not depend on its parameters; what a call returns from its own
  arguments is read at that call site from those arguments (a call's
  value includes its arguments, as before). One caller's secret no
  longer taints other callers' results, also through closures and
  lambdas.
- Methods, only where the callee is syntactic: self.m()/cls.m() bind to
  the class's own method or its nearest in-file base (a precomputed lookup
  over at most 8 classes), Class.m(obj, ...) with the unbound offset,
  super().m(), and Class(...)/cls(...) to __init__. Aliases of a class
  bind nothing, as on main. self.x/cls.x attributes are per class: a read
  sees stores in its class and in-file bases, never in siblings.
- One source per key and sink lane, the most severe for the lane:
  credential > file read > external input for network sinks, and
  external input > file read > credential for code execution and
  deserialization sinks. A helper's source can no longer turn a TT3 into
  TT2/TT4, nor a TT5/TT6 into TT2. When the lanes agree (almost always)
  they share one fact, which propagates once.
- Two-phase propagation. Phase A replays main's assignment flows in
  main's order; phase B adds argument binding, returns, attributes and
  defaults, and only fills clean keys or upgrades to a more severe source.
  A flow main reports keeps main's message, except where a more severe
  source now reaches the same variable.
- Messages name what the sink spells (never an attribute spelling from
  elsewhere in the file); identifiers longer than 120 characters are
  shortened, so message bytes stay bounded per finding.

Dropped, each a source of DoS paths or regressions:
- receiver type inference (per-scope type maps, annotations, with-as,
  cast and builtin results), MRO linearization beyond the bounded base
  lookup, and @Property reads;
- the untyped-receiver fallback into every same-named method;
- callback rules (Thread target, submit, run_in_executor): as on main;
- class hierarchy families and the shared-descendant attribute rule;
- per-source fact sets;
- comprehension iteration flows (main does not model iteration targets,
  and they changed which variable messages cite at sinks);
- lambda return summaries (a lambda's value is already read by name).

Bounds: indexing, name resolution, flow building, callee linking and
propagation are linear, and check_runtime runs in each of their loops.
Every flow fires at most once per (lane, rank, slot), 12 times whatever
the number of sources. On every DoS repro from both reviews, every file
completes and later files keep their findings; time is 1.0-2.2x main and
peak memory matches main, except +14% on a variable carrying all 23
sources.

Tests:
- main's 77 taint tests pass unchanged.
- New: name-clash classes found in real skills, direct-call recall,
  methods and attributes without sibling leakage, source lanes, message
  compatibility, deadlines in indexing, flow building, callee linking and
  propagation, and every DoS repro from both reviews as a work-scaling
  test (executed analyzer lines at 1x and 4x size, not wall time), a
  fact count or a byte count.
- TestNotResolved pins main-like behavior for the dropped features.
- Removed PR tests, with the reason:
  - receiver typing and the untyped fallback (library instance
    elsewhere, untyped parameter receiver, typed receivers reading or
    binding only their class, returns of constructed and attribute
    receivers, and their exec/file variants): the features are dropped;
    instance receivers now bind nothing, as on main;
  - callbacks on attribute, constructor and parameter receivers, Thread
    kwargs, Thread(target=Base.m): callback rules are dropped;
  - @Property reads, Config.TOKEN read through the class name, mixins and
    subclass overrides reached from a base: dropped with hierarchy
    families; overrides now assert main-like results;
  - comprehension over tainted data tainting its target: iteration flows
    are dropped;
  - the 1 s workflow-deadline wall-clock test: replaced by operation
    counts (it also passed on the unfixed code).
  The other PR tests were kept under new names (scope precision, helper
  returns, argument binding, closures, attributes, super, unbound calls,
  context sensitivity, lambda and comprehension scopes, deadlines).

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round-3 review of the narrowed design at 3c12146: five independent lenses (DoS bounds, recall vs main/2.12.0, Python semantics, output compatibility, test adequacy), each finding re-run by three skeptics on main and this branch.

17 new inline comments: 3 P1 (two lambda-shaped DoS cases, and *args/**kwargs wildcards binding into self/cls), 6 P2, 7 P3, 1 nit. Of the 18 earlier threads, 15 are now fixed, moot or documented residuals and are resolved with evidence; 3 stay open: return values through C()/cls() receivers, exact baseline/message compatibility wording plus a CHANGELOG line, and the binding mutants behind the 61/61 claim.

Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
github-actions Bot and others added 13 commits October 8, 2026 22:16
Address the third review round of the scoped taint analyzer.

Denial of service:
- A lambda is one callee shared by every name it is bound to; a name
  with another definition merges with it. Keyword names are computed
  once per callee, so a = b = ... = lambda p0, ..., pK costs
  O(names + params), not O(names x params).
- A default expression is scanned once, for its own function; a lambda
  nested in it is not walked again (was O(size x depth)).

Binding:
- *args and overflow arguments fill a wildcard slot keyed by the
  parameter they start at, and **mapping one keyed by the receiver
  offset, so a bound call never fills an implicit self/cls.
- A keyword that names a parameter binds only that parameter, never also
  **kwargs. A forwarded **kwargs binds as the keyword names its callers
  pass in, unless a caller passes a **mapping or it is reassigned.
- Class(...) also binds __new__ (cls implicit); Class().m(...) and
  cls().m(...) bind and read returns through the constructed class.

Scopes and attributes:
- A class-body read of a name the class binds also reads the module
  global (LOAD_NAME), restoring main's class-body findings.
- Class-body names are kept apart from self.x stores, and the nearest
  class whose body binds a name shadows its bases.
- The base lookup follows a bounded, iterative C3 linearization.

Messages:
- Keys keep a third, unranked fact: the first source to arrive in main's
  order. A sink cites it whenever it gives the same rule as the most
  severe source, so messages change only where the rule does.

Also drop the unused _ScopeIndex.type_map, add tests for each review
finding, deadline checks per stage, every sink spelling's length cap,
and scaling shapes that grow the expensive dimension. The scaling
metric now also counts jumps and branches, so loops inside one-line
comprehensions are measured.

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bring in the main merges the update-branch workflow added to the PR
branch while the third-round fixes were in progress.

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The update-branch merge of main (#721) kept both copies of the import,
which fails ruff (F811, I001).

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round-4 verification of the round-3 fixes at 78b571c. Each fixed item was re-run independently against main, two red-team passes probed the new code, and every problem below was upheld by two independent skeptics.

Verified: the chained-lambda and nested-lambda-default DoS fixes (linear, close to main); no */** binding into the implicit receiver; class-body reads; keyword double-binding; C()/cls() receivers; same-rule message ties; the 114/108 mutation count.

Still open: 2 P1 and 2 P2, plus 12 P3.

  • The P1s: forwarded **kwargs keyword tracking adds both a fan-out DoS and a one-blob false positive.
  • The P2s: a source call's own arguments raise its rule; a fourth same-rule message-change category.

Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py Outdated
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py
Comment thread src/skillspector/nodes/analyzers/behavioral_taint_tracking.py
Comment thread CHANGELOG.md Outdated
Comment thread CHANGELOG.md Outdated
github-actions Bot and others added 4 commits October 9, 2026 13:01
… semantics

Address the fourth review round.

- A forwarded **kwargs (a function's own ** parameter passed on) binds
  only into the callee's own ** parameter: one slot per callee, and it
  never fills named, keyword-only or receiver parameters. This removes
  the per-name expansion (up to 85 targets per **k) and the false
  positives from one tainted keyword reaching every forwarded one.
- A source call's value is what it reads, not its arguments, so
  json.load(open(os.getenv(...))) is TT4 again, as on main.
- Sinks cite variables main's flows taint (phase A) before parameters
  only the new flows bind, so same-rule messages keep main's wording.
- A class-body read of a name the class binds reads a view fed by the
  module and class keys; the class attribute keeps only the class's own
  value. Any class-body binding (annotated assignment, import, def)
  shadows a base's value; an annotation alone binds nothing.
- An immediately called lambda in a default yields its body's value.
- C3 merges every base and caps only the result.
- Deadline checks per argument and keyword in bind_arguments and every
  1,024 flows of one key in drain.
- CHANGELOG: exact fingerprints are version-bound; list every message
  change category and raised rule.

Tests pin each case, add non-root nested-default and many-base scaling
shapes, the forwarded-kwargs cost, the slot-cap boundary for *rest, the
multi-base ancestor cap, C3 local precedence, and non-constructor
self()/cls() calls.

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bring in the main merges the update-branch workflow added to the PR
branch (#791, #792) while the fixes were in progress.

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant