Python: add new shared-SSA-backed SSA adapter (additive) by yoff · Pull Request #21923 · github/codeql

yoff · 2026-06-01T12:04:29Z

Summary

Preparatory refactor for the shared-CFG dataflow migration (#21894).
Based on #21921 — merge that first.

Adds the new Python SSA adapter additively, without changing any production behaviour.

Library addition

semmle.python.dataflow.new.internal.SsaImpl — Python SSA implementation built on the new (shared) CFG.

Mirrors the Java SSA adapter at java/ql/lib/semmle/code/java/dataflow/internal/SsaImpl.qll: an InputSig is defined in terms of positional (BasicBlock, int) variable references, and the shared codeql.ssa.Ssa::Make<Location, Cfg, Input> module is then instantiated.

SourceVariable is the AST-level Py::Variable. Variable references are looked up via the new CFG facade's NameNode.defines/uses/deletes predicates (added in the preceding PR), which themselves are one-line bridges to AST-level Name.defines/uses/deletes.

Implicit-entry definitions are inserted for non-local/global/builtin reads, captured variables, and (when needed) parameters.

Test additions

library-tests/dataflow-new-ssa/ — exercises the new SSA over a representative test corpus and checks expected def/use chains.
library-tests/dataflow-new-ssa-vs-legacy/ — runs both new SSA and legacy ESSA over the same corpus and diffs the results, so any semantic divergence shows up as a test failure.

Production impact

None. The new SSA adapter has zero callers in lib/ and src/ — the legacy ESSA SSA (semmle/python/essa/*) remains the default. The dataflow library is not migrated yet; that lands in #21894 (which will shrink considerably once this stack merges).

Verification

All 367 lib/ + src/ + consistency-queries/ queries compile clean.
All 641 ControlFlow + PointsTo + dataflow + essa + consistency library-tests pass.
Both new dataflow-new-ssa[/vs-legacy] test packs pass.

Notes for reviewers

SsaImpl.qll is the only non-test addition; ~290 LOC. Mostly the InputSig glue + the implicit-entry-definition logic.
The vs-legacy test is the easiest way to spot semantic differences: any new divergence will show up as a diff in CmpTest.expected.

Copilot

Pull request overview

Adds an additive, shared-SSA-backed SSA adapter for Python on top of the new shared CFG facade, along with library test packs to validate the new SSA in isolation and against legacy ESSA as preparation for the upcoming dataflow migration.

Changes:

Introduce semmle.python.dataflow.new.internal.SsaImpl (shared-SSA instantiation + ESSA-shaped adapter surface).
Add a new inline-expectations SSA test pack (dataflow-new-ssa) and a “vs legacy ESSA” comparison pack (dataflow-new-ssa-vs-legacy).
Add a Python library change note documenting the new adapter.

Show a summary per file

File	Description
python/ql/test/library-tests/dataflow-new-ssa/test.py	New Python corpus for inline SSA def/use/phi expectations.
python/ql/test/library-tests/dataflow-new-ssa/SsaTest.ql	Inline-expectations driver query for the new SSA adapter.
python/ql/test/library-tests/dataflow-new-ssa/SsaTest.expected	Snapshot of current inline-expectations outcomes.
python/ql/test/library-tests/dataflow-new-ssa-vs-legacy/test.py	Shared corpus used to compare new SSA vs legacy ESSA.
python/ql/test/library-tests/dataflow-new-ssa-vs-legacy/CmpTest.ql	Query comparing def signatures between new SSA and legacy ESSA.
python/ql/test/library-tests/dataflow-new-ssa-vs-legacy/CmpTest.expected	Snapshot of current “def-only-old/new” diffs.
python/ql/lib/semmle/python/dataflow/new/internal/SsaImpl.qll	New shared-SSA-backed SSA adapter and ESSA-shaped compatibility layer.
python/ql/lib/change-notes/2026-05-19-add-shared-ssa.md	Change note entry for the new SSA adapter.

Copilot's findings

Files reviewed: 8/8 changed files
Comments generated: 3

+# The shared SSA implementation prunes its construction by liveness:
+# definitions of variables that are not read are never materialised.
+# This is by design — write-only variables would only bloat the SSA
+# graph. Tests therefore must always include a read of each variable
+# being verified.


+ *   - Captured / closure variables (gap: new SSA does not yet model
+ *     closure captures).


+---
+category: minorAnalysis
+---


Flips the Python dataflow trunk from the legacy CFG (semmle/python/Flow.qll) and legacy ESSA SSA (semmle/python/essa/*) to the new shared CFG facade (semmle.python.controlflow.internal.Cfg) and the new SSA adapter (semmle.python.dataflow.new.internal.SsaImpl), both introduced additively in the preceding PRs in this stack. This is the trunk-flip equivalent of the original draft PR #21894 (kept around as documentation), rebased on top of the four preparatory PRs: P1: Remove AstNode.getAFlowNode() and rewrite callers (#21919). P2: Qualify Flow.qll's AST references with Py:: prefix (#21920). P3: Add new shared-CFG-backed control flow graph (#21921). P4: Add new shared-SSA-backed SSA adapter (#21923). The Python dataflow library (semmle/python/dataflow/new/) now imports the new CFG facade and SSA adapter. All CFG-typed predicates (ControlFlowNode, CallNode, BasicBlock, NameNode, AttrNode, ...) are qualified with the Cfg:: prefix; SSA references switch from EssaVariable/EssaDefinition to SsaImpl::Definition/SourceVariable. GuardNode is redesigned to use the new CFG's outcome-node model (isAfterTrue / isAfterFalse) instead of the legacy ConditionBlock + flipped indirection. Only BarrierGuard<...> is preserved as public API. Framework files (Bottle, FastApi, Django, Tornado, Pyramid, Stdlib, ...) are updated to take CFG nodes from the new facade. A handful of dataflow consistency tweaks for the new CFG: - Augmented-assignment targets are treated as both load and store. - 'from X import *' produces uncertain SSA writes for unknown names. - CFG nodes are canonicalised so dataflow does not see equivalent pre/post-order pairs as distinct nodes. Two AST tweaks for the new CFG: - AstNodeImpl: omit PEP 695 type-parameter names from FunctionDefExpr / ClassDefExpr children. - ImportResolution: drop the legacy essa import. Test churn (~175 files): reblessed library- and query-test .expected files reflect slightly different CFG granularity, different toString output, and a handful of true alert deltas in security queries. Verification: all 367 lib + src + consistency-queries compile clean. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

aschackmull · 2026-07-08T10:42:15Z

+ *   - `WriteDefinition`, `UncertainWriteDefinition`, `PhiNode`
+ *   - the standard SSA predicates (`getAUse`, `getAnUltimateDefinition`, ...).
+ */
+module Ssa = SsaImplCommon::Make<Py::Location, CfgForSsa, SsaImplInput>;


This instantiation provides just the basic SSA construction where most functionality is in raw uncached top-level predicates. If you further instantiate Ssa::MakeSsa then you also get a nice class hierarchy where caching has been handled such that all the member predicates reference cached predicates. The result is intended to provide the public SSA class hierarchy, which can then be exposed as the python SSA.

yoff · 2026-07-28T09:13:44Z

(force pushed to rebase on 21921)

Preparatory refactor for the shared-CFG dataflow migration. Adds the new Python SSA adapter additively, without changing any production behaviour. Library additions: - semmle.python.dataflow.new.internal.SsaImpl — Python SSA implementation built on the new (shared) CFG. Mirrors the Java SSA adapter (java/ql/lib/semmle/code/java/dataflow/internal/SsaImpl.qll): an InputSig is defined in terms of positional (BasicBlock, int) variable references, and the shared codeql.ssa.Ssa::Make<Location, Cfg, Input> module is then instantiated. SourceVariable is the AST-level Py::Variable. Variable references are looked up via the new CFG facade's NameNode.defines/uses/deletes predicates (added in the preceding PR), which themselves are one-line bridges to AST-level Name.defines/uses/deletes. Implicit-entry definitions are inserted for non-local/global/builtin reads, captured variables, and (when needed) parameters. Test additions: - library-tests/dataflow-new-ssa/ — exercises the new SSA over a representative test corpus and checks expected def/use chains. - library-tests/dataflow-new-ssa-vs-legacy/ — runs both new SSA and legacy ESSA over the same corpus and diffs the results, so any semantic divergence shows up as a test failure. Production impact: None. The new SSA adapter has zero callers in lib/ and src/ — the legacy ESSA SSA (semmle/python/essa/*) remains the default. The dataflow library is not migrated yet; that lands in a follow-up PR. Verified by: - All 367 lib + src + consistency-queries compile clean. - All 641 ControlFlow + PointsTo + dataflow + essa + consistency library-tests pass. - Both new dataflow-new-ssa[/vs-legacy] test packs pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- part of the ESSA adapter layer still refers to the raw SSA (now called Impl)

yoff · 2026-07-28T14:52:18Z

(Force pushed to now be based on main -- I should start using stacks)

+      certain = true
+    )
+    or
+    // `del x` — removes the binding. Modelled as a certain write that


+    certain = true
+    or
+    // `from X import *` — possibly rebinds every name in the importing
+    // scope. Modelled as an uncertain write at the import-star's CFG


+/**
+ * An assignment definition: any binding where the value being assigned
+ * is statically known via `Cfg::DefinitionNode.getValue()`. Includes
+ * plain assignments, walrus, annotated assignments, augmented
+ * assignments, import aliases (`import x` / `from m import x [as y]`),
+ * `with ... as x`, and for-target bindings (where `getValue()` returns
+ * the iter expression's CFG node). Excludes parameter bindings —
+ * those are modelled by `ParameterDefinition`.
+ */


+  Ssa::SsaDefinition getAnInput() { Impl::phiHasInputFromBlock(this, result, _) }
+}
+
+/** Base class for all ESSA definitions (legacy-shaped). */


+module AdjacentUses {
+  /** Holds if `nodeFrom` and `nodeTo` are adjacent uses of the same SSA variable. */
+  predicate adjacentUseUse(Cfg::NameNode nodeFrom, Cfg::NameNode nodeTo) {
+    exists(SsaSourceVariable v, CfgImpl::BasicBlock bb1, int i1, CfgImpl::BasicBlock bb2, int i2 |


+/**
+ * Compares the new-CFG SSA against the legacy ESSA on the same Python
+ * sources. Reports definitions present in one implementation but not
+ * the other, identified by variable name + source position.
+ *
+ * The `.expected` file records the current diff as a snapshot: as the
+ * new SSA matures (closing captured-variable gap, exception bindings,
+ * etc.) and tracks more variables, the snapshot should monotonically
+ * shrink.
+ *
+ * Known categories of `def-only-old` mismatches:
+ *   - Function / class / global definitions with no in-scope read
+ *     (intentional: SSA is liveness-pruned, write-only variables are
+ *     not tracked).
+ *   - Captured / closure variables (gap: new SSA does not yet model
+ *     closure captures).
+ *   - Module variables `__name__`, `__package__`, `$` (legacy ESSA
+ *     adds implicit bindings the new SSA does not).
+ *   - Exception-handler `as` bindings (depend on raise modelling).
+ *
+ * `def-only-new` mismatches would indicate the new SSA produces spurious
+ * definitions; currently none are expected.
+ */


+from string kind, string sig
+where
+  kind = "def-only-new" and
+  exists(NewSsa::EssaNodeDefinition def |


+  kind = "def-only-new" and
+  exists(NewSsa::EssaNodeDefinition def |
+    sig = newDefSig(def) and
+    not exists(EssaNodeDefinition legacyDef | sig = legacyDefSig(legacyDef))


+  )
+  or
+  kind = "def-only-old" and
+  exists(EssaNodeDefinition legacyDef |


+  kind = "def-only-old" and
+  exists(EssaNodeDefinition legacyDef |
+    sig = legacyDefSig(legacyDef) and
+    not exists(NewSsa::EssaNodeDefinition def | sig = newDefSig(def))


Copilot AI review requested due to automatic review settings June 1, 2026 12:04

yoff requested a review from a team as a code owner June 1, 2026 12:04

github-actions Bot added documentation Python labels Jun 1, 2026

Copilot started reviewing on behalf of yoff June 1, 2026 12:04 View session

Copilot AI reviewed Jun 1, 2026

View reviewed changes

This was referenced Jun 1, 2026

Python: switch dataflow library to new (shared) CFG + SSA #21925

Open

Yoff/python use shared cfg for dataflow #21894

Draft

yoff force-pushed the yoff/python-add-new-cfg-library branch from b547f1b to 370dc98 Compare June 1, 2026 13:27

yoff requested a review from a team as a code owner June 1, 2026 13:27

yoff force-pushed the yoff/python-add-new-ssa branch from 28062e1 to 4d4b916 Compare June 1, 2026 13:28

yoff mentioned this pull request Jun 1, 2026

Python: simplify decorator-detection predicates to pure AST match #21926

Merged

yoff force-pushed the yoff/python-add-new-cfg-library branch from 370dc98 to 0ca4cab Compare June 1, 2026 14:05

yoff force-pushed the yoff/python-add-new-ssa branch from 4d4b916 to 6313e70 Compare June 1, 2026 14:05

yoff marked this pull request as draft June 2, 2026 07:49

yoff force-pushed the yoff/python-add-new-cfg-library branch from 0ca4cab to 99bcded Compare June 2, 2026 08:30

yoff force-pushed the yoff/python-add-new-ssa branch from 6313e70 to e3b9611 Compare June 2, 2026 08:30

yoff force-pushed the yoff/python-add-new-cfg-library branch from 99bcded to 45fef35 Compare June 2, 2026 08:47

yoff force-pushed the yoff/python-add-new-ssa branch from e3b9611 to 9be5c62 Compare June 2, 2026 08:47

yoff force-pushed the yoff/python-add-new-ssa branch from 9be5c62 to 944d56e Compare June 2, 2026 13:48

yoff force-pushed the yoff/python-add-new-cfg-library branch 2 times, most recently from 8ef008e to 57a5cad Compare June 2, 2026 13:56

yoff force-pushed the yoff/python-add-new-cfg-library branch 3 times, most recently from f4e50ca to 6ceb1ac Compare June 24, 2026 08:04

yoff force-pushed the yoff/python-add-new-ssa branch from b7f79f2 to bebe8c9 Compare June 24, 2026 08:04

yoff force-pushed the yoff/python-add-new-cfg-library branch from 6ceb1ac to c95c3fe Compare June 24, 2026 08:18

yoff force-pushed the yoff/python-add-new-ssa branch from bebe8c9 to b3f1d94 Compare June 24, 2026 08:18

yoff force-pushed the yoff/python-add-new-ssa branch from b3f1d94 to e51acdc Compare June 24, 2026 08:46

yoff force-pushed the yoff/python-add-new-cfg-library branch from c95c3fe to 4c8e6e3 Compare June 25, 2026 22:20

yoff force-pushed the yoff/python-add-new-ssa branch from e51acdc to 0389e01 Compare June 25, 2026 22:20

yoff force-pushed the yoff/python-add-new-ssa branch from 0389e01 to 6c6af73 Compare June 25, 2026 23:08

yoff force-pushed the yoff/python-add-new-cfg-library branch 2 times, most recently from 05aae21 to eccaccf Compare June 29, 2026 13:25

yoff force-pushed the yoff/python-add-new-ssa branch from 6c6af73 to 7afd645 Compare June 29, 2026 13:25

yoff force-pushed the yoff/python-add-new-cfg-library branch from d68784a to 30c56a8 Compare June 29, 2026 18:05

yoff force-pushed the yoff/python-add-new-ssa branch from 7afd645 to d059bf6 Compare June 29, 2026 18:05

aschackmull reviewed Jul 8, 2026

View reviewed changes

Comment thread python/ql/lib/change-notes/2026-05-19-add-shared-ssa.md Outdated

aschackmull reviewed Jul 8, 2026

View reviewed changes

Comment thread python/ql/lib/semmle/python/dataflow/new/internal/SsaImpl.qll Outdated

aschackmull reviewed Jul 8, 2026

View reviewed changes

Copilot AI and others added 4 commits July 28, 2026 16:51

Python: remove unnecessary change note

874653f

Python: remove superflous module

1b55291

Python: attempt at instantiating MakeSsa

d723cb4

- part of the ESSA adapter layer still refers to the raw SSA (now called Impl)

github-advanced-security AI found potential problems Jul 28, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Python: add new shared-SSA-backed SSA adapter (additive) - #21923

Python: add new shared-SSA-backed SSA adapter (additive)#21923
yoff wants to merge 4 commits into
mainfrom
yoff/python-add-new-ssa

yoff commented Jun 1, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

aschackmull Jul 8, 2026

Uh oh!

yoff commented Jul 28, 2026

Uh oh!

yoff commented Jul 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		* - Captured / closure variables (gap: new SSA does not yet model
		* closure captures).

Uh oh!

Conversation

yoff commented Jun 1, 2026

Summary

Library addition

Test additions

Production impact

Verification

Notes for reviewers

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

aschackmull Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

yoff commented Jul 28, 2026

Uh oh!

yoff commented Jul 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants