Skip to content

Re-emit the Java corpus graph at codeanalyzer-java 3.2.0: 7691 is below the attach floor and names none of its 76,791 parameter crossings #401

Description

@rahlk

Is your feature request related to a problem? Please describe.

The Java corpus graph on bolt://localhost:7691 (cldk-leg3-it, anchors can://daytrader8 and can://thingsboard) was emitted by codeanalyzer-java 3.1.1. That vintage now costs the Java live tiers two different things.

It carries no variable name on any parameter crossing. J_PARAM_IN has var on 0 of 76,791 edges. The cut predicate at cldk/analysis/commons/graphs.py:381 compares coalesce(rel.var, '') = c.var, and '' equals no legal cut, so a variable-shaped sanitizer cut cannot sever a Java parameter crossing on that graph at all. That absence is pinned deliberately, by test_a_variable_cut_does_not_reach_the_parameter_crossing (tests/analysis/java/test_java_taint_live.py:194), because under-cutting is the safe direction — a surviving path is over-reported, where an over-cut would certify a refutation and close a live alert. What no live test anywhere can do is exercise the positive case. The offline half has to synthesise the shape onto a 3.1.0 fixture to test it at all: _with_param_vars in tests/analysis/java/test_java_dataflow.py adds a var to every param edge of the committed a4 payload, and its param_in names are openly fabricated from each edge's own formal position, because a 3.1.0 payload records the argument's position and never its name.

Since #394 it is below the attach floor, not merely limited by it. _ANALYZER_FLOOR moved to (3, 2, 0) at cldk/analysis/java/neo4j/neo4j_backend.py:227 (on main and in v2.0.0-rc.6), so every Java live and scale module attached to 7691 is now refused at attach. The refusal is correct rather than over-strict: a 3.1.x graph carries text only on :JCallable.code, so under the 3.2.0 read path every other node's code would rebuild as "" and be served silently as if it were the real slice.

The re-emit is worth doing, and that is measured rather than assumed. The pinned codeanalyzer-java 3.1.2 jar, run at -a 4 over the same daytrader8 sources this corpus was cut from, names every parameter edge it emits:

edge kind edges carrying var what the name is
param_in 1,932 / 1,932 the actual argument's own name — message (252), conn (167), e (160), userID (84), req (82), sql (57)
param_out 909 / 909 $ret, uniformly

(schema_version 2.0.0, analyzer.version 3.1.2.) So a re-emitted graph would carry real formal names on its crossings, and the live positive case becomes writable for the first time.

Found while closing out #382 (leg 4b, taint()); the offline workaround it forced is in #393. Not a blocker for either — the offline suite covers the semantics, and the absence on 7691 is asserted rather than assumed — but until the graph is re-emitted, the Java half of the parameter-crossing cut is verified only against a payload the test itself constructs.

Describe the solution you'd like

  • Measure whether codeanalyzer-java 3.2.0 still names param edges. The probe above is 3.1.2, and 3.2.0's change is the additive text model, but var is the entire point of the re-emit, so confirm it on 3.2.0 output before emitting a graph.
  • Re-emit daytrader8 with codeanalyzer-java 3.2.0 --emit neo4j into a new disposable container on a port that is not 7687. Sources are in this repo: tests/resources/java/application/daytrader8-1.2.zip.
  • Re-emit thingsboard the same way, from thingsboard-v4.0.
  • Record the new graph's shape as a table alongside the existing corpus measurements: analyzer version, per-anchor scale, J_DDG var coverage, J_PARAM_IN / J_PARAM_OUT var coverage, and _ports_carry_dependence per anchor.
  • Update the environment recipes in the Java live module docstrings to the new port.
  • Add the live positive case: a variable cut naming a real formal that severs a J_PARAM_IN crossing, plus the same name under a sibling scope that it must not cut. This is the live counterpart of test_a_java_variable_cut_severs_a_call_boundary_and_only_the_scope_that_named_it in tests/analysis/java/test_java_dataflow.py, which today runs only against the synthesised fixture.
  • Re-measure the four Java lossiness entries and the 3.2.0 text-model reads against a real 3.2.0 graph, replacing what Java Neo4j: consume codeanalyzer-java 3.2.0's text model, floor at 3.2.0 #394 could only measure against JSON.
  • Replace the 0 of 76,791 prose wherever it appears (test_java_taint_live.py header and :194, and any docstring that repeats it) with the new measurement.

Describe alternatives you've considered

Not stated in the original issue.

Additional context

Scope boundary

Re-emits the two Java corpus graphs at the current floor and adds the live tests that only a named param edge makes possible. It does not:

  • change the traversal or the cut predicate — coalesce(rel.var, '') at cldk/analysis/commons/graphs.py:381 is correct, and the existing absence test keeps its job as the tripwire against "fixing" the asymmetry in the unsafe direction;
  • regenerate the offline fixtures under tests/resources/java/analysis_json/v2/, which stay at 3.1.0 for the reasons that directory's README already records;
  • touch the TypeScript graph on 7692. Its TS_PARAM_IN coverage is 3,576 / 11,137, and the cause is different — that emitter puts the parameter's name in the port's of rather than on the edge — so it is its own issue, not a rider on this one;
  • move any dependency pin. The pin is already 3.2.0 on main; this is about the graph catching up to it.

Caveats and known risks

  • Port 7687 must never be an emit target. It is an ssh tunnel on the development machine, and three stopped containers also publish it. Most Java live modules already default away from it, but tests/analysis/java/test_java_dataflow_live.py:65 defaults CLDK_TEST_NEO4J_URI to bolt://localhost:7687 and shares that namespace with three other modules — set it explicitly for every run rather than relying on the default.
  • CLDK_TEST_NEO4J_WRITE_* stays unset. The SDK's Neo4j backends are read-only; the write namespace exists only for throwaway fixtures and must never point at a corpus graph.
  • --emit neo4j forces level 4 and --external-calls, so the re-emitted graph is not shape-identical to an SDK-driven local -a 4. The existing cross-backend tolerance notes stay as they are.
  • daytrader8's L4 pass needs compiled classes. A whole-project -a 4 from the unpruned zip is known to work — that is how the 3.1.2 figures above were produced — but budget for the Maven build it runs.
  • thingsboard is large: the emit is slow, and "disposable container" means recreatable, not cheap.
  • Standing up the new graph beside the old one is the cheap rollback, but the two cannot share a port and the recipes name the port, so plan on a second container rather than replacing 7691 in place.
  • daytrader8's port lattice is currently connected (_ports_carry_dependence is True), so no Java live test may expect the codeanalyzer-java#227 refusal there. Re-measure it on the new graph rather than assuming it stays true.

Definition of done

  • The re-emitted daytrader8 anchor's J_PARAM_IN var coverage is recorded as an exact count (N / N), not as "non-empty", and the same for J_PARAM_OUT.
  • Every Java live and scale module attaches to the new graph instead of being refused by the 3.2.0 floor, with the previously-refused module count named in the PR.
  • A new live test fails against 7691 (the cut severs nothing) and passes against the re-emitted graph (the cut severs the crossing, and the same name under a sibling scope severs nothing). Demonstrated by running it against both, not argued.
  • test_a_variable_cut_does_not_reach_the_parameter_crossing either still passes with its reasoning updated to name what is now graph-vintage-specific, or is replaced by a test that pins the same safe direction on data that can express both outcomes. Deleting it outright is not done.
  • No occurrence of the 0 of 76,791 figure survives as a claim about current data.
  • The full offline gate stays green, and the Java live tier is reported with the graph up.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testAdded new tests

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions