perf(model): parse, index and validate a load's files on a pool of workers

[devin-ai-integration]

What and why

sysml -validate a.sysml b.sysml …, -satisfy and %load now open the files they are given as one batch and validate them on a pool of workers. This is the parallel batch pipeline of docs/project/large-model-scaling-design.md §5, built on #309 (one workspace document per loaded file), which this branch contains; merge #309 first, then this. Until #309 lands, the diff shown here includes its commits — the changes of this PR alone are git diff feature/per-file-documents...perf/parallel-batch-validation (32 files).

The branch is reconciled with the persistent semantic model and per-document gather cache of #316: the editor path (Workspace.Diagnostics) analyzes on the workspace's persistent resolver, model and gathers through passes.AnalyzeShared; the batch path (Workspace.DiagnosticsAll) analyzes on private per-worker contexts through passes.AnalyzeInBatch, with one gather shared by the batch.

The pipeline, in internal/core/model/batch.go:

type Input struct{ Name string; Content []byte; Version int }

func (w *Workspace) OpenAll(inputs []Input)
    // parse + symbols.Build on the pool, outside the lock
    // then, under the lock, in input order: installLocked(doc)
    //   = displaceLocked + index.AddBuiltDocument(doc.Name, doc.AST, doc.Scope) + standInLocked
    // then ExpandWildcardImports() once, invalidateLocked(installed...) once

func (w *Workspace) DiagnosticsAll(names []string) [][]passes.Diagnostic
    // cached entries served; the rest:
    //   batch := &passes.Batch{Documents: pending, Gathers: passes.NewGathers()}
    //   passes.PrepareBatch(index, batch)
    //   ParallelFor(workers, …, passes.AnalyzeInBatch(name, …, batch))   // private Resolver + Model per document
    //   results cached and recorded in w.batched; returned in `names` order

• Parse in parallel, index once. symbols.Index.AddBuiltDocument takes a scope tree the caller built with symbols.Build, so the batch builds its trees on the pool and the single writer only installs them. Wildcard imports are expanded once per batch rather than once per file added, which retires the quadratic per-file reindex cost the stress-test record noted.
• Analyze in parallel over a read-only index. Each document is analyzed by passes.AnalyzeInBatch in a context of its own; nothing takes a lock inside the resolver or the model. The one place resolution wrote to the shared scope tree — linking a metadata annotation body's owner (Scope.SetOwner) on first use — is done for all documents of the batch before the pool starts (resolve.(*Resolver).LinkMetadataBodies via passes.PrepareBatch, with the model-attached resolver analysis itself uses), so the workers only read it. To make that link computable outside a resolution, a metadata body scope records the declaration its annotation is written on (Scope.Annotated()), set by the builder.
• One gather per batch. passes.Batch.Gathers carries one passes.Gathers (the per-document gather cache of feat(model): persistent semantic model per workspace, invalidated per document #316) to every context of the batch. The first context that runs a workspace-wide audit (OOSEMMethodPass, IdentityMetadataPass, MOSAPass) gathers every document's facts into it under its lock; every context afterwards reads the same union. The three passes are untouched — they already read through Context.Gathers(); AnalyzeInBatch sets the context's gathers to the batch's. The gathers are the batch's, not the workspace's persistent ones, because a private resolver records no dependencies and facts gathered by one could not be invalidated per dependency.
• Batch-computed diagnostics drop on any edit. For the same reason, DiagnosticsAll records the names it analyzed in Workspace.batched; invalidateLocked deletes those diagCache entries before asking the resolver what a change invalidates, and invalidateAllLocked clears the set. A Diagnostics call after an edit re-analyzes on the persistent model, with dependencies, as before.
• Concurrent edits win over the batch. OpenAll parses outside the lock, so it records each input name's change count (Workspace.changes, bumped on every install and removal) before parsing and installs a document only where the count is still the one it reserved; a name opened, updated, closed or removed meanwhile — including one opened and removed again, absent both before and after — keeps its newer state (TestOpenAllKeepsAChangeMadeWhileItParsed).
• Deterministic output. Diagnostics are gathered in the order the names were given and are the same at any worker count; TestParallelBatchValidationMatchesSerial in internal/core/model runs every fixture directory, examples/ and the four OMG corpus roots at workers=1 and workers=GOMAXPROCS and asserts identical diagnostics (content and order), and identical to opening the files one by one.
• Workers setting. -workers N on sysml, or OPENSYSML_WORKERS (legacy SYSML_WORKERS via envvar.Lookup); the flag wins; default runtime.GOMAXPROCS(0); a value below one is rejected at startup before anything loads. OPENSYSML_JOBS is left to the runtime/check concurrency it already names. Documented in sysml -help, the environment listing and the regenerated man page.
• Streaming-ready, not streaming. Document stays immutable and analysis state lives in each context; nothing in the pipeline holds a tree past its analysis except the workspace's own docs map, so releasing trees once interface records exist (§4) is a change to that map alone. Nothing is dropped in this PR.
• Split-planes generator. cmd/stress-model -split-planes <dir> writes one .sysml per orbital plane plus library.sysml and constellation.sysml (ground segment, cross-plane network), staged and recorded in .stress-model-files so a later run removes only the plane files it owns. TestSatelliteNetworkSplitValidates checks the split declares the single file's network, validates clean at one worker and at several, through the batch and through the persistent workspace, and that every satisfy assertion holds across files.

Where the design and the implementation differ

• The CLI did not load N documents. Before feat(repl): analyze each loaded file as a workspace document of its own #309, -validate joined the files into one <repl> document, so per-document parallelism had nothing to parallelize; feat(repl): analyze each loaded file as a workspace document of its own #309 is the prerequisite this branch is stacked on.
• The split model's cost was not per-file reindexing; it was a quadratic gather. Before the gather was shared, each of the 34 analyses gathered all 34 documents afresh in its own model: 129 s on one worker, 30.9 s on eight, 75% of the CPU in the three audits, 7.24 GB peak RSS at eight workers for eight workspace-wide memoizations held live. With one gather per batch the split validates in 19.5 s on one worker and 7.33 s on eight, at the single file's peak RSS. That figure is measured, and it is not the ~5 s of §9: see below for what still bounds it.
• What bounds the pool now is the serial gather. Eight workers reach 394% CPU. The eight-worker profile puts 3.6 s of the 7.45 s wall in the gather (Gathers.oosemOf 2.7 s, identitiesOf 0.55 s, mosaOf 0.41 s), run by whichever context asks first over all 34 documents while the other workers wait at its lock; installing the scope trees and expanding wildcard imports before the pool (commitBatch, 1.1 s) is serial too. Gathering on the pool — each worker gathering its own document into the union before analysis starts, of which the per-document Gathers.Regather the editor path uses is the serial form — is the step left to ~5 s and is left as the follow-up; it touches the gather's locking and belongs in a change of its own.
• Cost of resolving shared names once per worker. With the gather shared, per-worker memoization is the document's own: allocation is 6.8 GiB for the split at any worker count against 5.4 GiB for the single file, and peak RSS 2.12 → 2.69 GB from one to eight workers. A shared read-mostly resolver would recover part of the 1.4 GiB; not proposed as a follow-up ahead of the gather-on-the-pool step, which is worth more.
• Per-document benchmark without a knob. passes.Options has no option to disable the audit passes and none was added; BenchmarkAnalyzeSplitPerDocument builds a passes.Registry from DefaultRegistry().Passes() minus the three and runs it over one index at one worker and one per CPU.
• Locking. DiagnosticsAll holds the workspace's write lock across the pool, as Diagnostics already does across one analysis. Batch parsing happens before the lock is taken.

Measurements

Intel Xeon Platinum 8559C, 8 CPUs, 31 GiB, no swap, Go 1.25.0 linux/amd64, GOMAXPROCS=8; /usr/bin/time -v sysml -validate -memstats; one run per row; CPU = (user + sys) / wall. Models from cmd/stress-model -planes 32 -satellites 50 -ground-stations 160 (1 600 satellites; 299 137 elements, 18.4 MB in one file, 34 files split) and -planes 8 -satellites 25 -ground-stations 20 (200 satellites, 10 files).

model	files	workers	wall	user	sys	CPU	allocated	allocs	peak RSS
200 sat, one file	1	—	1.95 s	2.35 s	0.15 s	127%	722 MiB	10.8 M	383 MB
200 sat, split	10	1	2.30 s	2.77 s	0.14 s	126%	861 MiB	11.5 M	346 MB
		2	1.51 s	3.03 s	0.18 s	210%	873 MiB	11.7 M	377 MB
		4	1.02 s	2.93 s	0.16 s	300%	875 MiB	11.7 M	398 MB
		8	0.89 s	3.19 s	0.20 s	368%	877 MiB	11.7 M	489 MB
1 600 sat, one file	1	—	18.5 s	23.85 s	0.96 s	134%	5.4 GiB	85.6 M	2.47 GB
1 600 sat, split	34	1	19.5 s	24.72 s	0.82 s	130%	6.7 GiB	89.3 M	2.12 GB
		2	12.3 s	26.42 s	0.99 s	221%	6.8 GiB	90.4 M	2.17 GB
		4	8.86 s	26.74 s	1.08 s	311%	6.8 GiB	90.4 M	2.55 GB
		8	7.33 s	28.01 s	1.25 s	394%	6.8 GiB	90.4 M	2.69 GB

The single file is unchanged from the figures taken before the gather was shared (18.5 s, 2.46 GB). Before it was shared, the 34-file split measured 129 s / 66.1 s / 38.6 s / 30.9 s at 1/2/4/8 workers, 39.1 GiB allocated, 1.98 → 7.24 GB peak RSS.

CPU profile, split 1 600, 8 workers (7.45 s wall, 28.6 s of samples, -cpuprofile): passes.(*Gathers).gather 3.5 s (12%) — oosemOf 2.7 s, identitiesOf 0.55 s, mosaOf 0.41 s; NameResolutionPass.Run 6.8 s (24%), W9CInheritedNameConflictPass.Run 3.3 s, DiagramLayoutPass.Run 1.3 s, TypeCheckPass.Run 1.2 s; OpenAll 1.1 s serial under commitBatch (ExpandWildcardImports 0.43 s); runtime.gcBgMarkWorker 5.0 s (17.5%), runtime.scanobject 5.1 s (18%). On one worker (19.6 s wall, 26.3 s of samples) the 34 analyses are 18.2 s of samples, 3.6 s of them the gather.

Benchmarks (internal/stressmodel, -benchtime 3x, four planes, six files):

BenchmarkValidateSplit/satellites=32/files=6/workers=1-8      468 ms/op   165 MB/op   2.15 M allocs/op
BenchmarkValidateSplit/satellites=32/files=6/workers=8-8      191 ms/op   168 MB/op   2.17 M allocs/op
BenchmarkValidateSplit/satellites=128/files=6/workers=1-8    1482 ms/op   531 MB/op   7.46 M allocs/op
BenchmarkValidateSplit/satellites=128/files=6/workers=8-8     437 ms/op   534 MB/op   7.49 M allocs/op
BenchmarkValidateSplit/satellites=512/files=6/workers=1-8    5773 ms/op  2010 MB/op  28.74 M allocs/op
BenchmarkValidateSplit/satellites=512/files=6/workers=8-8    1665 ms/op  2011 MB/op  28.75 M allocs/op
BenchmarkAnalyzeSplitPerDocument/satellites=32/files=6/workers=1-8    224 ms/op    80 MB/op   1.40 M allocs/op
BenchmarkAnalyzeSplitPerDocument/satellites=32/files=6/workers=8-8     57 ms/op    79 MB/op   1.40 M allocs/op
BenchmarkAnalyzeSplitPerDocument/satellites=128/files=6/workers=1-8   854 ms/op   283 MB/op   5.31 M allocs/op
BenchmarkAnalyzeSplitPerDocument/satellites=128/files=6/workers=8-8   222 ms/op   284 MB/op   5.31 M allocs/op
BenchmarkAnalyzeSplitPerDocument/satellites=512/files=6/workers=1-8  3651 ms/op  1114 MB/op  20.95 M allocs/op
BenchmarkAnalyzeSplitPerDocument/satellites=512/files=6/workers=8-8   942 ms/op  1115 MB/op  20.95 M allocs/op

The per-document benchmark (audits left out) shows the pool's own speedup, 3.9× over six files, the largest file being about a quarter of the work. With the audits in, BenchmarkValidateSplit went 10.3 s → 5.77 s serial and 2.77 s → 1.67 s on eight workers when the gather became once per batch.

Allocation follow-ups (listed, not implemented)

Parallelism leaves the allocation count unchanged (6.8 GiB, 90 M objects at any worker count). From the heap profile of the single-file 1 600-satellite run (62 M sampled objects, 4.3 GiB), by objects:

objects	site	what
12.8%	passes.(*w9cConflictChecker).specializes	a slice per conformance question of the inherited-name conflict pass
12.6%	passes.contributionsOf	the per-base member list the same pass compares
9.8%	symbols.FQNOf (strings.Builder)	a fully-qualified name string per call; 78% via symbols.(*Index).GetFQN, 18% via conflictingBases
7.3%	resolve.(*Resolver).specializationChain	a slice per walk of a type's generalizations
3.5%	semantics.(*Model).AllSupertypes	a slice per supertype closure
2.6%	parser.(*Parser).parseQualifiedNameRelaxed	a qualified-name node per reference
1.9%	parser.(*Parser).parseBase	a node per specialization clause

By bytes the parser leads: parseUsage and callees 25.6% of the 4.3 GiB (parseUsageValue 7%), parseQualifiedNameRelaxed 5.1%; then contributionsOf 6.6%, specializes 4.9%, FQNOf 4.4%. Each is to be measured on its own before it is changed; recorded in docs/internals/performance.md.

One parse per load is spent twice, as before: the REPL parses each file to accept it (declared names, whether it closes its own text) and the workspace parses the same bytes again as the document. The 34 split files (17 MB) parse in 1.03 s serially — ~1 s of the one-worker 19.5 s, ~0.13 s of the eight-worker wall. Carrying the accepted tree into the workspace batch is a change to what model.Input owns, listed as a follow-up in docs/internals/performance.md.

Edits to passes/pass.go, passes/analyze.go and model/workspace.go

pass.go: the Batch type (Documents, Gathers) and Context.Batch field. analyze.go: PrepareBatch and AnalyzeInBatch beside AnalyzeShared; all three funnel through one analyze(ctx, root) that runs the registry, drops escalated warnings and sorts. workspace.go: a workers field with DefaultWorkers(); the batched set; invalidateLocked(names ...string) drops batch-computed diagnostics before asking the resolver; Open/Update/SetOnDisk and OpenAll install through one installLocked(doc) — the library displacement, AddBuiltDocument with the document's already-built scope, and the library stand-in — then ExpandWildcardImports once per call; analyze(name, doc, batch) dispatches to AnalyzeInBatch when a batch is given and AnalyzeShared on the persistent resolver, model and gathers otherwise. Serial incremental behavior is otherwise unchanged.

symbols/index.go: adding a document the index already holds removes the old one without expanding wildcard imports (the public RemoveDocument still does) and records the document as changed for the persistent resolver's invalidation; a batch of N reloads expands once, as N fresh files do (TestReplacingDocumentsExpandsOnceEqualToFreshBuild). core/edit/edit.go: the reindexer, which relied on that incidental expansion, now expands explicitly.

How it was verified

• gofmt -l . — nothing; go build ./..., go vet ./... — clean; python3 scripts/changelog.py check, make docs-check, make docs-counts (test count 8,444) — clean.
• go test ./... and go test -race ./... — all packages ok.
• Corpus gates with the corpora present and the require variables set: OPENSYSML_REQUIRE_TRAINING_CORPUS=1 OPENSYSML_REQUIRE_PILOT_CORPORA=1 go test -count=1 ./internal/core/model -run 'TestTrainingExamples|TestPilotCorpora' — ok; training_examples_expected.txt untouched, no pilot ratchet moved.
• New tests: TestParallelBatchValidationMatchesSerial (fixtures, examples/, four OMG corpora; workers=1 vs N vs one-by-one), TestDiagnosticsAllAnswersInTheOrderAsked, TestOpenAllReplacesEarlierDocuments, TestOpenAllKeepsAChangeMadeWhileItParsed, TestWorkersSetting, the DiagnosticsAll subset race test in internal/core/model; TestWorkersFlagAndEnvironment in cmd/sysml; TestLinkMetadataBodies* in internal/core/resolve; TestSatelliteNetworkSplitValidates, BenchmarkValidateSplit, BenchmarkAnalyzeSplitPerDocument in internal/stressmodel.
• By hand: sysml -validate over the 200- and 1 600-satellite splits at workers 1/2/4/8 produces byte-identical stdout per model; the single-file output is byte-identical to develop's; every run reports no errors.

Checklist

• make test and make lint pass locally
• Tests added or updated for the change
• Documentation extended where it already covers the surface (see CONTRIBUTING.md)
• Changelog entry added as changes/unreleased/<slug>.<section>.md, not as an edit to CHANGELOG.md
• baselines regenerated and make docs-counts run if a gate count moved (compliance rows need nothing: the census is counted at docs build)
• No internal work-item labels (waves, slices, F4, K5) in the body, docs, or changelog

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 1 new potential issue.

2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

[devin-ai-integration]

🔴 Concurrent edits are overwritten

When a generated file changes after replaceable checks it, os.Rename replaces the new content without revalidation. A concurrent editor save is lost.

Learn more

The overwrite check and replacement are separate filesystem operations. replaceable verifies the destination before staging, but another process can change that destination before os.Rename replaces it. The manifest digest then records generated content even though user content was overwritten.

Example: An editor saves plane000.sysml after line 76 verifies the old generated digest. The generator then renames its staged plane000.sysml over the save and reports success, instead of preserving the edit and refusing the generation.

Recommended fix: Serialize generations and edits with an output-directory lock where supported, or use a compare-and-swap replacement protocol. Immediately before replacement, atomically preserve the existing file under a private name, verify its digest, install the staged file only on a match, and restore the preserved file on mismatch or failure. Apply the same transaction discipline to concurrent generator processes and manifest publication.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

The window exists: replaceable reads and digests each destination before staging, and the later os.Rename replaces whatever stands there at that moment, so a save landing on planeNNN.sysml between the two is overwritten and the final manifest records the generated digest.

What can close it is narrower than the note suggests. An editor writing into the output directory does not take any lock the generator could hold, so no single-process protocol makes the check and the replacement atomic against it: moving the standing file aside under a private name, digesting it and installing the staged file only on a match shrinks the window to the aside–install gap but cannot remove it, and a save that lands in that gap is still lost. A lock file in the output directory does serialize concurrent generator processes (the other party that would honour it), which is the case the manifest protocol was built to make safe.

Leaving this thread open for the maintainers' call: document that -split-planes expects sole ownership of its output directory while it runs (the current state, made explicit); add a lock file serializing concurrent generations; or add the aside-verify-install step per file as a best-effort narrowing on top of either.