Skip to content

Keep a dev session with an incremental TypeScript program in archinaut - #10123

Closed
kube wants to merge 2 commits into
cf/archinaut-retire-generatorfrom
cf/archinaut-dev-session
Closed

kube wants to merge 2 commits into
cf/archinaut-retire-generatorfrom
cf/archinaut-dev-session

Conversation

@kube

@kube kube commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Important

Dev tooling and docs, nothing user-facing.

Summary

Before this PR, every archinaut build ran scip-typescript over each covered package and dev reran the whole pipeline on every change, so a saved file in Zed came back on the site after tens of seconds and a content edit cost as much as a source edit.

Symbol facts for TypeScript now come from one TypeScript program over every covered package, walked once per build and cached as JSON. dev runs through a session that keeps each package's scan and facts in memory: a content edit rebuilds the model alone, a Python edit re-indexes that package, and a TypeScript edit goes to a language service built in the background after the first serve, which re-walks only the edited file and the files referencing what it defines. Only bundle files that differ are written and copied into the site tree.

Scenario, Petrinaut instance Before After
archinaut check, cold cache 28 s 10 s
archinaut check, warm cache 1.5 s 0.8 s
dev, source edit 30 s 1 s after a one-time 9 s service warm-up
dev, content edit 30 s 0.3 s
dev, role edit on core, 60 diagrams ~25 s 5.7 s

Links

Changes

Symbol facts

  • TypeScript facts come from the compiler's language service in src/pipeline/typescript-facts.ts

    One program holds every covered package, with paths mapping workspace packages to their source entries.
    A symbol is its defining file and name, resolved through aliases and re-exports; a module is file#, so bare and dynamic imports are references too.

  • Facts are cached as one JSON file keyed by the covered files' content

    scip-python keeps indexing Python packages. scip-typescript stays a dependency for its protobuf bindings only.

  • Alias resolution reaches further than the SCIP index did

    Re-exported symbols land on their original declaration, which adds a few edges the index missed.

Dev session

  • dev builds through src/dev/session.ts, which keeps per-package scan results and facts between rebuilds

    A rebuild rescans only the touched packages and reuses the facts of the others.

  • Language service is built after the first serve and answers every later TypeScript edit

    update bumps the edited files' versions and re-walks them plus their dependents.

  • Pipeline splits into runPipeline and assembleModel, and the build into buildArchitecture and writeBuild

    The session feeds in-memory facts to the same model and emit code the CLI commands run.

  • Site sync takes the list of changed bundle paths and copies or removes only those

    The first sync still compares whole directories.

  • Changed diagrams render in parallel, one d2 process per core

    A layer many diagrams show, like core, rewrites sixty of them on one role edit.

  • First build keeps the TypeScript program it walks when the cache is stale

    No second warm-up behind the serve; indexPackages takes a factory for it.

  • Facts cache is keyed by file content

    An edit undone finds its entry again.

  • Log line per rebuild names the stage times, the touched packages and the files written

Cleanup

  • scan.ts imports from ../model once

Test coverage

  • Existing @local/archinaut unit suite.
  • Cold and warm archinaut check, a full site build and a scripted dev session with source and content edits, by hand:

    Edge set matches the SCIP-built model apart from the deeper alias resolution.

How to test

  • yarn workspace @local/petrinaut-arch-docs lint:arch-docs

    Expect a summary line and reindexed typescript the first time, indexes from cache the second

  • yarn workspace @local/petrinaut-arch-docs dev

    Expect language service ready in N ms a few seconds after serving http://localhost:4322/

  • Add a comment to libs/@hashintel/petrinaut-core/src/handle/types.ts

    Expect rebuilt in around one second with the files walked, and the site unchanged

  • Edit a page under libs/@local/petrinaut-arch-docs/content

    Expect a rebuild well under a second and the page reloaded

@kube
kube added this pull request to stack #10118 October 20, 2026 16:08
@vercel

vercel Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
petrinaut Ready Ready Preview Oct 20, 2026 4:57pm UTC
petrinaut-docs Ready Ready Preview Oct 20, 2026 4:57pm UTC
2 Skipped Deployments
Project Deployment Actions Updated
hash Ignored Ignored Preview Oct 20, 2026 4:57pm UTC
hashdotdesign-tokens Ignored Ignored Preview Oct 20, 2026 4:57pm UTC

Request Review

@github-actions github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team labels Oct 20, 2026
@kube kube self-assigned this Oct 20, 2026
paths,
files,
}).documents();
await writeFile(cachePath, JSON.stringify(documents));
@kube

kube commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #10131, which carries the whole stack as one commit.

@kube kube closed this Oct 20, 2026
@kube
kube deleted the cf/archinaut-dev-session branch October 20, 2026 22:32

This branch was successfully deployed

3 active (1 outdated) deployments
Preview – petrinaut-docs — 8e14ec1e Deployed Oct 20, 2026 by vercel[bot]
Preview – petrinaut — 8e14ec1e Deployed Oct 20, 2026 by vercel[bot]
Preview – hash — cb40330b Deployed Oct 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

2 participants