Bound shutdown waits and retain Node cleanup

.agents/skills/duroxide-node-orchestrations/SKILL.md

@@ -327,7 +327,7 @@ npm run test:all            # Everything (52 tests)
 - Use `SqliteProvider.inMemory()` for fast isolated tests (SQLite smoketest only)
 - All PG tests need `DATABASE_URL` in `.env` (loaded by `dotenv`)
 - Each test file uses a separate PG schema for isolation
-- Use short `runtime.shutdown(100)` timeout — it waits the full duration
+- `runtime.shutdown(100)` permits 100 ms of grace plus five seconds of cleanup headroom. It can finish early; timeout rejects while cleanup remains owned and requires application/supervisor termination.
 - Set `RUST_LOG=info` to see traces in test output
 - Use `workerLockTimeoutMs: 2000` in tests needing fast activity cancellation detection
 

.github/copilot-instructions.md

@@ -203,6 +203,6 @@ validation so CI and published packages use registry crates.
 - All JS callbacks run on the Node.js main thread (single-threaded)
 - No `async function*` generators — breaks replay model
 - SQLite may hit "database is locked" under concurrency (retried automatically)
-- `Runtime.shutdown(timeoutMs)` waits the full timeout (no short-circuit)
+- `Runtime.shutdown(timeoutMs)` uses grace plus five seconds of bounded waiting; idle work can finish early, and failures reject while retaining cleanup.
 - Platform-specific `.node` binary — cross-platform needs per-platform builds
 - Activity cancellation detection latency depends on `workerLockTimeoutMs` (default 30s → ~15s detection)

.github/workflows/build.yml

@@ -83,6 +83,61 @@ jobs:
           name: binary-linux-x64-gnu
       - name: Run tests
         run: npm run test:all
+      - name: Stage isolated production lifecycle suite
+        run: node ci/stage-lifecycle.cjs . target/lifecycle-production/stage production
+      - name: Require production lifecycle and export boundary
+        run: node ci/run-lifecycle.cjs production target/lifecycle-production/stage
+      - name: Upload production lifecycle evidence
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: lifecycle-production-evidence-linux-x64
+          path: |
+            target/lifecycle-production/stage/lifecycle-native.json
+            target/lifecycle-production/stage/lifecycle-results.json
+            target/lifecycle-production/stage/lifecycle.tap
+            target/lifecycle-production/stage/lifecycle.stderr.log
+          if-no-files-found: error
+
+  lifecycle-instrumented:
+    name: Instrumented lifecycle (isolated)
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version: "24.22.1"
+      - name: Set up Rust from repository toolchain
+        run: |
+          test -f rust-toolchain.toml
+          rustup show active-toolchain
+          rustc --version --verbose
+      - name: Install dependencies
+        run: npm install --omit=optional --ignore-scripts --no-audit --no-fund --package-lock=false
+      # No fallback override: compatible published core APIs are a separate gate.
+      - name: Build isolated instrumented addon
+        env:
+          CARGO_TARGET_DIR: ${{ github.workspace }}/target/lifecycle-instrumented/cargo
+        run: >-
+          npx napi build --platform --features test-hooks --js false --dts bindings.d.ts
+          --cargo-flags="--locked" target/lifecycle-instrumented/native
+      - name: Stage matching source and native artifact
+        run: node ci/stage-lifecycle.cjs target/lifecycle-instrumented/native target/lifecycle-instrumented/stage instrumented
+      - name: Require every instrumented lifecycle case
+        run: node ci/run-lifecycle.cjs instrumented target/lifecycle-instrumented/stage
+      # Upload evidence only, never an instrumented binary-* packaging input.
+      - name: Upload instrumented lifecycle evidence
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: lifecycle-instrumented-evidence-linux-x64
+          path: |
+            target/lifecycle-instrumented/stage/lifecycle-native.json
+            target/lifecycle-instrumented/stage/lifecycle-results.json
+            target/lifecycle-instrumented/stage/lifecycle.tap
+            target/lifecycle-instrumented/stage/lifecycle.stderr.log
+          if-no-files-found: error
 
   # ── Pack tarballs for smoke (local install simulation) ─────
   pack:

CHANGELOG.md

@@ -7,6 +7,42 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- Add owned, metadata-only `RuntimeStopObserver` through `Runtime.stopObserver()`.
+  Retained stop/fault state supports independent AbortSignal waits, terminal
+  local close, and final-state delivery before source end. Observation does not
+  retain worker callbacks, keep Node alive by itself, or change shutdown
+  ownership, deadlines, and application policy.
+
+### Changed
+
+- Use fallible core startup and bounded shutdown waits while preserving `Promise<void>`.
+  Total waiting is the supplied grace plus five seconds.
+  Omitted grace is one second. Idle workers can finish early.
+  Invalid durations fail before ownership changes, including for no-work and repeated calls.
+  Repeated calls retain the original operation, errors, and deadlines.
+  They do not report success merely because the wrapper transferred its core.
+  Registry and preparation failures become ordinary retained startup errors.
+  Registration policy does not change.
+- Pre-start shutdown is terminal.
+  Start and registration calls fail after activation or stop.
+  Post-stop metrics remain unavailable.
+  Proven cleanup permits release of obsolete callback and core holders on the JavaScript thread.
+  The terminal result and deadlines remain available.
+- Timeouts require application- or supervisor-owned process termination while cleanup remains owned.
+  Later proven cleanup does not remove that requirement.
+  Finite grace does not guarantee bounded forced cleanup.
+  This adapter does not add .NET's foreign-continuation barrier or own detached application work.
+  Release requires a matching published core dependency.
+  Do not distribute local development overrides.
+
+### Tests
+
+- Add SQLite-only lifecycle coverage through the real wrapper.
+  Cases include feature-gated provider and fault controls, repeated ordered races,
+  isolated held-provider probes, and production-export checks.
+
 ## [0.2.4] - 2026-10-27
 
 ### Changed

CONTRIBUTING.md

@@ -36,4 +36,65 @@ npm run test:sessions
 cargo clippy --all-targets
 ```
 
-After Rust source changes (`src/*.rs`), re-run `npx napi build --platform` before running JavaScript tests.
+After Rust source changes (`src/*.rs`), re-run `npx napi build --platform` before running JavaScript tests.
+
+### Lifecycle checks without PostgreSQL
+
+The lifecycle suite requires a freshly built local native module.
+Do not use an optional prebuilt platform package.
+Production builds omit private test controls.
+Keep native outputs, Cargo targets, and staged sources separate for each mode.
+For example, from the repository root in PowerShell:
+
+```powershell
+$env:CARGO_TARGET_DIR = Join-Path $PWD 'target\lifecycle-production\cargo'
+npx napi build --platform --js false --dts bindings.d.ts --cargo-flags="--locked" target\lifecycle-production\native
+node ci\stage-lifecycle.cjs target\lifecycle-production\native target\lifecycle-production\stage production
+node ci\run-lifecycle.cjs production target\lifecycle-production\stage
+
+$env:CARGO_TARGET_DIR = Join-Path $PWD 'target\lifecycle-instrumented\cargo'
+npx napi build --platform --features test-hooks --js false --dts bindings.d.ts --cargo-flags="--locked" target\lifecycle-instrumented\native
+node ci\stage-lifecycle.cjs target\lifecycle-instrumented\native target\lifecycle-instrumented\stage instrumented
+node ci\run-lifecycle.cjs instrumented target\lifecycle-instrumented\stage
+Remove-Item Env:CARGO_TARGET_DIR
+```
+
+Instrumented cases exercise real provider waits and contained faults in owned tasks.
+The `lifecycle-instrumented` CI job runs on pull requests and main, alongside production tests.
+Every hook-dependent case must execute.
+Missing hooks, mode or hash mismatches, skips, TODOs, aborted tests, and empty selections fail validation.
+Production mode skips only explicitly marked instrumented cases.
+It also checks that private exports are absent.
+
+The stage manifest identifies the exact local addon, its SHA-256, the selected feature mode, and source hashes.
+Tests match the Duroxide loader's native class identities, not an arbitrary cached `.node`.
+Sources use a stage-local `node_modules` directory.
+This prevents root-level test discovery from running duplicate staged suites.
+The runner sets mode and provenance only in its child environment.
+It retains TAP, stderr, and JSON totals beside the manifest.
+Instrumented evidence artifacts do not use `binary-*` names or contain native binaries.
+They never feed packing or release jobs.
+`CARGO_TARGET_DIR` also tells napi where to find the compiled library.
+Cargo's `--target-dir` alone does not set napi's source path.
+The `--dts` filename is relative to napi's destination directory.
+
+Both ordered races still run **100 iterations**.
+Held-provider probes require `6000 <= elapsedMs < 7000` for omitted grace,
+`5000 <= elapsedMs < 6000` for zero grace, and `repeatedMs < 1000`.
+A separate message-delivery watchdog allows the first call, repeated call, and limited transport time.
+It cannot relax those measured SDK limits.
+
+The suite checks event-loop liveness during incomplete cleanup, without stdin or referenced timers.
+It also checks collection of activity-only self-capturing callbacks in a live process, after the creating frames return.
+Collection uses bounded WeakRef/FinalizationRegistry settling and a noncapturing control.
+It does not use natural process exit or zero task counts as proof.
+Cases cover successful, quiescent-error, and late cleanup, including late release without another stop caller.
+
+Harness and oracle checks that do not need native code can run before a native build:
+`node --test __tests__/lifecycle-oracles.test.js`.
+They include negative controls for child failure, malformed protocol, a zero-start counter, and CI skips.
+
+Do not publish instrumented assets or temporary local core overrides.
+A matching published core minimum remains a separate prerequisite.
+CI must fail when core APIs are missing.
+It must not silently skip coverage or substitute an unofficial dependency.

Cargo.toml

@@ -6,6 +6,9 @@ edition = "2021"
 [lib]
 crate-type = ["cdylib"]
 
+[features]
+test-hooks = ["duroxide/test-hooks"]
+
 [dependencies]
 duroxide = { version = "0.1.32", features = ["sqlite"] }
 duroxide-pg = "0.1.38"

README.md

@@ -25,6 +25,70 @@ Node.js/TypeScript SDK for the [Duroxide](https://github.com/microsoft/duroxide)
 - **SQLite & PostgreSQL** — pluggable storage backends
 - **KV store** — durable per-instance key-value state with snapshots and pruning via `ctx.getKvAllValues()` / `client.getKvAllValues()`
 
+## Runtime lifecycle (unreleased)
+
+`await runtime.start()` returns after startup. Core startup errors reject the Promise.
+`await runtime.shutdown(timeoutMs)` still resolves to `undefined` on success.
+The argument is a nonnegative safe-integer grace period in milliseconds.
+It must fit the native monotonic deadline range.
+Omitted grace is 1000 ms. Total waiting is grace plus 5000 ms.
+Idle work can finish early.
+At the first stop call, the supervisor's **remaining** time must exceed that total.
+Leave time for other shutdown work, result delivery, and process exit.
+Do not pass the entire outer budget as grace. See the
+[budget-sizing guidance](docs/user-guide.md#sizing-the-supervisor-budget).
+
+The first stop fixes both deadlines. Repeated calls observe the same operation.
+They can report timeout, operational errors, or actual late completion.
+Shutdown before start is terminal. Later start or registration calls fail.
+Metrics remain unavailable after stop.
+Serialize lifecycle and registration calls on each runtime.
+
+**Quiescence** means that tracked core execution and cleanup have ended.
+Only proven quiescence permits release of obsolete native callback and core holders.
+This includes quiescent failures and late cleanup without another caller.
+Stop results and the original deadlines remain available after release.
+Incomplete cleanup, unproven cleanup, or a callback-release failure retains ownership.
+
+A `lifecycle_shutdown_timed_out` rejection means cleanup is still owned and
+incomplete. Do not restart the worker.
+The application or supervisor must terminate the process, even if a later observation reports completion.
+The SDK does not terminate the process.
+`lifecycle_shutdown_failed` is an error category, not proof of quiescence.
+It also covers a lost cleanup owner and failure to release callback references after core cleanup.
+Do not infer release permission from that prefix.
+
+This adapter does not track arbitrary JavaScript continuations.
+It does not add the .NET SDK's additional managed foreign-retirement barrier.
+Independent provider and Client resources have their own lifetime.
+See the [lifecycle guide](docs/user-guide.md#runtime-lifecycle)
+for validation and an executable early-drain/repeated-stop example.
+
+These unreleased changes require a compatible core lifecycle release before distribution.
+A temporary local Cargo override provides development evidence, not a release dependency pin.
+
+### Passive stop observation
+
+`runtime.stopObserver()` returns an owned, metadata-only `RuntimeStopObserver`.
+Read `observer.state` first, then call `waitForChange(lastSeen, { signal })`.
+The states are `'NotRequested'`, `'Requested'`, and sticky `'Faulted'`.
+They report recorded stop/fault state, not readiness or completed cleanup.
+A late observer can already see a fault.
+
+Use one pending wait per observer. Independent consumers need separate observers.
+`AbortSignal` cancels only its wait, rejects with `signal.reason`, and permits retry.
+`close()` is terminal and idempotent; pending and later operations fail.
+A normal source end returns an unseen final state before `null`.
+Intermediate states can coalesce. Neither source end nor a settled Promise
+proves worker cleanup.
+
+Observation does not start/stop the worker, change shutdown deadlines, or retain
+its callbacks. Its native delivery does not keep Node alive by itself; the event
+loop must still run to deliver results. The existing limit on unregistered
+JavaScript continuations remains. See the
+[observer example](docs/user-guide.md#passive-stop-observation) for initial state,
+cancellation, local close, and normal source end.
+
 ## Quick Start
 
 ```bash