diff --git a/LESSONS.md b/LESSONS.md index 4807cb908..c437e12ff 100644 --- a/LESSONS.md +++ b/LESSONS.md @@ -654,3 +654,64 @@ its own role changes. Check the logical carrier as well as current human rights; keep separate private full-watch approval and Chief proposal authority. Changes stay invisible until the exact Effects intent is committed. A retired role keeps its history and does not silently move Tasks or invent worker exit evidence. + + +### 2026-11-03 — A deleted Bot is not proof its native work stopped + +Seen 1× (native exit regression). A deletion may leave a failed archive or outstanding +activity, and a missing worker handle can accidentally match undefined. Require exact +native binding/generation and actual Agent cancellation plus whenIdle before freeing +resources. Reuse SessionController rather than reading an uninjected agents service; +real DSH caught that integration error while factory tests passed. Keep unknown old +host execution held and retain the failed probe as evidence. + + +### 2026-11-03 — Artifact responsibilities are not logical Bot names + +Seen 1× (approved custom-role regression). A new developer-capability role passed +catalog governance but fixed-name policy/Workspace lookup prevented execution. Keep +logical identity for Session, Workspace, lease and provenance; use the pinned capability +for conventions, verification and SDK readOnly. Resolve artifact responsibilities +explicitly on the Attempt instead of aliasing or duplicating Workspaces. Retired sources +may finish retained claims, while new targets require the active catalog. + + +### 2026-11-03 — Restoring a Session needs both input identity and archive admission + +Seen 1× (actual DSH native pool). Object coercion made different handoffs identical +[object Object] strings, so old delivery dedupe swallowed a legitimate return. Serialize +structured inputs with their durable IDs, retaining stable exact replay content. A +restored archived Session also needs explicit unarchive after trusted binding: DSH +correctly blocks its first step otherwise. Neither issue appeared in factory-only +context tests; retain an actual model-request/history continuation regression. + + +### 2026-11-03 — Native archive admission is an external action + +Seen 1× (NoOp-save regression). Bind/cancel callbacks resolving does not prove their +metadata committed. Keep provisional native binding separate from activation, commit +its Effects marker with the Session/lease worker first, then allow input/unarchive. +The pre-step gate must cover startup inbox work as well as the ordinary delivery API. + + +### 2026-11-03 — Preserve history without loading all of it into every role + +Seen 1× (full role-policy budget regression). New native/context contracts pushed the +complete mandatory policy past its existing cap. Keep current owner rules and concise +state in PROJECT.md; archive completed implementation/probe detail verbatim behind +links. The mandatory source list and byte cap remain unchanged, and the existing +all-role budget regression must pass without silently truncating instructions. + + +### 2026-11-03 — Every exit consumer must use the same proof + +Seen 1× (actual pool adapter regression). Resource release had stopped trusting Bot +absence, but its caller still returned exitConfirmed=true for an unknown missing Bot. +Use one proof query at the initiative boundary too. Native Agent idle also excludes +background jobs: stop/query registered work across earlier parts and child lineage, +and preserve unknown occupancy until it settles. After restart reuse exact supervisor +facts without auto-resuming an old worker or confusing a reused PID with ownership. + +### 2026-11-03 — Preserve context and recheck current authority separately + +Seen 1× (native pre-step and existing scheduler regressions). Admission-time rights do not cover role/resource revocation during a long execution. Reuse one live authority check and the existing pool stop/acknowledgement contract. Keep frozen persona/model for continuation, permit only the retained active retired-role claim, and do not await an Agent's idle inside its own pre-step hook. A resumed physical Session also exists in history: select its current live carrier for authorization. Recheck the actual human sender after awaiting the initiating authority. diff --git a/PROJECT.md b/PROJECT.md index 82058460e..2bd4752a1 100644 --- a/PROJECT.md +++ b/PROJECT.md @@ -27,7 +27,12 @@ acceptance are #2183/#2185, compatibility #2186. This is source work, not live r ## Backend convergence — 2026-11-02 -Project convergence #2171 now adds explicit repository associations and non-destructive Owner double-confirmed closure on the delivery branch. It preserves unfinished Task records and memory; runtime source migration and new behavior are not deployed. Chief role proposals now require separate Maintainer/Owner approval with exact-hash/revision fencing and durable Effects proof; retirement preserves Task/Session and room history. Custom-role physical execution remains #2184 acceptance, and later convergence issues remain active. +Project convergence #2171 now adds explicit repository associations and non-destructive Owner double-confirmed closure on the delivery branch. It preserves unfinished Task records and memory; runtime source migration and new behavior are not deployed. Chief role proposals now require separate Maintainer/Owner approval with exact-hash/revision fencing and durable Effects proof; retirement preserves Task/Session and room history. Real-provider custom-role execution remains #2184 acceptance, and later convergence issues remain active. + +Native execution convergence #2184 now adds matching physical Session bindings and observed +Agent quiescence before Bot removal/resource release. Role retirement/capability changes +cannot restart pinned execution under stale authority. Deterministic runtime acceptance +and real-provider Task loops remain separate; this work is not deployed. Custom-role artifact verification and SDK capability policy now have actual Git/host-test and subprocess contract coverage. Verified supervisor exit now supports native prior-host recovery; unmanaged/unknown exit stays held. Canonical in-flight role/resource revocation now reuses current authority checks and the existing pool stop/scheduler; unknown exit retains capacity. Composed-stack acceptance and real-provider native continuation remain active acceptance. Native activation now requires committed Effects proof and pre-step admission. The actual DSH native return loop now passes with deterministic inference, retained prior conversation and one resource. ## Fork Project State — 2026-10-31 @@ -136,70 +141,11 @@ and safe-checkpoint deployment precede Chief resumption; no frontend code is edi ### Direct backend recovery — 2026-11-01 -The owner requires backend repairs in the foreground Codex session; do not -dispatch backend development through FlyBot. Existing frontend work is preserved and now continues autonomously while new -backend service work is developed independently (owner update, 2026-11-01). See the -[execution decision](decisions/2026-11-01-direct-backend-recovery.md). -R6 adds coalesced event-driven dispatch, manager-controlled retry of blocked queue -entries under the original requester, and operator-selected workspace preparation -before each pool receiver and after independent test cleanup. Failed allocation, -checkout or context preparation releases only proven unbound reservations and -retains its failure audit. These changes do not imply live UI or R8 acceptance. - - -This is `zy95-12/FlyBot`. Current scope is the [shared-project roadmap](https://github.com/zy95-12/FlyBot/issues/1) -and [fork decision](decisions/2026-10-31-project-first-fork.md). The inherited mission, -people and stage descriptions below are historical upstream context; upstream accounts -are not this fork's decision-makers merely because inherited docs name them. - -- Project is the target stable boundary for Task/Run/Workspace ownership. Bots and - Sessions are replaceable execution/interaction carriers. M1 implements direct ownership and explicit legacy resolution; see [Project domain](docs/subsystems/project-domain.md). -- Reuse control-plane identity, auth, invitation, instance and operations APIs unchanged - where possible. Put project business logic/adapters in FlyBot; justify a missing API - separately. The platform PR was withdrawn, not merged. -- Rebuild task navigation and the Project board; current UI/components are not constraints. -- M0 is implemented on [PR #9](https://github.com/zy95-12/FlyBot/pull/9), merged (2026-10-31): - real DSH/tool execution with a deterministic model, plus a FlyBot-owned HTTP adapter - exercising unchanged external platform source. This is not real-model PR delivery. -- The baseline identity, scoped Workflow-write and per-run cwd seams are implemented - through M1–M3; later milestone integration and live acceptance are recorded below. -- Durable repository memory is this page, ROADMAP, LESSONS and dated decisions. Update - them and matching architecture docs alongside code, with regression coverage in - verify/CI and tested-SHA evidence. Never edit live DSH/Bot memory for repository decisions. - -See [baseline ownership and tests](docs/subsystems/demo-baseline.md). - -- M1 implementation ([PR #10](https://github.com/zy95-12/FlyBot/pull/10), complete local verification, awaiting review) adds Project service/policy/repository boundaries, immutable work - ownership, explicit migration, archive/delete/member-removal rules and an owner-only - detail API. Workspace records are directory references; M3 supplies worktree isolation. -- The owner authorized continuing M2–M6, one PR per milestone, with memory, architecture - and tests updated in each. [M1 decision](decisions/2026-10-31-m1-project-ownership.md). - -- M2 ([PR #11](https://github.com/zy95-12/FlyBot/pull/11), complete local verification, awaiting review) implements the [shared access contract](docs/subsystems/project-access.md): unchanged - platform auth API adapter, stable subject mapping, human membership policy/invites, - immediate revocation and scoped Workflow gateway. It is explicitly configured; worktrees, - agent delegation/credentials and browser delivery are supplied by M3–M6 below. - -- M3 ([PR #13](https://github.com/zy95-12/FlyBot/pull/13), full local verify) implements the [commit-specific delivery contract](docs/subsystems/project-delivery.md) - on the existing Workflow service: versioned Attempts, real role worktrees/cwd, host test - evidence, Review/SHA gates and human acceptance. Actual DSH and draft artifact PR #12 - were first exercised with a deterministic model. Integrated real-model/GitHub/browser - acceptance subsequently passed in M6; [evidence](docs/live-acceptance.md). - -- M4 implements [scoped Project agents](docs/subsystems/project-agents.md): fresh public - Chief, private human/Project identities, separate memory/history/proposals, live delegated - tool checks, role/skill snapshots and exact GitHub credential scopes. Actual DSH fixture - validation covers creation/publication/revocation; real-model/browser acceptance - subsequently passed in M6. Runtime code-worker OS isolation is not claimed by this demo. - -- M7 is owner-authorized in [#20](https://github.com/zy95-12/FlyBot/issues/20), - implemented sequentially as #21–#25, one PR per issue. It explicitly authorizes native, - Codex SDK and Claude Agent SDK Harness choices; the inherited upstream removal of - Codex does not govern this fork. Control-plane remains unchanged. M7.1 introduces - explicit public role memory and bounded hashed execution contexts, durable Chief - identity/history and per-Attempt temporary Session bindings. M7.2–M7.5 code is on PRs #27–#30: resource leases, SDK Harnesses, Owner setup and - verified Git handoffs. Authenticated mixed-resource acceptance remains outstanding; - the later owner-authorized local Codex-only run passed, as recorded below. +The owner requires backend repairs in the foreground Codex session; do not dispatch +backend development through FlyBot. Preserve the existing autonomous frontend work +while implementing backend changes independently. The [owner decision](decisions/2026-11-01-direct-backend-recovery.md) +still governs; completed R6–R8 implementation/probe evidence is archived in +[execution history](docs/history/2026-11-03-execution-evidence.md#direct-backend-recovery--2026-11-01). ### Automatic development service — 2026-11-01 @@ -608,82 +554,14 @@ remain pending; native composer/tool/approval integration remains unfinished. ### Task-scoped SDK context and resident Chief memory — 2026-11-01 -Owner requires a resident Chief with durable Project memory and one logical Session per Task/role. Codex/Claude own their supported conversation persistence and automatic compaction; FlyBot does not inject a second model-history compactor. SDK role rework reuses a confirmed released Task-role cwd/thread under the same principal/resource/profile/model/persona binding, while preserving separate immutable Attempt/claim/log/test/Review evidence. Changed bindings rotate backend threads with recorded lineage. Chief memory uses the original own-Bot MEMORY.md/topics/journal, never global team or private-user memory. Context admission keeps current bounded facts and hash-bound full source archives for scoped retrieval. Backend changes are foreground work; frontend Task40/PR48 resumes through FlyBot after validation. - -Task40 continuation integrates trusted backend `91792c1b` into preserved candidate -`25607404` without committing or publishing. Room-scoped messages/streaming and role -identity are retained alongside Chief retrieval/memory and binding-checked SDK resume. -The inherited changelog failure reproduces (6/7) and the single-entry repair passes -(7/7). Generated patches match canonical core. Direct integration testing requires -DSH's registered loader: plain node cannot resolve `@deepseek-ai/dsh-llm` from the -new scheduled-task module. This is not a passing integration result. Native composer, -tool/approval/stop/stream, personal/group interactions, CSS-load synchronization and -actual deployed three-account/weak-network acceptance remain open. - -Follow-up local check used the required harness loader: -`cd core && node --import ./test/harness-resolve.mjs --test test/project-agents.test.js test/project-harness.test.js` -passed 40/40 on the staged merge draft. The earlier plain-node failure was a loader -invocation error, not a missing prepared dependency. Full verify, smoke, independent -Review and deployed acceptance are Not run for this draft. - -Task40 adds a CSS/JS readiness barrier (defect tests 0/2 → 2/2) and mounts the scoped -composer from original composer.js for Project agents and personal conversations. -The original dictation hook is reused with disposal; captured chat handles retain -scope revocation. A base-bundle composer test fails because the capability is absent; -current native render/client/asset checks pass 27/27. These are local draft checks, -not host full verification, independent Review or deployed three-account acceptance. - -The room composer now shares that native mount and existing registered-document -IDs, while personal groups restrict mention choices to their visible members. -Actual native workText labels render scoped live tool names. Final focused UI -checks pass 33/33. Full verify was attempted during editing and is not valid final -verification; a later process-control/result request was refused by sandbox -loopback policy. Request actual configured host verification through Tester, then -return this partial implementation to Developer for remaining tool-result, -approval, personal live/stop and full interaction work. Do not close Task40. - -Task40 continuation at base cbcad4c4 adds actual native question/approval/stop/continue -controls, original group MemberWork controls, bounded redacted tool results and current -own-chat streaming. Existing original components are reused; standing grants and raw -owner APIs are not exposed. SDK permission/message inputs now use the same original -approval/composer components with existing host worker-generation/current-rights gates. -The new personal HTTP capability test failed at base (1/2) before implementation and -passed afterward; additional boundary tests cover foreign approvals/workers, retired -handles, hidden turns, historical parts and stale questions. Host full verification, -independent Review and deployed three-account/weak-network screenshots remain required. - -The continuation also restores answerable group question/approval cards through -original handlers, exposing only public question fields rather than replyTo/origin. -Group member-stop tests use actual native membership and reject another owner's Bot. -Final local interaction checks, build/patch/lint results and outstanding host/browser -validation are reported at handoff; no candidate recommendation closes this Task. - -Task40 host verification at `33e482d6` reported four failures in install, secrets -and usage suites. All four reproduced locally (28/32). The personal stream observer -was also installed in legacy mode, shifting the established listener order used by -those protocol tests. Register it only for explicit shared-workbench mode; keep its -purpose/Session guards. The new mode-boundary regression fails without that fix; -affected suites plus personal-workbench coverage now pass 38/38. Final host full -verification and independent Review remain required; no deployed acceptance is claimed. - -Review return at `846378bf` reproduced current-part navigation leaving the live -conversation read-only; picker and explicit current-part links now normalize to the -live route (defect regression red→green). Human Markdown/mentions reuse original -PersonMessage. Attachment-only messages, scoped image preview/file download and -native file/image sends use public DSH attachment/Agent APIs with current-user checks. -The new human/attachment render test fails against the base bundle and passes on the -working build. Boundary tests cover foreign IDs, batch validation, historical reads, -revocation during admission and late download callbacks. Final host verification, -independent Review and real-service screenshots/weak-network acceptance remain open. - -Review return at `bde2cf7c` reproduced native composer remount/focus loss on same-route -refresh. The mounted regression now uses bounded boolean DOM-identity diagnostics -(the former object diff exhausted memory before reporting failure) and reproduces -red at that source. Native roots now update in place within the same scoped route and -source, retaining input, cursor and focus; navigation/revocation still dispose them. -Mounted personal/public/private/room regressions cover retained sends and scope teardown. -No resource limit was raised. Exact-candidate host verification, independent Review -and deployed three-account/weak-network/screenshot acceptance remain separate gates. + +Keep Chief durable memory and one logical Session per Task/role. SDKs retain their +supported persistence/compaction; FlyBot keeps bounded facts and exact binding/exit +proof, without a second model-history compactor. Rework resumes original context; +incompatible binding requires explicit recovery. Native continuation now has actual +DSH deterministic single-resource return/context coverage. Historical deployment and +probe details, including their original limitations, remain in +[execution history](docs/history/2026-11-03-execution-evidence.md#task-scoped-sdk-context-and-resident-chief-memory--2026-11-01). ### Task40 S1 compatibility integration — 2026-11-02 @@ -751,3 +629,16 @@ and its verified1811 production baseline; it does not merge or replace current upstream main3dde8e14. README checkout targets this repository/branch. This is a snapshot handoff, not a new main-targeting implementation PR or an upstream compatibility PASS. Runtime/private configuration and the frontend flow are untouched. + + +### Task-role execution convergence — 2026-11-03 (#2184) + +Task-role SDK retries retain pinned public persona/skills and the latest observed +backend context across Attempts. Dynamic input is not a definition identity. A +binding incompatibility or unknown old execution fails visibly; no silent fresh +Session or rollback to older context. Definition snapshots are shared by digest, +current rights still revoke execution, failed unused preparations release their +pins, and model input follows a durable-save/cancellation barrier. Existing +Harness/lease/Workflow services remain authoritative. Adapter capability inspection +reports exact native Task-role resume with committed activation/quiescence; actual DSH deterministic return/context coverage is recorded separately from real-provider validation. No +public service protocol, SQL transaction, deployment or frontend change is claimed. diff --git a/ROADMAP.md b/ROADMAP.md index c1a8cd29d..e7261716e 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -21,6 +21,11 @@ acceptance are #2183/#2185, compatibility #2186. This is source work, not live r Implement the 2026-11-02 convergence issues sequentially: Project boundaries, independent personal agents/authorization, execution and collaboration, context, Task/subtask lifecycle, SQLite/state, unified API/events, compatibility, then #2189 real deployment/README. Each issue has a separate main-repository branch/PR; no automatic merge or production rollout. #2171 includes repository relations, Owner closure and Chief-proposed/Owner-or-Maintainer-approved role governance. Execution integration for custom/retired roles remains in #2184; ancestor changes must propagate through the PR stack before final compatibility acceptance. +Original #2184 follow-up adds native Task-role restoration metadata, cancellation/quiescence +proofs and canonical-pool cwd retention. Custom-role verification now has direct artifact/SDK contract regressions; finish current-rights +in-flight stop/reconcile and real-provider native acceptance before propagating the +execution ancestor through collaboration/context/Task/state/API branches. + ## Active fork roadmap — 2026-10-31 Backend delivery branch `delivery/ai-centric-development-service` consolidates S1–S6 at1811e3b8f. @@ -405,3 +410,17 @@ Delivery documentation follow-up: root README links the operation guide and full v1 API reference (parameters/results/permissions/replay/errors). Current JSON CLI, in-process service and incomplete real watch/UI acceptance remain explicit; no standalone frontend, OpenAPI service or runtime migration is implied. + + +### Backend convergence #2184 — 2026-11-03 + +SDK Task-role binding hardening: pinned shared definition snapshots, latest observed +partial-thread recovery, verified legacy-key adoption, durable dispatch intent, +explicit incompatible recovery, current rights/resource checks and truthful adapter +capabilities. Native Task-role context restoration, parent waits, transactional Effects +and external command receipts remain dependent acceptance; no live-provider or deploy +claim from controlled process tests. README/API rewrite remains last (#2189). + +Native prior-host reconciliation now reuses verified supervisor process-group facts; native v2 registered-work/lineage and shared adapter-exit regressions guard against premature initiative transfer. In-flight current-rights abort and composed-stack acceptance remain active. + +Canonical ongoing role/resource authority checks now reuse native step/SDK input and the existing scheduler + pool stop. Frozen role context remains stable; current revoked rights stop execution, with unknown exit held. Compose descendants and verify parent Task resume/API projections next. diff --git a/changelog.d/2184.fixed.md b/changelog.d/2184.fixed.md new file mode 100644 index 000000000..f3c109581 --- /dev/null +++ b/changelog.d/2184.fixed.md @@ -0,0 +1 @@ +Preserve pinned Task-role definitions and latest SDK context across retries; reject incompatible or unconfirmed resumption, and persist execution intent before model input. Expose truthful adapter capability metadata. Preserve exact native Task-role Session bindings and context parts after confirmed quiescence, reject missing-Bot/undefined-handle exit assumptions, and fence retired or reclassified roles from stale restart. Execute approved custom roles with pinned capability conventions, SDK read/write policy and producer-attributed artifact verification; preserve retired active claims while blocking new retired-role work for Chief routing. Restore native pool returns through explicit archived-Session admission and identity-preserving structured input delivery, guarded by actual DSH single-resource context-continuation regression. Require committed native activation before model input or archive admission, including startup/inbox gates. Require shared physical-exit proof at initiative transfer, drain native registered work across parts/subagents, and reconcile exact previous host scopes without inferring exit from PID or Bot absence. Revalidate ongoing canonical role/resource authority through existing native steps, SDK input and scheduler passes; fence revoked claims and reuse confirmed lifecycle stop without replacing frozen role context. diff --git a/core/src/host/hooks.js b/core/src/host/hooks.js index 17537f3b1..378baa907 100644 --- a/core/src/host/hooks.js +++ b/core/src/host/hooks.js @@ -222,7 +222,11 @@ export function install(rt) { } if (record && (stepTools.length > 0 || caps.length > 0 || preset || executionProject)) await rt.save() if (executionWorkspace) { - try { await rt.projectDelivery?.bindSession(executionWorkspace, { id: bot.id, sessionId: rt.chatOf(bot.id) }) } + try { + if (roleSession) await rt.projectDelivery?.nativeSessions?.bind(roleSession, executionWorkspace, bot.id) + await rt.projectDelivery?.bindSession(executionWorkspace, { id: bot.id, sessionId: rt.chatOf(bot.id) }) + if (roleSession) await rt.projectDelivery?.nativeSessions?.activate(roleSession, bot.id) + } catch (error) { await rt.deleteBot(bot.id); throw error } } for (const [key, value] of Object.entries(fields ?? {})) fire('change', { botId: bot.id, plugin, key, value: clone(value), prev: undefined }) @@ -255,6 +259,7 @@ export function install(rt) { const last = view(rt.botOf(id)) const remote = rt.botOf(id)?.remote const remoteChat = remote !== undefined + await rt.projectDelivery?.nativeSessions?.stop(id) const result = await deleteBot(id, ...rest) await rt.projectDelivery?.releaseSession(id) if (!remoteChat) { @@ -787,6 +792,7 @@ export function install(rt) { } innerDeliver = rt.deliver rt.deliver = async (args) => { + if (rt.projectDelivery?.nativeSessions?.admits(args.toId) === false) throw Object.assign(new Error('Native execution is not admitted; confirm its durable activation or stop/recovery state before new input'), { code: 'bot/worker-stopping' }) // Everything that is not a first, checked send still carries its sender: resumed, retried, // deferred and re-sent deliveries (and notices) are judged by the owner policy // now, since the Bot they name may have had its grant revoked, replaced or removed since. @@ -1396,7 +1402,7 @@ export function install(rt) { }, async release(id) { await lifecycle.release(id) - return !rt.botOf(id) && (!rt.state.projectHarnessWorkers?.[id] || rt.state.projectHarnessWorkers[id].status === 'released') + return rt.projectDelivery.exitConfirmed(id) }, evidence({ attemptId }) { const attempt = rt.state.projectAttempts?.[attemptId] diff --git a/core/src/host/project-delivery-template.js b/core/src/host/project-delivery-template.js index c94f3a22e..b937c3d90 100644 --- a/core/src/host/project-delivery-template.js +++ b/core/src/host/project-delivery-template.js @@ -1,5 +1,5 @@ // One versioned path through the existing Workflow plugin; no second execution engine. -export const DELIVERY_VERSION = 'project-delivery/v6' +export const DELIVERY_VERSION = 'project-delivery/v7' export const DELIVERY_ROLES = ['developer', 'tester', 'reviewer'] export function deliveryGraph(model = '') { return { diff --git a/core/src/host/project-delivery.js b/core/src/host/project-delivery.js index 13399b0f9..7692f9a8c 100644 --- a/core/src/host/project-delivery.js +++ b/core/src/host/project-delivery.js @@ -11,11 +11,16 @@ import { createResourceWorkspaces, fileDigest } from './project-resource-workspa import { ROLE_POLICY_FILES, assembleRolePolicy, verifyRolePolicy, deliveryReviewReport, checkDevelopmentEvidence } from './project-role-policy.js' import { roleContext, contextHash } from './project-role-context.js' import { githubPublisher } from './project-github.js' -import { taskSessionId, confirmedWorkerExit } from './project-task-sessions.js' +import { taskSessionId, confirmedWorkerExit, pinTaskRoleDefinition, discardUnpreparedTaskDefinitions } from './project-task-sessions.js' +import { reconcileExecutionAuthority } from './project-execution-authority.js' +import { executionExitConfirmed } from './project-execution-exit.js' +import { createProjectNativeSessions, nativeExitConfirmed } from './project-native-sessions.js' import { testFailureExcerpt } from './project-test-diagnostics.js' import { EffectCancelled, EffectDeferred } from './project-effects.js' const exec = promisify(execFile) +const deliveryRole = (attempt, capability) => attempt.roleAssignments?.[capability] ?? capability +const workspaceFor = (attempt, capability) => attempt.workspaces[deliveryRole(attempt, capability)] const hash = value => createHash('sha256').update(value).digest('hex') const ongoing = new Set(['preparing', 'running']) const reject = message => Object.assign(new Error(message), { code: 'bot/delivery' }) @@ -34,6 +39,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { const { state, config } = rt const root = resolve(config.projectWorkspaceRoot ?? join(dirname(rt.dshHome), 'flybot-workspaces')) const artifacts = resolve(rt.home, 'project-artifacts') + const nativeSessions = createProjectNativeSessions(rt) const resourceWorkspaces = createResourceWorkspaces(rt, { hostGit: git, artifacts }) const workspaceGit = (workspace, args) => { if (workspace.resourceLease) rt.projectResources.requireBinding(workspace.projectId, workspace.resourceLease) @@ -71,7 +77,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { await verifyCandidate(attempt) const tests = attempt.evidence.tests, review = attempt.evidence.review if (!tests?.passed || tests.sha !== attempt.candidateSha || review?.sha !== attempt.candidateSha || review.verdict !== 'approve' || await fileDigest(tests.log) !== tests.sha256) throw new EffectCancelled('Publication evidence is missing or stale') - for (const role of DELIVERY_ROLES) await verifyExecution(attempt, role) + for (const role of DELIVERY_ROLES) await verifyExecution(attempt, deliveryRole(attempt, role)) context.assertCurrent() if (task.deliveryAttemptId !== attempt.id || attempt.cancelRequested || attempt.candidateSha !== effect.payload.candidateSha) throw new EffectCancelled('Publication changed during validation') } @@ -162,7 +168,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { for (const lease of Object.values(attempt.resourceLeases ?? {})) await rt.projectResources?.release(lease.id, lease.generation) await rt.save() } - async function allocate(attempt, project, settings, roles = DELIVERY_ROLES) { + async function allocate(attempt, project, settings, roles = DELIVERY_ROLES, capabilities = {}) { const repository = await realpath(settings.path) attempt.baseSha ??= await git(repository, ['rev-parse', 'HEAD']) attempt.branch = `flybot/delivery-${attempt.id.slice(-12)}` @@ -171,32 +177,37 @@ export function createProjectDelivery(rt, { publisher } = {}) { for (const role of roles) { if (attempt.workspaces[role]) { if (attempt.workspaces[role].status !== 'allocated') throw reject('Workspace allocation is incomplete; reconcile before retry'); continue } if (attempt.cancelRequested) throw reject('Delivery was stopped') + const policyRole = capabilities[role] ?? attempt.roleSessions?.[role]?.policyRole ?? Object.keys(attempt.roleAssignments ?? {}).find(capability => attempt.roleAssignments[capability] === role) ?? role const lease = attempt.resourceLeases?.[role] const resource = lease ? rt.projectResources.requireBinding(project.id, lease) : null const workspaceRoot = resource?.root ?? root const logicalSessionId = taskSessionId(project.id, attempt.taskId, role) - const reusable = lease && lease.harness !== 'flybot' ? Object.values(state.projectWorkspaces ?? {}).findLast(row => row.taskSessionId === logicalSessionId && row.resourceId === lease.resourceId && row.repository === repository && row.resourceRoot === workspaceRoot && row.status === 'allocated' && row.attemptId !== attempt.id) : null + const reusable = (lease && lease.harness !== 'flybot' || attempt.protocol === 'project-handoff/v1') ? Object.values(state.projectWorkspaces ?? {}).findLast(row => row.taskSessionId === logicalSessionId && (row.resourceId ?? null) === (lease?.resourceId ?? null) && row.repository === repository && row.resourceRoot === workspaceRoot && row.status === 'allocated' && row.attemptId !== attempt.id) : null if (reusable) { const previous = ownEntry(attempts(), reusable.attemptId) const workers = Object.values(state.projectHarnessWorkers ?? {}).filter(row => row.attemptId === reusable.attemptId && row.role === role) const leases = Object.values(state.projectResourceLeases ?? {}).filter(row => row.attemptId === reusable.attemptId && row.role === role) + const native = Object.values(state.projectNativeWorkers ?? {}).filter(row => row.attemptId === reusable.attemptId && row.role === role) + const liveNative = Object.values(state.bots ?? {}).some(row => row.ephemeral && row.attemptId === reusable.attemptId && row.workspaceId === reusable.id) + if (liveNative || native.some(row => !nativeExitConfirmed(state, row))) throw reject('Native Task workspace is still held by previous execution') if (!previous || previous.projectId !== project.id || previous.taskId !== attempt.taskId || ongoing.has(previous.status) || previous.status === 'waiting-acceptance' || workers.some(row => !confirmedWorkerExit(state, row)) || leases.some(row => row.status !== 'released')) throw reject('Task workspace is still held by previous execution') const dirty = await workspaceGit(reusable, ['status', '--porcelain']) const head = await workspaceGit(reusable, ['rev-parse', 'HEAD']) - if (head !== (previous.candidateSha ?? previous.seedSha ?? previous.baseSha) || dirty && (role !== 'developer' || previous.candidateSha)) throw reject('Task workspace changed after its checkpoint; preserve it before reuse') + if (head !== (previous.candidateSha ?? previous.seedSha ?? previous.baseSha) || dirty && (policyRole !== 'developer' || previous.candidateSha)) throw reject('Task workspace changed after its checkpoint; preserve it before reuse') reusable.retainedDraft = Boolean(dirty) } const path = reusable?.path ?? (resource ? join(workspaceRoot, 'delivery', attempt.id, role) : join(root, attempt.id, role)) if (!resource || resource.type === 'local') await mkdir(resource ? join(workspaceRoot, 'delivery', attempt.id) : join(root, attempt.id), { recursive: true, mode: 0o700 }) - const workspace = { id: `workspace-${randomUUID()}`, projectId: project.id, attemptId: attempt.id, role, path, repository, taskSessionId: logicalSessionId, ...(reusable ? { reusesWorkspaceId: reusable.id, storageAttemptId: reusable.storageAttemptId ?? reusable.attemptId, retainedDraft: reusable.retainedDraft } : {}), + const workspace = { id: `workspace-${randomUUID()}`, projectId: project.id, attemptId: attempt.id, role, policyRole, path, repository, taskSessionId: logicalSessionId, ...(reusable ? { reusesWorkspaceId: reusable.id, storageAttemptId: reusable.storageAttemptId ?? reusable.attemptId, retainedDraft: reusable.retainedDraft } : {}), kind: resource?.type === 'docker' ? 'resource-clone' : 'worktree', resourceId: lease?.resourceId ?? null, resourceRoot: workspaceRoot, ...(lease ? { resourceLease: lease } : {}), status: 'allocating', createdAt: Date.now() } setEntry(state.projectWorkspaces, workspace.id, workspace) attempt.workspaces[role] = workspace await rt.save() // Persist cleanup intent before git creates the worktree. - const args = role === 'developer' ? ['worktree', 'add', '-b', attempt.branch, path, attempt.baseSha] : ['worktree', 'add', '--detach', path, attempt.baseSha] + const branch = role === 'developer' ? attempt.branch : `${attempt.branch}-${role}` + const args = policyRole === 'developer' ? ['worktree', 'add', '-b', branch, path, attempt.baseSha] : ['worktree', 'add', '--detach', path, attempt.baseSha] if (reusable) { if (workspace.kind === 'worktree' && await realpath(path) !== path) throw reject('Task workspace resolves through a symlink') - if (role === 'developer' && workspace.kind === 'worktree') await workspaceGit(workspace, ['switch', '-c', attempt.branch]) + if (policyRole === 'developer' && workspace.kind === 'worktree') await workspaceGit(workspace, ['switch', '-c', branch]) reusable.status = 'retained' } else if (workspace.kind === 'resource-clone') await resourceWorkspaces.allocate(workspace, attempt) else await git(repository, args) @@ -239,7 +250,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { attempt.agentDefinitions = definitions if (config.projectDispatch !== 'workflow') { attempt.protocol = 'project-handoff/v1' - attempt.workflowVersion = 'project-delivery/pools-v3' + attempt.workflowVersion = 'project-delivery/pools-v4' attempt.baseSha ??= await git(settings.path, ['rev-parse', 'HEAD']) attempt.roleSessions = {}; attempt.rolePolicies = {}; attempt.resourceLeases = {} attempt.runId = `pool-run-${randomUUID()}` @@ -249,13 +260,14 @@ export function createProjectDelivery(rt, { publisher } = {}) { await rt.save() return { attempt: view(attempt) } } - attempt.resourceLeases = await rt.projectResources?.reserveRoles(project.id, attempt.id, definitions) ?? {} + const reservedDefinitions = definitions ? { ...definitions, roles: definitions.roles.map(definition => definition.role === 'chief' ? definition : pinTaskRoleDefinition(state, { projectId: project.id, taskId: task.id, role: definition.role }, definition, definitions.revision, rt.projectAgents.deliveryDefinitions(project.id)).definition) } : definitions + attempt.resourceLeases = await rt.projectResources?.reserveRoles(project.id, attempt.id, reservedDefinitions) ?? {} for (const lease of Object.values(attempt.resourceLeases)) { const resource = rt.projectResources.setting(project.id, lease.resourceId) if (lease.harness === 'flybot' && resource.type !== 'local') throw reject('Native Harness requires a registered local resource') } await allocate(attempt, project, settings) - if (attempt.seedSha) await workspaceGit(attempt.workspaces.developer, ['reset', '--hard', attempt.seedSha]) + if (attempt.seedSha) await workspaceGit(workspaceFor(attempt, 'developer'), ['reset', '--hard', attempt.seedSha]) if (attempt.cancelRequested) throw reject('Delivery was stopped') const model = settings.model ?? config.projectModel ?? await rt.defaultModel?.() if (typeof model !== 'string' || !model) throw reject('A served delivery model must be provisioned') @@ -265,7 +277,9 @@ export function createProjectDelivery(rt, { publisher } = {}) { attempt.rolePolicies = {} for (const node of graph.nodes) { const role = definitions?.roles.find(role => role.role === node.id) - const definition = role ?? { instructions: node.prompt, resolvedSkills: [], memory: [], execution: { harness: 'flybot', resourceIds: [] } } + let definition = role ?? { instructions: node.prompt, resolvedSkills: [], memory: [], execution: { harness: 'flybot', resourceIds: [] } } + const pinned = pinTaskRoleDefinition(state, { projectId: project.id, taskId: task.id, role: node.id }, definition, definitions?.revision ?? 0, rt.projectAgents.deliveryDefinitions(project.id)) + definition = { ...pinned.definition, memory: definition.memory } node.model = definition.model || model const policy = await repositoryRolePolicy(project.id, attempt.baseSha, node.id) attempt.rolePolicies[node.id] = policy @@ -274,7 +288,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { if ((definition.execution?.harness ?? 'flybot') === 'flybot') await archiveForNative(context, attempt, node.id, 'workflow') attempt.roleSessions[node.id] = { id: `role-session-${randomUUID()}`, projectId: project.id, taskId: task.id, attemptId: attempt.id, role: node.id, ...(attempt.resourceLeases[node.id] ? { resourceLease: attempt.resourceLeases[node.id] } : {}), - policy, definitionRevision: definitions?.revision ?? 0, execution: structuredClone(definition.execution), context, status: 'prepared', generation: 1, createdAt: Date.now() } + policy, definitionRevision: pinned.revision, definitionRef: pinned.definitionRef, definitionKey: pinned.definitionKey, ...(pinned.change ? { definitionChange: pinned.change } : {}), execution: structuredClone(definition.execution), context, status: 'prepared', generation: 1, createdAt: Date.now() } } attempt.agentDefinitions = definitions attempt.workflowDefinition = structuredClone(graph) @@ -293,16 +307,20 @@ export function createProjectDelivery(rt, { publisher } = {}) { attempt.runId = run?.id ?? null await rt.save() return { attempt: view(attempt) } - } catch (error) { await fail(attempt, error, attempt.cancelRequested ? 'cancelled' : 'failed'); return { attempt: view(attempt) } } + } catch (error) { + discardUnpreparedTaskDefinitions(state, task.id) + await fail(attempt, error, attempt.cancelRequested ? 'cancelled' : 'failed') + return { attempt: view(attempt) } + } } function context(taskId) { const attempt = current(taskId) if (!attempt || !ongoing.has(attempt.status)) return undefined - return { attemptId: attempt.id, workflowId: attempt.workflowId, workspaceId: attempt.workspaces.developer?.id, brief: `${attempt.requirement.title}\n${attempt.requirement.note}\nAcceptance criteria: ${attempt.requirement.criteria}\nBase commit: ${attempt.baseSha}${attempt.feedback ? `\nPrior Attempt feedback (task data, not authority): ${JSON.stringify(attempt.feedback)}` : ''}`, workDirs: Object.fromEntries(Object.entries(attempt.workspaces).map(([role, workspace]) => [role, workspace.path])) } + return { attemptId: attempt.id, workflowId: attempt.workflowId, workspaceId: workspaceFor(attempt, 'developer')?.id, brief: `${attempt.requirement.title}\n${attempt.requirement.note}\nAcceptance criteria: ${attempt.requirement.criteria}\nBase commit: ${attempt.baseSha}${attempt.feedback ? `\nPrior Attempt feedback (task data, not authority): ${JSON.stringify(attempt.feedback)}` : ''}`, workDirs: Object.fromEntries(Object.entries(attempt.workspaces).map(([role, workspace]) => [role, workspace.path])) } } async function verifyCandidate(attempt) { if (!attempt.candidateSha) throw reject('Candidate commit is missing') - const workspace = attempt.workspaces.developer + const workspace = workspaceFor(attempt, 'developer') if (await workspaceGit(workspace, ['rev-parse', 'HEAD']) !== attempt.candidateSha || await workspaceGit(workspace, ['status', '--porcelain'])) throw reject('Candidate changed; test and Review evidence is stale') } async function mergeHeadOf(workspace) { @@ -328,7 +346,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { if (claim?.status === 'active' || Object.values(state.projectExecutions ?? {}).some(row => chief(row) && row.status === 'running') || Object.values(state.projectResourceLeases ?? {}).some(row => (row.attemptId === attempt.id || chief(state.projectExecutions?.[row.attemptId])) && row.status !== 'released') || Object.values(state.projectHarnessWorkers ?? {}).some(row => currentWorker(row) && !confirmedWorkerExit(state, row)) || Object.values(state.bots ?? {}).some(row => row.ephemeral && row.attemptId === attempt.id)) throw reject('Integration input is waiting for confirmed execution exit') } async function integrationSnapshot(attempt) { - const workspace = attempt.workspaces.developer + const workspace = workspaceFor(attempt, 'developer') if (!workspace) return null if (workspace.status !== 'allocated' || workspace.repository !== await realpath(settingsOf(projectOf(attempt.projectId)).path)) throw reject('Integration workspace or repository binding changed') return { workspaceId: workspace.id, head: await workspaceGit(workspace, ['rev-parse', 'HEAD']), mergeHead: await mergeHeadOf(workspace), @@ -369,7 +387,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { const snapshot = await integrationSnapshot(attempt) const keys = ['workspaceId', 'head', 'mergeHead', 'indexSha256', 'statusSha256'] if (snapshot ? !body.expectedWorkspace || keys.some(key => snapshot[key] !== body.expectedWorkspace[key]) : body.expectedWorkspace !== null) throw reject('Integration workspace changed after the operator snapshot') - if (snapshot && await workspaceGit(attempt.workspaces.developer, ['ls-files', '--unmerged'])) throw reject('Integration has unresolved paths; preserve the index for the Developer') + if (snapshot && await workspaceGit(workspaceFor(attempt, 'developer'), ['ls-files', '--unmerged'])) throw reject('Integration has unresolved paths; preserve the index for the Developer') await verifyIntegrationInputs({ ...attempt, integrationInputs: desired }, { admissionRequired: false }) integrationReleased(attempt); rt.requireProjectAccess(actor, task.projectId, 'configure'); rt.projects.requireActive(task.projectId) if ((task.collaboration?.generation ?? 0) !== body.generation || task.deliveryAttemptId !== attempt.id || attempt.candidateSha || attempt.cancelRequested) throw reject('Integration admission changed during validation') @@ -405,7 +423,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { } async function seal(attempt) { const settings = settingsOf(projectOf(attempt.projectId)) - const workspace = attempt.workspaces.developer + const workspace = workspaceFor(attempt, 'developer') const parent = attempt.seedSha ?? attempt.baseSha, upstream = attempt.publicationBinding?.upstreamSha const approved = await verifyIntegrationInputs(attempt), mergeHead = await mergeHeadOf(workspace) const trusted = [...new Set([upstream, ...approved.map(row => row.sha)].filter(Boolean))] @@ -418,7 +436,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { const info = await lstat(path).catch(error => { if (error.code === 'ENOENT') return null; throw error }) if (info?.isSymbolicLink() && !inside(workspace.path, await realpath(path))) throw reject('Code changes contain a symbolic link outside the allocated workspace') } - checkDevelopmentEvidence(attempt.rolePolicies?.developer, names) + checkDevelopmentEvidence(attempt.rolePolicies?.[deliveryRole(attempt, 'developer')], names) const roots = settings.allowedPaths if (!Array.isArray(roots) || !roots.length || roots.some(value => typeof value !== 'string' || value.startsWith('/') || value.includes('..')) || names.some(name => !roots.some(prefix => name === prefix || name.startsWith(prefix.endsWith('/') ? prefix : `${prefix}/`)))) throw reject('Code changes exceed the provisioned repository paths') const ancestor = async (a, b) => { try { await workspaceGit(workspace, ['merge-base', '--is-ancestor', a, b]); return true } catch (error) { if (error.code === 1) return false; throw error } } @@ -446,9 +464,9 @@ export function createProjectDelivery(rt, { publisher } = {}) { attempt.candidateSha = await workspaceGit(workspace, ['rev-parse', 'HEAD']) if (attempt.candidateSha === attempt.baseSha) throw reject('Developer produced no code change') if (workspace.kind === 'resource-clone') await resourceWorkspaces.importCandidate(workspace, attempt) - attempt.evidence.implementation = { sha: attempt.candidateSha, paths: names, diffStat: await workspaceGit(workspace, ['diff', '--stat', attempt.baseSha, attempt.candidateSha]), at: Date.now() } + attempt.evidence.implementation = { logicalRole: workspace.role, roleSessionId: attempt.roleSessions?.[workspace.role]?.id ?? null, sha: attempt.candidateSha, paths: names, diffStat: await workspaceGit(workspace, ['diff', '--stat', attempt.baseSha, attempt.candidateSha]), at: Date.now() } for (const role of ['tester', 'reviewer', ...(attempt.workspaces['host-verifier'] ? ['host-verifier'] : [])]) { - const target = attempt.workspaces[role] + const target = attempt.workspaces[deliveryRole(attempt, role)] if (!target) continue if (target.kind === 'resource-clone') await resourceWorkspaces.checkout(target, attempt) else { await workspaceGit(target, ['reset', '--hard', attempt.candidateSha]); await workspaceGit(target, ['clean', '-ffdx']) } @@ -457,7 +475,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { async function testCandidate(attempt) { await verifyCandidate(attempt) const settings = settingsOf(projectOf(attempt.projectId)) - const workspace = attempt.workspaces['host-verifier'] ?? attempt.workspaces.tester + const workspace = attempt.workspaces['host-verifier'] ?? workspaceFor(attempt, 'tester') const path = workspace.path await workspaceGit(workspace, ['reset', '--hard', attempt.candidateSha]) await workspaceGit(workspace, ['clean', '-ffdx']) @@ -473,24 +491,24 @@ export function createProjectDelivery(rt, { publisher } = {}) { await mkdir(join(artifacts, attempt.id), { recursive: true, mode: 0o700 }) const log = join(artifacts, attempt.id, 'tests.log') await writeFile(log, output, { mode: 0o600 }) - attempt.evidence.tests = { executor: 'independent-host-controller', workspaceId: workspace.id, resourceId: workspace.resourceId, sha: attempt.candidateSha, passed, command: settings.testCommand, log, sha256: hash(output), at: Date.now() } + attempt.evidence.tests = { logicalRole: deliveryRole(attempt, 'tester'), roleSessionId: attempt.roleSessions?.[deliveryRole(attempt, 'tester')]?.id ?? null, executor: 'independent-host-controller', workspaceId: workspace.id, resourceId: workspace.resourceId, sha: attempt.candidateSha, passed, command: settings.testCommand, log, sha256: hash(output), at: Date.now() } if (!passed) throw reject('Configured tests failed; see Attempt evidence') } async function review(attempt, outputs) { await verifyCandidate(attempt) - const workspace = attempt.workspaces.reviewer + const workspace = workspaceFor(attempt, 'reviewer') if (await workspaceGit(workspace, ['rev-parse', 'HEAD']) !== attempt.candidateSha || await workspaceGit(workspace, ['status', '--porcelain'])) throw reject('Reviewer did not inspect the clean candidate commit') let result try { result = typeof outputs?.review === 'string' ? JSON.parse(outputs.review) : outputs?.review } catch { throw reject('Review output must be commit-specific JSON') } if (!result || result.commit !== attempt.candidateSha || !['approve', 'request-changes'].includes(result.verdict) || typeof result.summary !== 'string' || !result.summary.trim()) throw reject('Review verdict is missing or names a different commit') - const policy = attempt.rolePolicies?.reviewer - const execution = attempt.evidence.execution?.reviewer + const policy = attempt.rolePolicies?.[deliveryRole(attempt, 'reviewer')] + const execution = attempt.evidence.execution?.[deliveryRole(attempt, 'reviewer')] const commands = [] if (execution?.log) for (const line of (await readFile(execution.log, 'utf8')).split('\n').filter(Boolean)) { const item = JSON.parse(line).event?.item if (item?.type === 'command_execution' && typeof item.exit_code === 'number') commands.push({ command: item.command, exitCode: item.exit_code, executor: 'reviewer-sdk' }) } - attempt.evidence.review = { ...(policy ? { policySha256: policy.sha256, report: deliveryReviewReport({ verdict: result.verdict, sha: result.commit, commands }) } : {}), sha: result.commit, verdict: result.verdict, summary: result.summary.slice(0, 8000), at: Date.now() } + attempt.evidence.review = { logicalRole: workspace.role, roleSessionId: attempt.roleSessions?.[workspace.role]?.id ?? null, ...(policy ? { policySha256: policy.sha256, report: deliveryReviewReport({ verdict: result.verdict, sha: result.commit, commands }) } : {}), sha: result.commit, verdict: result.verdict, summary: result.summary.slice(0, 8000), at: Date.now() } if (result.verdict !== 'approve') throw reject('Reviewer requested changes') if (attempt.evidence.tests?.sha !== attempt.candidateSha || !attempt.evidence.tests.passed) throw reject('Matching passing tests are missing') if (attempt.cancelRequested) throw reject('Delivery was stopped') @@ -527,8 +545,8 @@ export function createProjectDelivery(rt, { publisher } = {}) { await verifyCandidate(attempt) for (const [role, policy] of Object.entries(attempt.rolePolicies ?? {})) verifyRolePolicy(policy, attempt.roleSessions[role]?.context.sources.policy?.sha256) for (const role of DELIVERY_ROLES) { - if (!attempt.roleSessions?.[role]) throw reject('Publication retry is missing the original role Session') - await verifyExecution(attempt, role) + if (!attempt.roleSessions?.[deliveryRole(attempt, role)]) throw reject('Publication retry is missing the original role Session') + await verifyExecution(attempt, deliveryRole(attempt, role)) } const tests = attempt.evidence.tests, review = attempt.evidence.review if (!tests?.passed || tests.sha !== attempt.candidateSha || review?.sha !== attempt.candidateSha || review.verdict !== 'approve') throw reject('Matching passing tests and independent Review are required') @@ -598,7 +616,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { if (task.title !== attempt.requirement.title || (task.criteria ?? '') !== attempt.requirement.criteria) throw reject('Requirements changed; delivery evidence is stale') if (!attempt.runId || ownEntry(state.projectExecutions, attempt.runId)?.status !== 'done') throw reject('Delivery Run has not finished successfully') if (!attempt.pr || !attempt.evidence.tests?.passed || attempt.evidence.tests.sha !== attempt.candidateSha || attempt.evidence.review?.sha !== attempt.candidateSha || attempt.evidence.review.verdict !== 'approve') throw reject('Delivery evidence is missing or stale') - for (const role of DELIVERY_ROLES) await verifyExecution(attempt, role) + for (const role of DELIVERY_ROLES) await verifyExecution(attempt, deliveryRole(attempt, role)) const testLog = await readFile(attempt.evidence.tests.log, 'utf8') if (hash(testLog) !== attempt.evidence.tests.sha256) throw reject('Test log changed; evidence is stale') const pr = await publisherOf(projectOf(attempt.projectId)).read(attempt.pr) @@ -757,48 +775,58 @@ export function createProjectDelivery(rt, { publisher } = {}) { const attempt = ownEntry(attempts(), claim.attemptId) if (!attempt || attempt.taskId !== task.id || attempt.protocol !== 'project-handoff/v1' || attempt.status !== 'running') throw reject('Pool Attempt is unavailable') const role = definition.role, project = projectOf(task.projectId) - const settings = await preflightRepository(rt, project) - if (role === 'developer') await refreshUnstartedPublication(attempt, project, claim) - const policy = await repositoryRolePolicy(project.id, attempt.baseSha, role) - const sessionId = `role-session-${randomUUID()}` - const execution = definition.execution - const lease = execution.resourceIds?.length ? await rt.projectResources.reserve({ projectId: project.id, attemptId: attempt.id, role, sessionKey: sessionId, ...execution }) : null - if (lease) attempt.resourceLeases[role] = lease - let context, workspace + const pinned = pinTaskRoleDefinition(state, { projectId: project.id, taskId: task.id, role }, definition, attempt.agentDefinitions.revision, rt.projectAgents.deliveryDefinitions(project.id)) try { - const freshWorkspace = !attempt.workspaces[role] - await allocate(attempt, project, settings, [role]) - workspace = attempt.workspaces[role] - if (lease) workspace.resourceLease = lease - if (attempt.seedSha && role === 'developer' && !attempt.candidateSha && freshWorkspace && !workspace.retainedDraft) await workspaceGit(workspace, ['reset', '--hard', attempt.seedSha]) - if (attempt.candidateSha) { - if (workspace.kind === 'resource-clone') await resourceWorkspaces.checkout(workspace, attempt) - else await workspaceGit(workspace, ['reset', '--hard', attempt.candidateSha]) + definition = { ...pinned.definition, memory: definition.memory } + const policyRole = definition.policyRole ?? role + if (!DELIVERY_ROLES.includes(policyRole)) throw reject('Task worker requires an approved execution capability') + const settings = await preflightRepository(rt, project) + if (policyRole === 'developer') await refreshUnstartedPublication(attempt, project, claim) + const policy = await repositoryRolePolicy(project.id, attempt.baseSha, role, policyRole) + const sessionId = `role-session-${randomUUID()}` + const execution = definition.execution + const lease = execution.resourceIds?.length ? await rt.projectResources.reserve({ projectId: project.id, attemptId: attempt.id, role, sessionKey: sessionId, ...execution }) : null + if (lease) attempt.resourceLeases[role] = lease + let context, workspace + try { + const freshWorkspace = !attempt.workspaces[role] + await allocate(attempt, project, settings, [role], { [role]: policyRole }) + workspace = attempt.workspaces[role] + if (lease) workspace.resourceLease = lease + if (attempt.seedSha && policyRole === 'developer' && !attempt.candidateSha && freshWorkspace && !workspace.retainedDraft) await workspaceGit(workspace, ['reset', '--hard', attempt.seedSha]) + if (attempt.candidateSha) { + if (workspace.kind === 'resource-clone') await resourceWorkspaces.checkout(workspace, attempt) + else await workspaceGit(workspace, ['reset', '--hard', attempt.candidateSha]) + } + await prepareWorkspace(attempt, workspace, settings) + await prepareIntegrations(attempt, workspace) + context = roleContext({ retrieval: true, policy, projectId: project.id, role, revision: pinned.revision, definition, memory: definition.memory, knowledge: ownEntry(state.projectKnowledge ?? {}, project.id) ?? [], requirement: attempt.requirement, feedback: attempt.feedback, projectFacts: { ...(pinned.change ? { roleDefinitionChange: pinned.change } : {}), integrationInputs: structuredClone(attempt.integrationInputs ?? []), testFailure: await failureContext(attempt), publicationBinding: attempt.publicationBinding ?? null, integrationInstruction: attempt.publicationBinding ? 'Merge the pinned upstreamSha into the published candidate using --no-commit, resolve and stage conflicts before handoff. Host sealing retains both trusted inputs; never fabricate parents or discard backend changes.' : null, workingDraft: !attempt.candidateSha && workspaceFor(attempt, 'developer') ? { kind: 'unverified-workspace-reference', resourceId: workspaceFor(attempt, 'developer').resourceId, path: workspaceFor(attempt, 'developer').resourceId === workspace.resourceId ? workspaceFor(attempt, 'developer').path : null, note: 'Unsealed source, not candidate or verification evidence. Cross-resource drafts require a sealed artifact before verification.' } : null, handoff: task.collaboration?.history.at(-1) ? { purpose: task.collaboration.history.at(-1).purpose, reason: task.collaboration.history.at(-1).reason } : null, candidateSha: attempt.candidateSha ?? null, tests: attempt.evidence.tests ? { sha: attempt.evidence.tests.sha, passed: attempt.evidence.tests.passed, sha256: attempt.evidence.tests.sha256, command: attempt.evidence.tests.command } : null, review: attempt.evidence.review ? { sha: attempt.evidence.review.sha, verdict: attempt.evidence.review.verdict, summary: attempt.evidence.review.summary } : null }, repository: await repositoryContext(project.id, attempt.baseSha) }) + } catch (error) { + const registered = lease && ownEntry(state.projectResourceLeases ?? {}, lease.id) + const unbound = registered && registered.generation === lease.generation && !registered.worker && !registered.processScope && !Object.values(state.projectHarnessWorkers ?? {}).some(row => row.leaseId === lease.id) && !Object.values(attempt.roleSessions ?? {}).some(row => row.id === sessionId) + if (unbound) await rt.projectResources.release(lease.id, lease.generation) + ;(attempt.preparationFailures ??= []).push({ role, sessionId, leaseId: lease?.id ?? null, message: String(error.message), releasedBeforeExternalStart: Boolean(unbound && registered.status === 'released' && registered.releaseProof?.kind === 'unbound-reservation-release'), at: Date.now() }) + await rt.save() + throw error } - await prepareWorkspace(attempt, workspace, settings) - await prepareIntegrations(attempt, workspace) - context = roleContext({ retrieval: true, policy, projectId: project.id, role, revision: attempt.agentDefinitions.revision, definition, memory: definition.memory, knowledge: ownEntry(state.projectKnowledge ?? {}, project.id) ?? [], requirement: attempt.requirement, feedback: attempt.feedback, projectFacts: { integrationInputs: structuredClone(attempt.integrationInputs ?? []), testFailure: await failureContext(attempt), publicationBinding: attempt.publicationBinding ?? null, integrationInstruction: attempt.publicationBinding ? 'Merge the pinned upstreamSha into the published candidate using --no-commit, resolve and stage conflicts before handoff. Host sealing retains both trusted inputs; never fabricate parents or discard backend changes.' : null, workingDraft: !attempt.candidateSha && attempt.workspaces.developer ? { kind: 'unverified-workspace-reference', resourceId: attempt.workspaces.developer.resourceId, path: attempt.workspaces.developer.resourceId === workspace.resourceId ? attempt.workspaces.developer.path : null, note: 'Unsealed source, not candidate or verification evidence. Cross-resource drafts require a sealed artifact before verification.' } : null, handoff: task.collaboration?.history.at(-1) ? { purpose: task.collaboration.history.at(-1).purpose, reason: task.collaboration.history.at(-1).reason } : null, candidateSha: attempt.candidateSha ?? null, tests: attempt.evidence.tests ? { sha: attempt.evidence.tests.sha, passed: attempt.evidence.tests.passed, sha256: attempt.evidence.tests.sha256, command: attempt.evidence.tests.command } : null, review: attempt.evidence.review ? { sha: attempt.evidence.review.sha, verdict: attempt.evidence.review.verdict, summary: attempt.evidence.review.summary } : null }, repository: await repositoryContext(project.id, attempt.baseSha) }) - } catch (error) { - const registered = lease && ownEntry(state.projectResourceLeases ?? {}, lease.id) - const unbound = registered && registered.generation === lease.generation && !registered.worker && !registered.processScope && !Object.values(state.projectHarnessWorkers ?? {}).some(row => row.leaseId === lease.id) && !Object.values(attempt.roleSessions ?? {}).some(row => row.id === sessionId) - if (unbound) await rt.projectResources.release(lease.id, lease.generation) - ;(attempt.preparationFailures ??= []).push({ role, sessionId, leaseId: lease?.id ?? null, message: String(error.message), releasedBeforeExternalStart: Boolean(unbound && registered.status === 'released' && registered.releaseProof?.kind === 'unbound-reservation-release'), at: Date.now() }) + attempt.rolePolicies[role] = policy + if ((execution.harness ?? 'flybot') === 'flybot') await archiveForNative(context, attempt, role, sessionId) + const session = { id: sessionId, logicalSessionId: taskSessionId(project.id, task.id, role), projectId: project.id, taskId: task.id, attemptId: attempt.id, role, policyRole, definitionRevision: pinned.revision, definitionRef: pinned.definitionRef, definitionKey: pinned.definitionKey, ...(pinned.change ? { definitionChange: pinned.change } : {}), execution: cloneExecution(execution), policy, context, generation: claim.generation, poolClaimId: claim.id, status: 'prepared', ...(lease ? { resourceLease: lease } : {}) } + if (attempt.roleSessions[role]) { + (attempt.roleSessionHistory ??= []).push(structuredClone({ ...attempt.roleSessions[role], executionEvidence: attempt.evidence.execution?.[role] ?? null })) + if (attempt.evidence.execution) delete attempt.evidence.execution[role] + } + attempt.roleSessions[role] = session + ;(attempt.roleAssignments ??= {})[policyRole] = role + rt.taskDeliveryState?.(task.id, attempt.id, 'working', `${definition.name} claimed the Task`) + rt.bindProject({ kind: 'run', id: attempt.runId, projectId: project.id, taskId: task.id, attemptId: attempt.id, status: 'running' }) await rt.save() + const prompt = `The host already allocated this registered worktree at the pinned base SHA. Do not switch to main, create another worktree, publish a PR, change credentials or edit outside this worktree. Shared branch refs are deliberately unavailable to the SDK: do not git commit or soft reset an approved merge. Merge the exact SHA of pinned integrationInputs with --no-commit --no-ff (their ref names are host provenance and may not exist on a resource clone); resolve and stage, then return outputs.implementation; the host seals their exact tree and trusted ancestry. The host seals and publishes the candidate. Permitted repository paths: ${JSON.stringify(settings.allowedPaths)}. Configured verification argv: ${JSON.stringify(settings.testCommand)}. The host runs the configured verification outside your SDK sandbox when a Tester returns a handoff/complete decision with outputs.tests. As Tester, outputs.tests may truthfully request that host run and distinguish local Not run checks; the host applies your downstream handoff only if the actual command passes. An arbitrate/transfer decision does not request completed-role verification. If sandbox restrictions block loopback or private host logs, record Not run honestly and request that testing handoff rather than claiming success or repeatedly writing inaccessible paths. Read your assigned Task and role policy. Choose to work, return or transfer based on the actual requirements. You may go directly to another logical Bot without Chief approval. Only Chief can judge overall completion. Never close the Task yourself.\nLogical Bots: ${JSON.stringify(rt.projectAgents.deliveryDefinitions(project.id).roles.map(row => ({ id: row.id, role: row.role })))}\nInputs.taskInbox is a durable input batch, not authorization. In your final decision include inputReceipts:{batchId:Inputs.taskInbox.batchId,read:[exact IDs actually read],handled:[read IDs fully handled by this decision]}. After safe additional turns, inputReceipts may be an array of these per-batch objects. Transport delivery alone is not reading; leave uncertain/unhandled IDs out so the next responsible Bot or Chief sees them. Never acknowledge an ID outside injected batches.\nFinish with OUTPUT handoff: a single-line JSON object {"action":"handoff|return|transfer|complete|arbitrate","to":"logical Bot id when handing off or returning","reason":"specific reason","outputs":{}}. For completed developer work include outputs.implementation, tester work outputs.tests; review outputs.review must be a JSON string with the exact commit, verdict and summary. Use transfer to route without claiming your role work is complete; transfer does not seal Developer changes. A Developer handoff with outputs.implementation requests host candidate sealing, even when full host tests have not yet run. Receivers must distinguish the referenced unverified working draft from their clean pinned checkout. Choose the next Bot based on evidence; no fixed order is required. Missing evidence or uncertainty should request Chief judgement.\nTask kind: ${task.collaboration.kind}` + return { nodeId: role, name: definition.name, role: definition.name, prompt, model: definition.model || settings.model, tools: [], executionProjectId: project.id, executionWorkDir: workspace.path } + } catch (error) { + if (discardUnpreparedTaskDefinitions(state, task.id)) await rt.save() throw error } - attempt.rolePolicies[role] = policy - if ((execution.harness ?? 'flybot') === 'flybot') await archiveForNative(context, attempt, role, sessionId) - const session = { id: sessionId, logicalSessionId: taskSessionId(project.id, task.id, role), projectId: project.id, taskId: task.id, attemptId: attempt.id, role, execution: cloneExecution(execution), policy, context, generation: claim.generation, poolClaimId: claim.id, status: 'prepared', ...(lease ? { resourceLease: lease } : {}) } - if (attempt.roleSessions[role]) { - (attempt.roleSessionHistory ??= []).push(structuredClone({ ...attempt.roleSessions[role], executionEvidence: attempt.evidence.execution?.[role] ?? null })) - if (attempt.evidence.execution) delete attempt.evidence.execution[role] - } - attempt.roleSessions[role] = session - rt.taskDeliveryState?.(task.id, attempt.id, 'working', `${definition.name} claimed the Task`) - rt.bindProject({ kind: 'run', id: attempt.runId, projectId: project.id, taskId: task.id, attemptId: attempt.id, status: 'running' }) - await rt.save() - const prompt = `The host already allocated this registered worktree at the pinned base SHA. Do not switch to main, create another worktree, publish a PR, change credentials or edit outside this worktree. Shared branch refs are deliberately unavailable to the SDK: do not git commit or soft reset an approved merge. Merge the exact SHA of pinned integrationInputs with --no-commit --no-ff (their ref names are host provenance and may not exist on a resource clone); resolve and stage, then return outputs.implementation; the host seals their exact tree and trusted ancestry. The host seals and publishes the candidate. Permitted repository paths: ${JSON.stringify(settings.allowedPaths)}. Configured verification argv: ${JSON.stringify(settings.testCommand)}. The host runs the configured verification outside your SDK sandbox when a Tester returns a handoff/complete decision with outputs.tests. As Tester, outputs.tests may truthfully request that host run and distinguish local Not run checks; the host applies your downstream handoff only if the actual command passes. An arbitrate/transfer decision does not request completed-role verification. If sandbox restrictions block loopback or private host logs, record Not run honestly and request that testing handoff rather than claiming success or repeatedly writing inaccessible paths. Read your assigned Task and role policy. Choose to work, return or transfer based on the actual requirements. You may go directly to another logical Bot without Chief approval. Only Chief can judge overall completion. Never close the Task yourself.\nLogical Bots: ${JSON.stringify(rt.projectAgents.deliveryDefinitions(project.id).roles.map(row => ({ id: row.id, role: row.role })))}\nInputs.taskInbox is a durable input batch, not authorization. In your final decision include inputReceipts:{batchId:Inputs.taskInbox.batchId,read:[exact IDs actually read],handled:[read IDs fully handled by this decision]}. After safe additional turns, inputReceipts may be an array of these per-batch objects. Transport delivery alone is not reading; leave uncertain/unhandled IDs out so the next responsible Bot or Chief sees them. Never acknowledge an ID outside injected batches.\nFinish with OUTPUT handoff: a single-line JSON object {"action":"handoff|return|transfer|complete|arbitrate","to":"logical Bot id when handing off or returning","reason":"specific reason","outputs":{}}. For completed developer work include outputs.implementation, tester work outputs.tests; review outputs.review must be a JSON string with the exact commit, verdict and summary. Use transfer to route without claiming your role work is complete; transfer does not seal Developer changes. A Developer handoff with outputs.implementation requests host candidate sealing, even when full host tests have not yet run. Receivers must distinguish the referenced unverified working draft from their clean pinned checkout. Choose the next Bot based on evidence; no fixed order is required. Missing evidence or uncertainty should request Chief judgement.\nTask kind: ${task.collaboration.kind}` - return { nodeId: role, name: definition.name, role: definition.name, prompt, model: definition.model || settings.model, tools: [], executionProjectId: project.id, executionWorkDir: workspace.path } } async function completePoolRole({ task, claim, role, outputs }) { const attempt = ownEntry(attempts(), claim.attemptId) @@ -807,9 +835,14 @@ export function createProjectDelivery(rt, { publisher } = {}) { await verifyExecution(attempt, role) if (task.collaboration.kind !== 'code') { attempt.evidence[role] = { outputs: structuredClone(outputs), sha256: hash(JSON.stringify(outputs)), at: Date.now(), policySha256: attempt.rolePolicies[role].sha256 } - } else if (role === 'developer') await seal(attempt) - else if (role === 'tester') await testCandidate(attempt) - else if (role === 'reviewer') await review(attempt, outputs) + } else { + const policyRole = attempt.roleSessions[role]?.policyRole ?? role + if (deliveryRole(attempt, policyRole) !== role) throw reject('Role artifact assignment changed before completion') + if (policyRole === 'developer') await seal(attempt) + else if (policyRole === 'tester') await testCandidate(attempt) + else if (policyRole === 'reviewer') await review(attempt, outputs) + else throw reject('Unknown worker artifact capability') + } await rt.save() } async function chiefAccept({ taskId, expectedSha, decision }, actor) { @@ -873,7 +906,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { const id = `attempt-${randomUUID()}` if (['running', 'waiting-acceptance'].includes(previous.status)) previous.status = 'returned' previous.returned = { at: Date.now(), by: claim.botId, reason } - const next = { id, protocol: previous.protocol, projectId: previous.projectId, taskId: task.id, requestKey: hash(`${task.projectId}\n${task.id}\n${claim.id}`), workflowVersion: previous.protocol === 'project-handoff/v1' ? 'project-delivery/pools-v3' : previous.workflowVersion, status: 'running', initiatedBy: previous.initiatedBy, executionOwnerId: previous.executionOwnerId, baseSha: !previous.candidateSha && (force || previous.status === 'interrupted') ? await git(settingsOf(projectOf(task.projectId)).path, ['rev-parse', 'HEAD']) : previous.baseSha, seedSha: previous.candidateSha ?? previous.seedSha, ...(publicationBinding ? { publicationBinding } : {}), requirement: structuredClone(previous.requirement), feedback: { previousAttemptId: previous.id, kind: 'bot-return', reason, candidateSha: previous.candidateSha }, agentDefinitions: rt.projectAgents.deliveryDefinitions(task.projectId), roleSessions: {}, rolePolicies: {}, resourceLeases: {}, workspaces: {}, evidence: {}, createdAt: Date.now(), updatedAt: Date.now(), runId: `pool-run-${randomUUID()}` } + const next = { id, protocol: previous.protocol, projectId: previous.projectId, taskId: task.id, requestKey: hash(`${task.projectId}\n${task.id}\n${claim.id}`), workflowVersion: previous.protocol === 'project-handoff/v1' ? 'project-delivery/pools-v4' : previous.workflowVersion, status: 'running', initiatedBy: previous.initiatedBy, executionOwnerId: previous.executionOwnerId, baseSha: !previous.candidateSha && (force || previous.status === 'interrupted') ? await git(settingsOf(projectOf(task.projectId)).path, ['rev-parse', 'HEAD']) : previous.baseSha, seedSha: previous.candidateSha ?? previous.seedSha, ...(publicationBinding ? { publicationBinding } : {}), requirement: structuredClone(previous.requirement), feedback: { previousAttemptId: previous.id, kind: 'bot-return', reason, candidateSha: previous.candidateSha }, agentDefinitions: rt.projectAgents.deliveryDefinitions(task.projectId), roleSessions: {}, rolePolicies: {}, resourceLeases: {}, workspaces: {}, evidence: {}, createdAt: Date.now(), updatedAt: Date.now(), runId: `pool-run-${randomUUID()}` } previous.supersededBy = id setEntry(attempts(), id, next); task.deliveryAttemptId = id rt.bindProject({ kind: 'run', id: previous.runId, projectId: task.projectId, taskId: task.id, attemptId: previous.id, status: 'failed' }) @@ -882,12 +915,12 @@ export function createProjectDelivery(rt, { publisher } = {}) { return id } const cloneExecution = execution => structuredClone(execution) - async function repositoryRolePolicy(projectId, commit, role) { + async function repositoryRolePolicy(projectId, commit, role, policyRole = role) { const project = projectOf(projectId) const settings = settingsOf(project) commit ??= await git(settings.path, ['rev-parse', 'HEAD']) - const configured = settings.rolePolicySources?.[role] - const names = configured ?? [...ROLE_POLICY_FILES.common, ...ROLE_POLICY_FILES[role]] + const configured = settings.rolePolicySources?.[role] ?? settings.rolePolicySources?.[policyRole] + const names = configured ?? [...ROLE_POLICY_FILES.common, ...ROLE_POLICY_FILES[policyRole]] if (!Array.isArray(names) || names.length > 40 || names.some(name => typeof name !== 'string' || name.startsWith('/') || name.includes('..') || /[\\\r\n\0]/.test(name))) throw reject('Role policy source list is invalid') const sources = [] for (const name of [...new Set(names)]) { @@ -901,8 +934,8 @@ export function createProjectDelivery(rt, { publisher } = {}) { const entry = JSON.parse(await git(settings.path, ['show', `${commit}:package.json`])) if (entry.name === 'flybot' && await git(settings.path, ['ls-tree', '--name-only', commit, '--', 'core/cordis.patch.yml'])) profile = 'flybot-development' } - const requiredFiles = settings.requiredRolePolicySources?.[role] ?? (profile === 'flybot-development' ? [...ROLE_POLICY_FILES.common.filter(name => !name.startsWith('decisions/')), ...ROLE_POLICY_FILES[role]] : configured ?? []) - return assembleRolePolicy({ role, commit, sources, requiredFiles, profile }) + const requiredFiles = settings.requiredRolePolicySources?.[role] ?? settings.requiredRolePolicySources?.[policyRole] ?? (profile === 'flybot-development' ? [...ROLE_POLICY_FILES.common.filter(name => !name.startsWith('decisions/')), ...ROLE_POLICY_FILES[policyRole]] : configured ?? []) + return assembleRolePolicy({ role, policyRole, commit, sources, requiredFiles, profile }) } async function repositoryContext(projectId, commit) { const project = projectOf(projectId) @@ -943,11 +976,12 @@ export function createProjectDelivery(rt, { publisher } = {}) { await rt.save() } } - const ownsWorkspace = (projectId, path) => Object.values(state.projectWorkspaces ?? {}).find(workspace => workspace.projectId === projectId && workspace.path === path && DELIVERY_ROLES.includes(workspace.role) && ['worktree', 'resource-clone'].includes(workspace.kind) && workspace.status === 'allocated' && ongoing.has(ownEntry(attempts(), workspace.attemptId)?.status)) - return { endpoint, context, preparePoolRole, completePoolRole, chiefAccept, reworkPool, recordExecution, repositoryContext, repositoryRolePolicy, bindSession, releaseSession, sessionFor, recover, runReported, ownsWorkspace, gate: input => serialAttempt(current(input.taskId)?.id ?? input.taskId, () => gate(input)) } + const ownsWorkspace = (projectId, path) => Object.values(state.projectWorkspaces ?? {}).find(workspace => workspace.projectId === projectId && workspace.path === path && DELIVERY_ROLES.includes(workspace.policyRole ?? workspace.role) && ['worktree', 'resource-clone'].includes(workspace.kind) && workspace.status === 'allocated' && ongoing.has(ownEntry(attempts(), workspace.attemptId)?.status)) + return { reconcileExecutions: () => reconcileExecutionAuthority(rt), exitConfirmed: workerId => executionExitConfirmed(rt, workerId), nativeSessions, endpoint, context, preparePoolRole, completePoolRole, chiefAccept, reworkPool, recordExecution, repositoryContext, repositoryRolePolicy, bindSession, releaseSession, sessionFor, recover, runReported, ownsWorkspace, gate: input => serialAttempt(current(input.taskId)?.id ?? input.taskId, () => gate(input)) } } export function install(rt) { rt.projectDelivery = createProjectDelivery(rt) + rt.ctx.on('agent/pre-step', async (payload, next) => await rt.projectDelivery.nativeSessions.validateStep(payload.agent) ? next() : { kind: 'reject' }) rt.taskNeedsDelivery = task => Boolean(task?.projectId && ownEntry(rt.config.projectRepositories ?? {}, rt.readProject(task.projectId)?.repository)) } diff --git a/core/src/host/project-execution-authority.js b/core/src/host/project-execution-authority.js new file mode 100644 index 000000000..4d2485dc1 --- /dev/null +++ b/core/src/host/project-execution-authority.js @@ -0,0 +1,65 @@ +// Current authority limits retained execution context; it never replaces its persona. +import { ownEntry } from './store.js' +import { pinTaskRoleDefinition } from './project-task-sessions.js' +const refuse = message => Object.assign(new Error(message), { code: 'bot/execution-revoked' }) +export async function requireExecutionAuthority(rt, session) { + if (rt.readOnly?.() !== undefined) throw refuse('Execution lost its writable host authority') + const task = ownEntry(rt.state.tasks ?? {}, session.taskId) + if (!task || task.projectId !== session.projectId) throw refuse('Execution Task scope changed') + const claim = session.poolClaimId && task.collaboration?.claim + const attempt = ownEntry(rt.state.projectAttempts ?? {}, session.attemptId) + const memberId = claim ? task.collaboration.initiatedBy : attempt?.initiatedBy + const sourceSessionId = task.collaboration?.sourceSessionId + if (claim && (typeof memberId !== 'string' || !memberId)) throw refuse('Execution initiating human is unavailable') + function current(afterAwait = false) { + if (ownEntry(rt.state.tasks ?? {}, session.taskId) !== task) throw refuse('Execution Task scope changed') + if (claim && (task.collaboration?.initiatedBy !== memberId || task.collaboration.sourceSessionId !== sourceSessionId || task.deliveryAttemptId !== session.attemptId || task.collaboration?.claim !== claim || claim.id !== session.poolClaimId || claim.generation !== session.generation || task.collaboration.generation !== session.generation || claim.status !== 'active')) throw refuse(afterAwait ? 'Execution claim was revoked during authority validation' : 'Execution claim changed before execution') + if (session.poolClaimId && !claim) throw refuse('Execution claim is unavailable') + if (['accepted', 'cancelled'].includes(task.status)) throw refuse('Execution Task has ended') + if (rt.readOnly?.() !== undefined) throw refuse('Execution lost its writable host authority') + } + current() + if (memberId) { + rt.requireProjectAccess({ memberId }, task.projectId, 'task-write', task) + await rt.projectDelegations?.background(memberId, task.collaboration?.sourceSessionId) + current(true) + rt.requireProjectAccess({ memberId }, task.projectId, 'task-write', task) + } + rt.projects?.requireActive(task.projectId) + if (session.definitionRef) { + const frozen = ownEntry(rt.state.projectRoleDefinitionSnapshots ?? {}, session.definitionRef) + if (!frozen) throw refuse('Pinned execution role definition is unavailable') + let catalog = rt.projectAgents?.deliveryDefinitions(session.projectId) + if (catalog && !catalog.roles.some(role => role.role === session.role) && claim) { + const retired = ownEntry(rt.state.projectRetiredRoles ?? {}, `${session.projectId}:${session.role}`) + if (retired?.id === claim.botId && (claim.role ?? session.role) === session.role) catalog = { ...catalog, roles: [...catalog.roles, retired] } + } + const retained = pinTaskRoleDefinition(rt.state, session, frozen.definition, session.definitionRevision, catalog) + const live = catalog?.roles.find(role => role.role === session.role) + if (!frozen.definition.execution?.resourceIds?.length && live?.execution?.resourceIds?.length) throw refuse('Implicit host execution is no longer authorized by the role resource pool') + const execution = retained.definition.execution ?? {} + if (session.resourceLease && !execution.resourceIds?.includes(session.resourceLease.resourceId)) throw refuse('The execution resource is no longer authorized') + if (!session.resourceLease && execution.resourceIds?.length) throw refuse('Implicit host execution is no longer authorized by the role resource pool') + } + if (session.resourceLease) { + const resource = rt.projectResources?.requireBinding(session.projectId, session.resourceLease) + const harness = session.execution?.harness ?? session.definition?.execution?.harness ?? 'flybot' + if (resource && (resource.disabled || !resource.harnesses?.includes(harness) || harness === 'flybot' && resource.type !== 'local')) throw refuse('Execution resource or Harness capability was revoked') + } + current() +} + +// Called by the existing serialized scheduler pass, never by a new timer. +export async function reconcileExecutionAuthority(rt) { + if (rt.readOnly?.() !== undefined) return + for (const attempt of Object.values(rt.state.projectAttempts ?? {})) { + for (const session of Object.values(attempt.roleSessions ?? {})) { + const task = ownEntry(rt.state.tasks ?? {}, session.taskId), claim = task?.collaboration?.claim + if (!session.poolClaimId || session.status === 'released' || claim?.id !== session.poolClaimId || claim.status !== 'active' || claim.workerId !== session.botId) continue + try { await requireExecutionAuthority(rt, session) } + catch (error) { + await rt.projectPools?.revokeExecution(task.id, claim.id, String(error.message).slice(0, 1000)) + } + } + } +} diff --git a/core/src/host/project-execution-exit.js b/core/src/host/project-execution-exit.js new file mode 100644 index 000000000..0ff345a58 --- /dev/null +++ b/core/src/host/project-execution-exit.js @@ -0,0 +1,16 @@ +// Physical exit is independent of Bot deletion and of whether context can resume. +import { ownEntry } from './store.js' +import { nativeExitConfirmed } from './project-native-sessions.js' +import { confirmedWorkerExit } from './project-task-sessions.js' +import { supervisorProofMatches } from './project-process-proof.js' +export function executionExitConfirmed(rt, workerId) { + if (rt.botOf(workerId)) return false + const state = rt.state, sdk = ownEntry(state.projectHarnessWorkers ?? {}, workerId) + if (sdk) return confirmedWorkerExit(state, sdk) && Boolean(!sdk.leaseId || ownEntry(state.projectResourceLeases ?? {}, sdk.leaseId)?.status === 'released') + const native = ownEntry(state.projectNativeWorkers ?? {}, workerId) + if (nativeExitConfirmed(state, native)) return !native.leaseId || ownEntry(state.projectResourceLeases ?? {}, native.leaseId)?.status === 'released' + const leases = Object.values(state.projectResourceLeases ?? {}).filter(row => row.worker?.kind === 'native' && row.worker.botId === workerId) + if (!leases.length) return false + for (const lease of leases) if (lease.status !== 'released' || !supervisorProofMatches(lease.processScope, lease.releaseProof)) return false + return true +} diff --git a/core/src/host/project-harness.js b/core/src/host/project-harness.js index 9fea0aeab..e0dbeccaa 100644 --- a/core/src/host/project-harness.js +++ b/core/src/host/project-harness.js @@ -8,10 +8,22 @@ import { verifyRolePolicy, roleExecutionPrompt } from './project-role-policy.js' import { fileDigest } from './project-resource-workspaces.js' import { gitWritableRoots } from './project-git-sandbox.js' import { ownEntry, setEntry } from './store.js' -import { taskSession, backendBindingKey, resumableBinding } from './project-task-sessions.js' +import { requireExecutionAuthority } from './project-execution-authority.js' +import { taskSession, backendBindingKey, resumableBinding, roleDefinitionKey } from './project-task-sessions.js' const refuse = message => Object.assign(new Error(message), { code: 'bot/harness' }) const sha = text => createHash('sha256').update(text).digest('hex') const VERSIONS = { codex: '0.166.0', 'claude-agent': '0.3.316' } +// Actual adapter support, independent of resource installation/authentication. +// Native continuation uses exact committed Task-role bindings and the existing +// Bot/parts lifecycle. Cross-Harness restoration is never inferred. +export function harnessCapabilities(harness) { + if (!['flybot', ...Object.keys(VERSIONS)].includes(harness)) throw refuse('Unknown execution Harness') + return { version: 'project-harness-capabilities/v1', harness, sdkVersion: VERSIONS[harness] ?? null, + taskRoleResume: true, input: harness === 'flybot' ? 'native-bot-turn' : 'between-sdk-turns', + cancellation: harness === 'flybot' ? 'native-bot-stop' : 'abort-and-worker-stop', + exitConfirmation: harness === 'flybot' ? 'native-agent-quiescence' : 'stop-ack-and-process-exit', crossHarnessResume: false } +} + export function handoffFrame(text, outputs) { if (typeof text !== 'string' || outputs?.length !== 1 || outputs[0] !== 'handoff' || /(?:^|\n)OUTPUT handoff:/.test(text)) return text try { @@ -73,8 +85,10 @@ export function createProjectHarness(rt) { const workerPath = selected.resource.runtimeDir ? join(selected.resource.runtimeDir, 'src/host/project-harness-worker.mjs') : resolve(import.meta.dirname, 'project-harness-worker.mjs') if (relative(selected.resource.root, workspace.path).startsWith('..')) throw refuse('SDK Workspace is outside the selected resource') const logical = !chief ? taskSession(rt.state, session) : null - const bindingKey = chief?.bindingKey ?? backendBindingKey({ harness: session.execution.harness, resourceId: session.resourceLease.resourceId, principalId: selected.principalId, cwd: workspace.path, model: session.execution.model ?? selected.selected.model, profile: selected.selected, definition: sha(spec.prompt ?? '') }) - const previous = logical ? resumableBinding(rt.state, logical, bindingKey) : null + const definitionKey = logical ? roleDefinitionKey(session) : null + const identity = { harness: session.execution.harness, resourceId: session.resourceLease.resourceId, principalId: selected.principalId, cwd: workspace.path, model: session.execution.model ?? selected.selected.model, profile: selected.selected } + const bindingKey = chief?.bindingKey ?? backendBindingKey({ ...identity, definition: definitionKey }) + const previous = logical ? resumableBinding(rt.state, logical, bindingKey, { legacyKey: backendBindingKey({ ...identity, definition: sha(spec.prompt ?? '') }), definitionKey }) : null const id = `sdk-worker-${randomUUID()}` // Verification may write fixtures, but never the sealed review checkout. const scratchDirectory = chief ? null : join(selected.resource.root, `.flybot-scratch-${id}`) @@ -83,11 +97,11 @@ export function createProjectHarness(rt) { Object.assign(selected.env, { TMPDIR: scratchDirectory, npm_config_cache: join(scratchDirectory, 'npm-cache'), npm_config_update_notifier: 'false', npm_config_offline: 'true' }) } const conversation = chief ? ownEntry(rt.state.projectAgents ?? {}, chief.agentId) : null - const row = { id, ...(conversation?.roomId ? { roomId: conversation.roomId } : {}), model: session.execution.model ?? selected.selected.model ?? null, chiefRunId: chief?.runId ?? null, visibility: conversation?.visibility ?? 'project', ...(conversation?.visibility === 'private' ? { memberId: conversation.memberId } : {}), projectId: session.projectId, taskId: session.taskId, attemptId: session.attemptId, role: session.role, sessionKey: session.id, + const row = { id, ...(conversation?.roomId ? { roomId: conversation.roomId } : {}), model: session.execution.model ?? selected.selected.model ?? null, chiefRunId: chief?.runId ?? null, visibility: conversation?.visibility ?? 'project', ...(conversation?.visibility === 'private' ? { memberId: conversation.memberId } : {}), projectId: session.projectId, taskId: session.taskId, attemptId: session.attemptId, role: session.role, policyRole: session.policyRole ?? session.definition?.policyRole ?? session.role, sessionKey: session.id, resourceId: session.resourceLease.resourceId, leaseId: session.resourceLease.id, generation: session.resourceLease.generation, harness: session.execution.harness, sdkVersion: null, expectedSdkVersion: VERSIONS[session.execution.harness], principalId: selected.principalId, - cwd: workspace.path, scratchDirectory, status: 'prepared', createdAt: Date.now(), eventCount: 0 } - if (logical) Object.assign(row, { logicalSessionId: logical.id, bindingKey, ...(previous ? { backendSessionId: previous.sessionId, continuedFrom: previous.workerId } : { rotationReason: logical.bindings.length ? 'Execution binding changed; retained logical Task history' : 'First Task-role execution' }) }) + cwd: workspace.path, scratchDirectory, status: 'prepared', executionStartedAt: null, createdAt: Date.now(), eventCount: 0 } + if (logical) Object.assign(row, { logicalSessionId: logical.id, bindingKey, bindingVersion: 'task-role-binding/v2', definitionKey, ...(previous ? { backendSessionId: previous.sessionId, continuedFrom: previous.workerId, ...(previous.key !== bindingKey ? { migratedBindingKey: previous.key } : {}) } : { rotationReason: 'First Task-role execution' }) }) else if (chief) Object.assign(row, { logicalSessionId: chief.agentId, bindingKey, ...(chief.resumeSessionId ? { backendSessionId: chief.resumeSessionId, continuedFrom: chief.previousWorkerId } : {}) }) setEntry(workers(), id, row) if (chief) await rt.projectResources.markStarted(row.leaseId, row.generation, { kind: 'sdk', handle: id }) @@ -150,7 +164,11 @@ export function createProjectHarness(rt) { if (claim?.id === session.poolClaimId && claim.status === 'active' && task.collaboration.generation === claim.generation) await rt.projectInputs?.transport(task.id, claim, control.inputBatch.batchId, { kind: 'sdk-output', sha256: row.lastEventHash }) } const backend = event.event.thread_id ?? event.event.session_id - if (backend) { if (row.backendSessionId && row.backendSessionId !== backend) throw refuse('SDK changed its bound Session'); row.backendSessionId = backend; session.backendSessionId = backend } + if (backend) { + if (row.backendSessionId && row.backendSessionId !== backend) throw refuse('SDK changed its bound Session') + row.backendSessionId = backend; session.backendSessionId = backend + if (logical && !logical.bindings.some(binding => binding.workerId === row.id)) logical.bindings.push({ version: 'task-role-binding/v2', definitionKey, key: bindingKey, sessionId: backend, workerId: row.id, policySha256: session.policy?.sha256 ?? null, attemptId: row.attemptId, at: Date.now(), observedBeforeCompletion: true }) + } } if (event.type === 'completed') { rt.projectResources.setting(row.projectId, row.resourceId) @@ -161,6 +179,7 @@ export function createProjectHarness(rt) { if (claim?.id === session.poolClaimId && claim.status === 'active' && task.collaboration.generation === claim.generation) { const next = await rt.projectInputs?.prepare(task.id, claim, { next: true }) if (next?.inputs.length) { + await requireExecutionAuthority(rt, session) row.backendSessionId = event.sessionId; session.backendSessionId = event.sessionId control.inputBatch = next; control.safeInputTurns = (control.safeInputTurns ?? 0) + 1 row.safeInputTurns = control.safeInputTurns @@ -189,7 +208,7 @@ export function createProjectHarness(rt) { if (event.text !== rawText) { row.rawResultHash = sha(rawText); row.outputFraming = 'json-handoff/v1' } row.backendSessionId = event.sessionId; session.backendSessionId = event.sessionId; row.usage = event.usage; row.resultHash = sha(event.text) if (logical) { - logical.bindings.push({ key: bindingKey, sessionId: event.sessionId, workerId: row.id, policySha256: session.policy?.sha256 ?? null, attemptId: row.attemptId, at: Date.now() }) + if (!logical.bindings.some(binding => binding.workerId === row.id)) logical.bindings.push({ version: 'task-role-binding/v2', definitionKey, key: bindingKey, sessionId: event.sessionId, workerId: row.id, policySha256: session.policy?.sha256 ?? null, attemptId: row.attemptId, at: Date.now() }) await rt.save() } if (control.outputs.every(name => new RegExp(`(?:^|\\n)OUTPUT ${name}:`).test(event.text))) { @@ -254,6 +273,7 @@ export function createProjectHarness(rt) { const next = await rt.projectInputs?.prepare(task.id, claim, { next: true }) if (!next?.inputs.length) return { woken: false } if (active.get(row.id) !== control || task.collaboration.claim !== claim || task.collaboration.generation !== claim.generation || claim.status !== 'active' || row.status !== 'waiting-input' || row.pending?.kind !== 'message' || lease.status === 'released') return { woken: false } + await requireExecutionAuthority(rt, control.session) rt.requireProjectAccess(actor, task.projectId, 'task-write', task) control.inputBatch = next; control.safeInputTurns = (control.safeInputTurns ?? 0) + 1 row.safeInputTurns = control.safeInputTurns; row.status = 'running'; delete row.pending @@ -272,14 +292,29 @@ export function createProjectHarness(rt) { if (!control || control.closed || !['prepared', 'idle'].includes(row.status)) throw refuse('SDK worker is unavailable or already dispatched') const lease = ownEntry(rt.state.projectResourceLeases ?? {}, row.leaseId) if (!lease || lease.generation !== row.generation || lease.status === 'released') throw refuse('SDK callback lease is stale') + if (control.session.poolClaimId) await requireExecutionAuthority(rt, control.session) control.outputs = (step.outputs ?? []).map(port => port.name) control.inputBatch = step.inputs?.taskInbox const prompt = `${roleExecutionPrompt(control.prompt, control.session, control.continuedPolicyHash)}${control.archivePath ? '\nExact scoped context archive: ' + control.archivePath + '. Read only needed entries with bounded output; do not print the whole archive. Archive SHA256: ' + control.session.context.archiveSha256 : ''}\n\nTask ${step.taskId}: ${step.brief}\nInputs: ${JSON.stringify(step.inputs ?? {})}\nFinish with OUTPUT : for ${control.outputs.join(', ')}.` - row.status = 'running' const workspace = rt.state.projectAttempts?.[row.attemptId]?.workspaces?.[row.role] - const writableRoots = row.role === 'developer' && row.harness === 'codex' && rt.projectResources.setting(row.projectId, row.resourceId).type === 'local' && workspace?.path === row.cwd ? await gitWritableRoots({ cwd: row.cwd, repository: workspace.repository, developer: true }) : [] + const writableRoots = (row.policyRole ?? row.role) === 'developer' && row.harness === 'codex' && rt.projectResources.setting(row.projectId, row.resourceId).type === 'local' && workspace?.path === row.cwd ? await gitWritableRoots({ cwd: row.cwd, repository: workspace.repository, developer: true }) : [] + row.status = 'running'; row.executionStartedAt = Date.now() + try { + const before = rt.savesQueued?.() + const saved = rt.save() + const queued = rt.savesQueued?.() + await saved + if (typeof before === 'number' && queued <= before) throw refuse('SDK dispatch did not commit a state snapshot') + const currentLease = ownEntry(rt.state.projectResourceLeases ?? {}, row.leaseId) + if (control.session.poolClaimId) await requireExecutionAuthority(rt, control.session) + if (active.get(row.id) !== control || control.closed || row.status !== 'running' || currentLease?.generation !== row.generation || currentLease.status === 'released') throw refuse('SDK dispatch was superseded before input delivery') + } catch (error) { + if (row.status === 'running') row.status = 'idle' + row.executionStartedAt = null; row.dispatchPreventedAt = Date.now() + throw error + } control.process.stdin.write(JSON.stringify({ type: 'start', settings: { harness: row.harness, cwd: row.cwd, sessionId: control.resumeSessionId, noTools: Boolean(control.chief), model: control.profile.model, env: control.env, - ...(control.profile.baseUrl ? { baseUrl: control.profile.baseUrl } : {}), readOnly: Boolean(control.chief) || row.role !== 'developer', writableRoots, scratchDirectory: row.scratchDirectory, maxBudgetUsd: control.profile.maxBudgetUsd ?? 1 }, prompt }) + '\n') + ...(control.profile.baseUrl ? { baseUrl: control.profile.baseUrl } : {}), readOnly: Boolean(control.chief) || (row.policyRole ?? row.role) !== 'developer', writableRoots, scratchDirectory: row.scratchDirectory, maxBudgetUsd: control.profile.maxBudgetUsd ?? 1 }, prompt }) + '\n') if (control.inputBatch?.inputs.length && control.session.poolClaimId) { const task = rt.state.tasks[row.taskId], claim = task?.collaboration?.claim if (claim?.id === control.session.poolClaimId) await rt.projectInputs?.transport(task.id, claim, control.inputBatch.batchId, { kind: 'adapter-written', sha256: sha(JSON.stringify(control.inputBatch)) }) @@ -354,7 +389,9 @@ export function createProjectHarness(rt) { if (!row || !control) throw refuse('Execution Session is unavailable') const task = ownEntry(rt.state.tasks, row.taskId) rt.requireProjectAccess(actor, row.projectId, 'task-write', task) - if (row.status !== 'waiting-input' || body.generation !== row.generation) throw refuse('Execution input is stale') + if (control.session.poolClaimId) await requireExecutionAuthority(rt, control.session) + rt.requireProjectAccess(actor, row.projectId, 'task-write', task) + if (row.status !== 'waiting-input' || control.closed || body.generation !== row.generation) throw refuse('Execution input is stale') if (row.pending.kind === 'permission') { rt.requireProjectAccess(actor, row.projectId, 'configure') if (body.requestId !== row.pending.id) throw refuse('Permission request is stale') @@ -366,6 +403,6 @@ export function createProjectHarness(rt) { delete row.pending; row.status = 'running'; await rt.save(); return { sent: true } } return { profile, spawn, dispatch, taskInputWake, release, input, chiefTurn, onResult: fn => { listeners.add(fn); return () => listeners.delete(fn) }, - owns: id => Boolean(ownEntry(workers(), id)), hasController: id => active.has(id) && !active.get(id).closed, versions: VERSIONS } + capabilities: harnessCapabilities, owns: id => Boolean(ownEntry(workers(), id)), hasController: id => active.has(id) && !active.get(id).closed, versions: VERSIONS } } export function install(rt) { rt.projectHarness = createProjectHarness(rt) } diff --git a/core/src/host/project-native-sessions.js b/core/src/host/project-native-sessions.js new file mode 100644 index 000000000..63d58f2c7 --- /dev/null +++ b/core/src/host/project-native-sessions.js @@ -0,0 +1,193 @@ +// Native execution keeps exact Task-role context; Bot deletion is not exit proof. +import { createHash, randomUUID } from 'node:crypto' +import { captureProcessScope, priorScopeExited, supervisorProofMatches } from './project-process-proof.js' +import { ownEntry, setEntry } from './store.js' +import { requireExecutionAuthority } from './project-execution-authority.js' +import { taskSession, taskSessionId, roleDefinitionKey } from './project-task-sessions.js' +const digest = value => createHash('sha256').update(JSON.stringify(value)).digest('hex') +const refuse = message => Object.assign(new Error(message), { code: 'bot/context-resume' }) +const clone = value => structuredClone(value) +export function nativeExitConfirmed(state, row) { + const proof = row?.releaseProof + if (!row || row.version !== 'project-native-worker/v2' || row.status !== 'quiescent' || !['native-agent-quiescent', 'systemd-prior-invocation-exited'].includes(proof?.kind) || proof.workerId !== row.id || proof.sessionId !== row.sessionId || proof.generation !== row.generation || proof.bindingKey !== row.bindingKey) return false + if (proof.kind === 'native-agent-quiescent' && proof.registeredWorkEmpty !== true) return false + if (!row.leaseId) return proof.kind === 'native-agent-quiescent' || supervisorProofMatches(row.processScope, proof) + const lease = ownEntry(state.projectResourceLeases ?? {}, row.leaseId) + return Boolean(lease && lease.projectId === row.projectId && lease.attemptId === row.attemptId && lease.role === row.role && lease.sessionKey === row.sessionKey && lease.resourceId === row.resourceId && lease.generation === row.generation && lease.worker?.kind === 'native' && lease.worker.botId === row.id && (proof.kind !== 'systemd-prior-invocation-exited' || supervisorProofMatches(lease.processScope, proof))) +} +export function createProjectNativeSessions(rt, { priorExit = priorScopeExited, captureScope = captureProcessScope } = {}) { + const hostEpoch = randomUUID() + const rows = () => (rt.state.projectNativeWorkers ??= {}) + const stopping = new Map() + const binding = new Set() + rt.projectEffects?.register('native.activate', { run: effect => ({ workerId: effect.payload.workerId, status: 'durable-native-binding' }) }, { replace: true }) + const bindingKey = (session, workspace) => digest({ projectId: session.projectId, taskId: session.taskId, role: session.role, + harness: 'flybot', resourceId: session.resourceLease?.resourceId ?? null, cwd: workspace.path, definitionKey: roleDefinitionKey(session) }) + function previous(session, workspace) { + const logical = taskSession(rt.state, session), last = logical.bindings.at(-1), key = bindingKey(session, workspace) + if (!last) { + const historical = Object.values(rt.state.projectAttempts ?? {}).filter(attempt => attempt.projectId === session.projectId && attempt.taskId === session.taskId).flatMap(attempt => [attempt.roleSessions?.[session.role], ...(attempt.roleSessionHistory ?? [])]).some(prior => prior && prior.id !== session.id && prior.role === session.role && prior.sessionId && (prior.execution?.harness ?? 'flybot') === 'flybot') + if (historical) throw refuse('Legacy native Task-role context has no confirmed binding; explicit recovery is required instead of a fresh Session') + return { logical, key, row: null } + } + const row = ownEntry(rows(), last.workerId) + const lease = row?.leaseId && ownEntry(rt.state.projectResourceLeases ?? {}, row.leaseId) + if (last.harness !== 'flybot' || !row || row.logicalSessionId !== logical.id || row.projectId !== session.projectId || row.taskId !== session.taskId || row.role !== session.role || last.key !== key || row.bindingKey !== key || last.sessionId !== row.sessionId || !nativeExitConfirmed(rt.state, row) || row.leaseId && lease?.status !== 'released' || rt.botOf(row.id)) throw refuse('Latest native Task-role Session is incompatible or its execution has not confirmed release; a fresh Session is forbidden') + return { logical, key, row } + } + const authority = session => requireExecutionAuthority(rt, session) + async function bind(session, workspace, botId) { + await authority(session) + const bot = rt.botOf(botId) + if (!bot || bot.workspaceId !== workspace.id || bot.attemptId !== session.attemptId || session.status === 'released') throw refuse('Native carrier is unavailable for its registered Workspace') + const selected = previous(session, workspace), original = selected.row + let sessionId = rt.chatOf(botId) + if (original) { + if ((original.model ?? null) !== (bot.model ?? null)) throw refuse('Native Task-role model changed; explicit recovery is required') + const resumed = await rt.resolveAgent(original.sessionId) + if (resumed.session?.id !== original.sessionId || resumed.status !== 'idle' || typeof resumed.whenIdle !== 'function') throw refuse('Original native Session cannot be resumed safely') + await resumed.whenIdle() + await authority(session) + const current = previous(session, workspace) + if (current.row !== original || rt.botOf(botId) !== bot || session.status === 'released') throw refuse('Native resume binding changed while loading context') + await rt.ctx.workspaceRegistry.archiveSession(sessionId, { stopActivity: true }) + rt.state.retiredSessions ??= {} + setEntry(rt.state.retiredSessions, sessionId, botId) + sessionId = original.sessionId + bot.sessionId = sessionId + bot.segments = clone(original.segments ?? []) + rt.indexSessions?.() + } + const row = { id: botId, hostEpoch, version: 'project-native-worker/v2', processScope: await captureScope(), projectId: session.projectId, taskId: session.taskId, attemptId: session.attemptId, + role: session.role, model: bot.model ?? null, ...(original ? { continuedFrom: original.id } : {}), logicalSessionId: selected.logical.id, sessionKey: session.id, sessionId, cwd: workspace.path, bindingKey: selected.key, + definitionKey: roleDefinitionKey(session), generation: session.resourceLease?.generation ?? session.generation ?? 0, + ...(session.resourceLease ? { leaseId: session.resourceLease.id, resourceId: session.resourceLease.resourceId } : {}), status: 'bound', createdAt: Date.now() } + setEntry(rows(), botId, row) + selected.logical.bindings.push({ harness: 'flybot', version: 'native-session/v1', key: row.bindingKey, sessionId, workerId: botId, attemptId: session.attemptId }) + session.botId = botId; session.sessionId = sessionId + await rt.save() + return { id: botId, sessionId } + } + async function activate(session, botId) { + await authority(session) + const row = ownEntry(rows(), botId) + if (!row || row.hostEpoch !== hostEpoch || row.sessionKey !== session.id || row.attemptId !== session.attemptId || row.sessionId !== rt.chatOf(botId) || row.status !== 'bound') throw refuse('Native activation no longer matches its registered execution Session') + if (!rt.projectEffects) throw refuse('Native activation requires durable Effects proof') + const token = rt.projectEffects.stage({ kind: 'native.activate', key: botId, projectId: row.projectId, taskId: row.taskId, + payload: { workerId: botId, sessionId: row.sessionId, sessionKey: row.sessionKey, bindingKey: row.bindingKey, attemptId: row.attemptId, generation: row.generation } }) + row.activationEffect = token + await rt.projectEffects.persist(token) + if (!rt.projectEffects.intentCommitted(token)) throw refuse('Native activation binding is not committed') + await authority(session) + if (row.continuedFrom) { + if (typeof rt.ctx.workspaceRegistry.unarchiveSession !== 'function') throw refuse('Native Session unarchive is unavailable; explicit recovery is required') + await rt.ctx.workspaceRegistry.unarchiveSession(row.sessionId) + await authority(session) + } + return { workerId: row.id, sessionId: row.sessionId } + } + function ownedSessions(row, bot, sessionId) { + const ids = new Set([sessionId, row.sessionId, ...(bot.segments ?? []).map(value => value.sessionId), ...(row.knownSessionIds ?? [])].filter(Boolean)) + const headers = (rt.ctx.sessions?.list?.() ?? []).map(value => value.header) + let changed + do { + changed = false + for (const header of headers) if (header.origin === 'subagent' && ids.has(header.parentSession) && !ids.has(header.id)) { ids.add(header.id); changed = true } + } while (changed) + row.knownSessionIds = [...ids] + return row.knownSessionIds + } + function recordExit(row, bot, details) { + const sessionId = rt.chatOf(row.id), logical = taskSession(rt.state, row), last = logical.bindings.at(-1) + if (last?.workerId !== row.id || last.key !== row.bindingKey) throw refuse('Native worker is not the latest Task-role binding') + row.version = 'project-native-worker/v2' + row.sessionId = sessionId; row.model = bot.model ?? null; row.segments = clone(bot.segments ?? []) + last.sessionId = sessionId + const attempt = ownEntry(rt.state.projectAttempts ?? {}, row.attemptId) + const session = [attempt?.roleSessions?.[row.role], ...(attempt?.roleSessionHistory ?? [])].find(value => value?.id === row.sessionKey && value.botId === row.id) + if (session) session.sessionId = sessionId + row.releaseProof = { ...details, workerId: row.id, sessionId, bindingKey: row.bindingKey, generation: row.generation, at: Date.now() } + row.status = 'quiescent' + } + async function stop(botId) { + const row = ownEntry(rows(), botId) + if (!row) return false + if (nativeExitConfirmed(rt.state, row)) return clone(row.releaseProof) + const bot = rt.botOf(botId), sessionId = bot && rt.chatOf(botId) + row.status = 'stopping'; await rt.save() + try { + if (row.hostEpoch !== hostEpoch) { + const lease = ownEntry(rt.state.projectResourceLeases ?? {}, row.leaseId), scope = lease?.processScope ?? row.processScope + const proof = { kind: 'systemd-prior-invocation-exited', previousInvocationId: scope?.invocationId } + if (!bot || !supervisorProofMatches(scope, proof) || !await priorExit(scope)) throw refuse('Native execution belongs to an earlier host; actual prior process-group exit is unconfirmed') + if (ownEntry(rows(), botId) !== row || rt.botOf(botId) !== bot || rt.chatOf(botId) !== sessionId || row.leaseId && (!lease || lease.generation !== row.generation || lease.worker?.kind !== 'native' || lease.worker.botId !== botId || lease.projectId !== row.projectId || lease.attemptId !== row.attemptId || lease.role !== row.role || lease.sessionKey !== row.sessionKey || lease.resourceId !== row.resourceId || lease.processScope !== scope) || !row.leaseId && row.processScope !== scope) throw refuse('Native prior-host scope or binding changed during exit inspection') + recordExit(row, bot, proof) + await rt.save() + return clone(row.releaseProof) + } + const agent = sessionId && await rt.resolveAgent(sessionId) + if (!bot || !agent || agent.session?.id !== sessionId || typeof agent.cancel !== 'function' || typeof agent.whenIdle !== 'function') throw refuse('Native worker stop is unconfirmed; keep its execution/resource held') + agent.cancel({ kind: 'hook', reason: 'Project native execution release' }) + if (typeof rt.ctx.parallel !== 'function' || typeof rt.ctx.waterfall !== 'function') throw refuse('Native registered session work cannot be inspected; keep execution held') + const sessions = ownedSessions(row, bot, sessionId) + await Promise.all(sessions.map(sessionId => rt.ctx.parallel('workspace/session-stop', { sessionId }))) + await agent.whenIdle() + const reports = await Promise.all(sessions.map(sessionId => rt.ctx.waterfall('workspace/session-activity', { sessionId }, () => Promise.resolve([])))) + const activity = reports.flatMap(report => Array.isArray(report) ? report : [{ kind: 'unknown' }]) + if (activity.length) { + row.pendingWork = activity.map(value => ({ kind: value.kind, count: value.items?.length ?? 1 })) + throw refuse('Native registered session work is still stopping; keep execution and resources held') + } + delete row.pendingWork + if (ownEntry(rows(), botId) !== row || rt.botOf(botId) !== bot || rt.chatOf(botId) !== sessionId || agent.status !== 'idle') throw refuse('Native worker changed before quiescence confirmation') + recordExit(row, bot, { kind: 'native-agent-quiescent', registeredWorkEmpty: true }) + await rt.save() + return clone(row.releaseProof) + } catch (error) { row.status = 'stop-unconfirmed'; delete row.releaseProof; row.failure = String(error.message).slice(0, 1000); await rt.save(); throw error } + } + function admitsAgent(agent, track = false) { + let header = agent.session.header ?? { id: agent.session.id } + const visited = new Set() + while (!visited.has(header.id)) { + visited.add(header.id) + const botId = rt.selfOf(header.id), bot = botId && rt.botOf(botId) + const row = ownEntry(rt.state.projectNativeWorkers ?? {}, botId) + if (row) { + const admitted = api.admits(botId) + if (admitted && track) row.knownSessionIds = [...new Set([...(row.knownSessionIds ?? []), ...visited])] + return admitted + } + if (bot?.ephemeral && bot.workspaceId && bot.attemptId) return false + if (header.origin !== 'subagent' || !header.parentSession) return true + const parent = rt.ctx.sessions.get(header.parentSession) + header = parent?.header ?? { id: header.parentSession } + } + return false + } + const api = { previous, activate, admitsAgent, async validateStep(agent) { + if (!api.admitStep(agent)) return false + const row = Object.values(rows()).find(value => value.hostEpoch === hostEpoch && value.status === 'bound' && rt.botOf(value.id) && value.knownSessionIds?.includes(agent.session.id)) + if (!row) return true + const session = ownEntry(rt.state.projectAttempts ?? {}, row.attemptId)?.roleSessions?.[row.role] + if (!session?.poolClaimId) return true + try { await authority(session); return api.admits(row.id) } + catch (error) { + // Do not wait for this same Agent to become idle inside its pre-step hook. + void rt.projectPools?.revokeExecution(row.taskId, session.poolClaimId, String(error.message).slice(0, 1000)).catch(failure => rt.warn('Execution revocation: %s', failure.message)) + return false + } + }, admitStep: agent => admitsAgent(agent, true), currentHost: botId => Boolean(rt.botOf(botId) && ownEntry(rt.state.projectNativeWorkers ?? {}, botId)?.hostEpoch === hostEpoch), async bind(session, workspace, botId) { + const key = taskSessionId(session.projectId, session.taskId, session.role) + if (binding.has(key)) throw refuse('Native Task-role binding is already in progress') + binding.add(key) + try { return await bind(session, workspace, botId) } finally { binding.delete(key) } + }, admits(botId) { + const row = ownEntry(rt.state.projectNativeWorkers ?? {}, botId) + return !row || row.hostEpoch === hostEpoch && row.status === 'bound' && rt.readOnly?.() === undefined && Boolean(row.activationEffect && rt.projectEffects?.intentCommitted(row.activationEffect)) + }, + stop(botId) { + if (stopping.has(botId)) return stopping.get(botId) + const operation = stop(botId).finally(() => stopping.delete(botId)); stopping.set(botId, operation); return operation + } } + return api +} diff --git a/core/src/host/project-pools.js b/core/src/host/project-pools.js index 947f397a8..b38663262 100644 --- a/core/src/host/project-pools.js +++ b/core/src/host/project-pools.js @@ -19,6 +19,14 @@ export function createProjectPools(rt) { if (!target) throw refuse('Target logical Bot is not in this Project') return target } + const sourceOf = (task, botId) => { + const active = definitions(task.projectId).roles.find(row => row.id === botId) + if (active) return active + const claim = task.collaboration?.claim + const retired = Object.values(rt.state.projectRetiredRoles ?? {}).find(row => row.projectId === task.projectId && row.id === botId) + if (!retired || claim?.botId !== botId || !['active', 'completed', 'intervened'].includes(claim.status)) throw refuse('Source logical Bot has no retained execution claim') + return { ...retired, role: claim.role ?? retired.role, policyRole: claim.policyRole ?? retired.policyRole ?? retired.role } + } const dispatchIntent = (task, row) => rt.projectEffects?.stage({ kind: 'pool.dispatch', key: row.id, projectId: task.projectId, taskId: task.id, payload: { handoffId: row.id, botId: row.to, initiatedBy: row.initiatedBy } }) async function authority(task) { @@ -39,7 +47,7 @@ export function createProjectPools(rt) { async function checkAdmission(task, { from, to, attemptId, requestId, purpose }) { if (!adapter) throw refuse('Handoff execution adapter is unavailable') const c = task.collaboration, target = targetOf(task.projectId, to) - const fromRole = from ? targetOf(task.projectId, from).role : '' + const fromRole = from ? sourceOf(task, from).role : '' // A privileged, audited Chief decision resolves a routing loop. Keep global // usage/history, but bound autonomous repetitions after that intervention. const epoch = c.history.findLastIndex(row => row.purpose === 'chief-intervention') @@ -108,14 +116,16 @@ export function createProjectPools(rt) { async function claim(taskId, botId, handoffId) { const task = taskOf(taskId), c = task.collaboration if (!c || c.botId !== botId || c.status !== 'queued' || handoffId && c.currentHandoffId !== handoffId) return null - await authority(task); targetOf(task.projectId, botId) + await authority(task) + const definition = definitions(task.projectId).roles.find(row => row.id === botId) + if (!definition) { c.status = 'blocked'; c.block = { reason: 'Target role is retired; Chief must explicitly select an active role', at: Date.now() }; await rt.save(); return null } if (Object.values(rt.state.tasks).some(row => row.collaboration?.claim?.botId === botId && row.collaboration.claim.status === 'active')) return null - if (adapter.ready && !adapter.ready({ task, definition: targetOf(task.projectId, botId) })) return null + if (adapter.ready && !adapter.ready({ task, definition })) return null const receipt = c.history.find(row => row.id === c.currentHandoffId) if (!receipt || receipt.status !== 'pending') throw refuse('Task has no pending handoff receipt') const limit = rt.config.projectHandoff?.quota?.steps if (Number.isFinite(limit) && (c.usage?.steps ?? 0) >= limit) { c.status = 'blocked'; c.block = { reason: 'Handoff refused (quota): execution step quota is exhausted', at: Date.now() }; await rt.save(); throw refuse(c.block.reason) } - const claim = { id: `claim-${randomUUID()}`, botId, generation: ++c.generation, taskId: task.id, projectId: task.projectId, attemptId: c.attemptId, status: 'active', createdAt: Date.now() } + const claim = { id: `claim-${randomUUID()}`, botId, role: definition.role, policyRole: definition.policyRole ?? definition.role, generation: ++c.generation, taskId: task.id, projectId: task.projectId, attemptId: c.attemptId, status: 'active', createdAt: Date.now() } c.usage ??= {}; c.usage.steps = (c.usage.steps ?? 0) + 1 c.claim = claim; c.status = 'claimed'; receipt.status = 'accepted'; receipt.acceptedAt = Date.now(); receipt.claimId = claim.id await rt.save() @@ -133,7 +143,7 @@ export function createProjectPools(rt) { const claim = await current(task, token) const action = decision?.action, reason = reasonOf(decision?.reason) if (!['handoff', 'return', 'transfer', 'complete', 'arbitrate'].includes(action)) throw refuse('Bot must hand off, return, transfer or request Chief judgement') - const role = targetOf(task.projectId, claim.botId).role + const role = sourceOf(task, claim.botId).role const next = ['complete', 'arbitrate'].includes(action) ? definitions(task.projectId).roles.find(row => row.role === 'chief') : targetOf(task.projectId, decision.to) if (!next) throw refuse('Chief is not configured') // Stop confirmation precedes relinquishing the claim and capacity. @@ -150,7 +160,7 @@ export function createProjectPools(rt) { c.usage ??= {}; c.usage.ms = (c.usage.ms ?? 0) + Date.now() - claim.createdAt const usage = rt.state.projectHarnessWorkers?.[claim.workerId]?.usage c.usage.tokens = (c.usage.tokens ?? 0) + (usage?.input_tokens ?? 0) + (usage?.output_tokens ?? 0) - if (next.role === 'developer' && c.kind === 'code') c.attemptId = await adapter.rework?.({ task, claim, reason }) ?? c.attemptId + if ((next.policyRole ?? next.role) === 'developer' && c.kind === 'code') c.attemptId = await adapter.rework?.({ task, claim, reason }) ?? c.attemptId try { return await offer(task, { from: claim.botId, to: next.id, reason, requestId: decision.requestId ?? claim.id, attemptId: c.attemptId, evidence: decision.evidence ?? [], purpose: action, expectedGeneration: claim.generation }) } catch (error) { @@ -226,15 +236,23 @@ export function createProjectPools(rt) { } } } + async function revokeExecution(taskId, claimId, reason) { + const task = taskOf(taskId), c = task.collaboration + if (!c || c.claim?.id !== claimId || c.claim.status !== 'active') return false + const claim = c.claim + c.status = 'blocked'; ++c.generation; c.block = { reason, at: Date.now() } + claim.status = 'revoked' + await rt.save() + if (claim.workerId) { try { claim.exitConfirmed = await adapter.release(claim.workerId) } catch { claim.exitConfirmed = false } } + await rt.save() + return true + } async function revokePrincipal(projectId, memberId) { for (const task of Object.values(rt.state.tasks)) { const c = task.collaboration if (task.projectId !== projectId || c?.initiatedBy !== memberId || !['queued', 'claimed'].includes(c.status)) continue - c.status = 'blocked'; ++c.generation; c.block = { reason: 'Initiating human authority was revoked', at: Date.now() } - if (c.claim) c.claim.status = 'revoked' - await rt.save() - if (c.claim?.workerId) { try { c.claim.exitConfirmed = await adapter.release(c.claim.workerId) } catch { c.claim.exitConfirmed = false } } - await rt.save() + if (c.claim?.status === 'active') await revokeExecution(task.id, c.claim.id, 'Initiating human authority was revoked') + else { c.status = 'blocked'; ++c.generation; c.block = { reason: 'Initiating human authority was revoked', at: Date.now() }; await rt.save() } } } async function validateClosure(taskId, actor, evidence) { @@ -281,7 +299,7 @@ export function createProjectPools(rt) { ++c.generation if (oldClaim) oldClaim.status = 'intervened' c.status = 'stopping'; await rt.save() - if (oldClaim?.workerId && targetOf(task.projectId, oldClaim.botId).role !== 'chief') { + if (oldClaim?.workerId && sourceOf(task, oldClaim.botId).role !== 'chief') { let stopped = false try { stopped = await adapter.release(oldClaim.workerId) } catch { /* retain unknown worker capacity */ } oldClaim.exitConfirmed = stopped @@ -293,7 +311,7 @@ export function createProjectPools(rt) { if (currentReceipt) { currentReceipt.status = 'intervened'; currentReceipt.finishedAt = Date.now(); if (oldClaim) currentReceipt.execution = clone(oldClaim); currentReceipt.chiefDecisionId = body.decisionId } c.initiatedBy = actor.memberId; c.sourceSessionId = null const previous = rt.state.projectAttempts?.[c.attemptId] - if (target.role === 'developer') c.attemptId = await adapter.rework({ task, claim: oldClaim ?? { attemptId: c.attemptId, id: body.decisionId, botId: `${task.projectId}:chief` }, reason: body.reason, force: body.rework !== false, decisionId: body.decisionId }) + if ((target.policyRole ?? target.role) === 'developer') c.attemptId = await adapter.rework({ task, claim: oldClaim ?? { attemptId: c.attemptId, id: body.decisionId, botId: `${task.projectId}:chief` }, reason: body.reason, force: body.rework !== false, decisionId: body.decisionId }) else if (previous && previous.status !== 'running') throw refuse('Interrupted delivery must return to Developer in a new Attempt') return await offer(task, { from: `${task.projectId}:chief`, to: target.id, reason: body.reason, requestId: body.decisionId, attemptId: c.attemptId, purpose: 'chief-intervention' }) } catch (error) { @@ -325,7 +343,7 @@ export function createProjectPools(rt) { return changed } return { connect(value) { adapter = value; return () => { if (adapter === value) adapter = null } }, admit: (attempt, actor, body) => serial(() => admit(attempt, actor, body)), available, owns, - claim: (id, botId) => serial(() => claim(id, botId)), decide: (id, token, decision) => serial(() => decide(id, token, decision)), result: row => serial(() => result(row)), pump, recover, revokePrincipal, validateClosure, retryChief: (id, actor) => serial(() => retryChief(id, actor)), chiefClaim: (id, actor) => serial(() => chiefClaim(id, actor)), chiefRoute: (body, actor) => serial(() => chiefRoute(body, actor)) } + claim: (id, botId) => serial(() => claim(id, botId)), decide: (id, token, decision) => serial(() => decide(id, token, decision)), result: row => serial(() => result(row)), pump, recover, revokePrincipal: (projectId, memberId) => serial(() => revokePrincipal(projectId, memberId)), revokeExecution: (id, claimId, reason) => serial(() => revokeExecution(id, claimId, reason)), validateClosure, retryChief: (id, actor) => serial(() => retryChief(id, actor)), chiefClaim: (id, actor) => serial(() => chiefClaim(id, actor)), chiefRoute: (body, actor) => serial(() => chiefRoute(body, actor)) } } export function install(rt) { rt.projectPools = createProjectPools(rt) diff --git a/core/src/host/project-process-proof.js b/core/src/host/project-process-proof.js index 5da21cdf6..d641c943b 100644 --- a/core/src/host/project-process-proof.js +++ b/core/src/host/project-process-proof.js @@ -24,3 +24,7 @@ export async function priorScopeExited(scope, options = {}) { // Different machines/boots, unmanaged launchers or changed kill policy remain unknown. return current.kind === 'systemd-control-group' && current.bootId === scope.bootId && current.unit === scope.unit && current.controlGroup === scope.controlGroup && current.invocationId !== scope.invocationId } +export function supervisorProofMatches(scope, proof) { + return Boolean(scope?.kind === 'systemd-control-group' && scope.killMode === 'control-group' && /^[a-f0-9]{32}$/.test(scope.invocationId ?? '') && + proof?.kind === 'systemd-prior-invocation-exited' && proof.previousInvocationId === scope.invocationId) +} diff --git a/core/src/host/project-resources.js b/core/src/host/project-resources.js index 3602f9328..3541227b8 100644 --- a/core/src/host/project-resources.js +++ b/core/src/host/project-resources.js @@ -1,5 +1,6 @@ // Operator-provisioned execution capabilities; Project configuration only references IDs. -import { captureProcessScope, priorScopeExited } from './project-process-proof.js' +import { nativeExitConfirmed } from './project-native-sessions.js' +import { captureProcessScope, priorScopeExited, supervisorProofMatches } from './project-process-proof.js' import { execFile, spawn } from 'node:child_process' import { realpath, copyFile, stat } from 'node:fs/promises' import { randomUUID, createHash } from 'node:crypto' @@ -13,7 +14,7 @@ const active = lease => lease.status !== 'released' const contains = (root, path) => { const part = relative(root, path); return part === '' || part !== '..' && !part.startsWith(`..${sep}`) && !isAbsolute(part) } const safeEnv = () => ({ PATH: process.env.PATH ?? '/usr/local/bin:/usr/bin:/bin', LANG: 'C.UTF-8' }) -export function createProjectResources(rt) { +export function createProjectResources(rt, { priorExit = priorScopeExited } = {}) { const { state, config } = rt const leases = () => (state.projectResourceLeases ??= {}) const resources = () => config.projectResources ?? {} @@ -120,9 +121,14 @@ export function createProjectResources(rt) { const lease = requireLease(id, generation) if (lease.status === 'released') return const worker = lease.worker - const confirmed = !worker || worker.kind === 'native' && !ownEntry(state.bots ?? {}, worker.botId) || confirmation?.handle === worker.handle && confirmation?.exited === true + const native = worker?.kind === 'native' && ownEntry(state.projectNativeWorkers ?? {}, worker.botId) + const scope = lease.processScope + const supervisor = worker?.kind === 'native' && supervisorProofMatches(scope, confirmation?.proof) && await priorExit(scope) + if (requireLease(id, generation) !== lease || lease.worker !== worker || lease.processScope !== scope) throw failure('Execution resource binding changed during exit inspection') + if (lease.status === 'released') return + const confirmed = supervisor || !worker || worker.kind === 'native' && !ownEntry(state.bots ?? {}, worker.botId) && native?.leaseId === lease.id && nativeExitConfirmed(state, native) || worker.kind !== 'native' && typeof worker.handle === 'string' && worker.handle.length > 0 && confirmation?.handle === worker.handle && confirmation?.exited === true lease.status = confirmed ? 'released' : 'stopping' - if (confirmed) { lease.releasedAt = Date.now(); lease.releaseProof ??= confirmation?.proof ?? { kind: worker ? 'host-confirmed-worker-exit' : 'unbound-reservation-release', at: lease.releasedAt } } + if (confirmed) { lease.releasedAt = Date.now(); lease.releaseProof ??= confirmation?.proof ?? native?.releaseProof ?? { kind: worker ? 'host-confirmed-worker-exit' : 'unbound-reservation-release', at: lease.releasedAt } } await rt.save() if (confirmed) rt.projectScheduler?.wake() } @@ -139,8 +145,8 @@ export function createProjectResources(rt) { async function reconcile() { let changed = false for (const lease of Object.values(leases()).filter(active)) { - if (lease.processScope?.hostPid === process.pid && (lease.worker?.kind === 'native' || lease.worker?.kind === 'sdk' && rt.projectHarness?.hasController(lease.worker.handle) || lease.worker?.kind === 'native-turn' && rt.projectAgents?.hasExchange(lease.worker.handle))) continue - const priorExited = lease.processScope && await priorScopeExited(lease.processScope) + if (lease.processScope?.hostPid === process.pid && (lease.worker?.kind === 'native' && rt.projectDelivery?.nativeSessions?.currentHost(lease.worker.botId) || lease.worker?.kind === 'sdk' && rt.projectHarness?.hasController(lease.worker.handle) || lease.worker?.kind === 'native-turn' && rt.projectAgents?.hasExchange(lease.worker.handle))) continue + const priorExited = lease.processScope && await priorExit(lease.processScope) // The supervisor read yields: a confirmed concurrent release is final. if (!active(lease)) continue if (priorExited) { @@ -150,14 +156,15 @@ export function createProjectResources(rt) { if (!bot.ephemeral || !bot.attemptId) continue await rt.flybot.scope('flybot-workflow').bot.remove(bot.id) } - lease.releaseProof = { kind: 'systemd-prior-invocation-exited', previousInvocationId: lease.processScope.invocationId, at: Date.now() } - await release(lease.id, lease.generation, { handle: lease.worker?.handle, exited: true }) + if (!active(lease)) { changed = true; continue } + const proof = { kind: 'systemd-prior-invocation-exited', previousInvocationId: lease.processScope.invocationId, at: Date.now() } + await release(lease.id, lease.generation, { handle: lease.worker?.handle, exited: true, proof }) const row = rt.state.projectHarnessWorkers?.[lease.worker?.handle] if (row) { row.status = 'released'; row.releasedAt = Date.now(); row.releaseProof = lease.releaseProof; await rt.projectDelivery?.releaseSession(row.id); changed = true } continue } if (lease.processScope?.hostPid === process.pid) { - if (lease.worker?.kind === 'native' || lease.worker?.kind === 'sdk' && rt.projectHarness?.hasController(lease.worker.handle) || lease.worker?.kind === 'native-turn' && rt.projectAgents?.hasExchange(lease.worker.handle)) continue + if (lease.worker?.kind === 'native' && rt.projectDelivery?.nativeSessions?.currentHost(lease.worker.botId) || lease.worker?.kind === 'sdk' && rt.projectHarness?.hasController(lease.worker.handle) || lease.worker?.kind === 'native-turn' && rt.projectAgents?.hasExchange(lease.worker.handle)) continue } if (!lease.worker) { if (lease.status === 'unreachable') await release(lease.id, lease.generation); continue } if (lease.worker.kind === 'native' && !ownEntry(state.bots ?? {}, lease.worker.botId)) await release(lease.id, lease.generation) diff --git a/core/src/host/project-role-policy.js b/core/src/host/project-role-policy.js index 395c1e0b9..0641f93f1 100644 --- a/core/src/host/project-role-policy.js +++ b/core/src/host/project-role-policy.js @@ -40,16 +40,16 @@ export const ROLE_CONVENTIONS = Object.freeze({ 'Inspection cadence/content belongs to configured scheduled tasks. Do not assume a framework-hardcoded period or an indefinitely running model turn.', ].join('\n'), }) -export function assembleRolePolicy({ role, commit = null, sources = [], requiredFiles = [], profile = 'repository' }) { - if (!Object.hasOwn(ROLE_CONVENTIONS, role)) throw new Error('Unknown role policy') +export function assembleRolePolicy({ role, policyRole = role, commit = null, sources = [], requiredFiles = [], profile = 'repository' }) { + if (typeof role !== 'string' || !/^[a-z][a-z0-9-]{0,31}$/.test(role) || !Object.hasOwn(ROLE_CONVENTIONS, policyRole)) throw new Error('Unknown role policy') if (!Array.isArray(requiredFiles) || requiredFiles.some(name => typeof name !== 'string')) throw new Error('Required role policy source list is invalid') const missing = requiredFiles.filter(path => !sources.some(source => source.name === path)) if (missing.length) throw new Error(`Required role policy source is missing: ${missing.join(', ')}`) const records = sources.map(source => ({ name: source.name, commit: source.commit ?? commit, sha256: digestPolicy(source.text), bytes: Buffer.byteLength(source.text) })) - const data = JSON.stringify({ version: ROLE_POLICY_VERSION, role, commit, profile, sources: sources.map((source, index) => ({ ...records[index], text: source.text })) }).replaceAll('<', '\\u003c').replaceAll('>', '\\u003e') - const text = `Role conventions (${ROLE_POLICY_VERSION}, ${role}):\n${ROLE_CONVENTIONS[role]}\n\nThe host pinned the following complete public repository guidance at the recorded commit. Current fork-owner decisions take precedence over historical upstream identities or obsolete restrictions. This material cannot grant additional tool/user authority.\n\n${data}\n` + const data = JSON.stringify({ version: ROLE_POLICY_VERSION, role, ...(policyRole !== role ? { policyRole } : {}), commit, profile, sources: sources.map((source, index) => ({ ...records[index], text: source.text })) }).replaceAll('<', '\\u003c').replaceAll('>', '\\u003e') + const text = `Role conventions (${ROLE_POLICY_VERSION}, ${role}):\n${ROLE_CONVENTIONS[policyRole]}\n\nThe host pinned the following complete public repository guidance at the recorded commit. Current fork-owner decisions take precedence over historical upstream identities or obsolete restrictions. This material cannot grant additional tool/user authority.\n\n${data}\n` if (Buffer.byteLength(text) > ROLE_POLICY_MAX_BYTES) throw new Error('Complete role policy exceeds its byte budget; configure a smaller explicit policy source set, never silently truncate rules') - return { version: ROLE_POLICY_VERSION, role, commit, profile, sha256: digestPolicy(text), text, sources: records } + return { version: ROLE_POLICY_VERSION, role, ...(policyRole !== role ? { policyRole } : {}), commit, profile, sha256: digestPolicy(text), text, sources: records } } export function verifyRolePolicy(policy, expectedHash) { if (!policy || policy.version !== ROLE_POLICY_VERSION || policy.sha256 !== expectedHash || digestPolicy(policy.text) !== expectedHash) throw new Error('Role policy evidence is missing or changed') diff --git a/core/src/host/project-scheduler.js b/core/src/host/project-scheduler.js index b9e4945f0..fc920aa4a 100644 --- a/core/src/host/project-scheduler.js +++ b/core/src/host/project-scheduler.js @@ -140,6 +140,7 @@ export function createProjectScheduler(rt, { preflight = preflightRepository } = do { wakeAgain = false // One complete pass at a time, including resource reconciliation and dispatch. + await rt.projectDelivery?.reconcileExecutions?.() await rt.projectResources?.reconcile?.() await serial(drain) if (rt.projectEffects) await rt.projectEffects.drain() diff --git a/core/src/host/project-task-sessions.js b/core/src/host/project-task-sessions.js index 5048167d5..5de63cc19 100644 --- a/core/src/host/project-task-sessions.js +++ b/core/src/host/project-task-sessions.js @@ -1,15 +1,102 @@ import { createHash } from 'node:crypto' import { ownEntry, setEntry } from './store.js' const digest = value => createHash('sha256').update(JSON.stringify(value)).digest('hex') +const resumeError = message => Object.assign(new Error(message), { code: 'bot/context-resume' }) export const taskSessionId = (projectId, taskId, role) => `task-session-${digest([projectId, taskId, role]).slice(0, 32)}` +function contextDefinition(session) { + const text = session.context?.text + const start = '\n', end = '\n' + if (typeof text !== 'string' || !text.includes(start) || !text.endsWith(end)) throw resumeError('Original role context is unavailable; cannot infer a new Session') + if (!/^[a-f0-9]{64}$/.test(session.context.sha256 ?? '') || createHash('sha256').update(text).digest('hex') !== session.context.sha256) throw resumeError('Original role context hash is unavailable or changed') + let data + try { data = JSON.parse(text.slice(text.indexOf(start) + start.length, -end.length)) } catch { throw resumeError('Original role context cannot be decoded') } + if (data.version !== 'project-role-context/v1' || data.projectId !== session.projectId || data.role !== session.role || typeof data.persona !== 'string' || !Array.isArray(data.skills)) throw resumeError('Original role context has another scope or contract') + return { role: session.role, instructions: data.persona, resolvedSkills: data.skills, skills: data.skills.map(skill => skill.name), revision: data.revision } +} +export function roleDefinitionKey(session) { + if (session.definitionKey !== undefined) { + if (typeof session.definitionKey !== 'string' || !/^[a-f0-9]{64}$/.test(session.definitionKey)) throw resumeError('Role definition fingerprint is invalid') + return session.definitionKey + } + const definition = session.definition ?? contextDefinition(session) + return digest({ role: session.role, ...(definition.policyRole && definition.policyRole !== session.role ? { policyRole: definition.policyRole } : {}), persona: definition.instructions ?? '', skills: (definition.resolvedSkills ?? definition.skills ?? []).map(skill => typeof skill === 'string' ? skill : { name: skill.name, sha256: skill.sha256 }) }) +} +function workerSession(state, worker) { + const attempt = ownEntry(state.projectAttempts ?? {}, worker.attemptId) + if (!attempt || attempt.projectId !== worker.projectId || attempt.taskId !== worker.taskId) return null + return [attempt.roleSessions?.[worker.role], ...(attempt.roleSessionHistory ?? [])].find(session => session?.id === worker.sessionKey && session.projectId === worker.projectId && session.taskId === worker.taskId && session.role === worker.role) ?? null +} +function latestRoleWorker(state, record) { + return Object.entries(state.projectHarnessWorkers ?? {}).filter(([, worker]) => worker.logicalSessionId === record.id && worker.projectId === record.projectId && worker.taskId === record.taskId && worker.role === record.role) + .sort((a, b) => (a[1].createdAt ?? 0) - (b[1].createdAt ?? 0)).at(-1) +} +export function pinTaskRoleDefinition(state, scope, definition, revision = 0, catalog) { + catalog ??= ownEntry(state.projectAgentDefinitions ?? {}, scope.projectId) + const currentRevision = catalog?.revision ?? revision + revision = currentRevision + const logical = taskSession(state, scope, catalog) + state.projectRoleDefinitionSnapshots ??= {} + const snapshots = state.projectRoleDefinitionSnapshots + let frozen = logical.definitionRef && ownEntry(snapshots, logical.definitionRef) + if (logical.definitionRef && (!frozen || frozen.version !== 'project-role-definition/v1' || frozen.projectId !== scope.projectId || frozen.role !== scope.role || frozen.sha256 !== digest(frozen.definition))) throw resumeError('Pinned Task-role definition is unavailable or changed') + if (!frozen) { + const initial = Object.fromEntries(Object.entries(definition).filter(([key]) => key !== 'memory')) + let selected = initial, origin = 'initial-definition' + const previous = logical.bindings.at(-1) + const observed = latestRoleWorker(state, logical) + const worker = observed?.[1].backendSessionId ? observed[1] : previous && ownEntry(state.projectHarnessWorkers ?? {}, previous.workerId) + if (worker) { + const session = worker && workerSession(state, worker) + if (!session) throw resumeError('Original Task-role Session is unavailable for definition migration') + const old = session.definition ?? contextDefinition(session) + selected = { ...initial, ...Object.fromEntries(Object.entries(old).filter(([key]) => key !== 'revision')), execution: session.execution ?? initial.execution } + revision = session.definitionRevision ?? old.revision ?? revision + origin = 'legacy-latest-role-context' + } + const sha256 = digest(selected), id = `role-definition-${digest([scope.projectId, scope.role, sha256])}` + frozen = ownEntry(snapshots, id) + if (!frozen) { frozen = { version: 'project-role-definition/v1', id, projectId: scope.projectId, role: scope.role, revision, sha256, definition: structuredClone(selected), origin, createdAt: Date.now() }; setEntry(snapshots, id, frozen) } + logical.definitionRef = id + } + const registered = catalog?.roles.find(row => row.role === scope.role) + if (catalog && !registered || registered?.status === 'retired') throw resumeError('Task role was retired; explicit reassignment is required') + const live = registered ?? definition + const current = live.execution ?? {}, pinned = frozen.definition.execution ?? {} + const available = (pinned.resourceIds ?? []).filter(id => (current.resourceIds ?? []).includes(id)) + if ((live.policyRole ?? live.role) !== (frozen.definition.policyRole ?? frozen.definition.role) || (current.harness ?? 'flybot') !== (pinned.harness ?? 'flybot') || pinned.resourceIds?.length && !available.length || frozen.definition.toolPolicy?.shell === true && live.toolPolicy?.shell === false) throw resumeError('Task-role execution permission or resource binding was revoked; explicit recovery is required') + const retained = structuredClone(frozen.definition) + if (retained.execution) retained.execution.resourceIds = available + const observed = latestRoleWorker(state, logical) + const priorWorker = observed?.[1].backendSessionId ? observed[1] : ownEntry(state.projectHarnessWorkers ?? {}, logical.bindings.at(-1)?.workerId) + if (priorWorker && available.length) { + if (!available.includes(priorWorker.resourceId)) throw resumeError('The prior Session resource is no longer authorized') + retained.execution.resourceIds = [priorWorker.resourceId] + } + return { definition: retained, revision: frozen.revision, definitionRef: frozen.id, definitionKey: roleDefinitionKey({ role: scope.role, definition: frozen.definition }), + ...(frozen.sha256 !== digest(Object.fromEntries(Object.entries(definition).filter(([key]) => key !== 'memory'))) ? { change: { pinnedRevision: frozen.revision, currentRevision, retainedOriginalDefinition: true } } : {}) } +} +export function discardUnpreparedTaskDefinitions(state, taskId) { + let changed = false + for (const logical of Object.values(state.projectTaskSessions ?? {})) { + if (logical.taskId !== taskId || !logical.definitionRef || logical.bindings.length || latestRoleWorker(state, logical)) continue + const prepared = Object.values(state.projectAttempts ?? {}).some(attempt => attempt.taskId === taskId && attempt.projectId === logical.projectId && (attempt.roleSessions?.[logical.role] || attempt.roleSessionHistory?.some(session => session.role === logical.role))) + if (prepared) continue + delete logical.definitionRef + changed = true + } + return changed +} + // Logical Sessions outlive processes and Attempts. Backend bindings never cross cwd/profile. -export function taskSession(state, session) { +export function taskSession(state, session, currentCatalog) { const task = ownEntry(state.tasks ?? {}, session.taskId) - if (!task || task.projectId !== session.projectId || !['developer', 'tester', 'reviewer'].includes(session.role)) throw new Error('Task Session scope changed') - state.projectTaskSessions ??= {} const id = taskSessionId(session.projectId, session.taskId, session.role) - let record = ownEntry(state.projectTaskSessions, id) + let record = ownEntry(state.projectTaskSessions ?? {}, id) + const catalog = currentCatalog ?? ownEntry(state.projectAgentDefinitions ?? {}, session.projectId) + const registered = catalog?.roles.find(row => row.role === session.role && row.status !== 'retired') + if (!task || task.projectId !== session.projectId || typeof session.role !== 'string' || !/^[a-z][a-z0-9-]{0,31}$/.test(session.role) || session.role === 'chief' || !record && (catalog ? !registered : !['developer', 'tester', 'reviewer'].includes(session.role))) throw new Error('Task Session scope changed') + state.projectTaskSessions ??= {} if (!record) { record = { id, projectId: session.projectId, taskId: session.taskId, role: session.role, bindings: [], createdAt: Date.now() }; setEntry(state.projectTaskSessions, id, record) } if (record.projectId !== session.projectId || record.taskId !== session.taskId || record.role !== session.role) throw new Error('Task Session identity changed') return record @@ -35,11 +122,32 @@ export function confirmedWorkerExit(state, worker) { lease.worker?.kind === 'sdk' && lease.worker.handle === worker.id && lease.processScope?.kind === 'systemd-control-group' && lease.processScope.killMode === 'control-group' && lease.processScope.invocationId === proof.previousInvocationId && lease.releaseProof?.kind === proof.kind && lease.releaseProof.previousInvocationId === proof.previousInvocationId) } -export function resumableBinding(state, record, key) { - const previous = record.bindings.findLast(row => row.key === key) +export function resumableBinding(state, record, key, { legacyKey, definitionKey } = {}) { + let previous = record.bindings.at(-1) + if (previous && !ownEntry(state.projectHarnessWorkers ?? {}, previous.workerId)) throw resumeError('Latest Task Session binding is unconfirmed') + const latest = latestRoleWorker(state, record) + if (latest && latest[0] !== previous?.workerId) { + const [workerId, worker] = latest + const lease = ownEntry(state.projectResourceLeases ?? {}, worker.leaseId) + const released = lease?.status === 'released' && lease.projectId === record.projectId && lease.generation === worker.generation && lease.resourceId === worker.resourceId + const cold = !worker.backendSessionId && (worker.executionStartedAt === null || worker.executionStartedAt === undefined && worker.releaseProof?.kind === 'sdk-worker-spawn-failed') + if (cold && released && worker.status === 'released' && worker.releaseProof?.generation === worker.generation && worker.releaseProof?.workerId === workerId) { + // A persist-before-dispatch marker (or proven failed spawn) establishes that + // no model context was started. Keep any earlier real context authoritative. + } else { + if (!confirmedWorkerExit(state, worker) || !released) throw resumeError('Latest Task Session exit is unconfirmed') + if (!worker.backendSessionId) throw resumeError('Latest Task-role context has no recoverable backend Session') + previous = { version: worker.bindingVersion, definitionKey: worker.definitionKey, key: worker.bindingKey, sessionId: worker.backendSessionId, workerId, attemptId: worker.attemptId, policySha256: workerSession(state, worker)?.policy?.sha256 ?? null, recoveredFromWorker: true } + } + } if (!previous) return null + if (previous.key !== key) { + const worker = ownEntry(state.projectHarnessWorkers ?? {}, previous.workerId) + const session = worker && workerSession(state, worker) + if (previous.version || previous.key !== legacyKey || !session || roleDefinitionKey(session) !== definitionKey) throw resumeError('Latest Task-role context cannot resume with this execution binding; a fresh Session is forbidden') + } const worker = ownEntry(state.projectHarnessWorkers ?? {}, previous.workerId) const lease = worker && ownEntry(state.projectResourceLeases ?? {}, worker.leaseId) - if (!worker || worker.projectId !== record.projectId || worker.taskId !== record.taskId || worker.role !== record.role || worker.logicalSessionId !== record.id || worker.bindingKey !== key || !confirmedWorkerExit(state, worker) || lease?.status !== 'released' || lease.projectId !== record.projectId || lease.resourceId !== worker.resourceId || lease.generation !== worker.generation || worker.backendSessionId !== previous.sessionId) throw new Error('Previous Task Session exit or binding is unconfirmed') + if (!worker || worker.projectId !== record.projectId || worker.taskId !== record.taskId || worker.role !== record.role || worker.logicalSessionId !== record.id || worker.bindingKey !== previous.key || !confirmedWorkerExit(state, worker) || lease?.status !== 'released' || lease.projectId !== record.projectId || lease.resourceId !== worker.resourceId || lease.generation !== worker.generation || worker.backendSessionId !== previous.sessionId) throw resumeError('Previous Task Session exit or binding is unconfirmed') return previous } diff --git a/core/test/install.test.js b/core/test/install.test.js index 74de6c2e3..d248075d9 100644 --- a/core/test/install.test.js +++ b/core/test/install.test.js @@ -37,6 +37,8 @@ test('apply registers listeners, tools, and effects in a fixed order', (t) => { assert.deepEqual(ctx.calls, [ 'effect ds-bot: state lock', 'effect ds-bot: Project effects', + // Native activation/epoch admission precedes every model step and stream observer. + 'on agent/pre-step', // Project streaming is observed before the session/delegation guards. 'on llm/stream', diff --git a/core/test/project-delivery.test.js b/core/test/project-delivery.test.js index ea094763b..0eff1cf8a 100644 --- a/core/test/project-delivery.test.js +++ b/core/test/project-delivery.test.js @@ -210,7 +210,7 @@ test('managed delivery definitions and role/model snapshots cannot be edited or const attempt = fixture.attempt() assert.equal(attempt.workflowDefinition.nodes[0].model, 'fixture/model') for (const operation of ['graph', 'set-model', 'concurrency', 'reset', 'delete']) await assert.rejects(fixture.rt.projectEndpoint('project-workflow', { projectId: 'p1', workflowId: attempt.workflowId, operation, nodes: [] }, alice), /immutable/) - assert.equal(attempt.workflowVersion, 'project-delivery/v6') + assert.equal(attempt.workflowVersion, 'project-delivery/v7') }) @@ -292,7 +292,9 @@ test('temporary role context freezes memory, repository instructions and session fixture.rt.bindProject({ kind: 'run', id: first.runId, projectId: 'p1', status: 'failed', taskId: fixture.task.id, attemptId: first.id }) await fixture.start('session-second') assert.notEqual(fixture.attempt().roleSessions.developer.id, descriptor.id) - assert.match(fixture.attempt().roleSessions.developer.context.text, /New persona/) + assert.doesNotMatch(fixture.attempt().roleSessions.developer.context.text, /New persona/) + assert.equal(fixture.attempt().roleSessions.developer.definitionRef, descriptor.definitionRef) + assert.equal(fixture.attempt().roleSessions.developer.definitionChange.retainedOriginalDefinition, true) assert.match(fixture.attempt().roleSessions.developer.context.text, /previousAttemptId/) }) @@ -790,7 +792,7 @@ test('failed context admission preserves an existing unsealed developer workspac const path=a.workspaces.developer.path writeFileSync(join(path,'code.mjs'),'export const preserved = true;\n') a.seedSha=a.baseSha - dev.instructions='x'.repeat(49000) // Required persona cannot be summarized away. + a.feedback='x'.repeat(49000) // Required current Task input cannot be summarized away. await assert.rejects(f.rt.projectDelivery.preparePoolRole(task,{id:'second',attemptId:a.id,generation:2},dev),/exceeds/) assert.equal(readFileSync(join(path,'code.mjs'),'utf8'),'export const preserved = true;\n') assert.equal(a.preparationFailures.at(-1).releasedBeforeExternalStart,false) @@ -1350,3 +1352,49 @@ test('Chief non-rework recovery retains a clean interrupted Attempt and its appr assert.equal(git(path,['rev-parse','HEAD']),before.head);assert.equal(git(path,['status','--porcelain']),before.status) assert.equal(Object.keys(f.rt.state.projectAttempts).length,1) }) + +test('approved custom role capabilities preserve logical identities while sealing testing and reviewing real code', async t => { + const f = await setup(t, undefined, { pool: true }) + f.rt.projectPools.connect({ validate: () => ({ ok: true }) }) + const original = f.rt.projectAgents.deliveryDefinitions('p1') + const names = { developer: 'implementer', tester: 'validator', reviewer: 'auditor' } + // Governance approval is tested through real host APIs separately. This fixture + // starts from an approved catalog and exercises the actual artifact controller. + for (const [policyRole, role] of Object.entries(names)) { + const definition = structuredClone(original.roles.find(row => row.role === policyRole)) + delete definition.memory; delete definition.resolvedSkills; delete definition.definitionRevision + f.rt.state.projectAgentDefinitions.p1.roles.push({ ...definition, id: `p1:${role}`, role, policyRole, name: role }) + } + ++f.rt.state.projectAgentDefinitions.p1.revision + await f.start('custom-role-artifacts') + const attempt = f.attempt(), task = f.rt.state.tasks[f.task.id] + const prepare = async (policyRole, generation) => { + const role = names[policyRole], definition = f.rt.projectAgents.deliveryDefinitions('p1').roles.find(row => row.role === role) + await f.rt.projectDelivery.preparePoolRole(task, { id: `custom-${role}`, attemptId: attempt.id, generation }, definition) + return role + } + const dev = await prepare('developer', 1) + const workspace = attempt.workspaces[dev] + assert.equal(workspace.role, 'implementer') + const backup = { ...structuredClone(f.rt.state.projectAgentDefinitions.p1.roles.find(row => row.role === 'implementer')), id: 'p1:backup', role: 'backup' } + f.rt.state.projectAgentDefinitions.p1.roles.push(backup); ++f.rt.state.projectAgentDefinitions.p1.revision + f.rt.config.projectRepositories['example/demo'].requiredRolePolicySources = { backup: ['MISSING.md'] } + await assert.rejects(f.rt.projectDelivery.preparePoolRole(task, { id: 'bad-backup', attemptId: attempt.id, generation: 2 }, f.rt.projectAgents.deliveryDefinitions('p1').roles.find(row => row.role === 'backup')), /Required role policy source is missing/) + assert.equal(attempt.roleAssignments.developer, 'implementer', 'Failed preparation cannot replace the actual artifact producer') + assert.match(attempt.roleSessions[dev].policy.text, /Never invent passing tests/) + writeFileSync(join(workspace.path, 'code.mjs'), 'export const add = (a,b) => a+b;\n') + await f.rt.projectDelivery.completePoolRole({ task, claim: { attemptId: attempt.id }, role: dev, outputs: { implementation: 'Correct addition' } }) + assert.equal(attempt.evidence.implementation.sha, attempt.candidateSha) + const tester = await prepare('tester', 2) + await f.rt.projectDelivery.completePoolRole({ task, claim: { attemptId: attempt.id }, role: tester, outputs: { tests: 'Request actual configured host checks' } }) + assert.equal(attempt.evidence.tests.passed, true) + const reviewer = await prepare('reviewer', 3) + await f.rt.projectDelivery.completePoolRole({ task, claim: { attemptId: attempt.id }, role: reviewer, outputs: { review: JSON.stringify({ commit: attempt.candidateSha, verdict: 'approve', summary: 'Fixture candidate recommendation, not signed forge review' }) } }) + assert.equal(attempt.evidence.review.sha, attempt.candidateSha) + assert.equal(attempt.pr.sha, attempt.candidateSha) + assert.deepEqual(attempt.roleAssignments, names) + assert.equal(attempt.evidence.implementation.logicalRole, dev) + assert.equal(attempt.evidence.tests.logicalRole, tester) + assert.equal(attempt.evidence.review.logicalRole, reviewer) + for (const role of Object.values(names)) assert.equal(attempt.roleSessions[role].logicalSessionId, Object.values(f.rt.state.projectTaskSessions).find(row => row.role === role).id) +}) diff --git a/core/test/project-execution-authority.test.js b/core/test/project-execution-authority.test.js new file mode 100644 index 000000000..f1da6f7d0 --- /dev/null +++ b/core/test/project-execution-authority.test.js @@ -0,0 +1,78 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { pinTaskRoleDefinition } from '../src/host/project-task-sessions.js' +import { requireExecutionAuthority, reconcileExecutionAuthority } from '../src/host/project-execution-authority.js' +function fixture() { + const definition = { id: 'p:developer', role: 'developer', policyRole: 'developer', instructions: 'Retain original persona', toolPolicy: { shell: true }, execution: { harness: 'codex', resourceIds: ['local'] } } + const claim = { id: 'claim', status: 'active', role: 'developer', botId: definition.id, generation: 1, workerId: 'worker' } + const session = { id: 'role-session', projectId: 'p', taskId: 't', attemptId: 'a', role: 'developer', generation: 1, poolClaimId: claim.id, botId: claim.workerId, resourceLease: { id: 'lease', resourceId: 'local' }, status: 'running' } + const task = { id: 't', projectId: 'p', status: 'open', deliveryAttemptId: 'a', collaboration: { initiatedBy: 'human', generation: 1, claim } } + const catalog = { revision: 1, roles: [definition] } + const rt = { state: { tasks: { t: task }, projectAttempts: { a: { roleSessions: { developer: session } } } }, readOnly: () => undefined, projects: { requireActive() {} }, requireProjectAccess() {}, projectDelegations: { async background() {} }, projectAgents: { deliveryDefinitions: () => catalog }, projectResources: { requireBinding() {} } } + Object.assign(session, pinTaskRoleDefinition(rt.state, session, definition, 1, catalog)) + return { rt, definition, claim, session, task, catalog } +} +test('ongoing execution retains persona but refuses live role capability and resource revocation', async () => { + const f = fixture() + f.definition.instructions = 'New persona'; f.definition.model = 'new-model'; f.catalog.revision++ + await requireExecutionAuthority(f.rt, f.session) + assert.equal(f.session.definition.instructions, 'Retain original persona') + f.definition.toolPolicy.shell = false + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /revoked/) + f.definition.toolPolicy.shell = true; f.definition.execution.resourceIds = [] + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /revoked/) +}) +test('retired role may finish its retained active claim but cannot borrow another role identity', async () => { + const f = fixture(); f.catalog.roles = [] + f.rt.state.projectRetiredRoles = { 'p:developer': structuredClone(f.definition) } + await requireExecutionAuthority(f.rt, f.session) + f.rt.state.projectRetiredRoles['p:developer'].id = 'p:foreign' + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /retired/) +}) +test('live authority rechecks membership and claim after asynchronous delegation', async () => { + const f = fixture() + f.rt.projectDelegations.background = async () => { f.claim.status = 'revoked' } + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /revoked during/) + f.claim.status = 'active'; f.rt.projectDelegations.background = async () => {} + f.rt.requireProjectAccess = () => { throw Error('Member was removed') } + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /Member was removed/) +}) +test('current implicit native host cannot survive a newly explicit role resource restriction', async () => { + const f = fixture(); delete f.session.resourceLease + f.definition.execution = { harness: 'flybot', resourceIds: [] } + delete f.session.definitionRef + f.rt.state.projectTaskSessions = {}; f.rt.state.projectRoleDefinitionSnapshots = {} + Object.assign(f.session, pinTaskRoleDefinition(f.rt.state, f.session, f.definition, 1, f.catalog)) + await requireExecutionAuthority(f.rt, f.session) + f.definition.execution.resourceIds = ['local'] + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /Implicit host/) +}) +test('scheduler reconciliation stops only the exact current claim, preserving released sessions and other tasks', async () => { + const f = fixture(), stopped = [] + f.rt.projectPools = { async revokeExecution(...args) { stopped.push(args); f.claim.status = 'revoked' } } + await reconcileExecutionAuthority(f.rt); assert.equal(stopped.length, 0) + f.definition.execution.resourceIds = [] + await reconcileExecutionAuthority(f.rt) + assert.equal(stopped.length, 1); assert.deepEqual(stopped[0].slice(0, 2), ['t', 'claim']) + await reconcileExecutionAuthority(f.rt); assert.equal(stopped.length, 1) + f.claim.status = 'active'; f.session.status = 'released' + await reconcileExecutionAuthority(f.rt); assert.equal(stopped.length, 1) +}) + +test('missing initiator and Task identity replacement fail closed during authority checks', async () => { + const f = fixture(); delete f.task.collaboration.initiatedBy + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /initiating human/) + f.task.collaboration.initiatedBy = 'human' + f.rt.projectDelegations.background = async () => { f.rt.state.tasks.t = structuredClone(f.task) } + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /Task scope changed/) +}) + +test('operator disabled resources and removed Harness capability stop retained execution', async () => { + const f = fixture(), resource = { harnesses: ['codex'], type: 'local' } + f.rt.projectResources.requireBinding = () => resource + await requireExecutionAuthority(f.rt, f.session) + resource.disabled = true + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /resource.*revoked/) + resource.disabled = false; resource.harnesses = ['flybot'] + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /resource.*revoked/) +}) diff --git a/core/test/project-execution-exit.test.js b/core/test/project-execution-exit.test.js new file mode 100644 index 000000000..7bc1a2f96 --- /dev/null +++ b/core/test/project-execution-exit.test.js @@ -0,0 +1,38 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { mkdtemp, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { createRuntime } from '../index.js' +import { fakeContext } from './fake-ctx.js' + +test('the actual pool host adapter cannot confirm an unknown missing worker from Bot absence alone', async t => { + const home = await mkdtemp(join(tmpdir(), 'flybot-exit-adapter-')) + const ctx = fakeContext() + const rt = createRuntime(ctx, { home }) + t.after(async () => { await ctx.dispose(); await rm(home, { recursive: true, force: true }) }) + let adapter + rt.projectPools.connect = value => { adapter = value; return () => {} } + rt.flybot.scope('flybot-workflow').projects.execution.connectPools({ release: async () => {} }, () => ({ ok: true })) + assert.equal(await adapter.release('unknown-native-carrier'), false) +}) + +test('physical exit query accepts only matching persisted facts and never infers an unknown worker', async () => { + const { executionExitConfirmed } = await import('../src/host/project-execution-exit.js') + const rt = { state: {}, botOf: () => undefined } + assert.equal(executionExitConfirmed(rt, 'unknown'), false) + const scope = { kind: 'systemd-control-group', killMode: 'control-group', invocationId: 'a'.repeat(32) } + rt.state.projectResourceLeases = { legacy: { id: 'legacy', status: 'released', worker: { kind: 'native', botId: 'legacy-native' }, processScope: scope, releaseProof: { kind: 'systemd-prior-invocation-exited', previousInvocationId: scope.invocationId } } } + assert.equal(executionExitConfirmed(rt, 'legacy-native'), true) + rt.state.projectResourceLeases.legacy.releaseProof.previousInvocationId = 'b'.repeat(32) + assert.equal(executionExitConfirmed(rt, 'legacy-native'), false) + rt.state.projectHarnessWorkers = { sdk: { id: 'sdk', status: 'released', generation: 3, leaseId: 'sdk-lease', releaseProof: { kind: 'sdk-stop-ack-and-process-exit', generation: 3 } } } + rt.state.projectResourceLeases['sdk-lease'] = { status: 'running' } + assert.equal(executionExitConfirmed(rt, 'sdk'), false) + rt.state.projectResourceLeases['sdk-lease'].status = 'released' + assert.equal(executionExitConfirmed(rt, 'sdk'), true) + rt.state.projectNativeWorkers = { native: { id: 'native', version: 'project-native-worker/v2', status: 'quiescent', generation: 1, sessionId: 'native-session', bindingKey: 'key', releaseProof: { kind: 'native-agent-quiescent', registeredWorkEmpty: true, workerId: 'native', generation: 1, sessionId: 'native-session', bindingKey: 'key' } } } + assert.equal(executionExitConfirmed(rt, 'native'), true) + rt.botOf = id => id === 'native' ? { id } : undefined + assert.equal(executionExitConfirmed(rt, 'native'), false) +}) diff --git a/core/test/project-harness.test.js b/core/test/project-harness.test.js index 6dcda2c61..07fa9617c 100644 --- a/core/test/project-harness.test.js +++ b/core/test/project-harness.test.js @@ -6,7 +6,7 @@ import { join } from 'node:path' import { createProjectEffects } from '../src/host/project-effects.js' import { install as installProjectInputs } from '../src/host/project-inputs.js' import { createProjectResources } from '../src/host/project-resources.js' -import { createProjectHarness, handoffFrame } from '../src/host/project-harness.js' +import { createProjectHarness, handoffFrame, harnessCapabilities } from '../src/host/project-harness.js' async function fixture(t) { const root = await mkdtemp(join(tmpdir(), 'flybot-harness-contract-')) @@ -21,17 +21,17 @@ lines.on('line',line=>{const m=JSON.parse(line); if(m.type==='stop'){write({type:'stopped',released:true});setTimeout(()=>process.exit(0),20);} if(m.type==='start') { write({type:'sdk-event',event:{type:'thread.started',thread_id:'bound-thread'}}); - write({type:'sdk-event',event:{type:'debug',noTools:m.settings.noTools,readOnly:m.settings.readOnly,value:m.settings.env.OPENAI_API_KEY,taskId:'task-1234567890abcdef',unknownCredential:'sk-unitsecret123456789012345'}}); + write({type:'sdk-event',event:{type:'debug',noTools:m.settings.noTools,readOnly:m.settings.readOnly,resumeSessionId:m.settings.sessionId??null,value:m.settings.env.OPENAI_API_KEY,taskId:'task-1234567890abcdef',unknownCredential:'sk-unitsecret123456789012345'}}); write({type:'completed',sessionId:'bound-thread',text:'OUTPUT implementation: actual contract worker output',usage:{input_tokens:1}}); } });`) - const session = { id: 's', projectId: 'p', taskId: 't', attemptId: 'a', role: 'developer', execution: { harness: 'codex' }, context: { text: 'Frozen criteria and role memory' }, resourceLease: { id: 'l', resourceId: 'r', generation: 1 } } + const session = { id: 's', projectId: 'p', taskId: 't', attemptId: 'a', role: 'developer', definition: { role: 'developer', instructions: 'Stable developer convention', skills: [] }, execution: { harness: 'codex' }, context: { text: 'Frozen criteria and role memory' }, resourceLease: { id: 'l', resourceId: 'r', generation: 1 } } const workspace = { id: 'w', projectId: 'p', attemptId: 'a', role: 'developer', path: cwd } const rt = { home: root, state: { tasks: { t: { id: 't', projectId: 'p' } }, projectResourceLeases: { l: { id: 'l', resourceId: 'r', projectId: 'p', attemptId: 'a', generation: 1, status: 'reserved' } } }, - config: { projectResources: { r: { type: 'local', root, capacity: 1, runtimeDir, allowedProjectOwnerIds: ['owner'], profiles: { owner: { codex: { home: join(root, 'profile'), configDir: join(root, 'profile/codex'), model: 'test-model', envRefs: { OPENAI_API_KEY: 'FLYBOT_TEST_HARNESS_SECRET' } } } } } } }, + config: { projectResources: { r: { type: 'local', root, capacity: 1, runtimeDir, harnesses: ['codex'], allowedProjectOwnerIds: ['owner'], profiles: { owner: { codex: { home: join(root, 'profile'), configDir: join(root, 'profile/codex'), model: 'test-model', envRefs: { OPENAI_API_KEY: 'FLYBOT_TEST_HARNESS_SECRET' } } } } } } }, save: async () => {}, warn() {}, readProject: () => ({ id: 'p', ownerUserId: 'owner' }), requireProjectAccess(actor) { if (actor.memberId !== 'owner') throw new Error('Access denied') }, projectDelivery: { ownsWorkspace: (projectId, path) => projectId === 'p' && path === cwd ? workspace : null, sessionFor: () => session, - bindSession: async (_workspace, bot) => rt.projectResources.markStarted('l', 1, { kind: 'sdk', handle: bot.id }), releaseSession: async () => {} } } + bindSession: async (_workspace, bot) => rt.projectResources.markStarted(session.resourceLease.id, session.resourceLease.generation, { kind: 'sdk', handle: bot.id }), releaseSession: async () => {} } } process.env.FLYBOT_TEST_HARNESS_SECRET = 'unit-credential-opaque-value' rt.projectResources = createProjectResources(rt) const harness = createProjectHarness(rt) @@ -178,7 +178,7 @@ test('busy SDK queues feedback until a safe next turn in the same worker, thread const session = rt.projectDelivery.sessionFor() session.poolClaimId = 'claim' const task = rt.state.tasks.t - task.status = 'working' + task.status = 'working'; task.deliveryAttemptId = session.attemptId; session.generation = 2 task.collaboration = { status: 'claimed', generation: 2, botId: 'p:developer', currentHandoffId: 'h', initiatedBy: 'owner', claim: { id: 'claim', generation: 2, botId: 'p:developer', status: 'active' } } rt.projects = { requireActive() {} } rt.projectEffects = createProjectEffects(rt); installProjectInputs(rt) @@ -234,7 +234,7 @@ createInterface({input:process.stdin}).on('line',line=>{const m=JSON.parse(line) test('safe SDK input turns are bounded and excess feedback remains for the next responsible Bot', async t => { const {rt,harness,cwd}=await fixture(t),session=rt.projectDelivery.sessionFor(),task=rt.state.tasks.t - session.poolClaimId='bounded-claim';task.status='working' + session.poolClaimId='bounded-claim';task.status='working';task.deliveryAttemptId=session.attemptId;session.generation=2 task.collaboration={status:'claimed',generation:2,botId:'p:developer',initiatedBy:'owner',claim:{id:session.poolClaimId,generation:2,botId:'p:developer',status:'active'}} rt.projects={requireActive(){}};rt.projectEffects=createProjectEffects(rt);installProjectInputs(rt) rt.save=async()=>{rt.projectEffects.saved(rt.projectEffects.capture())} @@ -278,7 +278,7 @@ write({type:'completed',sessionId:'bounded-thread',text:'OUTPUT handoff: '+JSON. async function taskInputFixture(t, workerSource) { const f=await fixture(t),{rt,harness}=f,session=rt.projectDelivery.sessionFor(),task=rt.state.tasks.t - session.poolClaimId='idle-claim';task.status='working' + session.poolClaimId='idle-claim';task.status='working';task.deliveryAttemptId=session.attemptId;session.generation=2 task.collaboration={status:'claimed',generation:2,botId:'p:developer',initiatedBy:'owner',claim:{id:session.poolClaimId,generation:2,botId:'p:developer',status:'active'}} rt.projects={requireActive(){}};rt.projectEffects=createProjectEffects(rt);installProjectInputs(rt);rt.projectHarness=harness rt.save=async()=>{const proof=rt.projectEffects.capture();await writeFile(join(rt.home,'inbox.json'),JSON.stringify(rt.state));rt.projectEffects.saved(proof)} @@ -349,3 +349,147 @@ test('a result callback failing after confirmed release cannot poison the worker assert.equal(rt.projectResources.occupied('r'),0) assert.equal(row.failure,undefined) }) + + +test('changed caller prompt resumes the Task-role SDK thread when its role definition is unchanged', async t => { + const { rt, harness, cwd } = await fixture(t) + const first = await harness.spawn({ executionProjectId: 'p', executionWorkDir: cwd, prompt: 'Initial Task requirement' }) + const completed = new Promise(resolve => harness.onResult(resolve)) + await harness.dispatch({ botId: first, taskId: 't', brief: 'Initial work', outputs: [{ name: 'implementation' }] }) + await completed + await harness.release(first) + const session = rt.projectDelivery.sessionFor() + session.id = 'returned-role-session' + session.attemptId = 'second-attempt' + rt.state.projectResourceHealth = { r: { status: 'ready' } } + rt.state.projectHarnessHealth = { 'p:r:codex': { status: 'ready' } } + session.resourceLease = await rt.projectResources.reserve({ projectId: 'p', attemptId: session.attemptId, role: 'developer', sessionKey: session.id, harness: 'codex', resourceIds: ['r'] }) + const next = await harness.spawn({ executionProjectId: 'p', executionWorkDir: cwd, prompt: 'Tester returned the Task with new feedback and changed candidate facts' }) + assert.equal(rt.state.projectHarnessWorkers[next].continuedFrom, first) + assert.equal(rt.state.projectHarnessWorkers[next].backendSessionId, 'bound-thread') + const returned = new Promise(resolve => harness.onResult(resolve)) + await harness.dispatch({ botId: next, taskId: 't', brief: 'Continue previous work', outputs: [{ name: 'implementation' }] }) + await returned + assert.match(await readFile(rt.state.projectHarnessWorkers[next].eventLog, 'utf8'), /"resumeSessionId":"bound-thread"/) +}) + +test('a no-op state save cannot authorize SDK input, and the unstarted process remains releasable', async t => { + const { rt, harness, cwd } = await fixture(t) + const id = await harness.spawn({ executionProjectId: 'p', executionWorkDir: cwd, prompt: 'No uncommitted input' }) + rt.savesQueued = () => 0 + await assert.rejects(harness.dispatch({ botId: id, taskId: 't', brief: 'Must not be sent', outputs: [{ name: 'implementation' }] }), /commit a state snapshot/) + assert.equal(rt.state.projectHarnessWorkers[id].backendSessionId, undefined) + assert.equal(rt.state.projectHarnessWorkers[id].executionStartedAt, null) + await harness.release(id) + assert.equal(rt.projectResources.occupied('r'), 0) +}) + + +test('Harness capability inspection distinguishes SDK support, native limitations and unknown adapters', () => { + for (const harness of ['codex', 'claude-agent']) { + const capabilities = harnessCapabilities(harness) + assert.equal(capabilities.taskRoleResume, true) + assert.equal(capabilities.crossHarnessResume, false) + assert.equal(capabilities.input, 'between-sdk-turns') + capabilities.taskRoleResume = false + assert.equal(harnessCapabilities(harness).taskRoleResume, true) + } + assert.equal(harnessCapabilities('flybot').taskRoleResume, true) + assert.equal(harnessCapabilities('flybot').exitConfirmation, 'native-agent-quiescence') + assert.equal(harnessCapabilities('flybot').sdkVersion, null) + assert.throws(() => harnessCapabilities('unknown'), /Unknown/) +}) + +test('SDK sandbox policy follows pinned capability while custom logical role identity remains distinct', async t => { + for (const [role, policyRole, readOnly] of [['implementer', 'developer', false], ['auditor', 'reviewer', true]]) { + await t.test(role, async t => { + const f = await fixture(t), session = f.rt.projectDelivery.sessionFor() + session.role = role; session.policyRole = policyRole + session.definition = { ...session.definition, role, policyRole } + f.rt.state.projectAgentDefinitions = { p: { revision: 1, roles: [session.definition] } } + f.rt.state.projectResourceLeases.l.role = role + f.rt.projectDelivery.ownsWorkspace('p', f.cwd).role = role + let resolve; const result = new Promise(done => { resolve = done }) + f.harness.onResult(resolve) + const id = await f.harness.spawn({ executionProjectId: 'p', executionWorkDir: f.cwd, prompt: 'Execute the pinned capability contract' }) + await f.harness.dispatch({ botId: id, brief: 'Inspect assigned work', inputs: {} }) + await result; await f.harness.release(id) + const row = f.rt.state.projectHarnessWorkers[id] + const events = (await readFile(row.sealedLog, 'utf8')).split('\n').filter(Boolean).map(line => JSON.parse(line)) + const debug = events.find(item => item.event?.type === 'debug') + assert.equal(debug.event.readOnly, readOnly) + assert.equal(row.role, role); assert.equal(row.policyRole, policyRole) + assert.equal(f.rt.state.projectTaskSessions[row.logicalSessionId].role, role) + }) + } +}) + +test('an actual prepared SDK subprocess receives no start after its role resource is revoked', async t => { + const { pinTaskRoleDefinition } = await import('../src/host/project-task-sessions.js') + const { rt, harness, cwd } = await fixture(t), session = rt.projectDelivery.sessionFor() + const task = rt.state.tasks.t + session.poolClaimId = 'revoked-start'; session.generation = 2; task.deliveryAttemptId = session.attemptId + task.collaboration = { initiatedBy: 'owner', generation: 2, claim: { id: session.poolClaimId, generation: 2, status: 'active', botId: 'p:developer', role: 'developer' } } + const role = { id: 'p:developer', ...session.definition, execution: { harness: 'codex', resourceIds: ['r'] } } + const catalog = { revision: 1, roles: [role] } + rt.projectAgents = { deliveryDefinitions: () => catalog } + Object.assign(session, pinTaskRoleDefinition(rt.state, session, role, 1, catalog)) + const id = await harness.spawn({ executionProjectId: 'p', executionWorkDir: cwd, prompt: 'Do not execute after revocation' }) + task.collaboration.claim.workerId = id + role.execution.resourceIds = [] + await assert.rejects(harness.dispatch({ botId: id, taskId: task.id, brief: 'Current Task', outputs: [{ name: 'handoff' }] }), /revoked/) + const row = rt.state.projectHarnessWorkers[id] + assert.equal(row.executionStartedAt, null) + assert.equal(row.status, 'prepared') + await harness.release(id) + assert.equal(row.releaseProof.kind, 'sdk-stop-ack-and-process-exit') + assert.equal(rt.state.projectResourceLeases.l.status, 'released') +}) + +test('existing pool revocation stops an actual live SDK worker before resource release after role rights change', async t => { + for (const revocation of ['role-resource', 'operator-disable']) await t.test(revocation, async t => { + const { pinTaskRoleDefinition } = await import('../src/host/project-task-sessions.js') + const { reconcileExecutionAuthority } = await import('../src/host/project-execution-authority.js') + const { createProjectPools } = await import('../src/host/project-pools.js') + const { rt, harness, cwd } = await fixture(t), session = rt.projectDelivery.sessionFor(), task = rt.state.tasks.t + session.poolClaimId = 'live-revoke'; session.generation = 2; task.deliveryAttemptId = session.attemptId + task.collaboration = { status: 'claimed', initiatedBy: 'owner', generation: 2, claim: { id: session.poolClaimId, generation: 2, status: 'active', botId: 'p:developer', role: 'developer' } } + const role = { id: 'p:developer', ...session.definition, execution: { harness: 'codex', resourceIds: ['r'] } } + const catalog = { revision: 1, roles: [role] } + rt.projectAgents = { deliveryDefinitions: () => catalog } + rt.state.projectAttempts = { [session.attemptId]: { roleSessions: { developer: session } } } + Object.assign(session, pinTaskRoleDefinition(rt.state, session, role, 1, catalog)) + rt.projectPools = createProjectPools(rt) + rt.projectPools.connect({ release: id => harness.release(id) }) + const id = await harness.spawn({ executionProjectId: 'p', executionWorkDir: cwd, prompt: 'Current Task' }) + session.botId = id; task.collaboration.claim.workerId = id + await harness.dispatch({ botId: id, taskId: task.id, brief: 'Current Task', outputs: [{ name: 'handoff' }] }) + for (let n = 0; n < 200 && rt.state.projectHarnessWorkers[id].status !== 'waiting-input'; n++) await new Promise(resolve => setTimeout(resolve, 5)) + assert.equal(rt.state.projectHarnessWorkers[id].status, 'waiting-input') + if (revocation === 'role-resource') role.execution.resourceIds = [] + else rt.config.projectResources.r.disabled = true + await reconcileExecutionAuthority(rt) + assert.equal(task.collaboration.status, 'blocked') + assert.equal(task.collaboration.claim.status, 'revoked') + assert.equal(task.collaboration.claim.exitConfirmed, true) + assert.equal(rt.state.projectHarnessWorkers[id].releaseProof.kind, 'sdk-stop-ack-and-process-exit') + assert.equal(rt.state.projectResourceLeases.l.status, 'released') + assert.equal(harness.hasController(id), false) + }) +}) + +test('SDK human input rechecks the sending actor after awaited initiator authorization', async t => { + const { rt, harness, task, id } = await taskInputFixture(t, ` +import {createInterface} from 'node:readline';const write=v=>process.stdout.write(JSON.stringify(v)+'\\n');write({type:'ready',sdk:{codex:'0.166.0'}}); +createInterface({input:process.stdin}).on('line',line=>{const m=JSON.parse(line);if(m.type==='stop'){write({type:'stopped',released:true});setTimeout(()=>process.exit(0),20);return;} +if(m.type==='start')write({type:'completed',sessionId:'input-actor-thread',text:'Need clarification'}); +});`) + await harness.dispatch({ botId: id, taskId: task.id, brief: 'Current Task', outputs: [{ name: 'handoff' }] }) + for (let n = 0; n < 200 && rt.state.projectHarnessWorkers[id].status !== 'waiting-input'; n++) await new Promise(resolve => setTimeout(resolve, 5)) + assert.equal(rt.state.projectHarnessWorkers[id].status, 'waiting-input') + let senderAllowed = true + rt.requireProjectAccess = actor => { if (actor.memberId !== 'owner' && !senderAllowed) throw Error('Sending member was removed') } + rt.projectDelegations = { async background() { senderAllowed = false } } + await assert.rejects(harness.input({ workerId: id, generation: rt.state.projectHarnessWorkers[id].generation, text: 'Late unauthorized input' }, { memberId: 'sender' }), /Sending member was removed/) + assert.equal(rt.state.projectHarnessWorkers[id].status, 'waiting-input') +}) diff --git a/core/test/project-native-exit.test.js b/core/test/project-native-exit.test.js new file mode 100644 index 000000000..04ac66ca2 --- /dev/null +++ b/core/test/project-native-exit.test.js @@ -0,0 +1,24 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { createProjectResources } from '../src/host/project-resources.js' +async function fixture(botExists) { + const rt = { state: { bots: botExists ? { worker: { id: 'worker' } } : {}, projectResourceHealth: { local: { status: 'ready' } } }, + config: { projectResources: { local: { type: 'local', root: '/tmp/native-exit-test', capacity: 1, allowedProjectOwnerIds: ['owner'], harnesses: ['flybot'] } } }, + readProject: () => ({ id: 'p', ownerUserId: 'owner' }), save: async () => {} } + const service = createProjectResources(rt) + const lease = await service.reserve({ projectId: 'p', attemptId: 'a', role: 'developer', sessionKey: 's', harness: 'flybot', resourceIds: ['local'] }) + await service.markStarted(lease.id, lease.generation, { kind: 'native', botId: 'worker' }) + return { rt, service, lease } +} +test('a missing native Bot keeps its resource held until actual quiescence proof', async () => { + const f = await fixture(false) + await f.service.release(f.lease.id, f.lease.generation) + assert.equal(f.service.occupied('local'), 1) + assert.equal(f.rt.state.projectResourceLeases[f.lease.id].status, 'stopping') +}) +test('an undefined handle plus exited flag cannot manufacture native exit acknowledgement', async () => { + const f = await fixture(true) + await f.service.release(f.lease.id, f.lease.generation, { exited: true }) + assert.equal(f.service.occupied('local'), 1) + assert.equal(f.rt.state.projectResourceLeases[f.lease.id].releaseProof, undefined) +}) diff --git a/core/test/project-native-sessions.test.js b/core/test/project-native-sessions.test.js new file mode 100644 index 000000000..12cf5463a --- /dev/null +++ b/core/test/project-native-sessions.test.js @@ -0,0 +1,299 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { createProjectEffects } from '../src/host/project-effects.js' +import { pinTaskRoleDefinition } from '../src/host/project-task-sessions.js' +import { createProjectNativeSessions, nativeExitConfirmed } from '../src/host/project-native-sessions.js' +function fixture() { + let cancelled = 0, archived = [], unarchived = [], hold, failure = false + const session = { id: 'role-1', projectId: 'p', taskId: 't', attemptId: 'a', role: 'developer', generation: 1, definition: { role: 'developer', instructions: 'Original convention' }, status: 'prepared' } + const workspace = { id: 'workspace-1', path: '/registered/task', attemptId: 'a' } + const agent = { session: { id: 'physical-1' }, status: 'idle', cancel() { cancelled++ }, async whenIdle() { if (hold) await hold; if (failure) throw Error('native drain failed') } } + const rt = { state: { tasks: { t: { projectId: 'p' } }, bots: { bot1: { id: 'bot1', sessionId: 'physical-1', workspaceId: workspace.id, attemptId: 'a' } } }, + ctx: { async parallel() {}, async waterfall(_event, _payload, next) { return next() }, sessions: { get: () => undefined }, agents: { get: id => id === 'physical-1' ? agent : undefined }, workspaceRegistry: { async archiveSession(id) { archived.push(id) }, async unarchiveSession(id) { unarchived.push(id) } } }, + botOf(id) { return rt.state.bots[id] }, chatOf(id) { return rt.state.bots[id]?.sessionId ?? id }, indexSessions() {}, resolveAgent: async () => agent, async save() { const captured = rt.projectEffects.capture(); rt.projectEffects.saved(captured) } } + rt.projectEffects = createProjectEffects(rt) + return { rt, session, workspace, agent, service: createProjectNativeSessions(rt), cancelled: () => cancelled, archived: () => archived, unarchived: () => unarchived, hold(value) { hold = value }, fail(value) { failure = value } } +} +test('native deletion or cancellation acknowledgement alone does not prove quiescence', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1') + let release; f.hold(new Promise(resolve => { release = resolve })) + const stopping = f.service.stop('bot1') + await new Promise(resolve => setImmediate(resolve)) + assert.equal(f.cancelled(), 1); assert.equal(f.service.admits('bot1'), false) + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), false) + release(); await stopping + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), true) + assert.equal((await f.service.stop('bot1')).workerId, 'bot1'); assert.equal(f.cancelled(), 1) +}) +test('a fresh native carrier resumes only the exact latest released Task-role physical Session', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.stop('bot1') + delete f.rt.state.bots.bot1 + f.rt.state.bots.bot2 = { id: 'bot2', workspaceId: 'workspace-2', attemptId: 'a2' } + const session = { ...f.session, id: 'role-2', attemptId: 'a2', status: 'prepared' }, workspace = { ...f.workspace, id: 'workspace-2', attemptId: 'a2' } + const result = await f.service.bind(session, workspace, 'bot2') + await f.service.activate(session, 'bot2') + assert.equal(result.sessionId, 'physical-1'); assert.deepEqual(f.archived(), ['bot2']) + assert.deepEqual(f.unarchived(), ['physical-1'], 'The original archived Session must be admitted by DSH only after exact binding') + assert.equal(f.rt.state.retiredSessions.bot2, 'bot2', 'Unused carrier Session stays managed and cannot become an unowned plain Session') + assert.equal(f.rt.state.bots.bot1, undefined) + assert.equal(f.rt.state.projectTaskSessions[Object.keys(f.rt.state.projectTaskSessions)[0]].bindings.length, 2) +}) +test('native incompatible cwd, definition, unknown exit and foreign bindings refuse silent fresh sessions', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1') + assert.throws(() => f.service.previous(f.session, f.workspace), /forbidden/) + await f.service.stop('bot1'); delete f.rt.state.bots.bot1 + for (const [session, workspace] of [[f.session, { ...f.workspace, path: '/other' }], [{ ...f.session, definition: { role: 'developer', instructions: 'Changed convention' } }, f.workspace], [{ ...f.session, role: 'reviewer' }, f.workspace]]) { + if (session.role === 'reviewer') assert.equal(f.service.previous(session, workspace).row, null) + else assert.throws(() => f.service.previous(session, workspace), /forbidden/) + } + f.rt.state.projectNativeWorkers.bot1.releaseProof.generation++ + assert.throws(() => f.service.previous(f.session, f.workspace), /forbidden/) +}) +test('failed native drain keeps execution held and denies later input', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1'); f.fail(true) + await assert.rejects(f.service.stop('bot1'), /drain failed/) + assert.equal(f.rt.state.projectNativeWorkers.bot1.status, 'stop-unconfirmed') + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), false) + assert.equal(f.service.admits('bot1'), false) +}) + + +test('native restoration preserves current context parts and refuses a changed execution model', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1') + f.rt.state.bots.bot1.sessionId = 'physical-part-2' + f.rt.state.bots.bot1.segments = [{ sessionId: 'physical-1', endedAt: 1 }] + f.agent.session.id = 'physical-part-2' + await f.service.stop('bot1'); delete f.rt.state.bots.bot1 + f.rt.state.bots.bot2 = { id: 'bot2', workspaceId: 'workspace-2', attemptId: 'a2', model: 'other-model' } + const session = { ...f.session, id: 'role-2', attemptId: 'a2', status: 'prepared' }, workspace = { ...f.workspace, id: 'workspace-2', attemptId: 'a2' } + await assert.rejects(f.service.bind(session, workspace, 'bot2'), /model changed/) + delete f.rt.state.bots.bot2.model + const result = await f.service.bind(session, workspace, 'bot2') + await f.service.activate(session, 'bot2') + assert.equal(result.sessionId, 'physical-part-2') + assert.deepEqual(f.rt.state.bots.bot2.segments, [{ sessionId: 'physical-1', endedAt: 1 }]) +}) +test('unknown prior host and legacy native context remain explicitly fenced without reopening a Session', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1') + const restarted = createProjectNativeSessions(f.rt) + await assert.rejects(restarted.stop('bot1'), /earlier host/) + assert.equal(restarted.admits('bot1'), false); assert.equal(f.cancelled(), 0) + const g = fixture() + g.rt.state.projectAttempts = { old: { projectId: 'p', taskId: 't', roleSessions: { developer: { id: 'legacy-role', role: 'developer', sessionId: 'legacy-session', execution: { harness: 'flybot' } } } } } + assert.throws(() => g.service.previous(g.session, g.workspace), /Legacy native/) +}) +test('current pool claim authority is required before any native carrier binding', async () => { + const f = fixture() + f.session.poolClaimId = 'claim'; f.rt.state.tasks.t.deliveryAttemptId = 'a' + f.rt.state.tasks.t.collaboration = { initiatedBy: 'owner', generation: 1, claim: { id: 'other', generation: 1, status: 'active' } } + f.rt.requireProjectAccess = () => {} + await assert.rejects(f.service.bind(f.session, f.workspace, 'bot1'), /claim changed/) + assert.deepEqual(f.rt.state.projectNativeWorkers, undefined) +}) +test('overlapping native binds cannot attach two carriers to the same restored physical Session', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.stop('bot1'); delete f.rt.state.bots.bot1 + let release; const gate = new Promise(resolve => { release = resolve }) + f.rt.resolveAgent = async () => { await gate; return f.agent } + f.rt.state.bots.bot2 = { id: 'bot2', workspaceId: 'workspace-2', attemptId: 'a2' } + f.rt.state.bots.bot3 = { id: 'bot3', workspaceId: 'workspace-2', attemptId: 'a2' } + const session = { ...f.session, id: 'role-2', attemptId: 'a2', status: 'prepared' }, workspace = { ...f.workspace, id: 'workspace-2', attemptId: 'a2' } + const first = f.service.bind(session, workspace, 'bot2') + await new Promise(resolve => setImmediate(resolve)) + await assert.rejects(f.service.bind({ ...session }, workspace, 'bot3'), /already in progress/) + release(); await first + const logical = Object.values(f.rt.state.projectTaskSessions)[0] + assert.equal(logical.bindings.length, 2); assert.equal(logical.bindings.at(-1).workerId, 'bot2') +}) + + +test('native bind rechecks pinned current permissions and rejects a claim revoked during authorization', async () => { + const f = fixture() + f.rt.state.projectAgentDefinitions = { p: { revision: 1, roles: [f.session.definition] } } + const pinned = pinTaskRoleDefinition(f.rt.state, f.session, f.session.definition, 1) + f.session.definitionRef = pinned.definitionRef + f.session.poolClaimId = 'claim'; f.rt.state.tasks.t.deliveryAttemptId = 'a' + f.rt.state.tasks.t.collaboration = { initiatedBy: 'owner', generation: 1, claim: { id: 'claim', generation: 1, status: 'active' } } + let checks = 0; f.rt.requireProjectAccess = () => { checks++ } + f.rt.projectDelegations = { background: async () => {} } + await f.service.bind(f.session, f.workspace, 'bot1') + assert.equal(checks, 2) + const g = fixture() + g.session.poolClaimId = 'claim'; g.rt.state.tasks.t.deliveryAttemptId = 'a' + g.rt.state.tasks.t.collaboration = { initiatedBy: 'owner', generation: 1, claim: { id: 'claim', generation: 1, status: 'active' } } + g.rt.requireProjectAccess = () => {} + g.rt.projectDelegations = { background: async () => { g.rt.state.tasks.t.collaboration.claim.status = 'revoked' } } + await assert.rejects(g.service.bind(g.session, g.workspace, 'bot1'), /revoked during/) + assert.equal(g.rt.state.projectNativeWorkers, undefined) + const h = fixture(); h.session.definitionRef = 'missing-definition' + await assert.rejects(h.service.bind(h.session, h.workspace, 'bot1'), /definition is unavailable/) +}) + + +test('a noncommitting native binding cannot lift archive admission or authorize new model input', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.stop('bot1'); delete f.rt.state.bots.bot1 + f.rt.state.bots.bot2 = { id: 'bot2', workspaceId: 'workspace-2', attemptId: 'a2' } + const session = { ...f.session, id: 'role-2', attemptId: 'a2', status: 'prepared' }, workspace = { ...f.workspace, id: 'workspace-2', attemptId: 'a2' } + f.rt.save = async () => {} // Resolving is not evidence that the binding reached disk. + await f.service.bind(session, workspace, 'bot2') + assert.deepEqual(f.unarchived(), [], 'Uncommitted restoration must leave the original DSH archive gate intact') + assert.equal(f.service.admits('bot2'), false) + await assert.rejects(f.service.activate(session, 'bot2'), /commit/) + assert.deepEqual(f.unarchived(), []) +}) + + +test('native activation needs committed binding and current writable epoch before input admission', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1') + assert.equal(f.service.admits('bot1'), false) + await f.service.activate(f.session, 'bot1') + assert.equal(f.service.admits('bot1'), true) + const restarted = createProjectNativeSessions(f.rt) + assert.equal(restarted.admits('bot1'), false, 'A previous host epoch cannot silently restart native input') + f.rt.readOnly = () => 'Another writer holds the home' + assert.equal(f.service.admits('bot1'), false) + await assert.rejects(f.service.activate(f.session, 'bot1'), /writable host/) + assert.equal(f.service.admits('unrelated-personal-chief'), true) +}) + +function supervisedLease(f) { + const lease = { id: 'native-lease', projectId: 'p', taskId: 't', attemptId: 'a', role: 'developer', sessionKey: f.session.id, resourceId: 'native-resource', generation: 7, status: 'running', + worker: { kind: 'native', botId: 'bot1' }, processScope: { kind: 'systemd-control-group', killMode: 'control-group', invocationId: 'a'.repeat(32), bootId: 'same-boot', unit: 'flybot.service', controlGroup: '/system.slice/flybot.service', hostPid: 12345 } } + f.session.resourceLease = { ...lease } + f.rt.state.projectResourceLeases = { [lease.id]: lease } + f.rt.state.projectAttempts = { a: { id: 'a', projectId: 'p', taskId: 't', roleSessions: { developer: f.session } } } + return lease +} +test('supervisor-confirmed old host exit preserves the latest native context without resuming a stale worker', async () => { + const f = fixture(), lease = supervisedLease(f) + await f.service.bind(f.session, f.workspace, 'bot1') + f.rt.state.bots.bot1.sessionId = 'latest-native-part'; f.rt.state.bots.bot1.segments = [{ sessionId: 'physical-1', endedAt: 1 }] + f.rt.resolveAgent = async () => { throw Error('Must not resume old activity to manufacture exit proof') } + let checks = 0 + const restarted = createProjectNativeSessions(f.rt, { priorExit: async scope => { checks++; return scope.invocationId === 'a'.repeat(32) } }) + const proof = await restarted.stop('bot1') + assert.equal(checks, 1); assert.equal(f.cancelled(), 0) + assert.equal(proof.kind, 'systemd-prior-invocation-exited'); assert.equal(proof.previousInvocationId, lease.processScope.invocationId) + assert.equal(proof.sessionId, 'latest-native-part'); assert.equal(f.session.sessionId, 'latest-native-part') + const row = f.rt.state.projectNativeWorkers.bot1 + assert.equal(nativeExitConfirmed(f.rt.state, row), true) + lease.status = 'released'; lease.releaseProof = proof; delete f.rt.state.bots.bot1 + assert.equal(restarted.previous({ ...f.session, id: 'next-role' }, f.workspace).row, row) +}) +test('native old-host recovery rejects missing process-group proof and scope changes during inspection', async () => { + for (const variant of ['unconfirmed', 'unmanaged', 'changed']) { + const f = fixture(), lease = supervisedLease(f) + await f.service.bind(f.session, f.workspace, 'bot1') + if (variant === 'unmanaged') lease.processScope.kind = 'unmanaged-process' + const restarted = createProjectNativeSessions(f.rt, { priorExit: async () => { + if (variant === 'changed') lease.generation++ + return variant !== 'unconfirmed' + } }) + await assert.rejects(restarted.stop('bot1'), /host|scope|binding|generation/) + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), false) + assert.equal(lease.status, 'running') + } +}) + +test('native exit remains unconfirmed while registered jobs or subagents are still stopping', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1') + let requested = 0 + f.rt.ctx.parallel = async (event, body) => { assert.equal(event, 'workspace/session-stop'); assert.equal(body.sessionId, 'physical-1'); requested++ } + f.rt.ctx.waterfall = async () => [{ kind: 'job', items: [{ id: 'still-stopping-job' }] }, { kind: 'subagent', items: [{ id: 'child' }] }] + await assert.rejects(f.service.stop('bot1'), /registered session work/) + const row = f.rt.state.projectNativeWorkers.bot1 + assert.equal(nativeExitConfirmed(f.rt.state, row), false) + assert.deepEqual(row.pendingWork, [{ kind: 'job', count: 1 }, { kind: 'subagent', count: 1 }]) + f.rt.ctx.waterfall = async () => [] + await f.service.stop('bot1') + assert.equal(requested, 2); assert.equal(nativeExitConfirmed(f.rt.state, row), true) + assert.equal(row.releaseProof.registeredWorkEmpty, true) +}) +test('native admission fences subagent lineage and missing legacy execution bindings while preserving ordinary Bots', async () => { + const f = fixture() + f.rt.selfOf = id => id === 'physical-1' ? 'bot1' : id === 'legacy-session' ? 'legacy' : id === 'personal-session' ? 'personal' : undefined + f.rt.state.bots.legacy = { id: 'legacy', ephemeral: { plugin: 'flybot-workflow' }, workspaceId: 'w', attemptId: 'old' } + f.rt.state.bots.personal = { id: 'personal' } + await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.activate(f.session, 'bot1') + const child = { session: { id: 'child', header: { id: 'child', origin: 'subagent', parentSession: 'physical-1' } } } + assert.equal(f.service.admitsAgent(child), true) + await f.service.stop('bot1') + assert.equal(f.service.admitsAgent(child), false) + assert.equal(f.service.admitsAgent({ session: { id: 'legacy-session' } }), false) + assert.equal(f.service.admitsAgent({ session: { id: 'personal-session' } }), true) +}) +test('native host scope is captured without a resource and supports only actual supervisor exit recovery', async () => { + const f = fixture() + const scope = { kind: 'systemd-control-group', killMode: 'control-group', invocationId: 'b'.repeat(32), unit: 'flybot.service', controlGroup: '/system.slice/flybot.service', bootId: 'same-boot' } + const original = createProjectNativeSessions(f.rt, { captureScope: async () => scope }) + await original.bind(f.session, f.workspace, 'bot1') + const restarted = createProjectNativeSessions(f.rt, { priorExit: async value => value === scope }) + const proof = await restarted.stop('bot1') + assert.equal(proof.previousInvocationId, scope.invocationId) + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), true) +}) + + +test('native drain includes idle subagent jobs and earlier context parts before proving exit', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.activate(f.session, 'bot1') + f.rt.selfOf = id => id === 'physical-1' ? 'bot1' : undefined + const child = { session: { id: 'child-session', header: { id: 'child-session', origin: 'subagent', parentSession: 'physical-1' } } } + assert.equal(f.service.admitStep(child), true) + f.rt.state.bots.bot1.segments = [{ sessionId: 'earlier-part' }] + const stopped = [] + f.rt.ctx.parallel = async (_event, body) => { stopped.push(body.sessionId) } + f.rt.ctx.waterfall = async (_event, body) => body.sessionId === 'child-session' ? [{ kind: 'job', items: [{ id: 'idle-child-job' }] }] : [] + await assert.rejects(f.service.stop('bot1'), /registered session work/) + assert.ok(stopped.includes('child-session')); assert.ok(stopped.includes('earlier-part')) + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), false) + f.rt.ctx.waterfall = async () => [] + await f.service.stop('bot1') + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), true) +}) + +test('actual native pre-step refuses revoked role rights while retaining the original role context', async () => { + const f = fixture(), revoked = [] + const role = { id: 'p:developer', ...structuredClone(f.session.definition), toolPolicy: { shell: true }, execution: { harness: 'flybot', resourceIds: [] } } + const task = f.rt.state.tasks.t + task.deliveryAttemptId = 'a'; task.collaboration = { initiatedBy: 'owner', generation: 1, claim: { id: 'claim', botId: role.id, role: 'developer', generation: 1, status: 'active' } } + f.session.poolClaimId = 'claim' + f.rt.requireProjectAccess = () => {} + f.rt.projectAgents = { deliveryDefinitions: () => ({ revision: 1, roles: [role] }) } + f.rt.projectPools = { async revokeExecution(...args) { revoked.push(args) } } + f.rt.state.projectAttempts = { a: { roleSessions: { developer: f.session } } } + f.rt.selfOf = id => id === f.agent.session.id ? 'bot1' : undefined + Object.assign(f.session, pinTaskRoleDefinition(f.rt.state, f.session, role, 1, f.rt.projectAgents.deliveryDefinitions())) + await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.activate(f.session, 'bot1') + const check = agent => f.service.validateStep ? f.service.validateStep(agent) : f.service.admitStep(agent) + assert.equal(await check(f.agent), true) + role.toolPolicy.shell = false + assert.equal(await check(f.agent), false) + assert.equal(revoked.length, 1) + assert.deepEqual(revoked[0].slice(0, 2), ['t', 'claim']) + assert.match(revoked[0][2], /revoked/) + assert.equal(f.session.definition.instructions, 'Original convention') +}) + +test('native ongoing authority follows the live carrier when a resumed physical Session also appears in history', async () => { + const f = fixture() + f.rt.selfOf = id => id === f.agent.session.id ? (f.rt.state.bots.bot2 ? 'bot2' : 'bot1') : undefined + const role = { id: 'p:developer', ...structuredClone(f.session.definition), execution: { harness: 'flybot', resourceIds: [] } } + f.rt.projectAgents = { deliveryDefinitions: () => ({ revision: 1, roles: [role] }) } + Object.assign(f.session, pinTaskRoleDefinition(f.rt.state, f.session, role, 1, f.rt.projectAgents.deliveryDefinitions())) + f.session.poolClaimId = 'old-claim' + f.rt.state.tasks.t.deliveryAttemptId = 'a'; f.rt.state.tasks.t.collaboration = { initiatedBy: 'owner', generation: 1, claim: { id: 'old-claim', botId: role.id, role: 'developer', generation: 1, status: 'active' } } + f.rt.requireProjectAccess = () => {} + f.rt.state.projectAttempts = { a: { roleSessions: { developer: f.session } } } + await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.activate(f.session, 'bot1') + assert.equal(await f.service.validateStep(f.agent), true) + await f.service.stop('bot1'); delete f.rt.state.bots.bot1 + const session = { ...f.session, id: 'role-2', attemptId: 'a2', generation: 2, poolClaimId: 'new-claim', status: 'prepared' } + const task = f.rt.state.tasks.t + task.deliveryAttemptId = 'a2'; task.collaboration = { initiatedBy: 'owner', generation: 2, claim: { id: session.poolClaimId, botId: role.id, role: 'developer', generation: 2, status: 'active' } } + f.rt.requireProjectAccess = () => {} + f.rt.state.projectAttempts.a2 = { roleSessions: { developer: session } } + f.rt.state.bots.bot2 = { id: 'bot2', workspaceId: 'w2', attemptId: 'a2' } + await f.service.bind(session, { ...f.workspace, id: 'w2', attemptId: 'a2' }, 'bot2') + await f.service.activate(session, 'bot2') + assert.equal(f.rt.state.projectNativeWorkers.bot1.sessionId, f.rt.state.projectNativeWorkers.bot2.sessionId) + assert.equal(await f.service.validateStep(f.agent), true) +}) diff --git a/core/test/project-pools.test.js b/core/test/project-pools.test.js index 52ab2109e..3cd70ec3b 100644 --- a/core/test/project-pools.test.js +++ b/core/test/project-pools.test.js @@ -222,3 +222,41 @@ test('Chief intervention does not reset global usage or bypass quota', async () await assert.rejects(f.pool.chiefRoute({ taskId: 'task', targetBotId: 'p:developer', expectedGeneration: f.task.collaboration.generation, reason: 'No quota remains', decisionId: 'quota-chief' }, { memberId: 'owner' }), /quota/) assert.deepEqual(f.task, before) }) + + +test('a retired source can finish its frozen active claim but cannot receive new offers', async () => { + const f = setup(); await f.admit(); const claim = await f.claim() + const roles = f.rt.projectAgents.deliveryDefinitions().roles + const index = roles.findIndex(row => row.role === 'developer'), retired = roles.splice(index, 1)[0] + f.rt.state.projectRetiredRoles = { 'p:developer': { ...retired, projectId: 'p', retiredAt: 1 } } + await f.pool.decide('task', claim, { action: 'handoff', to: 'p:tester', reason: 'Complete original responsibility and request actual testing' }) + assert.equal(f.completed[0].role, 'developer') + assert.equal(f.task.collaboration.botId, 'p:tester') + const tester = await f.pool.claim('task', 'p:tester') + await assert.rejects(f.pool.decide('task', tester, { action: 'return', to: 'p:developer', reason: 'Cannot hand off to an inactive role' }), /Target/) +}) +test('queued work for a retired role becomes visibly blocked without reassignment or a new worker', async () => { + const f = setup(); await f.admit() + const roles = f.rt.projectAgents.deliveryDefinitions().roles + roles.splice(roles.findIndex(row => row.role === 'developer'), 1) + assert.equal(await f.claim(), null) + assert.equal(f.task.collaboration.status, 'blocked') + assert.equal(f.task.collaboration.botId, 'p:developer') + assert.match(f.task.collaboration.block.reason, /Chief must explicitly/) + assert.equal(f.task.collaboration.claim, undefined) +}) + +test('host execution revocation fences the exact claim before stopping and never infers exit or reassigns', async () => { + const f = setup(); await f.admit(); const token = await f.claim() + f.task.collaboration.claim.workerId = 'worker' + const events = [] + f.rt.save = async () => { events.push('saved') } + f.adapter.release = async workerId => { events.push('stop'); assert.equal(workerId, 'worker'); assert.equal(f.task.collaboration.claim.status, 'revoked'); assert.equal(events[0], 'saved'); return false } + assert.equal(await f.pool.revokeExecution('task', 'stale-claim', 'Wrong worker'), false) + assert.equal(await f.pool.revokeExecution('task', token.id, 'Role resource permission revoked'), true) + assert.equal(f.task.collaboration.status, 'blocked') + assert.equal(f.task.collaboration.claim.exitConfirmed, false) + assert.equal(f.task.collaboration.history.length, 1) + assert.equal(await f.pool.revokeExecution('task', token.id, 'Repeated observation'), false) + await assert.rejects(f.pool.decide('task', token, { action: 'handoff', to: 'p:tester', reason: 'Stale callback' }), /stale/) +}) diff --git a/core/test/project-resources.test.js b/core/test/project-resources.test.js index b078d6a4f..8fe3e03ec 100644 --- a/core/test/project-resources.test.js +++ b/core/test/project-resources.test.js @@ -4,6 +4,7 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { once } from 'node:events' import { test } from 'node:test' +import { createProjectNativeSessions } from '../src/host/project-native-sessions.js' import { createProjectResources } from '../src/host/project-resources.js' function fixture() { @@ -29,6 +30,19 @@ test('resource leases reserve real capacity once and never release an unconfirme assert.equal(resources.occupied('local'), 1) delete rt.state.bots.worker await resources.release(first.id, first.generation) + assert.equal(resources.occupied('local'), 1, 'Bot absence alone is not exit proof') + await resources.release(first.id, first.generation, { exited: true }) + assert.equal(resources.occupied('local'), 1, 'Missing handles cannot manufacture native exit') + rt.state.bots.worker = { id: 'worker', workspaceId: 'workspace', attemptId: 'a1' } + rt.state.tasks = { task: { projectId: 'p1' } } + rt.botOf = id => rt.state.bots[id]; rt.chatOf = id => id + rt.resolveAgent = async () => ({ session: { id: 'worker' }, status: 'idle', cancel() {}, async whenIdle() {} }) + rt.ctx = { async parallel() {}, async waterfall(_event, _payload, next) { return next() } } + const native = createProjectNativeSessions(rt) + await native.bind({ id: 's1', projectId: 'p1', taskId: 'task', attemptId: 'a1', role: 'developer', status: 'prepared', definition: { role: 'developer' }, resourceLease: first }, { id: 'workspace', path: '/registered' }, 'worker') + await native.stop('worker') + delete rt.state.bots.worker + await resources.release(first.id, first.generation) await resources.release(first.id, first.generation) assert.equal(resources.occupied('local'), 0) await assert.rejects(resources.markStarted(first.id, first.generation, { kind: 'native', botId: 'late' }), /released/) @@ -118,3 +132,32 @@ test('reconciliation cannot resurrect a concurrently confirmed released SDK leas assert.equal(rt.state.projectResourceLeases[lease.id].status, 'released') assert.equal(resources.occupied('local'), 0) }) + +test('verified supervisor exit can free a legacy native resource without claiming its context is recoverable', async () => { + const f = fixture(), resources = createProjectResources(f.rt, { priorExit: async () => true }) + const lease = await resources.reserve({ projectId: 'p1', attemptId: 'a', role: 'developer', sessionKey: 's', harness: 'flybot', resourceIds: ['local'] }) + await resources.markStarted(lease.id, lease.generation, { kind: 'native', botId: 'missing-legacy-bot' }) + const stored = f.rt.state.projectResourceLeases[lease.id] + stored.processScope = { kind: 'systemd-control-group', killMode: 'control-group', invocationId: 'a'.repeat(32), hostPid: 12345 } + await resources.reconcile() + assert.equal(stored.status, 'released') + assert.equal(stored.releaseProof.kind, 'systemd-prior-invocation-exited') + assert.equal(f.rt.state.projectNativeWorkers, undefined, 'Capacity release does not manufacture a Session binding') +}) + +test('supervisor inspection cannot release a binding changed while the manager yields', async () => { + const f = fixture() + let inspect, inspected + const started = new Promise(resolve => { inspected = resolve }) + const resources = createProjectResources(f.rt, { priorExit: () => { inspected(); return new Promise(resolve => { inspect = resolve }) } }) + const lease = await resources.reserve({ projectId: 'p1', attemptId: 'a', role: 'developer', sessionKey: 's', harness: 'flybot', resourceIds: ['local'] }) + await resources.markStarted(lease.id, lease.generation, { kind: 'native', botId: 'old-worker' }) + const stored = f.rt.state.projectResourceLeases[lease.id] + stored.processScope = { kind: 'systemd-control-group', killMode: 'control-group', invocationId: 'a'.repeat(32) } + const releasing = resources.release(lease.id, lease.generation, { proof: { kind: 'systemd-prior-invocation-exited', previousInvocationId: stored.processScope.invocationId } }) + await started + stored.worker = { kind: 'native', botId: 'replacement' } + inspect(true) + await assert.rejects(releasing, /binding changed/) + assert.equal(stored.status, 'running') +}) diff --git a/core/test/project-role-policy.test.js b/core/test/project-role-policy.test.js index b518246c4..69c961700 100644 --- a/core/test/project-role-policy.test.js +++ b/core/test/project-role-policy.test.js @@ -45,3 +45,15 @@ test('complete current FlyBot repository and subsystem rules fit every developme assert.ok(policy.sources.some(source => source.name === 'docs/subsystems/project-access.md')) } }) + + +test('custom logical roles inherit the exact approved capability conventions while default policy bytes stay compatible', () => { + const original = assembleRolePolicy({ role: 'developer' }) + assert.deepEqual(assembleRolePolicy({ role: 'developer', policyRole: 'developer' }), original) + const custom = assembleRolePolicy({ role: 'analyst', policyRole: 'reviewer' }) + assert.equal(custom.role, 'analyst'); assert.equal(custom.policyRole, 'reviewer') + assert.ok(custom.text.includes(ROLE_CONVENTIONS.reviewer)) + assert.equal(verifyRolePolicy(custom, custom.sha256), true) + assert.throws(() => assembleRolePolicy({ role: 'analyst', policyRole: 'unapproved' }), /Unknown role/) + assert.throws(() => assembleRolePolicy({ role: '', policyRole: 'reviewer' }), /Unknown role/) +}) diff --git a/core/test/project-scheduler.test.js b/core/test/project-scheduler.test.js index c2365bf19..45bea4881 100644 --- a/core/test/project-scheduler.test.js +++ b/core/test/project-scheduler.test.js @@ -186,3 +186,12 @@ test('queue recovery rechecks original requester authority and refuses terminal state.projectAttempts.active = { id: 'active', taskId: 'one', status: 'running' } await assert.rejects(scheduler.retry({ queueId: 'q' }, { memberId: 'owner-1' }), /active Attempt/) }) + +test('the existing scheduler checks in-flight execution authority before resource release and new dispatch', async () => { + const { rt, scheduler } = fixture(), calls = [] + rt.projectDelivery.reconcileExecutions = async () => { calls.push('authority') } + rt.projectResources = { async reconcile() { calls.push('resources') } } + rt.projectPools = { async pump() { calls.push('dispatch') } } + await scheduler.wake() + assert.deepEqual(calls, ['authority', 'resources', 'dispatch']) +}) diff --git a/core/test/project-task-sessions.test.js b/core/test/project-task-sessions.test.js index d4ea2cb16..59ae57aab 100644 --- a/core/test/project-task-sessions.test.js +++ b/core/test/project-task-sessions.test.js @@ -1,6 +1,6 @@ import assert from 'node:assert/strict' import {test} from 'node:test' -import {taskSession,taskSessionId,backendBindingKey,resumableBinding,confirmedWorkerExit} from '../src/host/project-task-sessions.js' +import {taskSession,taskSessionId,backendBindingKey,resumableBinding,confirmedWorkerExit,pinTaskRoleDefinition,discardUnpreparedTaskDefinitions,roleDefinitionKey} from '../src/host/project-task-sessions.js' test('Task-role identity persists across Attempts and separates projects, tasks and roles', () => { const state={tasks:{t:{projectId:'p'},t2:{projectId:'p'}}} @@ -16,7 +16,7 @@ test('Task-role identity persists across Attempts and separates projects, tasks assert.throws(()=>taskSession(state,session),/identity/) }) -test('SDK bindings rotate on cwd, principal, resource, profile, model or persona changes', () => { +test('SDK binding fingerprints distinguish cwd, principal, resource, profile, model and persona changes', () => { const input={harness:'codex',resourceId:'r',principalId:'owner',cwd:'/task',model:'model',profile:{home:'/home',configDir:'/config'},definition:'persona'} const key=backendBindingKey(input) assert.equal(backendBindingKey(structuredClone(input)),key) @@ -29,7 +29,7 @@ test('resumption requires same-scope backend identity and confirmed worker and l const record={id:'logical',projectId:'p',taskId:'t',role:'developer',bindings:[{key:'key',workerId:'w',sessionId:'thread'}]} const original={projectHarnessWorkers:{w:{projectId:'p',taskId:'t',role:'developer',logicalSessionId:'logical',bindingKey:'key',leaseId:'l',resourceId:'r',generation:2,status:'released',backendSessionId:'thread',releaseProof:{kind:'sdk-stop-ack-and-process-exit',generation:2}}},projectResourceLeases:{l:{projectId:'p',resourceId:'r',generation:2,status:'released'}}} assert.equal(resumableBinding(original,record,'key').sessionId,'thread') - assert.equal(resumableBinding(original,record,'different'),null) + assert.throws(()=>resumableBinding(original,record,'different'),/resume|binding/) for(const [target,key,value] of [['worker','projectId','foreign'],['worker','taskId','other'],['worker','role','reviewer'],['worker','logicalSessionId','other'],['worker','bindingKey','other'],['worker','status','unreachable'],['worker','backendSessionId','foreign'],['worker','releaseProof',null],['lease','status','reserved'],['lease','generation',3],['lease','projectId','other'],['lease','resourceId','other']]) { const state=structuredClone(original) ;(target==='worker'?state.projectHarnessWorkers.w:state.projectResourceLeases.l)[key]=value @@ -86,3 +86,163 @@ test('late callback failure cannot erase a fully fenced SDK stop and lease exit const l=structuredClone(lease);mutate(l);assert.equal(confirmedWorkerExit({projectResourceLeases:{l}},worker),false) } }) + +test('an incompatible latest Task-role binding cannot silently start a fresh Session or select older context', () => { + const record = { id: 'logical', projectId: 'p', taskId: 't', role: 'developer', bindings: [{ key: 'old', workerId: 'w', sessionId: 'original-thread' }] } + assert.throws(() => resumableBinding({}, record, 'new'), /resume|binding|context/i) + const state = { projectHarnessWorkers: { w: { projectId: 'p', taskId: 't', role: 'developer', logicalSessionId: 'logical', bindingKey: 'old', leaseId: 'l', resourceId: 'r', generation: 2, status: 'released', backendSessionId: 'original-thread', releaseProof: { kind: 'sdk-stop-ack-and-process-exit', generation: 2 } } }, projectResourceLeases: { l: { projectId: 'p', resourceId: 'r', generation: 2, status: 'released' } } } + record.bindings.push({ key: 'new', workerId: 'later', sessionId: 'latest-thread' }) + assert.throws(() => resumableBinding(state, record, 'old'), /resume|binding|context/i) +}) + + +function pinnedFixture() { + const scope = { projectId: 'p', taskId: 't', role: 'developer' } + const definition = { role: 'developer', instructions: 'Original persona', resolvedSkills: [{ name: 'verify', sha256: 'a'.repeat(64) }], memory: ['Current fact'], toolPolicy: { shell: true }, execution: { harness: 'codex', resourceIds: ['r', 'backup'] } } + const state = { tasks: { t: { projectId: 'p' }, other: { projectId: 'p' } }, projectAgentDefinitions: { p: { revision: 1, roles: [structuredClone(definition)] } } } + return { state, scope, definition } +} + +test('a Task pins its definition once, shares immutable snapshots, and keeps current memory outside them', () => { + const { state, scope, definition } = pinnedFixture() + const first = pinTaskRoleDefinition(state, scope, definition) + assert.equal(first.definition.memory, undefined) + assert.equal(first.revision, 1) + const same = pinTaskRoleDefinition(state, { ...scope, taskId: 'other' }, definition) + assert.equal(same.definitionRef, first.definitionRef) + state.projectAgentDefinitions.p.revision = 2 + definition.instructions = 'Changed persona' + definition.memory = ['New fact'] + const retry = pinTaskRoleDefinition(state, scope, definition) + assert.equal(retry.definition.instructions, 'Original persona') + assert.equal(retry.definitionKey, first.definitionKey) + assert.deepEqual(retry.change, { pinnedRevision: 1, currentRevision: 2, retainedOriginalDefinition: true }) + state.tasks.fresh = { projectId: 'p' } + const next = pinTaskRoleDefinition(state, { ...scope, taskId: 'fresh' }, definition) + assert.notEqual(next.definitionRef, first.definitionRef) + assert.equal(next.definition.instructions, 'Changed persona') + assert.equal(next.revision, 2) + retry.definition.instructions = 'Mutating a returned copy' + assert.equal(pinTaskRoleDefinition(state, scope, definition).definition.instructions, 'Original persona') +}) + +test('pinned role definitions do not preserve revoked live permissions or tolerate changed snapshots', () => { + const fixture = pinnedFixture() + pinTaskRoleDefinition(fixture.state, fixture.scope, fixture.definition) + for (const mutate of [ + role => { role.status = 'retired' }, + role => { role.execution.harness = 'claude-agent' }, + role => { role.execution.resourceIds = [] }, + role => { role.toolPolicy.shell = false }, + ]) { + const state = structuredClone(fixture.state) + mutate(state.projectAgentDefinitions.p.roles[0]) + assert.throws(() => pinTaskRoleDefinition(state, fixture.scope, fixture.definition), { code: 'bot/context-resume' }) + } + const snapshot = Object.values(fixture.state.projectRoleDefinitionSnapshots)[0] + snapshot.definition.instructions = 'Changed stored persona' + assert.throws(() => pinTaskRoleDefinition(fixture.state, fixture.scope, fixture.definition), /changed/) +}) + +test('failed preparation discards only unused definition pins and preserves established Session history', () => { + const { state, scope, definition } = pinnedFixture() + const first = pinTaskRoleDefinition(state, scope, definition) + assert.equal(discardUnpreparedTaskDefinitions(state, 'other'), false) + assert.equal(discardUnpreparedTaskDefinitions(state, 't'), true) + assert.equal(Object.keys(state.projectRoleDefinitionSnapshots).length, 1) + definition.instructions = 'Repaired initial definition' + assert.notEqual(pinTaskRoleDefinition(state, scope, definition).definitionRef, first.definitionRef) + state.projectAttempts = { attempt: { projectId: 'p', taskId: 't', roleSessionHistory: [{ role: 'developer' }] } } + assert.equal(discardUnpreparedTaskDefinitions(state, 't'), false) +}) + +function observedFixture() { + const scope = { projectId: 'p', taskId: 't', role: 'developer' } + const logical = { ...scope, id: 'logical', bindings: [] } + const worker = { ...scope, id: 'w', logicalSessionId: 'logical', attemptId: 'a', sessionKey: 's', bindingKey: 'key', backendSessionId: 'partial-thread', leaseId: 'l', resourceId: 'r', generation: 2, status: 'released', releaseProof: { kind: 'sdk-stop-ack-and-process-exit', workerId: 'w', generation: 2 } } + const state = { projectHarnessWorkers: { w: worker }, projectResourceLeases: { l: { ...scope, attemptId: 'a', status: 'released', resourceId: 'r', generation: 2 } } } + return { state, logical, worker, scope } +} + +test('the latest partially observed SDK Session supersedes old completed bindings, but unknown context never resets', () => { + const { state, logical, worker } = observedFixture() + assert.equal(resumableBinding(state, logical, 'key').sessionId, 'partial-thread') + logical.bindings.push({ key: 'old-key', workerId: 'old', sessionId: 'old-thread' }) + state.projectHarnessWorkers.old = { ...worker, id: 'old', bindingKey: 'old-key', backendSessionId: 'old-thread', createdAt: -1 } + assert.equal(resumableBinding(state, logical, 'key').sessionId, 'partial-thread') + worker.backendSessionId = null + assert.throws(() => resumableBinding(state, logical, 'key'), /no recoverable/) + worker.executionStartedAt = null + assert.throws(() => resumableBinding(state, logical, 'key'), /binding|resume/) + logical.bindings = [] + delete state.projectHarnessWorkers.old + assert.equal(resumableBinding(state, logical, 'key'), null) + worker.status = 'unreachable' + assert.throws(() => resumableBinding(state, logical, 'key'), /unconfirmed/) +}) + +test('legacy definition migration verifies original typed context and preserves its binding proof', async () => { + const { roleContext } = await import('../src/host/project-role-context.js') + const { state, scope, definition } = pinnedFixture() + const logical = taskSession(state, scope) + const context = roleContext({ ...scope, definition, revision: 1 }) + const session = { ...scope, id: 's', context, execution: definition.execution } + const proof = { kind: 'sdk-stop-ack-and-process-exit', workerId: 'w', generation: 2 } + state.projectAttempts = { a: { projectId: 'p', taskId: 't', roleSessionHistory: [session] } } + state.projectHarnessWorkers = { w: { ...scope, id: 'w', attemptId: 'a', sessionKey: 's', logicalSessionId: logical.id, backendSessionId: 'legacy-thread', bindingKey: 'legacy-key', resourceId: 'r', leaseId: 'l', generation: 2, status: 'released', releaseProof: proof } } + state.projectResourceLeases = { l: { projectId: 'p', resourceId: 'r', generation: 2, status: 'released' } } + logical.bindings.push({ key: 'legacy-key', workerId: 'w', sessionId: 'legacy-thread' }) + definition.instructions = 'New definition after legacy execution' + const pinned = pinTaskRoleDefinition(state, scope, definition) + assert.equal(pinned.definition.instructions, 'Original persona') + assert.deepEqual(pinned.definition.execution.resourceIds, ['r']) + const oldProof = structuredClone(logical.bindings) + assert.equal(resumableBinding(state, logical, 'v2-key', { legacyKey: 'legacy-key', definitionKey: pinned.definitionKey }).sessionId, 'legacy-thread') + assert.deepEqual(logical.bindings, oldProof) + session.context.sha256 = '0'.repeat(64) + assert.throws(() => roleDefinitionKey(session), /hash/) + assert.throws(() => resumableBinding(state, logical, 'v2-key', { legacyKey: 'legacy-key', definitionKey: pinned.definitionKey }), /hash/) +}) + +test('approved removal or capability reclassification cannot restart a pinned role with its old authority', () => { + const f = pinnedFixture() + pinTaskRoleDefinition(f.state, f.scope, f.definition) + const removed = structuredClone(f.state) + removed.projectAgentDefinitions.p.roles = [] + removed.projectRetiredRoles = { 'p:developer': { ...f.definition, retiredAt: 1 } } + assert.throws(() => pinTaskRoleDefinition(removed, f.scope, f.definition), { code: 'bot/context-resume' }) + const changed = structuredClone(f.state) + changed.projectAgentDefinitions.p.roles[0].policyRole = 'reviewer' + assert.throws(() => pinTaskRoleDefinition(changed, f.scope, f.definition), { code: 'bot/context-resume' }) + assert.equal(f.state.projectRoleDefinitionSnapshots[Object.keys(f.state.projectRoleDefinitionSnapshots)[0]].definition.instructions, 'Original persona') +}) + +test('custom logical roles fingerprint their capability class without invalidating original default-role keys', () => { + const base = { role: 'developer', instructions: 'Convention', resolvedSkills: [] } + const original = roleDefinitionKey({ role: 'developer', definition: base }) + assert.equal(roleDefinitionKey({ role: 'developer', definition: { ...base, policyRole: 'developer' } }), original) + const reviewer = roleDefinitionKey({ role: 'analyst', definition: { ...base, role: 'analyst', policyRole: 'reviewer' } }) + const developer = roleDefinitionKey({ role: 'analyst', definition: { ...base, role: 'analyst', policyRole: 'developer' } }) + assert.notEqual(reviewer, developer) +}) + + +test('approved custom roles have distinct Task-role Sessions while retired Sessions remain stoppable', () => { + const f = pinnedFixture(), analyst = { ...f.definition, role: 'analyst', policyRole: 'reviewer' } + f.state.projectAgentDefinitions.p.roles.push(analyst) + const logical = taskSession(f.state, { ...f.scope, role: 'analyst' }) + assert.notEqual(logical.id, taskSession(f.state, f.scope).id) + f.state.projectAgentDefinitions.p.roles = [] + assert.equal(taskSession(f.state, { ...f.scope, role: 'analyst' }), logical) + assert.throws(() => taskSession(f.state, { ...f.scope, role: 'unapproved' }), /scope changed/) +}) + + +test('staged governance mutations cannot replace the execution-visible committed catalog', () => { + const f = pinnedFixture(), committed = structuredClone(f.state.projectAgentDefinitions.p) + f.state.projectAgentDefinitions.p = { revision: 2, roles: [] } + const pinned = pinTaskRoleDefinition(f.state, f.scope, f.definition, 1, committed) + assert.equal(pinned.revision, 1) + assert.equal(pinned.definition.instructions, 'Original persona') + assert.throws(() => pinTaskRoleDefinition(f.state, f.scope, f.definition, 2, f.state.projectAgentDefinitions.p), /retired/) +}) diff --git a/dev/demo/delivery.mjs b/dev/demo/delivery.mjs index c65c18512..bf6d91759 100644 --- a/dev/demo/delivery.mjs +++ b/dev/demo/delivery.mjs @@ -67,6 +67,13 @@ export async function checkDelivery({ api, call, saved, waitFor, repository, git await waitFor(() => Object.values(saved().projectAttempts[id].roleSessions).every(row => row.status === 'released'), 'temporary role Session release') const sessions = Object.values(saved().projectAttempts[id].roleSessions) assert.equal(new Set(sessions.map(row => row.sessionId)).size, 3) + for (const session of sessions) { + const native = saved().projectNativeWorkers?.[session.botId] + assert.equal(native?.status, 'quiescent', 'Actual DSH must settle before native worker deletion') + assert.equal(native.releaseProof.kind, 'native-agent-quiescent') + assert.equal(native.releaseProof.sessionId, session.sessionId) + assert.equal(saved().bots[session.botId], undefined) + } assert.ok(sessions.every(row => row.context.text.includes('Published role memory marker'))) assert.equal(attempt.evidence.review.verdict, 'approve') assert.equal(attempt.evidence.tests.sha, attempt.evidence.review.sha) @@ -93,7 +100,7 @@ export async function checkDelivery({ api, call, saved, waitFor, repository, git const remaining = execFileSync('git', ['worktree', 'list', '--porcelain'], { cwd: repository, encoding: 'utf8' }) assert.equal((remaining.match(/^worktree /gm) ?? []).length, 1) return { milestone: 'M3', runtime: 'actual DSH', model: 'deterministic mock', - checks: ['model-visible role memory and repository context', 'temporary role Sessions released', 'isolated role worktrees', 'actual code edit', 'independent configured tests', 'commit-specific reviewer output', + checks: ['actual native quiescence proof before removal', 'model-visible role memory and repository context', 'temporary role Sessions released', 'isolated role worktrees', 'actual code edit', 'independent configured tests', 'commit-specific reviewer output', ...(githubRepository ? ['real GitHub PR head verified', 'authorized human acceptance'] : ['missing GitHub credential blocks publication/acceptance', 'duplicate start idempotent']), 'registered worktrees cleaned'], ...(githubRepository ? { pr: attempt.pr } : {}), notRun: ['real model', ...(!githubRepository ? ['real GitHub publication'] : []), 'browser acceptance'] } diff --git a/dev/demo/native-pools.mjs b/dev/demo/native-pools.mjs new file mode 100644 index 000000000..98844b8fc --- /dev/null +++ b/dev/demo/native-pools.mjs @@ -0,0 +1,82 @@ +// Actual DSH/host/pool continuation; model transport is deliberately deterministic. +import assert from 'node:assert/strict' +import { readFileSync } from 'node:fs' +import { join } from 'node:path' +const marker = 'NATIVE_PRIOR_CONVERSATION_2184' +export function nativePoolModel() { + const observed = { developerTurns: 0, testerTurns: 0, resumedContext: false } + function reply({ body, result, tools }) { + const system = JSON.stringify(body.system ?? '') + const developer = system.includes('Implement the task in your assigned worktree') + const tester = system.includes('Inspect the implementation in your assigned worktree') + if (!developer && !tester) return undefined + if (developer && observed.developerTurns === 1 && !result) { + assert.ok(JSON.stringify(body.messages).includes(marker), 'The actual second native request must contain its prior conversation') + assert.ok(!system.includes(marker), 'The marker must not be reconstructed from the current system/context snapshot') + assert.ok(!JSON.stringify(body.messages.at(-1)).includes(marker), 'The marker must not be supplied as new input') + observed.resumedContext = true + } + if (tools && !result) { + const source = 'export const add = (a,b) => a+b;\n' + (observed.developerTurns ? '// native continuation verified\n' : '') + const command = developer + ? `node --input-type=module -e 'import {writeFileSync} from "node:fs"; writeFileSync("code.mjs",${JSON.stringify(source)}); console.log("implemented")'` + : 'node --test code.test.mjs' + return [{ type: 'tool_use', id: `native-${developer ? 'dev' : 'test'}-${developer ? observed.developerTurns : observed.testerTurns}`, name: 'bash', input: { description: 'Execute the assigned native pool work', command } }] + } + if (developer) { + ++observed.developerTurns + return [{ type: 'text', text: (observed.developerTurns === 1 ? `Private conversation marker: ${marker}\n` : '') + 'OUTPUT handoff: ' + JSON.stringify({ action: 'handoff', to: observed.projectId + ':tester', reason: 'Inspect the actual sealed implementation', outputs: { implementation: 'Correct addition and preserve the requested continuation checkpoint' } }) }] + } + ++observed.testerTurns + return [{ type: 'text', text: 'OUTPUT handoff: ' + JSON.stringify(observed.testerTurns === 1 + ? { action: 'return', to: observed.projectId + ':developer', reason: 'Add an explicit continuation checkpoint comment before final host verification', outputs: {} } + : { action: 'handoff', to: observed.projectId + ':chief', reason: 'Actual host verification is ready for Chief judgement; no Review or project closure is claimed', outputs: { tests: 'Request the configured independent host tests' } }) }] + } + return { observed, reply } +} +export async function checkNativePools({ api, saved, waitFor, repository, model }) { + const invitation = await api('auth-invite') + const human = await api('auth-redeem', { token: invitation.token, displayName: 'Native pool owner' }) + const call = (endpoint, body = {}) => api(endpoint, body, human.sessionToken) + const state = await call('state') + const route = state.models.find(row => /deepseek-flash/.test(row.ref))?.ref ?? state.models.at(-1)?.ref + const { project } = await call('project-create', { name: 'Native pool continuation acceptance', repository: 'example/delivery-fixture' }) + model.observed.projectId = project.id + assert.equal((await call('project-resource-environment-check', { projectId: project.id, resourceId: 'native-local' })).status, 'ready') + for (const role of ['chief', 'developer', 'tester', 'reviewer']) await call('project-agent-definition', { projectId: project.id, role, model: route, ...(role === 'chief' ? {} : { execution: { harness: 'flybot', resourceIds: ['native-local'] } }) }) + const botId = saved().projects[project.id].chief.botId + const task = await call('task-create', { botId, projectId: project.id, title: 'Fix addition with explicit native continuation', criteria: 'Addition passes actual configured tests and native role context survives a return' }) + const started = await call('project-task-run', { taskId: task.id, requestId: 'native-pool-continuation' }) + assert.equal(started.attempt.protocol, 'project-handoff/v1') + await waitFor(() => { + const current = saved().tasks[task.id]?.collaboration + if (current?.status === 'blocked') throw new Error('Native pool blocked: ' + current.block?.reason) + return current?.status === 'arbitration' + }, 'native Developer → Tester → original Developer → Tester → Chief pool', 90_000) + const snapshot = saved(), attempts = Object.values(snapshot.projectAttempts).filter(row => row.taskId === task.id) + const developer = Object.values(snapshot.projectTaskSessions).find(row => row.taskId === task.id && row.role === 'developer') + assert.equal(attempts.length, 2) + assert.equal(developer.bindings.length, 2) + const [first, second] = developer.bindings.map(binding => snapshot.projectNativeWorkers[binding.workerId]) + assert.notEqual(first.id, second.id, 'Execution carriers are temporary') + assert.equal(first.sessionId, second.sessionId, 'The original physical native context must resume') + assert.equal(first.cwd, second.cwd, 'Resume keeps the original execution cwd') + assert.ok(model.observed.resumedContext) + assert.equal(model.observed.developerTurns, 2); assert.equal(model.observed.testerTurns, 2) + assert.ok(Object.values(snapshot.projectNativeWorkers).every(row => row.status === 'quiescent' && row.releaseProof?.kind === 'native-agent-quiescent')) + const leases = Object.values(snapshot.projectResourceLeases).filter(row => row.projectId === project.id).sort((a, b) => a.createdAt - b.createdAt) + assert.equal(leases.length, 4) + assert.ok(leases.every(row => row.status === 'released' && row.releaseProof?.kind === 'native-agent-quiescent')) + for (let index = 1; index < leases.length; ++index) assert.ok(leases[index - 1].releasedAt <= leases[index].createdAt, 'One actual resource must never authorize overlapping execution') + const current = snapshot.projectAttempts[snapshot.tasks[task.id].deliveryAttemptId] + assert.equal(current.evidence.tests.passed, true) + assert.equal(current.evidence.tests.sha, current.candidateSha) + assert.notEqual(attempts[0].candidateSha, current.candidateSha) + assert.notEqual(snapshot.tasks[task.id].status, 'accepted', 'Chief has not adjudicated completion') + assert.match(readFileSync(join(repository, 'code.mjs'), 'utf8'), /a-b/, 'The base repository stays intact') + await call('project-attempt-stop', { attemptId: current.id }) + for (const attempt of attempts) await call('project-attempt-cleanup', { attemptId: attempt.id }) + return { milestone: '2184-native-pools', runtime: 'actual DSH', model: 'deterministic mock', + checks: ['native per-Bot pool return/handoff', 'same Task and original physical role Session', 'fresh carrier identities and same cwd', 'prior conversation present only in retained context', 'actual two code revisions and configured host tests', 'native quiescence and single-resource release before downstream execution', 'Chief arbitration inbox without fake closure', 'disposable cleanup'], + notRun: ['real model', 'Reviewer and forge publication in this continuation probe', 'Chief acceptance', 'native crash recovery'] } +} diff --git a/dev/demo/run.mjs b/dev/demo/run.mjs index 4db3de835..230bb9627 100644 --- a/dev/demo/run.mjs +++ b/dev/demo/run.mjs @@ -7,6 +7,7 @@ import { createServer } from 'node:net' import { tmpdir } from 'node:os' import { join, resolve } from 'node:path' import { mockModel } from './mock.mjs' +import { nativePoolModel, checkNativePools } from './native-pools.mjs' import { checkLiveWorkbench } from './live-workbench.mjs' import { serveWorkbench } from './serve.mjs' import { checkWorkbench } from './workbench.mjs' @@ -43,6 +44,8 @@ async function waitFor(read, label, milliseconds = 90_000) { } async function main() { + const nativePools = process.argv.includes('--native-pools') + const nativeModel = nativePools ? nativePoolModel() : null const agents = process.argv.includes('--agents') const serve = process.argv.includes('--workbench-serve') const live = process.argv.includes('--live-check') @@ -53,7 +56,7 @@ async function main() { const requestStop = () => shutdown.abort() if (serve) { process.on('SIGINT', requestStop); process.on('SIGTERM', requestStop) } const queue = process.argv.includes('--queue') - const delivery = process.argv.includes('--delivery') || queue || workbench || Boolean(codex) + const delivery = nativePools || process.argv.includes('--delivery') || queue || workbench || Boolean(codex) const githubRepository = process.argv.includes('--github-repo') ? process.argv[process.argv.indexOf('--github-repo') + 1] : undefined const githubConfigDir = process.argv.includes('--github-config-dir') ? process.argv[process.argv.indexOf('--github-config-dir') + 1] : undefined if (githubRepository && (!delivery || !githubConfigDir)) throw new Error('GitHub demo requires --delivery and explicit --github-config-dir') @@ -85,7 +88,7 @@ async function main() { '@deepseek-ai/dsh-base': '0.2.1-rc.2', '@deepseek-ai/dsh-web-app': '0.2.1-rc.2' }, dsh: { profile: { bundles: ['@deepseek-ai/dsh-base', '@deepseek-ai/dsh-web-app', 'ds-bot', '@flybot/workflow'] } } })) const observations = [] - const model = mockModel(observations, { reply: workbench ? input => deliveryReply(input) ?? agentReply(input) : delivery ? deliveryReply : agents ? agentReply : undefined }) + const model = mockModel(observations, { reply: nativeModel?.reply ?? (workbench ? input => deliveryReply(input) ?? agentReply(input) : delivery ? deliveryReply : agents ? agentReply : undefined) }) model.listen(0, '127.0.0.1') await once(model, 'listening') const port = await freePort() @@ -101,11 +104,12 @@ async function main() { if (shared || delivery || agents) { patch = join(directory, 'shared.patch.yml') const botConfig = { projectWorkbench: true, + ...(nativePools ? { projectResources: { 'native-local': { type: 'local', root: directory, capacity: 1, allowedProjectOwnerIds: ['owner-1'], harnesses: ['flybot'] } } } : {}), ...(codex ? { projectResources: { 'codex-local': { type: 'local', root: directory, capacity: 3, nodePath: process.execPath, allowedProjectOwnerIds: ['owner-1'], adminMemberIds: ['owner-1'], harnesses: ['codex'], profiles: { 'owner-1': { codex: { home: codex.home, configDir: codex.configDir, model: codex.model } } } } } } : {}), ...(workbench && !live ? { projectResources: { 'demo-local': { type: 'local', root: directory, profileRoot: join(directory, 'private-sdk-profiles'), capacity: 2, allowedProjectOwnerIds: ['owner-1'], adminMemberIds: ['owner-1'], harnesses: ['codex'], profiles: { 'owner-1': { codex: { home: join(directory, 'sdk-home'), configDir: join(directory, 'sdk-config'), model: 'fixture-sdk-probe', baseUrl: 'http://127.0.0.1:1' } } } } } } : {}), ...(workbench ? { projectDemoMode: realModel ? 'real-provider' : 'deterministic-fixture' } : {}), ...(shared ? { controlPlane: { origin: shared.origin, ownerSubject: shared.ownerSubject } } : {}), - ...(delivery ? { projectDispatch: 'workflow', projectRepositories: { [githubRepository ?? 'example/delivery-fixture']: { ...(githubRepository ? { github: { credential: 'gh-cli', configDir: githubConfigDir, ...(codex ? { login: codex.login } : {}) } } : {}), path: workspace, testCommand: live || codex ? [process.execPath, golden] : [process.execPath, '--test', 'code.test.mjs'], allowedPaths: ['code.mjs', 'code.test.mjs'] }, ...(live ? { 'example/intentional-failure': { path: workspace, testCommand: [process.execPath, golden, '--force-failure'], allowedPaths: ['code.mjs', 'code.test.mjs'] } } : {}) } } : {}) } + ...(delivery ? { projectDispatch: nativePools ? 'pools' : 'workflow', projectRepositories: { [githubRepository ?? 'example/delivery-fixture']: { ...(githubRepository ? { github: { credential: 'gh-cli', configDir: githubConfigDir, ...(codex ? { login: codex.login } : {}) } } : {}), path: workspace, testCommand: live || codex ? [process.execPath, golden] : [process.execPath, '--test', 'code.test.mjs'], allowedPaths: ['code.mjs', 'code.test.mjs'] }, ...(live ? { 'example/intentional-failure': { path: workspace, testCommand: [process.execPath, golden, '--force-failure'], allowedPaths: ['code.mjs', 'code.test.mjs'] } } : {}) } } : {}) } writeFileSync(patch, readFileSync(join(root, 'docker/web.patch.yml'), 'utf8') + (live ? '\n- id: llm-deepseek\n config: { thinking: disabled, reasoningEffort: off, maxTokens: 8192 }\n' : '') + `\n- id: bot\n config: ${JSON.stringify(botConfig)}\n`, { mode: 0o600 }) } const child = spawn(process.execPath, [cli, '--profile', 'web', '--patch', patch, @@ -173,6 +177,10 @@ async function main() { console.log(JSON.stringify(await checkQueue({ api, call, saved, waitFor }))) return } + if (nativePools) { + console.log(JSON.stringify(await checkNativePools({ api, saved, waitFor, repository: workspace, model: nativeModel }))) + return + } if (delivery) { console.log(JSON.stringify(await checkDelivery({ api, call, saved, waitFor, repository: workspace, githubRepository }))) return diff --git a/dev/demo/test/runtime.test.mjs b/dev/demo/test/runtime.test.mjs index db22ffbec..b11668f13 100644 --- a/dev/demo/test/runtime.test.mjs +++ b/dev/demo/test/runtime.test.mjs @@ -70,3 +70,20 @@ test('real DSH delivery queue enforces quota before allocation and rechecks memb assert.ok(report.checks.includes('confirmed worker release before workspace cleanup')) assert.deepEqual(readdirSync(directory), []) }) + +test('real DSH resumes native Task-role context after a pool return and serializes a single resource', { timeout: 150_000 }, t => { + const directory = mkdtempSync(join(tmpdir(), 'flybot-native-pool-runtime-')) + t.after(() => rmSync(directory, { recursive: true, force: true })) + const result = spawnSync(process.execPath, [resolve(import.meta.dirname, '../run.mjs'), '--native-pools'], { + encoding: 'utf8', timeout: 140_000, env: { ...process.env, TMPDIR: directory, M0_REPORT: '' }, + }) + assert.equal(result.status, 0, result.stderr) + const report = JSON.parse(result.stdout) + assert.equal(report.milestone, '2184-native-pools') + assert.equal(report.runtime, 'actual DSH') + assert.ok(report.checks.includes('prior conversation present only in retained context')) + assert.ok(report.checks.includes('native quiescence and single-resource release before downstream execution')) + assert.ok(report.checks.includes('Chief arbitration inbox without fake closure')) + assert.ok(report.notRun.includes('real model')) + assert.deepEqual(readdirSync(directory), [], 'Disposable native workers, profiles and worktrees must be removed') +}) diff --git a/docs/architecture.md b/docs/architecture.md index b843bc2a5..9dfc0a866 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -173,3 +173,6 @@ it does not grant SDK shared-ref access, replace a PR or substitute for verifica Project role governance (#2171) separates resident Chief proposals, Maintainer/Owner decisions, the current revisioned role catalog, and archived definitions/audit. Durable Effects proof gates visible changes; logical role keys and execution capability classes remain distinct. See [role governance](subsystems/project-agents.md#project-role-governance-2171). + + +Native execution (#2184) uses the original Workflow lifecycle and SessionController, with exact Task-role physical bindings and durable quiescence facts. A new ephemeral carrier can restore prior context without reviving a deleted Bot identity; new input is fenced during stopping and resources are never released from Bot absence alone. See [native adapter](subsystems/project-harness.md#native-physical-session-binding-and-quiescence-2184). diff --git a/docs/history/2026-11-03-execution-evidence.md b/docs/history/2026-11-03-execution-evidence.md new file mode 100644 index 000000000..123c0bfaf --- /dev/null +++ b/docs/history/2026-11-03-execution-evidence.md @@ -0,0 +1,148 @@ +# Historical backend execution evidence + +Archived verbatim from PROJECT.md on 2026-11-03 to keep the active role-policy context bounded. These dated implementation/probe records retain their original claims and limitations; current owner decisions and PROJECT.md govern current behavior and permissions. No active operating rule or test budget is removed. + +### Direct backend recovery — 2026-11-01 + +The owner requires backend repairs in the foreground Codex session; do not +dispatch backend development through FlyBot. Existing frontend work is preserved and now continues autonomously while new +backend service work is developed independently (owner update, 2026-11-01). See the +[execution decision](decisions/2026-11-01-direct-backend-recovery.md). +R6 adds coalesced event-driven dispatch, manager-controlled retry of blocked queue +entries under the original requester, and operator-selected workspace preparation +before each pool receiver and after independent test cleanup. Failed allocation, +checkout or context preparation releases only proven unbound reservations and +retains its failure audit. These changes do not imply live UI or R8 acceptance. + + +This is `zy95-12/FlyBot`. Current scope is the [shared-project roadmap](https://github.com/zy95-12/FlyBot/issues/1) +and [fork decision](decisions/2026-10-31-project-first-fork.md). The inherited mission, +people and stage descriptions below are historical upstream context; upstream accounts +are not this fork's decision-makers merely because inherited docs name them. + +- Project is the target stable boundary for Task/Run/Workspace ownership. Bots and + Sessions are replaceable execution/interaction carriers. M1 implements direct ownership and explicit legacy resolution; see [Project domain](docs/subsystems/project-domain.md). +- Reuse control-plane identity, auth, invitation, instance and operations APIs unchanged + where possible. Put project business logic/adapters in FlyBot; justify a missing API + separately. The platform PR was withdrawn, not merged. +- Rebuild task navigation and the Project board; current UI/components are not constraints. +- M0 is implemented on [PR #9](https://github.com/zy95-12/FlyBot/pull/9), merged (2026-10-31): + real DSH/tool execution with a deterministic model, plus a FlyBot-owned HTTP adapter + exercising unchanged external platform source. This is not real-model PR delivery. +- The baseline identity, scoped Workflow-write and per-run cwd seams are implemented + through M1–M3; later milestone integration and live acceptance are recorded below. +- Durable repository memory is this page, ROADMAP, LESSONS and dated decisions. Update + them and matching architecture docs alongside code, with regression coverage in + verify/CI and tested-SHA evidence. Never edit live DSH/Bot memory for repository decisions. + +See [baseline ownership and tests](docs/subsystems/demo-baseline.md). + +- M1 implementation ([PR #10](https://github.com/zy95-12/FlyBot/pull/10), complete local verification, awaiting review) adds Project service/policy/repository boundaries, immutable work + ownership, explicit migration, archive/delete/member-removal rules and an owner-only + detail API. Workspace records are directory references; M3 supplies worktree isolation. +- The owner authorized continuing M2–M6, one PR per milestone, with memory, architecture + and tests updated in each. [M1 decision](decisions/2026-10-31-m1-project-ownership.md). + +- M2 ([PR #11](https://github.com/zy95-12/FlyBot/pull/11), complete local verification, awaiting review) implements the [shared access contract](docs/subsystems/project-access.md): unchanged + platform auth API adapter, stable subject mapping, human membership policy/invites, + immediate revocation and scoped Workflow gateway. It is explicitly configured; worktrees, + agent delegation/credentials and browser delivery are supplied by M3–M6 below. + +- M3 ([PR #13](https://github.com/zy95-12/FlyBot/pull/13), full local verify) implements the [commit-specific delivery contract](docs/subsystems/project-delivery.md) + on the existing Workflow service: versioned Attempts, real role worktrees/cwd, host test + evidence, Review/SHA gates and human acceptance. Actual DSH and draft artifact PR #12 + were first exercised with a deterministic model. Integrated real-model/GitHub/browser + acceptance subsequently passed in M6; [evidence](docs/live-acceptance.md). + +- M4 implements [scoped Project agents](docs/subsystems/project-agents.md): fresh public + Chief, private human/Project identities, separate memory/history/proposals, live delegated + tool checks, role/skill snapshots and exact GitHub credential scopes. Actual DSH fixture + validation covers creation/publication/revocation; real-model/browser acceptance + subsequently passed in M6. Runtime code-worker OS isolation is not claimed by this demo. + +- M7 is owner-authorized in [#20](https://github.com/zy95-12/FlyBot/issues/20), + implemented sequentially as #21–#25, one PR per issue. It explicitly authorizes native, + Codex SDK and Claude Agent SDK Harness choices; the inherited upstream removal of + Codex does not govern this fork. Control-plane remains unchanged. M7.1 introduces + explicit public role memory and bounded hashed execution contexts, durable Chief + identity/history and per-Attempt temporary Session bindings. M7.2–M7.5 code is on PRs #27–#30: resource leases, SDK Harnesses, Owner setup and + verified Git handoffs. Authenticated mixed-resource acceptance remains outstanding; + the later owner-authorized local Codex-only run passed, as recorded below. + +### Task-scoped SDK context and resident Chief memory — 2026-11-01 +Owner requires a resident Chief with durable Project memory and one logical Session per Task/role. Codex/Claude own their supported conversation persistence and automatic compaction; FlyBot does not inject a second model-history compactor. SDK role rework reuses a confirmed released Task-role cwd/thread under the same principal/resource/profile/model/persona binding, while preserving separate immutable Attempt/claim/log/test/Review evidence. Changed bindings rotate backend threads with recorded lineage. Chief memory uses the original own-Bot MEMORY.md/topics/journal, never global team or private-user memory. Context admission keeps current bounded facts and hash-bound full source archives for scoped retrieval. Backend changes are foreground work; frontend Task40/PR48 resumes through FlyBot after validation. + +Task40 continuation integrates trusted backend `91792c1b` into preserved candidate +`25607404` without committing or publishing. Room-scoped messages/streaming and role +identity are retained alongside Chief retrieval/memory and binding-checked SDK resume. +The inherited changelog failure reproduces (6/7) and the single-entry repair passes +(7/7). Generated patches match canonical core. Direct integration testing requires +DSH's registered loader: plain node cannot resolve `@deepseek-ai/dsh-llm` from the +new scheduled-task module. This is not a passing integration result. Native composer, +tool/approval/stop/stream, personal/group interactions, CSS-load synchronization and +actual deployed three-account/weak-network acceptance remain open. + +Follow-up local check used the required harness loader: +`cd core && node --import ./test/harness-resolve.mjs --test test/project-agents.test.js test/project-harness.test.js` +passed 40/40 on the staged merge draft. The earlier plain-node failure was a loader +invocation error, not a missing prepared dependency. Full verify, smoke, independent +Review and deployed acceptance are Not run for this draft. + +Task40 adds a CSS/JS readiness barrier (defect tests 0/2 → 2/2) and mounts the scoped +composer from original composer.js for Project agents and personal conversations. +The original dictation hook is reused with disposal; captured chat handles retain +scope revocation. A base-bundle composer test fails because the capability is absent; +current native render/client/asset checks pass 27/27. These are local draft checks, +not host full verification, independent Review or deployed three-account acceptance. + +The room composer now shares that native mount and existing registered-document +IDs, while personal groups restrict mention choices to their visible members. +Actual native workText labels render scoped live tool names. Final focused UI +checks pass 33/33. Full verify was attempted during editing and is not valid final +verification; a later process-control/result request was refused by sandbox +loopback policy. Request actual configured host verification through Tester, then +return this partial implementation to Developer for remaining tool-result, +approval, personal live/stop and full interaction work. Do not close Task40. + +Task40 continuation at base cbcad4c4 adds actual native question/approval/stop/continue +controls, original group MemberWork controls, bounded redacted tool results and current +own-chat streaming. Existing original components are reused; standing grants and raw +owner APIs are not exposed. SDK permission/message inputs now use the same original +approval/composer components with existing host worker-generation/current-rights gates. +The new personal HTTP capability test failed at base (1/2) before implementation and +passed afterward; additional boundary tests cover foreign approvals/workers, retired +handles, hidden turns, historical parts and stale questions. Host full verification, +independent Review and deployed three-account/weak-network screenshots remain required. + +The continuation also restores answerable group question/approval cards through +original handlers, exposing only public question fields rather than replyTo/origin. +Group member-stop tests use actual native membership and reject another owner's Bot. +Final local interaction checks, build/patch/lint results and outstanding host/browser +validation are reported at handoff; no candidate recommendation closes this Task. + +Task40 host verification at `33e482d6` reported four failures in install, secrets +and usage suites. All four reproduced locally (28/32). The personal stream observer +was also installed in legacy mode, shifting the established listener order used by +those protocol tests. Register it only for explicit shared-workbench mode; keep its +purpose/Session guards. The new mode-boundary regression fails without that fix; +affected suites plus personal-workbench coverage now pass 38/38. Final host full +verification and independent Review remain required; no deployed acceptance is claimed. + +Review return at `846378bf` reproduced current-part navigation leaving the live +conversation read-only; picker and explicit current-part links now normalize to the +live route (defect regression red→green). Human Markdown/mentions reuse original +PersonMessage. Attachment-only messages, scoped image preview/file download and +native file/image sends use public DSH attachment/Agent APIs with current-user checks. +The new human/attachment render test fails against the base bundle and passes on the +working build. Boundary tests cover foreign IDs, batch validation, historical reads, +revocation during admission and late download callbacks. Final host verification, +independent Review and real-service screenshots/weak-network acceptance remain open. + +Review return at `bde2cf7c` reproduced native composer remount/focus loss on same-route +refresh. The mounted regression now uses bounded boolean DOM-identity diagnostics +(the former object diff exhausted memory before reporting failure) and reproduces +red at that source. Native roots now update in place within the same scoped route and +source, retaining input, cursor and focus; navigation/revocation still dispose them. +Mounted personal/public/private/room regressions cover retained sends and scope teardown. +No resource limit was raised. Exact-candidate host verification, independent Review +and deployed three-account/weak-network/screenshot acceptance remain separate gates. diff --git a/docs/subsystems/demo-baseline.md b/docs/subsystems/demo-baseline.md index f469ecba5..cb8a675e9 100644 --- a/docs/subsystems/demo-baseline.md +++ b/docs/subsystems/demo-baseline.md @@ -164,3 +164,9 @@ and the legacy per-Project persona. It still exercises scoped context, private T proposal, explicit publication and revoked Session refusal through actual DSH. A fixture regression pins both personas; this is model-transport compatibility, not real external-provider or paid model acceptance. + + +#2184 extends the original actual-DSH delivery probe to inspect persisted native-agent-quiescent proofs for Developer, Tester and Reviewer before removal. It keeps deterministic inference and the existing real code edit, configured host tests, missing-credential refusal and disposable cleanup assertions; this is not a real-provider or canonical return-loop acceptance. + + +`node dev/demo/run.mjs --native-pools` adds actual DSH single-resource Developer/Tester return/context restoration acceptance with a deterministic model. The marker is emitted only into the first Developer conversation outside the durable handoff JSON; the second request must find it in retained history, absent fresh system/input. It verifies fresh carriers, same physical Session/cwd, actual second code revision/host tests, lease intervals and Chief arbitration without fake closure; cleanup stops its disposable Attempt and removes worktrees. The required runtime suite now includes this fifth case. Initial failures exposed implicit [object Object] delivery deduplication and DSH archived-session admission; retained private evidence distinguishes both from the passing probe. diff --git a/docs/subsystems/project-agents.md b/docs/subsystems/project-agents.md index e6393b784..94df2920c 100644 --- a/docs/subsystems/project-agents.md +++ b/docs/subsystems/project-agents.md @@ -414,3 +414,6 @@ messages to a retired role fail visibly; historical rooms remain readable. These governance tests use real host APIs/Store plus deterministic Bot carriers and Effects fault fixtures, not a real model. Custom-role physical execution and in-flight revocation acceptance remain in #2184; this catalog alone does not prove that loop. + + +Execution integration now preserves approved custom role keys through actual Workspaces, role-policy sources and artifact verification. Capability classes choose inherited conventions and SDK readOnly behavior; they do not grant human authority. A retired role can finish its already acquired frozen claim and explicitly hand off to an active target. Queued retired-role work becomes blocked with a Chief-routing reason; retirement never silently selects a replacement. New claims/offers to retired roles remain refused. diff --git a/docs/subsystems/project-chief-control.md b/docs/subsystems/project-chief-control.md index 14e3c9523..55b008adc 100644 --- a/docs/subsystems/project-chief-control.md +++ b/docs/subsystems/project-chief-control.md @@ -149,3 +149,9 @@ audit and workspace remain intact; no ownership is granted by the status change. Validated Chief completion accepts up to four role proposals alongside Task decisions. The host supplies stable run-scoped proposal keys and current Chief-job authority; proposals cannot apply role changes. A malformed proposal list fails the job rather than granting configuration rights. + + +Custom-role targets use their approved capability class for Developer rework semantics. The active catalog still controls new targets; an existing claim's frozen source identity may be retained after retirement for completion/Chief intervention. Chief must explicitly choose an active destination for blocked retired-role work. + + +Native prior-host recovery reuses the existing supervisor proof helper and exact lease/generation data. A matching known exit permits safe cleanup/resumption; unresolved platform jobs, old unmanaged scopes or lost context remain explicit held conditions for Chief/operator reconciliation. No new inspection timer or parallel execution coordinator is introduced. diff --git a/docs/subsystems/project-delivery.md b/docs/subsystems/project-delivery.md index 9eb2bd0cd..2a630e029 100644 --- a/docs/subsystems/project-delivery.md +++ b/docs/subsystems/project-delivery.md @@ -461,3 +461,29 @@ Attempt after all role workers/leases/native occupancy are confirmed released, whether its worktree is clean or dirty. A clean operator-restart checkpoint must not manufacture a successor and discard its Attempt-scoped integration admission. Explicit returns/rework and unknown worker occupancy retain existing isolation. + + +### Pinned Task-role definitions — #2184 + +Pool and legacy Workflow preparation pin public role definitions before resource +reservation, with shared content-addressed snapshots and scalar Session references. +Retries use the original persona/skills, current memory and explicit change metadata; +they still enforce current execution permission/resource availability. Preparation +failure clears an unused definition pin only if no Session/worker exists. Workspace +drafts, original prepared contexts and Attempt evidence remain untouched. SDK resume +and unknown-exit recovery follow [the Harness contract](project-harness.md). + + +Native Sessions now use the existing Task-role binding service through projectDelivery.nativeSessions. Canonical pool worktrees can be retained across Attempts only after exact native quiescence and released leases; original DAG workspace behavior remains compatible. Definition capability changes and removed roles block stale restart instead of inheriting authority. + + +New v7/pools-v4 Attempts distinguish logical role keys from developer/tester/reviewer capability classes. `roleAssignments` maps each artifact responsibility to the actual logical producer; Workspaces, Sessions, leases and execution evidence remain keyed by that logical role. Implementation/test/review evidence records its producer and role Session. Artifact sealing, host tests, Review, publication/retry and SDK proof validation resolve those assignments; no duplicate Workspace aliases or rewritten historical records are introduced. Historical Attempts without assignments retain default-role behavior. A custom Developer gets its own local branch while the canonical publication branch remains stable. The artifact-controller regression uses actual Git edits, configured host tests and a labelled fixture publisher; it does not claim signed Review or a whole bot loop. + + +Native host creation now prepares the exact context binding, invokes the existing Session/resource bind, then activates through a committed native.activate Effects marker. This orders all lease/carrier/Session metadata before archive admission or model input. The existing pre-step middleware closes startup/inbox bypasses for uncommitted or obsolete native bindings; ordinary Bots are unaffected. + + +The pool host release adapter now delegates to one physical-exit query for SDK and native execution, requiring actual recorded proof and released leases before initiative can move. Native stop follows current parts and known subagent work, and supervisor recovery updates the exact historical/current role Session physical ID. Missing Bot records alone cannot advance a Task. + + +Current canonical execution is revalidated through one authority helper before new input and by the existing scheduler before resource reconciliation/new dispatch. Revocation reuses the pool lifecycle stop, with an exact claim fence, visible blocked reason and confirmed-exit requirement. Role snapshots retain conventions while current rights constrain their use. diff --git a/docs/subsystems/project-harness.md b/docs/subsystems/project-harness.md index 97dd0c77d..6f22c18b0 100644 --- a/docs/subsystems/project-harness.md +++ b/docs/subsystems/project-harness.md @@ -1,6 +1,13 @@ --- subsystem: project-harness owned_paths: + - core/src/host/project-execution-authority.js + - core/test/project-execution-authority.test.js + - core/src/host/project-execution-exit.js + - core/test/project-execution-exit.test.js + - core/src/host/project-native-sessions.js + - core/test/project-native-sessions.test.js + - core/test/project-native-exit.test.js - core/src/host/project-task-sessions.js - core/test/project-task-sessions.test.js - core/src/host/project-git-sandbox.js @@ -57,7 +64,7 @@ installation in the target environment. An environment probe is not an auth prob One temporary SDK worker maps to one existing Workflow node and one resource lease. Saved worker evidence names Project/Task/Attempt/role/Harness/version/resource/principal, lease generation, cwd, backend Session/thread and event log. Input continuation resumes -that backend Session in the same bound cwd. New Attempts create fresh workers/threads. +that backend Session in the same bound cwd. New Attempts create fresh worker processes; compatible Task-role bindings retain the original backend thread. Codex uses streamed events, read-only/workspace-write sandbox by role, external command network and web search denied and approvalPolicy=never. The exec SDK does not provide an interactive approval @@ -183,7 +190,7 @@ are provisioned by the host, rather than asking a read-only Reviewer to install ## Task-role SDK continuation and scoped context -`project-task-sessions.js` owns stable Project/Task/role logical IDs and binding checks; state.projectTaskSessions persists backend bindings through ordinary store snapshots. Worker records retain logicalSessionId, bindingKey, continuedFrom or rotationReason. Same binding resumes a thread only after prior worker stop-ack/process-exit or an exact host-confirmed prior systemd invocation exit, with a matching released lease. Binding includes Harness/resource/principal/cwd/profile locations/baseURL/model/persona. A changed binding starts a new backend thread under the same logical Session; no cwd or principal migration. Worker/claim/generation/log identities remain distinct. Mandatory current candidate/criteria/context are supplied on every turn; a complete unchanged policy already in the resumed thread is referenced by verified digest instead of duplicated. +`project-task-sessions.js` owns stable Project/Task/role logical IDs and binding checks; state.projectTaskSessions persists backend bindings through ordinary store snapshots. Worker records retain logicalSessionId, bindingKey, continuedFrom and an optional legacy-key migration marker. Same binding resumes a thread only after prior worker stop-ack/process-exit or an exact host-confirmed prior systemd invocation exit, with a matching released lease. Binding includes Harness/resource/principal/cwd/profile locations/baseURL/model/persona. An incompatible latest binding raises `bot/context-resume`; it cannot silently reset or select an older matching thread. No cwd or principal migration is implied. Worker/claim/generation/log identities remain distinct. Mandatory current candidate/criteria/context are supplied on every turn; a complete unchanged policy already in the resumed thread is referenced by verified digest instead of duplicated. Codex exec/SDK retains its own automatic history compaction, with no custom compact threshold or second DSH engine. Native DSH retains original parts/steward behavior. Native per-execution carriers retain Task logical identity and source archives; this does not claim persistent native-carrier reuse across Attempts. @@ -216,3 +223,140 @@ only when worker ID/generation, Project/Attempt/role/resource and both worker/le stop proofs match a released SDK lease exactly. Unknown/unreachable exits, missing proofs and mismatched leases remain refused. This preserves historical failure attribution without blocking every future integration admission for the same Task. + + +## Execution convergence — #2184 + +Task-role identity remains stable across Attempts. A content-addressed public +`projectRoleDefinitionSnapshots` record pins persona, resolved skills and execution +contract on first preparation; Session/Task records reference its id/hash/revision. +Dynamic memory and current Task feedback remain per-turn inputs. A failed preparation +without any prepared Session or observed worker drops only the unused pin; history +and shared snapshots are retained. Existing prepared Sessions keep their version. +Current role retirement, revoked shell permission, Harness change or removal of the +previous resource from the approved pool prevents further execution. Definition +changes appear in the returned Session context rather than replacing the persona. + +`spawn` prepares a worker and records the exact Task/Attempt/role/logical Session, +resource/profile/Harness and lease generation. `dispatch` persists the running marker +before writing model input and rechecks cancellation/lease generation after saving. +A no-op save cannot authorize dispatch. The first observed backend Session is indexed, +including failed/partial turns; a later retry cannot fall back to older completed +context. If execution might have started but no backend id is known, recovery is +explicitly blocked. Only a proven unstarted, released process may be skipped. +Legacy prompt-based keys migrate only with matching environment and validated, +scoped original role context; original worker/binding proofs are preserved. + +The existing service methods remain the lifecycle seam: delivery `preparePoolRole`, +Harness `spawn`/`dispatch`, `input`/`taskInputWake`, `release`, and existing query views. +`capabilities(harness)` reports implementation support independently of installation, +login or successful inference: Codex/Claude support same-binding SDK continuation, +between-turn input and abort/stop with acknowledged process exit. Native uses existing +Bot turns/removal; native Task-role restoration is not yet advertised. Cross-Harness +restoration is unsupported. An SDK capability declaration is not real-provider proof. + +This stage has subprocess/SDK-adapter regression coverage, not a new real-model run. +Native Task-role continuation, unified public receipts/events (#2185), transactionally +committed Effects (#2182), parent resumption (#2179), and final real-task validation +remain explicit acceptance work. No second executor or scheduler is introduced. + + +## Native physical Session binding and quiescence (#2184) + +The original Workflow lifecycle still owns native Bot creation, dispatch and removal. +Its registered-Workspace host adapter now records `projectNativeWorkers` and a +`native-session/v1` binding on the existing Task-role logical Session. This is adapter +metadata, not a second executor. A fresh ephemeral Bot may bind only the latest exact +native physical Session after prior execution is quiescent, the old Bot is gone and +its resource lease is released. Project/Task/role, cwd, model, definition fingerprint +and resource must match. New native parts retain previous segments; tombstoned Bot +IDs are never resurrected. A saved legacy native Session without a trusted binding +fails with explicit recovery instead of silently starting over. + +Removal first fences new input, requests cancellation through the existing resolved +DSH Agent, and awaits `whenIdle()` before recording `native-agent-quiescent` with +worker, latest physical Session, binding key and generation. Bot absence and a bare +`exited` flag are not proof. A stop failure keeps visible `stop-unconfirmed` state and +capacity held. A process-local host epoch prevents accidentally resuming an unknown +old-host worker merely to cancel it. Quiescence survives restart as a recorded fact; +unknown old-host executions still require actual prior-exit reconciliation. No timeout +fabricates exit. Stop acknowledgement alone cannot release a native lease. + +Native continuation rechecks the live pool claim, initiating human/delegation, role +permissions and resource before and after asynchronous Session loading. Overlapping +binds for one Task-role cannot attach two carriers. Canonical pool rework may retain +the same physical worktree only after matching native quiescence and released leases; +legacy DAG workspace rules remain separate. Existing Task definitions remain pinned, +but a removed role or changed execution capability cannot restart under stale rights. +Approved custom role keys have distinct logical Session IDs; default-role fingerprints +remain compatible while a nondefault capability class participates in the fingerprint. + +Actual DSH delivery acceptance checks all three native roles' quiescence proofs before +removal, actual code edits/host tests, and credential refusal; inference is deterministic. +Factory tests cover exact restoration/parts, concurrency, stale claims, model/cwd changes +and unknown exit. They do not establish a real-provider canonical pool return loop. +Custom-role artifact verification, native crash reconciliation, in-flight capability +revocation and the final composed service acceptance remain tracked under #2184. +The capability query now advertises native Task-role resume and native-agent-quiescence exit confirmation for exact supported bindings; unknown exit still refuses continuation. + + +Approved custom roles now pin `policyRole` on their Session and SDK worker record. Native/SDK Workspace ownership accepts an approved capability while preserving logical role identity. Codex writable roots and SDK readOnly derive from the pinned capability, not a literal Bot name. Repository source overrides prefer the logical role and otherwise inherit its capability's complete rules; the custom policy hash records both, while default policy bytes remain compatible. SDK subprocess contract tests inspect actual transmitted readOnly settings for a custom Developer and Reviewer; they are deterministic workers, not provider execution. + + +The actual DSH `--native-pools` probe now executes Developer → Tester return → original Developer → Tester → Chief inbox with one native resource. It verifies a fresh carrier restores the same physical Session/cwd and a marker present only in the prior conversation, two real code revisions, configured host tests and nonoverlapping released leases. Exact restoration explicitly unarchives the prior Session through the existing Workspace registry after the new binding is saved; DSH otherwise rejects archived Sessions before their first model step. The unused fresh carrier Session remains managed/tombstoned. Inference is deterministic; Chief closure/provider/forge completion are not claimed. + + +Native activation is separate from provisional binding. After the original Session bind +records the native resource worker, `native.activate` stages and persists an Effects +marker with the exact Task/Attempt/Session/binding/generation. Only confirmed commit +allows input or lifts archive admission. A resolving noncommitting save is not evidence; +its regression leaves the old Session archived and native input fenced. The existing +DSH pre-step hook also rejects unactivated, previous-host-epoch, stopping/released or +read-only native work, including startup/inbox paths that bypass ordinary delivery. +Unrelated personal Chief/group Bots retain their original admission behavior. No new +executor or polling timer is involved. Current-role in-flight abort/reconciliation and +unknown old-host exit remain separate tracked acceptance. + + +## Physical exit and host recovery (#2184) + +The actual pool host adapter now asks the shared execution-exit checker after release; +a missing Bot/unknown worker alone cannot transfer execution authority. SDK facts use +the existing confirmed-worker-exit contract plus a released lease. Native facts use the +matching quiescence record and released lease, or a persisted verified supervisor +process-group exit for a legacy lease. Physical exit does not invent recoverable context. + +New `project-native-worker/v2` records capture the existing host process scope even +without an explicit resource. Normal exit requests all registered work to stop, waits +for Agent idle, and checks Workspace activity for current/earlier context parts and +known subagent Sessions. Idle-child background jobs still block release. Native pre-step +admission follows durable subagent lineage and records its Session IDs; an obsolete or +unactivated ancestor cannot resume through a child. Missing legacy execution bindings +are fenced, while ordinary personal Chiefs/groups retain their original behavior. +Registered work still stopping leaves bounded kind/count facts and held capacity; +no timeout fabricates exit. The existing Task/Chief retry path can recheck after actual +work has settled. This is registered-platform work proof in the trusted host profile, +not a claim of adversarial OS isolation. + +For a previous host epoch, native stop never resumes an old Agent to infer safety. +It reuses `priorScopeExited` with the exact recorded systemd control-group/invocation +and validates lease/worker/generation before and after inspection. Matching proof +preserves latest context parts and updates the exact Attempt role Session metadata. +Unknown/unmanaged groups, missing/latest context or changed binding remain held. +Capacity may be released from an actually verified legacy supervisor fact while +context restoration still refuses an unknown binding. Reconciliation respects actual +host epoch rather than PID equality and preserves an already confirmed nested release. +Tests inject manager observations; they do not claim a real systemd restart acceptance. + + +## In-flight authority (#2184) + +Native bind/activate and canonical SDK start/input use the same current execution-authority check: exact Task/Attempt/claim/generation, current initiating human/delegation, committed role catalog and actual Resource binding. Frozen persona/Skills/model stay frozen; current capability/Harness/resource revocations limit that context. A removed role may finish only its exact retained active claim under archived conventions; it cannot acquire fresh work or borrow another retired identity. An implicit native host cannot bypass a newly explicit resource pool. + +The existing serialized scheduler checks ongoing canonical role Sessions before resource reconciliation and new dispatch; no new timer is installed. Native pre-step also checks current authority for the bound Agent/child lineage and asynchronously requests the existing pool stop so it never waits for its own idle inside pre-step. SDK safe next-turn feedback and human input/permission responses revalidate before writing frames. Already running SDK operations stop cooperatively via the existing acknowledgement + process-exit contract; this does not assert retroactive prevention of an external operation already executed. + +`projectPools.revokeExecution` is a private host seam, not a new public task endpoint. It fences the exact claim, persists a blocked reason, then invokes the same lifecycle release used for member revocation. Unknown exit keeps occupancy and prevents reassignment; a late result cannot move a revoked claim. Chief's configured inspection can observe and route the blocked Task after existing exit/authority gates; this path does not impersonate a human or automatically close a Task. Tests cover native step, existing scheduler wiring, stale claim, current catalog, prepared and live real SDK subprocesses using an explicit deterministic worker fixture. They do not imply a new real-provider workflow acceptance. + +When a physical native Session is resumed under a fresh carrier, ongoing checks select only the current host/live bound record, never an earlier historical binding for that same Session ID. SDK human input checks both initiating authority and the sending actor again after any awaited validation. These two boundaries have failure regressions; the actual DSH return probe reproduced the former and passes after correction. + +Current operator disabling of a Resource or removal of its Harness support also requests the same confirmed stop; it cannot be bypassed by a retained role grant. Live deterministic SDK subprocess regression covers both role-pool removal and operator disable. diff --git a/docs/subsystems/project-pools.md b/docs/subsystems/project-pools.md index f4fb2d46c..6c2dd864d 100644 --- a/docs/subsystems/project-pools.md +++ b/docs/subsystems/project-pools.md @@ -121,3 +121,6 @@ Each offered handoff saves its exact pool.dispatch intent in the same state snap ## Durable Task inbox integration (S3) S3 stores a handoff input with each pool.dispatch offer, rolls it back only with an uncommitted offer and migrates old pending offers without changing ids. Current-claim injected-batch acknowledgements follow successful completed-role gates. Routing receipts resolve handoff transport without manufacturing a model read; remaining feedback follows current responsibility. Closure checks unhandled inputs. See [inputs](project-inputs.md). + + +#2184 records logical role and pinned capability class on a claim. Active targets alone can receive new offers/claims; a removed queued target becomes blocked with a Chief-routing reason while its Task/initiative/history remain intact. An already acquired source may finish its retained frozen claim after retirement, then explicitly hand off to an active target. Developer-capability destinations use existing rework semantics even when named differently. Artifact responsibilities stay in the existing Attempt's roleAssignments; no new queue or scheduler is added. diff --git a/docs/subsystems/project-resources.md b/docs/subsystems/project-resources.md index 681e3a735..eec034a1a 100644 --- a/docs/subsystems/project-resources.md +++ b/docs/subsystems/project-resources.md @@ -131,3 +131,12 @@ released lease. Current owned controllers avoid unnecessary supervisor I/O. Confirmed releases record their evidence kind. SDK releases carry bound worker and generation proof from stop acknowledgement plus process exit; unknown releases still retain capacity. Historical matching SDK released records remain host facts. + + +Native resource release now requires a matching projectNativeWorkers native-agent-quiescent record, generation/Session/lease scope and absent old Bot. Missing Bot records or undefined handles with exited=true cannot free capacity. The existing native-turn delegation path still requires its actual nonempty matching handle. Actual stopping uses the original SessionController-resolved Agent and awaits whenIdle; unknown prior-host execution remains held for explicit reconciliation. + + +#2184 separates physical release from context recovery. Native normal release needs v2 registered-work quiescence; supervisor release additionally verifies the exact process-group/invocation through the existing manager proof service. A legacy missing carrier can free capacity after actual manager verification without manufacturing native context. Reconciliation preserves a nested confirmed release and uses native host-epoch ownership rather than PID equality to recognize current controllers. No missing PID or timeout is a release proof. + + +Removing a current role resource grant or changing its registered Resource binding revokes ongoing canonical execution at the existing scheduler check. The same SDK/native stop and physical-exit proof govern release; current permissions never justify a silent new Session or an assumed exit. diff --git a/docs/subsystems/workflow.md b/docs/subsystems/workflow.md index 67cd329be..d46afe675 100644 --- a/docs/subsystems/workflow.md +++ b/docs/subsystems/workflow.md @@ -174,3 +174,6 @@ Fork M7.3 lifecycle first offers spawn/dispatch/release to the host SDK adapter, Repair R3 (#36): Project task pools now enforce the existing pure handoff validator via the trusted host adapter and reuse spawn/dispatch/release and result interception. New Project deliveries do not instantiate a static Workflow graph. `projectDispatch: workflow` retains explicit legacy graph execution and fixtures. See [pool contract](project-pools.md). + + +#2184 fixes native lifecycle input delivery: structured inputs use JSON rather than implicit [object Object] conversion, preserving handoff IDs/evidence and false/zero values. Distinct returned handoffs therefore produce distinct content for the existing delivery deduplicator, while an exact repeated dispatch remains stable. The regression uses the original plugin lifecycle seam; the real DSH native-pool probe reproduces and guards same-Session return delivery. No alternate Workflow or scheduler is introduced. diff --git a/patches/ds-bot/0001-core-fixes.patch b/patches/ds-bot/0001-core-fixes.patch index 7cfcd4f1b..a11efd588 100644 --- a/patches/ds-bot/0001-core-fixes.patch +++ b/patches/ds-bot/0001-core-fixes.patch @@ -9716,7 +9716,7 @@ diff --git a/test/images.test.js b/test/images.test.js diff --git a/test/install.test.js b/test/install.test.js --- a/test/install.test.js +++ b/test/install.test.js -@@ -33,10 +33,24 @@ test('apply registers listeners, tools, and effects in a fixed order', (t) => { +@@ -33,10 +33,26 @@ test('apply registers listeners, tools, and effects in a fixed order', (t) => { const { ctx, stop, applyResult } = start() t.after(stop) assert.equal(applyResult, undefined) @@ -9724,6 +9724,8 @@ diff --git a/test/install.test.js b/test/install.test.js assert.deepEqual(ctx.calls, [ 'effect ds-bot: state lock', + 'effect ds-bot: Project effects', ++ // Native activation/epoch admission precedes every model step and stream observer. ++ 'on agent/pre-step', + + // Project streaming is observed before the session/delegation guards. + 'on llm/stream', diff --git a/patches/ds-bot/files/src/host/hooks.js b/patches/ds-bot/files/src/host/hooks.js index 17537f3b1..378baa907 100644 --- a/patches/ds-bot/files/src/host/hooks.js +++ b/patches/ds-bot/files/src/host/hooks.js @@ -222,7 +222,11 @@ export function install(rt) { } if (record && (stepTools.length > 0 || caps.length > 0 || preset || executionProject)) await rt.save() if (executionWorkspace) { - try { await rt.projectDelivery?.bindSession(executionWorkspace, { id: bot.id, sessionId: rt.chatOf(bot.id) }) } + try { + if (roleSession) await rt.projectDelivery?.nativeSessions?.bind(roleSession, executionWorkspace, bot.id) + await rt.projectDelivery?.bindSession(executionWorkspace, { id: bot.id, sessionId: rt.chatOf(bot.id) }) + if (roleSession) await rt.projectDelivery?.nativeSessions?.activate(roleSession, bot.id) + } catch (error) { await rt.deleteBot(bot.id); throw error } } for (const [key, value] of Object.entries(fields ?? {})) fire('change', { botId: bot.id, plugin, key, value: clone(value), prev: undefined }) @@ -255,6 +259,7 @@ export function install(rt) { const last = view(rt.botOf(id)) const remote = rt.botOf(id)?.remote const remoteChat = remote !== undefined + await rt.projectDelivery?.nativeSessions?.stop(id) const result = await deleteBot(id, ...rest) await rt.projectDelivery?.releaseSession(id) if (!remoteChat) { @@ -787,6 +792,7 @@ export function install(rt) { } innerDeliver = rt.deliver rt.deliver = async (args) => { + if (rt.projectDelivery?.nativeSessions?.admits(args.toId) === false) throw Object.assign(new Error('Native execution is not admitted; confirm its durable activation or stop/recovery state before new input'), { code: 'bot/worker-stopping' }) // Everything that is not a first, checked send still carries its sender: resumed, retried, // deferred and re-sent deliveries (and notices) are judged by the owner policy // now, since the Bot they name may have had its grant revoked, replaced or removed since. @@ -1396,7 +1402,7 @@ export function install(rt) { }, async release(id) { await lifecycle.release(id) - return !rt.botOf(id) && (!rt.state.projectHarnessWorkers?.[id] || rt.state.projectHarnessWorkers[id].status === 'released') + return rt.projectDelivery.exitConfirmed(id) }, evidence({ attemptId }) { const attempt = rt.state.projectAttempts?.[attemptId] diff --git a/patches/ds-bot/files/src/host/project-delivery-template.js b/patches/ds-bot/files/src/host/project-delivery-template.js index c94f3a22e..b937c3d90 100644 --- a/patches/ds-bot/files/src/host/project-delivery-template.js +++ b/patches/ds-bot/files/src/host/project-delivery-template.js @@ -1,5 +1,5 @@ // One versioned path through the existing Workflow plugin; no second execution engine. -export const DELIVERY_VERSION = 'project-delivery/v6' +export const DELIVERY_VERSION = 'project-delivery/v7' export const DELIVERY_ROLES = ['developer', 'tester', 'reviewer'] export function deliveryGraph(model = '') { return { diff --git a/patches/ds-bot/files/src/host/project-delivery.js b/patches/ds-bot/files/src/host/project-delivery.js index 13399b0f9..7692f9a8c 100644 --- a/patches/ds-bot/files/src/host/project-delivery.js +++ b/patches/ds-bot/files/src/host/project-delivery.js @@ -11,11 +11,16 @@ import { createResourceWorkspaces, fileDigest } from './project-resource-workspa import { ROLE_POLICY_FILES, assembleRolePolicy, verifyRolePolicy, deliveryReviewReport, checkDevelopmentEvidence } from './project-role-policy.js' import { roleContext, contextHash } from './project-role-context.js' import { githubPublisher } from './project-github.js' -import { taskSessionId, confirmedWorkerExit } from './project-task-sessions.js' +import { taskSessionId, confirmedWorkerExit, pinTaskRoleDefinition, discardUnpreparedTaskDefinitions } from './project-task-sessions.js' +import { reconcileExecutionAuthority } from './project-execution-authority.js' +import { executionExitConfirmed } from './project-execution-exit.js' +import { createProjectNativeSessions, nativeExitConfirmed } from './project-native-sessions.js' import { testFailureExcerpt } from './project-test-diagnostics.js' import { EffectCancelled, EffectDeferred } from './project-effects.js' const exec = promisify(execFile) +const deliveryRole = (attempt, capability) => attempt.roleAssignments?.[capability] ?? capability +const workspaceFor = (attempt, capability) => attempt.workspaces[deliveryRole(attempt, capability)] const hash = value => createHash('sha256').update(value).digest('hex') const ongoing = new Set(['preparing', 'running']) const reject = message => Object.assign(new Error(message), { code: 'bot/delivery' }) @@ -34,6 +39,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { const { state, config } = rt const root = resolve(config.projectWorkspaceRoot ?? join(dirname(rt.dshHome), 'flybot-workspaces')) const artifacts = resolve(rt.home, 'project-artifacts') + const nativeSessions = createProjectNativeSessions(rt) const resourceWorkspaces = createResourceWorkspaces(rt, { hostGit: git, artifacts }) const workspaceGit = (workspace, args) => { if (workspace.resourceLease) rt.projectResources.requireBinding(workspace.projectId, workspace.resourceLease) @@ -71,7 +77,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { await verifyCandidate(attempt) const tests = attempt.evidence.tests, review = attempt.evidence.review if (!tests?.passed || tests.sha !== attempt.candidateSha || review?.sha !== attempt.candidateSha || review.verdict !== 'approve' || await fileDigest(tests.log) !== tests.sha256) throw new EffectCancelled('Publication evidence is missing or stale') - for (const role of DELIVERY_ROLES) await verifyExecution(attempt, role) + for (const role of DELIVERY_ROLES) await verifyExecution(attempt, deliveryRole(attempt, role)) context.assertCurrent() if (task.deliveryAttemptId !== attempt.id || attempt.cancelRequested || attempt.candidateSha !== effect.payload.candidateSha) throw new EffectCancelled('Publication changed during validation') } @@ -162,7 +168,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { for (const lease of Object.values(attempt.resourceLeases ?? {})) await rt.projectResources?.release(lease.id, lease.generation) await rt.save() } - async function allocate(attempt, project, settings, roles = DELIVERY_ROLES) { + async function allocate(attempt, project, settings, roles = DELIVERY_ROLES, capabilities = {}) { const repository = await realpath(settings.path) attempt.baseSha ??= await git(repository, ['rev-parse', 'HEAD']) attempt.branch = `flybot/delivery-${attempt.id.slice(-12)}` @@ -171,32 +177,37 @@ export function createProjectDelivery(rt, { publisher } = {}) { for (const role of roles) { if (attempt.workspaces[role]) { if (attempt.workspaces[role].status !== 'allocated') throw reject('Workspace allocation is incomplete; reconcile before retry'); continue } if (attempt.cancelRequested) throw reject('Delivery was stopped') + const policyRole = capabilities[role] ?? attempt.roleSessions?.[role]?.policyRole ?? Object.keys(attempt.roleAssignments ?? {}).find(capability => attempt.roleAssignments[capability] === role) ?? role const lease = attempt.resourceLeases?.[role] const resource = lease ? rt.projectResources.requireBinding(project.id, lease) : null const workspaceRoot = resource?.root ?? root const logicalSessionId = taskSessionId(project.id, attempt.taskId, role) - const reusable = lease && lease.harness !== 'flybot' ? Object.values(state.projectWorkspaces ?? {}).findLast(row => row.taskSessionId === logicalSessionId && row.resourceId === lease.resourceId && row.repository === repository && row.resourceRoot === workspaceRoot && row.status === 'allocated' && row.attemptId !== attempt.id) : null + const reusable = (lease && lease.harness !== 'flybot' || attempt.protocol === 'project-handoff/v1') ? Object.values(state.projectWorkspaces ?? {}).findLast(row => row.taskSessionId === logicalSessionId && (row.resourceId ?? null) === (lease?.resourceId ?? null) && row.repository === repository && row.resourceRoot === workspaceRoot && row.status === 'allocated' && row.attemptId !== attempt.id) : null if (reusable) { const previous = ownEntry(attempts(), reusable.attemptId) const workers = Object.values(state.projectHarnessWorkers ?? {}).filter(row => row.attemptId === reusable.attemptId && row.role === role) const leases = Object.values(state.projectResourceLeases ?? {}).filter(row => row.attemptId === reusable.attemptId && row.role === role) + const native = Object.values(state.projectNativeWorkers ?? {}).filter(row => row.attemptId === reusable.attemptId && row.role === role) + const liveNative = Object.values(state.bots ?? {}).some(row => row.ephemeral && row.attemptId === reusable.attemptId && row.workspaceId === reusable.id) + if (liveNative || native.some(row => !nativeExitConfirmed(state, row))) throw reject('Native Task workspace is still held by previous execution') if (!previous || previous.projectId !== project.id || previous.taskId !== attempt.taskId || ongoing.has(previous.status) || previous.status === 'waiting-acceptance' || workers.some(row => !confirmedWorkerExit(state, row)) || leases.some(row => row.status !== 'released')) throw reject('Task workspace is still held by previous execution') const dirty = await workspaceGit(reusable, ['status', '--porcelain']) const head = await workspaceGit(reusable, ['rev-parse', 'HEAD']) - if (head !== (previous.candidateSha ?? previous.seedSha ?? previous.baseSha) || dirty && (role !== 'developer' || previous.candidateSha)) throw reject('Task workspace changed after its checkpoint; preserve it before reuse') + if (head !== (previous.candidateSha ?? previous.seedSha ?? previous.baseSha) || dirty && (policyRole !== 'developer' || previous.candidateSha)) throw reject('Task workspace changed after its checkpoint; preserve it before reuse') reusable.retainedDraft = Boolean(dirty) } const path = reusable?.path ?? (resource ? join(workspaceRoot, 'delivery', attempt.id, role) : join(root, attempt.id, role)) if (!resource || resource.type === 'local') await mkdir(resource ? join(workspaceRoot, 'delivery', attempt.id) : join(root, attempt.id), { recursive: true, mode: 0o700 }) - const workspace = { id: `workspace-${randomUUID()}`, projectId: project.id, attemptId: attempt.id, role, path, repository, taskSessionId: logicalSessionId, ...(reusable ? { reusesWorkspaceId: reusable.id, storageAttemptId: reusable.storageAttemptId ?? reusable.attemptId, retainedDraft: reusable.retainedDraft } : {}), + const workspace = { id: `workspace-${randomUUID()}`, projectId: project.id, attemptId: attempt.id, role, policyRole, path, repository, taskSessionId: logicalSessionId, ...(reusable ? { reusesWorkspaceId: reusable.id, storageAttemptId: reusable.storageAttemptId ?? reusable.attemptId, retainedDraft: reusable.retainedDraft } : {}), kind: resource?.type === 'docker' ? 'resource-clone' : 'worktree', resourceId: lease?.resourceId ?? null, resourceRoot: workspaceRoot, ...(lease ? { resourceLease: lease } : {}), status: 'allocating', createdAt: Date.now() } setEntry(state.projectWorkspaces, workspace.id, workspace) attempt.workspaces[role] = workspace await rt.save() // Persist cleanup intent before git creates the worktree. - const args = role === 'developer' ? ['worktree', 'add', '-b', attempt.branch, path, attempt.baseSha] : ['worktree', 'add', '--detach', path, attempt.baseSha] + const branch = role === 'developer' ? attempt.branch : `${attempt.branch}-${role}` + const args = policyRole === 'developer' ? ['worktree', 'add', '-b', branch, path, attempt.baseSha] : ['worktree', 'add', '--detach', path, attempt.baseSha] if (reusable) { if (workspace.kind === 'worktree' && await realpath(path) !== path) throw reject('Task workspace resolves through a symlink') - if (role === 'developer' && workspace.kind === 'worktree') await workspaceGit(workspace, ['switch', '-c', attempt.branch]) + if (policyRole === 'developer' && workspace.kind === 'worktree') await workspaceGit(workspace, ['switch', '-c', branch]) reusable.status = 'retained' } else if (workspace.kind === 'resource-clone') await resourceWorkspaces.allocate(workspace, attempt) else await git(repository, args) @@ -239,7 +250,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { attempt.agentDefinitions = definitions if (config.projectDispatch !== 'workflow') { attempt.protocol = 'project-handoff/v1' - attempt.workflowVersion = 'project-delivery/pools-v3' + attempt.workflowVersion = 'project-delivery/pools-v4' attempt.baseSha ??= await git(settings.path, ['rev-parse', 'HEAD']) attempt.roleSessions = {}; attempt.rolePolicies = {}; attempt.resourceLeases = {} attempt.runId = `pool-run-${randomUUID()}` @@ -249,13 +260,14 @@ export function createProjectDelivery(rt, { publisher } = {}) { await rt.save() return { attempt: view(attempt) } } - attempt.resourceLeases = await rt.projectResources?.reserveRoles(project.id, attempt.id, definitions) ?? {} + const reservedDefinitions = definitions ? { ...definitions, roles: definitions.roles.map(definition => definition.role === 'chief' ? definition : pinTaskRoleDefinition(state, { projectId: project.id, taskId: task.id, role: definition.role }, definition, definitions.revision, rt.projectAgents.deliveryDefinitions(project.id)).definition) } : definitions + attempt.resourceLeases = await rt.projectResources?.reserveRoles(project.id, attempt.id, reservedDefinitions) ?? {} for (const lease of Object.values(attempt.resourceLeases)) { const resource = rt.projectResources.setting(project.id, lease.resourceId) if (lease.harness === 'flybot' && resource.type !== 'local') throw reject('Native Harness requires a registered local resource') } await allocate(attempt, project, settings) - if (attempt.seedSha) await workspaceGit(attempt.workspaces.developer, ['reset', '--hard', attempt.seedSha]) + if (attempt.seedSha) await workspaceGit(workspaceFor(attempt, 'developer'), ['reset', '--hard', attempt.seedSha]) if (attempt.cancelRequested) throw reject('Delivery was stopped') const model = settings.model ?? config.projectModel ?? await rt.defaultModel?.() if (typeof model !== 'string' || !model) throw reject('A served delivery model must be provisioned') @@ -265,7 +277,9 @@ export function createProjectDelivery(rt, { publisher } = {}) { attempt.rolePolicies = {} for (const node of graph.nodes) { const role = definitions?.roles.find(role => role.role === node.id) - const definition = role ?? { instructions: node.prompt, resolvedSkills: [], memory: [], execution: { harness: 'flybot', resourceIds: [] } } + let definition = role ?? { instructions: node.prompt, resolvedSkills: [], memory: [], execution: { harness: 'flybot', resourceIds: [] } } + const pinned = pinTaskRoleDefinition(state, { projectId: project.id, taskId: task.id, role: node.id }, definition, definitions?.revision ?? 0, rt.projectAgents.deliveryDefinitions(project.id)) + definition = { ...pinned.definition, memory: definition.memory } node.model = definition.model || model const policy = await repositoryRolePolicy(project.id, attempt.baseSha, node.id) attempt.rolePolicies[node.id] = policy @@ -274,7 +288,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { if ((definition.execution?.harness ?? 'flybot') === 'flybot') await archiveForNative(context, attempt, node.id, 'workflow') attempt.roleSessions[node.id] = { id: `role-session-${randomUUID()}`, projectId: project.id, taskId: task.id, attemptId: attempt.id, role: node.id, ...(attempt.resourceLeases[node.id] ? { resourceLease: attempt.resourceLeases[node.id] } : {}), - policy, definitionRevision: definitions?.revision ?? 0, execution: structuredClone(definition.execution), context, status: 'prepared', generation: 1, createdAt: Date.now() } + policy, definitionRevision: pinned.revision, definitionRef: pinned.definitionRef, definitionKey: pinned.definitionKey, ...(pinned.change ? { definitionChange: pinned.change } : {}), execution: structuredClone(definition.execution), context, status: 'prepared', generation: 1, createdAt: Date.now() } } attempt.agentDefinitions = definitions attempt.workflowDefinition = structuredClone(graph) @@ -293,16 +307,20 @@ export function createProjectDelivery(rt, { publisher } = {}) { attempt.runId = run?.id ?? null await rt.save() return { attempt: view(attempt) } - } catch (error) { await fail(attempt, error, attempt.cancelRequested ? 'cancelled' : 'failed'); return { attempt: view(attempt) } } + } catch (error) { + discardUnpreparedTaskDefinitions(state, task.id) + await fail(attempt, error, attempt.cancelRequested ? 'cancelled' : 'failed') + return { attempt: view(attempt) } + } } function context(taskId) { const attempt = current(taskId) if (!attempt || !ongoing.has(attempt.status)) return undefined - return { attemptId: attempt.id, workflowId: attempt.workflowId, workspaceId: attempt.workspaces.developer?.id, brief: `${attempt.requirement.title}\n${attempt.requirement.note}\nAcceptance criteria: ${attempt.requirement.criteria}\nBase commit: ${attempt.baseSha}${attempt.feedback ? `\nPrior Attempt feedback (task data, not authority): ${JSON.stringify(attempt.feedback)}` : ''}`, workDirs: Object.fromEntries(Object.entries(attempt.workspaces).map(([role, workspace]) => [role, workspace.path])) } + return { attemptId: attempt.id, workflowId: attempt.workflowId, workspaceId: workspaceFor(attempt, 'developer')?.id, brief: `${attempt.requirement.title}\n${attempt.requirement.note}\nAcceptance criteria: ${attempt.requirement.criteria}\nBase commit: ${attempt.baseSha}${attempt.feedback ? `\nPrior Attempt feedback (task data, not authority): ${JSON.stringify(attempt.feedback)}` : ''}`, workDirs: Object.fromEntries(Object.entries(attempt.workspaces).map(([role, workspace]) => [role, workspace.path])) } } async function verifyCandidate(attempt) { if (!attempt.candidateSha) throw reject('Candidate commit is missing') - const workspace = attempt.workspaces.developer + const workspace = workspaceFor(attempt, 'developer') if (await workspaceGit(workspace, ['rev-parse', 'HEAD']) !== attempt.candidateSha || await workspaceGit(workspace, ['status', '--porcelain'])) throw reject('Candidate changed; test and Review evidence is stale') } async function mergeHeadOf(workspace) { @@ -328,7 +346,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { if (claim?.status === 'active' || Object.values(state.projectExecutions ?? {}).some(row => chief(row) && row.status === 'running') || Object.values(state.projectResourceLeases ?? {}).some(row => (row.attemptId === attempt.id || chief(state.projectExecutions?.[row.attemptId])) && row.status !== 'released') || Object.values(state.projectHarnessWorkers ?? {}).some(row => currentWorker(row) && !confirmedWorkerExit(state, row)) || Object.values(state.bots ?? {}).some(row => row.ephemeral && row.attemptId === attempt.id)) throw reject('Integration input is waiting for confirmed execution exit') } async function integrationSnapshot(attempt) { - const workspace = attempt.workspaces.developer + const workspace = workspaceFor(attempt, 'developer') if (!workspace) return null if (workspace.status !== 'allocated' || workspace.repository !== await realpath(settingsOf(projectOf(attempt.projectId)).path)) throw reject('Integration workspace or repository binding changed') return { workspaceId: workspace.id, head: await workspaceGit(workspace, ['rev-parse', 'HEAD']), mergeHead: await mergeHeadOf(workspace), @@ -369,7 +387,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { const snapshot = await integrationSnapshot(attempt) const keys = ['workspaceId', 'head', 'mergeHead', 'indexSha256', 'statusSha256'] if (snapshot ? !body.expectedWorkspace || keys.some(key => snapshot[key] !== body.expectedWorkspace[key]) : body.expectedWorkspace !== null) throw reject('Integration workspace changed after the operator snapshot') - if (snapshot && await workspaceGit(attempt.workspaces.developer, ['ls-files', '--unmerged'])) throw reject('Integration has unresolved paths; preserve the index for the Developer') + if (snapshot && await workspaceGit(workspaceFor(attempt, 'developer'), ['ls-files', '--unmerged'])) throw reject('Integration has unresolved paths; preserve the index for the Developer') await verifyIntegrationInputs({ ...attempt, integrationInputs: desired }, { admissionRequired: false }) integrationReleased(attempt); rt.requireProjectAccess(actor, task.projectId, 'configure'); rt.projects.requireActive(task.projectId) if ((task.collaboration?.generation ?? 0) !== body.generation || task.deliveryAttemptId !== attempt.id || attempt.candidateSha || attempt.cancelRequested) throw reject('Integration admission changed during validation') @@ -405,7 +423,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { } async function seal(attempt) { const settings = settingsOf(projectOf(attempt.projectId)) - const workspace = attempt.workspaces.developer + const workspace = workspaceFor(attempt, 'developer') const parent = attempt.seedSha ?? attempt.baseSha, upstream = attempt.publicationBinding?.upstreamSha const approved = await verifyIntegrationInputs(attempt), mergeHead = await mergeHeadOf(workspace) const trusted = [...new Set([upstream, ...approved.map(row => row.sha)].filter(Boolean))] @@ -418,7 +436,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { const info = await lstat(path).catch(error => { if (error.code === 'ENOENT') return null; throw error }) if (info?.isSymbolicLink() && !inside(workspace.path, await realpath(path))) throw reject('Code changes contain a symbolic link outside the allocated workspace') } - checkDevelopmentEvidence(attempt.rolePolicies?.developer, names) + checkDevelopmentEvidence(attempt.rolePolicies?.[deliveryRole(attempt, 'developer')], names) const roots = settings.allowedPaths if (!Array.isArray(roots) || !roots.length || roots.some(value => typeof value !== 'string' || value.startsWith('/') || value.includes('..')) || names.some(name => !roots.some(prefix => name === prefix || name.startsWith(prefix.endsWith('/') ? prefix : `${prefix}/`)))) throw reject('Code changes exceed the provisioned repository paths') const ancestor = async (a, b) => { try { await workspaceGit(workspace, ['merge-base', '--is-ancestor', a, b]); return true } catch (error) { if (error.code === 1) return false; throw error } } @@ -446,9 +464,9 @@ export function createProjectDelivery(rt, { publisher } = {}) { attempt.candidateSha = await workspaceGit(workspace, ['rev-parse', 'HEAD']) if (attempt.candidateSha === attempt.baseSha) throw reject('Developer produced no code change') if (workspace.kind === 'resource-clone') await resourceWorkspaces.importCandidate(workspace, attempt) - attempt.evidence.implementation = { sha: attempt.candidateSha, paths: names, diffStat: await workspaceGit(workspace, ['diff', '--stat', attempt.baseSha, attempt.candidateSha]), at: Date.now() } + attempt.evidence.implementation = { logicalRole: workspace.role, roleSessionId: attempt.roleSessions?.[workspace.role]?.id ?? null, sha: attempt.candidateSha, paths: names, diffStat: await workspaceGit(workspace, ['diff', '--stat', attempt.baseSha, attempt.candidateSha]), at: Date.now() } for (const role of ['tester', 'reviewer', ...(attempt.workspaces['host-verifier'] ? ['host-verifier'] : [])]) { - const target = attempt.workspaces[role] + const target = attempt.workspaces[deliveryRole(attempt, role)] if (!target) continue if (target.kind === 'resource-clone') await resourceWorkspaces.checkout(target, attempt) else { await workspaceGit(target, ['reset', '--hard', attempt.candidateSha]); await workspaceGit(target, ['clean', '-ffdx']) } @@ -457,7 +475,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { async function testCandidate(attempt) { await verifyCandidate(attempt) const settings = settingsOf(projectOf(attempt.projectId)) - const workspace = attempt.workspaces['host-verifier'] ?? attempt.workspaces.tester + const workspace = attempt.workspaces['host-verifier'] ?? workspaceFor(attempt, 'tester') const path = workspace.path await workspaceGit(workspace, ['reset', '--hard', attempt.candidateSha]) await workspaceGit(workspace, ['clean', '-ffdx']) @@ -473,24 +491,24 @@ export function createProjectDelivery(rt, { publisher } = {}) { await mkdir(join(artifacts, attempt.id), { recursive: true, mode: 0o700 }) const log = join(artifacts, attempt.id, 'tests.log') await writeFile(log, output, { mode: 0o600 }) - attempt.evidence.tests = { executor: 'independent-host-controller', workspaceId: workspace.id, resourceId: workspace.resourceId, sha: attempt.candidateSha, passed, command: settings.testCommand, log, sha256: hash(output), at: Date.now() } + attempt.evidence.tests = { logicalRole: deliveryRole(attempt, 'tester'), roleSessionId: attempt.roleSessions?.[deliveryRole(attempt, 'tester')]?.id ?? null, executor: 'independent-host-controller', workspaceId: workspace.id, resourceId: workspace.resourceId, sha: attempt.candidateSha, passed, command: settings.testCommand, log, sha256: hash(output), at: Date.now() } if (!passed) throw reject('Configured tests failed; see Attempt evidence') } async function review(attempt, outputs) { await verifyCandidate(attempt) - const workspace = attempt.workspaces.reviewer + const workspace = workspaceFor(attempt, 'reviewer') if (await workspaceGit(workspace, ['rev-parse', 'HEAD']) !== attempt.candidateSha || await workspaceGit(workspace, ['status', '--porcelain'])) throw reject('Reviewer did not inspect the clean candidate commit') let result try { result = typeof outputs?.review === 'string' ? JSON.parse(outputs.review) : outputs?.review } catch { throw reject('Review output must be commit-specific JSON') } if (!result || result.commit !== attempt.candidateSha || !['approve', 'request-changes'].includes(result.verdict) || typeof result.summary !== 'string' || !result.summary.trim()) throw reject('Review verdict is missing or names a different commit') - const policy = attempt.rolePolicies?.reviewer - const execution = attempt.evidence.execution?.reviewer + const policy = attempt.rolePolicies?.[deliveryRole(attempt, 'reviewer')] + const execution = attempt.evidence.execution?.[deliveryRole(attempt, 'reviewer')] const commands = [] if (execution?.log) for (const line of (await readFile(execution.log, 'utf8')).split('\n').filter(Boolean)) { const item = JSON.parse(line).event?.item if (item?.type === 'command_execution' && typeof item.exit_code === 'number') commands.push({ command: item.command, exitCode: item.exit_code, executor: 'reviewer-sdk' }) } - attempt.evidence.review = { ...(policy ? { policySha256: policy.sha256, report: deliveryReviewReport({ verdict: result.verdict, sha: result.commit, commands }) } : {}), sha: result.commit, verdict: result.verdict, summary: result.summary.slice(0, 8000), at: Date.now() } + attempt.evidence.review = { logicalRole: workspace.role, roleSessionId: attempt.roleSessions?.[workspace.role]?.id ?? null, ...(policy ? { policySha256: policy.sha256, report: deliveryReviewReport({ verdict: result.verdict, sha: result.commit, commands }) } : {}), sha: result.commit, verdict: result.verdict, summary: result.summary.slice(0, 8000), at: Date.now() } if (result.verdict !== 'approve') throw reject('Reviewer requested changes') if (attempt.evidence.tests?.sha !== attempt.candidateSha || !attempt.evidence.tests.passed) throw reject('Matching passing tests are missing') if (attempt.cancelRequested) throw reject('Delivery was stopped') @@ -527,8 +545,8 @@ export function createProjectDelivery(rt, { publisher } = {}) { await verifyCandidate(attempt) for (const [role, policy] of Object.entries(attempt.rolePolicies ?? {})) verifyRolePolicy(policy, attempt.roleSessions[role]?.context.sources.policy?.sha256) for (const role of DELIVERY_ROLES) { - if (!attempt.roleSessions?.[role]) throw reject('Publication retry is missing the original role Session') - await verifyExecution(attempt, role) + if (!attempt.roleSessions?.[deliveryRole(attempt, role)]) throw reject('Publication retry is missing the original role Session') + await verifyExecution(attempt, deliveryRole(attempt, role)) } const tests = attempt.evidence.tests, review = attempt.evidence.review if (!tests?.passed || tests.sha !== attempt.candidateSha || review?.sha !== attempt.candidateSha || review.verdict !== 'approve') throw reject('Matching passing tests and independent Review are required') @@ -598,7 +616,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { if (task.title !== attempt.requirement.title || (task.criteria ?? '') !== attempt.requirement.criteria) throw reject('Requirements changed; delivery evidence is stale') if (!attempt.runId || ownEntry(state.projectExecutions, attempt.runId)?.status !== 'done') throw reject('Delivery Run has not finished successfully') if (!attempt.pr || !attempt.evidence.tests?.passed || attempt.evidence.tests.sha !== attempt.candidateSha || attempt.evidence.review?.sha !== attempt.candidateSha || attempt.evidence.review.verdict !== 'approve') throw reject('Delivery evidence is missing or stale') - for (const role of DELIVERY_ROLES) await verifyExecution(attempt, role) + for (const role of DELIVERY_ROLES) await verifyExecution(attempt, deliveryRole(attempt, role)) const testLog = await readFile(attempt.evidence.tests.log, 'utf8') if (hash(testLog) !== attempt.evidence.tests.sha256) throw reject('Test log changed; evidence is stale') const pr = await publisherOf(projectOf(attempt.projectId)).read(attempt.pr) @@ -757,48 +775,58 @@ export function createProjectDelivery(rt, { publisher } = {}) { const attempt = ownEntry(attempts(), claim.attemptId) if (!attempt || attempt.taskId !== task.id || attempt.protocol !== 'project-handoff/v1' || attempt.status !== 'running') throw reject('Pool Attempt is unavailable') const role = definition.role, project = projectOf(task.projectId) - const settings = await preflightRepository(rt, project) - if (role === 'developer') await refreshUnstartedPublication(attempt, project, claim) - const policy = await repositoryRolePolicy(project.id, attempt.baseSha, role) - const sessionId = `role-session-${randomUUID()}` - const execution = definition.execution - const lease = execution.resourceIds?.length ? await rt.projectResources.reserve({ projectId: project.id, attemptId: attempt.id, role, sessionKey: sessionId, ...execution }) : null - if (lease) attempt.resourceLeases[role] = lease - let context, workspace + const pinned = pinTaskRoleDefinition(state, { projectId: project.id, taskId: task.id, role }, definition, attempt.agentDefinitions.revision, rt.projectAgents.deliveryDefinitions(project.id)) try { - const freshWorkspace = !attempt.workspaces[role] - await allocate(attempt, project, settings, [role]) - workspace = attempt.workspaces[role] - if (lease) workspace.resourceLease = lease - if (attempt.seedSha && role === 'developer' && !attempt.candidateSha && freshWorkspace && !workspace.retainedDraft) await workspaceGit(workspace, ['reset', '--hard', attempt.seedSha]) - if (attempt.candidateSha) { - if (workspace.kind === 'resource-clone') await resourceWorkspaces.checkout(workspace, attempt) - else await workspaceGit(workspace, ['reset', '--hard', attempt.candidateSha]) + definition = { ...pinned.definition, memory: definition.memory } + const policyRole = definition.policyRole ?? role + if (!DELIVERY_ROLES.includes(policyRole)) throw reject('Task worker requires an approved execution capability') + const settings = await preflightRepository(rt, project) + if (policyRole === 'developer') await refreshUnstartedPublication(attempt, project, claim) + const policy = await repositoryRolePolicy(project.id, attempt.baseSha, role, policyRole) + const sessionId = `role-session-${randomUUID()}` + const execution = definition.execution + const lease = execution.resourceIds?.length ? await rt.projectResources.reserve({ projectId: project.id, attemptId: attempt.id, role, sessionKey: sessionId, ...execution }) : null + if (lease) attempt.resourceLeases[role] = lease + let context, workspace + try { + const freshWorkspace = !attempt.workspaces[role] + await allocate(attempt, project, settings, [role], { [role]: policyRole }) + workspace = attempt.workspaces[role] + if (lease) workspace.resourceLease = lease + if (attempt.seedSha && policyRole === 'developer' && !attempt.candidateSha && freshWorkspace && !workspace.retainedDraft) await workspaceGit(workspace, ['reset', '--hard', attempt.seedSha]) + if (attempt.candidateSha) { + if (workspace.kind === 'resource-clone') await resourceWorkspaces.checkout(workspace, attempt) + else await workspaceGit(workspace, ['reset', '--hard', attempt.candidateSha]) + } + await prepareWorkspace(attempt, workspace, settings) + await prepareIntegrations(attempt, workspace) + context = roleContext({ retrieval: true, policy, projectId: project.id, role, revision: pinned.revision, definition, memory: definition.memory, knowledge: ownEntry(state.projectKnowledge ?? {}, project.id) ?? [], requirement: attempt.requirement, feedback: attempt.feedback, projectFacts: { ...(pinned.change ? { roleDefinitionChange: pinned.change } : {}), integrationInputs: structuredClone(attempt.integrationInputs ?? []), testFailure: await failureContext(attempt), publicationBinding: attempt.publicationBinding ?? null, integrationInstruction: attempt.publicationBinding ? 'Merge the pinned upstreamSha into the published candidate using --no-commit, resolve and stage conflicts before handoff. Host sealing retains both trusted inputs; never fabricate parents or discard backend changes.' : null, workingDraft: !attempt.candidateSha && workspaceFor(attempt, 'developer') ? { kind: 'unverified-workspace-reference', resourceId: workspaceFor(attempt, 'developer').resourceId, path: workspaceFor(attempt, 'developer').resourceId === workspace.resourceId ? workspaceFor(attempt, 'developer').path : null, note: 'Unsealed source, not candidate or verification evidence. Cross-resource drafts require a sealed artifact before verification.' } : null, handoff: task.collaboration?.history.at(-1) ? { purpose: task.collaboration.history.at(-1).purpose, reason: task.collaboration.history.at(-1).reason } : null, candidateSha: attempt.candidateSha ?? null, tests: attempt.evidence.tests ? { sha: attempt.evidence.tests.sha, passed: attempt.evidence.tests.passed, sha256: attempt.evidence.tests.sha256, command: attempt.evidence.tests.command } : null, review: attempt.evidence.review ? { sha: attempt.evidence.review.sha, verdict: attempt.evidence.review.verdict, summary: attempt.evidence.review.summary } : null }, repository: await repositoryContext(project.id, attempt.baseSha) }) + } catch (error) { + const registered = lease && ownEntry(state.projectResourceLeases ?? {}, lease.id) + const unbound = registered && registered.generation === lease.generation && !registered.worker && !registered.processScope && !Object.values(state.projectHarnessWorkers ?? {}).some(row => row.leaseId === lease.id) && !Object.values(attempt.roleSessions ?? {}).some(row => row.id === sessionId) + if (unbound) await rt.projectResources.release(lease.id, lease.generation) + ;(attempt.preparationFailures ??= []).push({ role, sessionId, leaseId: lease?.id ?? null, message: String(error.message), releasedBeforeExternalStart: Boolean(unbound && registered.status === 'released' && registered.releaseProof?.kind === 'unbound-reservation-release'), at: Date.now() }) + await rt.save() + throw error } - await prepareWorkspace(attempt, workspace, settings) - await prepareIntegrations(attempt, workspace) - context = roleContext({ retrieval: true, policy, projectId: project.id, role, revision: attempt.agentDefinitions.revision, definition, memory: definition.memory, knowledge: ownEntry(state.projectKnowledge ?? {}, project.id) ?? [], requirement: attempt.requirement, feedback: attempt.feedback, projectFacts: { integrationInputs: structuredClone(attempt.integrationInputs ?? []), testFailure: await failureContext(attempt), publicationBinding: attempt.publicationBinding ?? null, integrationInstruction: attempt.publicationBinding ? 'Merge the pinned upstreamSha into the published candidate using --no-commit, resolve and stage conflicts before handoff. Host sealing retains both trusted inputs; never fabricate parents or discard backend changes.' : null, workingDraft: !attempt.candidateSha && attempt.workspaces.developer ? { kind: 'unverified-workspace-reference', resourceId: attempt.workspaces.developer.resourceId, path: attempt.workspaces.developer.resourceId === workspace.resourceId ? attempt.workspaces.developer.path : null, note: 'Unsealed source, not candidate or verification evidence. Cross-resource drafts require a sealed artifact before verification.' } : null, handoff: task.collaboration?.history.at(-1) ? { purpose: task.collaboration.history.at(-1).purpose, reason: task.collaboration.history.at(-1).reason } : null, candidateSha: attempt.candidateSha ?? null, tests: attempt.evidence.tests ? { sha: attempt.evidence.tests.sha, passed: attempt.evidence.tests.passed, sha256: attempt.evidence.tests.sha256, command: attempt.evidence.tests.command } : null, review: attempt.evidence.review ? { sha: attempt.evidence.review.sha, verdict: attempt.evidence.review.verdict, summary: attempt.evidence.review.summary } : null }, repository: await repositoryContext(project.id, attempt.baseSha) }) - } catch (error) { - const registered = lease && ownEntry(state.projectResourceLeases ?? {}, lease.id) - const unbound = registered && registered.generation === lease.generation && !registered.worker && !registered.processScope && !Object.values(state.projectHarnessWorkers ?? {}).some(row => row.leaseId === lease.id) && !Object.values(attempt.roleSessions ?? {}).some(row => row.id === sessionId) - if (unbound) await rt.projectResources.release(lease.id, lease.generation) - ;(attempt.preparationFailures ??= []).push({ role, sessionId, leaseId: lease?.id ?? null, message: String(error.message), releasedBeforeExternalStart: Boolean(unbound && registered.status === 'released' && registered.releaseProof?.kind === 'unbound-reservation-release'), at: Date.now() }) + attempt.rolePolicies[role] = policy + if ((execution.harness ?? 'flybot') === 'flybot') await archiveForNative(context, attempt, role, sessionId) + const session = { id: sessionId, logicalSessionId: taskSessionId(project.id, task.id, role), projectId: project.id, taskId: task.id, attemptId: attempt.id, role, policyRole, definitionRevision: pinned.revision, definitionRef: pinned.definitionRef, definitionKey: pinned.definitionKey, ...(pinned.change ? { definitionChange: pinned.change } : {}), execution: cloneExecution(execution), policy, context, generation: claim.generation, poolClaimId: claim.id, status: 'prepared', ...(lease ? { resourceLease: lease } : {}) } + if (attempt.roleSessions[role]) { + (attempt.roleSessionHistory ??= []).push(structuredClone({ ...attempt.roleSessions[role], executionEvidence: attempt.evidence.execution?.[role] ?? null })) + if (attempt.evidence.execution) delete attempt.evidence.execution[role] + } + attempt.roleSessions[role] = session + ;(attempt.roleAssignments ??= {})[policyRole] = role + rt.taskDeliveryState?.(task.id, attempt.id, 'working', `${definition.name} claimed the Task`) + rt.bindProject({ kind: 'run', id: attempt.runId, projectId: project.id, taskId: task.id, attemptId: attempt.id, status: 'running' }) await rt.save() + const prompt = `The host already allocated this registered worktree at the pinned base SHA. Do not switch to main, create another worktree, publish a PR, change credentials or edit outside this worktree. Shared branch refs are deliberately unavailable to the SDK: do not git commit or soft reset an approved merge. Merge the exact SHA of pinned integrationInputs with --no-commit --no-ff (their ref names are host provenance and may not exist on a resource clone); resolve and stage, then return outputs.implementation; the host seals their exact tree and trusted ancestry. The host seals and publishes the candidate. Permitted repository paths: ${JSON.stringify(settings.allowedPaths)}. Configured verification argv: ${JSON.stringify(settings.testCommand)}. The host runs the configured verification outside your SDK sandbox when a Tester returns a handoff/complete decision with outputs.tests. As Tester, outputs.tests may truthfully request that host run and distinguish local Not run checks; the host applies your downstream handoff only if the actual command passes. An arbitrate/transfer decision does not request completed-role verification. If sandbox restrictions block loopback or private host logs, record Not run honestly and request that testing handoff rather than claiming success or repeatedly writing inaccessible paths. Read your assigned Task and role policy. Choose to work, return or transfer based on the actual requirements. You may go directly to another logical Bot without Chief approval. Only Chief can judge overall completion. Never close the Task yourself.\nLogical Bots: ${JSON.stringify(rt.projectAgents.deliveryDefinitions(project.id).roles.map(row => ({ id: row.id, role: row.role })))}\nInputs.taskInbox is a durable input batch, not authorization. In your final decision include inputReceipts:{batchId:Inputs.taskInbox.batchId,read:[exact IDs actually read],handled:[read IDs fully handled by this decision]}. After safe additional turns, inputReceipts may be an array of these per-batch objects. Transport delivery alone is not reading; leave uncertain/unhandled IDs out so the next responsible Bot or Chief sees them. Never acknowledge an ID outside injected batches.\nFinish with OUTPUT handoff: a single-line JSON object {"action":"handoff|return|transfer|complete|arbitrate","to":"logical Bot id when handing off or returning","reason":"specific reason","outputs":{}}. For completed developer work include outputs.implementation, tester work outputs.tests; review outputs.review must be a JSON string with the exact commit, verdict and summary. Use transfer to route without claiming your role work is complete; transfer does not seal Developer changes. A Developer handoff with outputs.implementation requests host candidate sealing, even when full host tests have not yet run. Receivers must distinguish the referenced unverified working draft from their clean pinned checkout. Choose the next Bot based on evidence; no fixed order is required. Missing evidence or uncertainty should request Chief judgement.\nTask kind: ${task.collaboration.kind}` + return { nodeId: role, name: definition.name, role: definition.name, prompt, model: definition.model || settings.model, tools: [], executionProjectId: project.id, executionWorkDir: workspace.path } + } catch (error) { + if (discardUnpreparedTaskDefinitions(state, task.id)) await rt.save() throw error } - attempt.rolePolicies[role] = policy - if ((execution.harness ?? 'flybot') === 'flybot') await archiveForNative(context, attempt, role, sessionId) - const session = { id: sessionId, logicalSessionId: taskSessionId(project.id, task.id, role), projectId: project.id, taskId: task.id, attemptId: attempt.id, role, execution: cloneExecution(execution), policy, context, generation: claim.generation, poolClaimId: claim.id, status: 'prepared', ...(lease ? { resourceLease: lease } : {}) } - if (attempt.roleSessions[role]) { - (attempt.roleSessionHistory ??= []).push(structuredClone({ ...attempt.roleSessions[role], executionEvidence: attempt.evidence.execution?.[role] ?? null })) - if (attempt.evidence.execution) delete attempt.evidence.execution[role] - } - attempt.roleSessions[role] = session - rt.taskDeliveryState?.(task.id, attempt.id, 'working', `${definition.name} claimed the Task`) - rt.bindProject({ kind: 'run', id: attempt.runId, projectId: project.id, taskId: task.id, attemptId: attempt.id, status: 'running' }) - await rt.save() - const prompt = `The host already allocated this registered worktree at the pinned base SHA. Do not switch to main, create another worktree, publish a PR, change credentials or edit outside this worktree. Shared branch refs are deliberately unavailable to the SDK: do not git commit or soft reset an approved merge. Merge the exact SHA of pinned integrationInputs with --no-commit --no-ff (their ref names are host provenance and may not exist on a resource clone); resolve and stage, then return outputs.implementation; the host seals their exact tree and trusted ancestry. The host seals and publishes the candidate. Permitted repository paths: ${JSON.stringify(settings.allowedPaths)}. Configured verification argv: ${JSON.stringify(settings.testCommand)}. The host runs the configured verification outside your SDK sandbox when a Tester returns a handoff/complete decision with outputs.tests. As Tester, outputs.tests may truthfully request that host run and distinguish local Not run checks; the host applies your downstream handoff only if the actual command passes. An arbitrate/transfer decision does not request completed-role verification. If sandbox restrictions block loopback or private host logs, record Not run honestly and request that testing handoff rather than claiming success or repeatedly writing inaccessible paths. Read your assigned Task and role policy. Choose to work, return or transfer based on the actual requirements. You may go directly to another logical Bot without Chief approval. Only Chief can judge overall completion. Never close the Task yourself.\nLogical Bots: ${JSON.stringify(rt.projectAgents.deliveryDefinitions(project.id).roles.map(row => ({ id: row.id, role: row.role })))}\nInputs.taskInbox is a durable input batch, not authorization. In your final decision include inputReceipts:{batchId:Inputs.taskInbox.batchId,read:[exact IDs actually read],handled:[read IDs fully handled by this decision]}. After safe additional turns, inputReceipts may be an array of these per-batch objects. Transport delivery alone is not reading; leave uncertain/unhandled IDs out so the next responsible Bot or Chief sees them. Never acknowledge an ID outside injected batches.\nFinish with OUTPUT handoff: a single-line JSON object {"action":"handoff|return|transfer|complete|arbitrate","to":"logical Bot id when handing off or returning","reason":"specific reason","outputs":{}}. For completed developer work include outputs.implementation, tester work outputs.tests; review outputs.review must be a JSON string with the exact commit, verdict and summary. Use transfer to route without claiming your role work is complete; transfer does not seal Developer changes. A Developer handoff with outputs.implementation requests host candidate sealing, even when full host tests have not yet run. Receivers must distinguish the referenced unverified working draft from their clean pinned checkout. Choose the next Bot based on evidence; no fixed order is required. Missing evidence or uncertainty should request Chief judgement.\nTask kind: ${task.collaboration.kind}` - return { nodeId: role, name: definition.name, role: definition.name, prompt, model: definition.model || settings.model, tools: [], executionProjectId: project.id, executionWorkDir: workspace.path } } async function completePoolRole({ task, claim, role, outputs }) { const attempt = ownEntry(attempts(), claim.attemptId) @@ -807,9 +835,14 @@ export function createProjectDelivery(rt, { publisher } = {}) { await verifyExecution(attempt, role) if (task.collaboration.kind !== 'code') { attempt.evidence[role] = { outputs: structuredClone(outputs), sha256: hash(JSON.stringify(outputs)), at: Date.now(), policySha256: attempt.rolePolicies[role].sha256 } - } else if (role === 'developer') await seal(attempt) - else if (role === 'tester') await testCandidate(attempt) - else if (role === 'reviewer') await review(attempt, outputs) + } else { + const policyRole = attempt.roleSessions[role]?.policyRole ?? role + if (deliveryRole(attempt, policyRole) !== role) throw reject('Role artifact assignment changed before completion') + if (policyRole === 'developer') await seal(attempt) + else if (policyRole === 'tester') await testCandidate(attempt) + else if (policyRole === 'reviewer') await review(attempt, outputs) + else throw reject('Unknown worker artifact capability') + } await rt.save() } async function chiefAccept({ taskId, expectedSha, decision }, actor) { @@ -873,7 +906,7 @@ export function createProjectDelivery(rt, { publisher } = {}) { const id = `attempt-${randomUUID()}` if (['running', 'waiting-acceptance'].includes(previous.status)) previous.status = 'returned' previous.returned = { at: Date.now(), by: claim.botId, reason } - const next = { id, protocol: previous.protocol, projectId: previous.projectId, taskId: task.id, requestKey: hash(`${task.projectId}\n${task.id}\n${claim.id}`), workflowVersion: previous.protocol === 'project-handoff/v1' ? 'project-delivery/pools-v3' : previous.workflowVersion, status: 'running', initiatedBy: previous.initiatedBy, executionOwnerId: previous.executionOwnerId, baseSha: !previous.candidateSha && (force || previous.status === 'interrupted') ? await git(settingsOf(projectOf(task.projectId)).path, ['rev-parse', 'HEAD']) : previous.baseSha, seedSha: previous.candidateSha ?? previous.seedSha, ...(publicationBinding ? { publicationBinding } : {}), requirement: structuredClone(previous.requirement), feedback: { previousAttemptId: previous.id, kind: 'bot-return', reason, candidateSha: previous.candidateSha }, agentDefinitions: rt.projectAgents.deliveryDefinitions(task.projectId), roleSessions: {}, rolePolicies: {}, resourceLeases: {}, workspaces: {}, evidence: {}, createdAt: Date.now(), updatedAt: Date.now(), runId: `pool-run-${randomUUID()}` } + const next = { id, protocol: previous.protocol, projectId: previous.projectId, taskId: task.id, requestKey: hash(`${task.projectId}\n${task.id}\n${claim.id}`), workflowVersion: previous.protocol === 'project-handoff/v1' ? 'project-delivery/pools-v4' : previous.workflowVersion, status: 'running', initiatedBy: previous.initiatedBy, executionOwnerId: previous.executionOwnerId, baseSha: !previous.candidateSha && (force || previous.status === 'interrupted') ? await git(settingsOf(projectOf(task.projectId)).path, ['rev-parse', 'HEAD']) : previous.baseSha, seedSha: previous.candidateSha ?? previous.seedSha, ...(publicationBinding ? { publicationBinding } : {}), requirement: structuredClone(previous.requirement), feedback: { previousAttemptId: previous.id, kind: 'bot-return', reason, candidateSha: previous.candidateSha }, agentDefinitions: rt.projectAgents.deliveryDefinitions(task.projectId), roleSessions: {}, rolePolicies: {}, resourceLeases: {}, workspaces: {}, evidence: {}, createdAt: Date.now(), updatedAt: Date.now(), runId: `pool-run-${randomUUID()}` } previous.supersededBy = id setEntry(attempts(), id, next); task.deliveryAttemptId = id rt.bindProject({ kind: 'run', id: previous.runId, projectId: task.projectId, taskId: task.id, attemptId: previous.id, status: 'failed' }) @@ -882,12 +915,12 @@ export function createProjectDelivery(rt, { publisher } = {}) { return id } const cloneExecution = execution => structuredClone(execution) - async function repositoryRolePolicy(projectId, commit, role) { + async function repositoryRolePolicy(projectId, commit, role, policyRole = role) { const project = projectOf(projectId) const settings = settingsOf(project) commit ??= await git(settings.path, ['rev-parse', 'HEAD']) - const configured = settings.rolePolicySources?.[role] - const names = configured ?? [...ROLE_POLICY_FILES.common, ...ROLE_POLICY_FILES[role]] + const configured = settings.rolePolicySources?.[role] ?? settings.rolePolicySources?.[policyRole] + const names = configured ?? [...ROLE_POLICY_FILES.common, ...ROLE_POLICY_FILES[policyRole]] if (!Array.isArray(names) || names.length > 40 || names.some(name => typeof name !== 'string' || name.startsWith('/') || name.includes('..') || /[\\\r\n\0]/.test(name))) throw reject('Role policy source list is invalid') const sources = [] for (const name of [...new Set(names)]) { @@ -901,8 +934,8 @@ export function createProjectDelivery(rt, { publisher } = {}) { const entry = JSON.parse(await git(settings.path, ['show', `${commit}:package.json`])) if (entry.name === 'flybot' && await git(settings.path, ['ls-tree', '--name-only', commit, '--', 'core/cordis.patch.yml'])) profile = 'flybot-development' } - const requiredFiles = settings.requiredRolePolicySources?.[role] ?? (profile === 'flybot-development' ? [...ROLE_POLICY_FILES.common.filter(name => !name.startsWith('decisions/')), ...ROLE_POLICY_FILES[role]] : configured ?? []) - return assembleRolePolicy({ role, commit, sources, requiredFiles, profile }) + const requiredFiles = settings.requiredRolePolicySources?.[role] ?? settings.requiredRolePolicySources?.[policyRole] ?? (profile === 'flybot-development' ? [...ROLE_POLICY_FILES.common.filter(name => !name.startsWith('decisions/')), ...ROLE_POLICY_FILES[policyRole]] : configured ?? []) + return assembleRolePolicy({ role, policyRole, commit, sources, requiredFiles, profile }) } async function repositoryContext(projectId, commit) { const project = projectOf(projectId) @@ -943,11 +976,12 @@ export function createProjectDelivery(rt, { publisher } = {}) { await rt.save() } } - const ownsWorkspace = (projectId, path) => Object.values(state.projectWorkspaces ?? {}).find(workspace => workspace.projectId === projectId && workspace.path === path && DELIVERY_ROLES.includes(workspace.role) && ['worktree', 'resource-clone'].includes(workspace.kind) && workspace.status === 'allocated' && ongoing.has(ownEntry(attempts(), workspace.attemptId)?.status)) - return { endpoint, context, preparePoolRole, completePoolRole, chiefAccept, reworkPool, recordExecution, repositoryContext, repositoryRolePolicy, bindSession, releaseSession, sessionFor, recover, runReported, ownsWorkspace, gate: input => serialAttempt(current(input.taskId)?.id ?? input.taskId, () => gate(input)) } + const ownsWorkspace = (projectId, path) => Object.values(state.projectWorkspaces ?? {}).find(workspace => workspace.projectId === projectId && workspace.path === path && DELIVERY_ROLES.includes(workspace.policyRole ?? workspace.role) && ['worktree', 'resource-clone'].includes(workspace.kind) && workspace.status === 'allocated' && ongoing.has(ownEntry(attempts(), workspace.attemptId)?.status)) + return { reconcileExecutions: () => reconcileExecutionAuthority(rt), exitConfirmed: workerId => executionExitConfirmed(rt, workerId), nativeSessions, endpoint, context, preparePoolRole, completePoolRole, chiefAccept, reworkPool, recordExecution, repositoryContext, repositoryRolePolicy, bindSession, releaseSession, sessionFor, recover, runReported, ownsWorkspace, gate: input => serialAttempt(current(input.taskId)?.id ?? input.taskId, () => gate(input)) } } export function install(rt) { rt.projectDelivery = createProjectDelivery(rt) + rt.ctx.on('agent/pre-step', async (payload, next) => await rt.projectDelivery.nativeSessions.validateStep(payload.agent) ? next() : { kind: 'reject' }) rt.taskNeedsDelivery = task => Boolean(task?.projectId && ownEntry(rt.config.projectRepositories ?? {}, rt.readProject(task.projectId)?.repository)) } diff --git a/patches/ds-bot/files/src/host/project-execution-authority.js b/patches/ds-bot/files/src/host/project-execution-authority.js new file mode 100644 index 000000000..4d2485dc1 --- /dev/null +++ b/patches/ds-bot/files/src/host/project-execution-authority.js @@ -0,0 +1,65 @@ +// Current authority limits retained execution context; it never replaces its persona. +import { ownEntry } from './store.js' +import { pinTaskRoleDefinition } from './project-task-sessions.js' +const refuse = message => Object.assign(new Error(message), { code: 'bot/execution-revoked' }) +export async function requireExecutionAuthority(rt, session) { + if (rt.readOnly?.() !== undefined) throw refuse('Execution lost its writable host authority') + const task = ownEntry(rt.state.tasks ?? {}, session.taskId) + if (!task || task.projectId !== session.projectId) throw refuse('Execution Task scope changed') + const claim = session.poolClaimId && task.collaboration?.claim + const attempt = ownEntry(rt.state.projectAttempts ?? {}, session.attemptId) + const memberId = claim ? task.collaboration.initiatedBy : attempt?.initiatedBy + const sourceSessionId = task.collaboration?.sourceSessionId + if (claim && (typeof memberId !== 'string' || !memberId)) throw refuse('Execution initiating human is unavailable') + function current(afterAwait = false) { + if (ownEntry(rt.state.tasks ?? {}, session.taskId) !== task) throw refuse('Execution Task scope changed') + if (claim && (task.collaboration?.initiatedBy !== memberId || task.collaboration.sourceSessionId !== sourceSessionId || task.deliveryAttemptId !== session.attemptId || task.collaboration?.claim !== claim || claim.id !== session.poolClaimId || claim.generation !== session.generation || task.collaboration.generation !== session.generation || claim.status !== 'active')) throw refuse(afterAwait ? 'Execution claim was revoked during authority validation' : 'Execution claim changed before execution') + if (session.poolClaimId && !claim) throw refuse('Execution claim is unavailable') + if (['accepted', 'cancelled'].includes(task.status)) throw refuse('Execution Task has ended') + if (rt.readOnly?.() !== undefined) throw refuse('Execution lost its writable host authority') + } + current() + if (memberId) { + rt.requireProjectAccess({ memberId }, task.projectId, 'task-write', task) + await rt.projectDelegations?.background(memberId, task.collaboration?.sourceSessionId) + current(true) + rt.requireProjectAccess({ memberId }, task.projectId, 'task-write', task) + } + rt.projects?.requireActive(task.projectId) + if (session.definitionRef) { + const frozen = ownEntry(rt.state.projectRoleDefinitionSnapshots ?? {}, session.definitionRef) + if (!frozen) throw refuse('Pinned execution role definition is unavailable') + let catalog = rt.projectAgents?.deliveryDefinitions(session.projectId) + if (catalog && !catalog.roles.some(role => role.role === session.role) && claim) { + const retired = ownEntry(rt.state.projectRetiredRoles ?? {}, `${session.projectId}:${session.role}`) + if (retired?.id === claim.botId && (claim.role ?? session.role) === session.role) catalog = { ...catalog, roles: [...catalog.roles, retired] } + } + const retained = pinTaskRoleDefinition(rt.state, session, frozen.definition, session.definitionRevision, catalog) + const live = catalog?.roles.find(role => role.role === session.role) + if (!frozen.definition.execution?.resourceIds?.length && live?.execution?.resourceIds?.length) throw refuse('Implicit host execution is no longer authorized by the role resource pool') + const execution = retained.definition.execution ?? {} + if (session.resourceLease && !execution.resourceIds?.includes(session.resourceLease.resourceId)) throw refuse('The execution resource is no longer authorized') + if (!session.resourceLease && execution.resourceIds?.length) throw refuse('Implicit host execution is no longer authorized by the role resource pool') + } + if (session.resourceLease) { + const resource = rt.projectResources?.requireBinding(session.projectId, session.resourceLease) + const harness = session.execution?.harness ?? session.definition?.execution?.harness ?? 'flybot' + if (resource && (resource.disabled || !resource.harnesses?.includes(harness) || harness === 'flybot' && resource.type !== 'local')) throw refuse('Execution resource or Harness capability was revoked') + } + current() +} + +// Called by the existing serialized scheduler pass, never by a new timer. +export async function reconcileExecutionAuthority(rt) { + if (rt.readOnly?.() !== undefined) return + for (const attempt of Object.values(rt.state.projectAttempts ?? {})) { + for (const session of Object.values(attempt.roleSessions ?? {})) { + const task = ownEntry(rt.state.tasks ?? {}, session.taskId), claim = task?.collaboration?.claim + if (!session.poolClaimId || session.status === 'released' || claim?.id !== session.poolClaimId || claim.status !== 'active' || claim.workerId !== session.botId) continue + try { await requireExecutionAuthority(rt, session) } + catch (error) { + await rt.projectPools?.revokeExecution(task.id, claim.id, String(error.message).slice(0, 1000)) + } + } + } +} diff --git a/patches/ds-bot/files/src/host/project-execution-exit.js b/patches/ds-bot/files/src/host/project-execution-exit.js new file mode 100644 index 000000000..0ff345a58 --- /dev/null +++ b/patches/ds-bot/files/src/host/project-execution-exit.js @@ -0,0 +1,16 @@ +// Physical exit is independent of Bot deletion and of whether context can resume. +import { ownEntry } from './store.js' +import { nativeExitConfirmed } from './project-native-sessions.js' +import { confirmedWorkerExit } from './project-task-sessions.js' +import { supervisorProofMatches } from './project-process-proof.js' +export function executionExitConfirmed(rt, workerId) { + if (rt.botOf(workerId)) return false + const state = rt.state, sdk = ownEntry(state.projectHarnessWorkers ?? {}, workerId) + if (sdk) return confirmedWorkerExit(state, sdk) && Boolean(!sdk.leaseId || ownEntry(state.projectResourceLeases ?? {}, sdk.leaseId)?.status === 'released') + const native = ownEntry(state.projectNativeWorkers ?? {}, workerId) + if (nativeExitConfirmed(state, native)) return !native.leaseId || ownEntry(state.projectResourceLeases ?? {}, native.leaseId)?.status === 'released' + const leases = Object.values(state.projectResourceLeases ?? {}).filter(row => row.worker?.kind === 'native' && row.worker.botId === workerId) + if (!leases.length) return false + for (const lease of leases) if (lease.status !== 'released' || !supervisorProofMatches(lease.processScope, lease.releaseProof)) return false + return true +} diff --git a/patches/ds-bot/files/src/host/project-harness.js b/patches/ds-bot/files/src/host/project-harness.js index 9fea0aeab..e0dbeccaa 100644 --- a/patches/ds-bot/files/src/host/project-harness.js +++ b/patches/ds-bot/files/src/host/project-harness.js @@ -8,10 +8,22 @@ import { verifyRolePolicy, roleExecutionPrompt } from './project-role-policy.js' import { fileDigest } from './project-resource-workspaces.js' import { gitWritableRoots } from './project-git-sandbox.js' import { ownEntry, setEntry } from './store.js' -import { taskSession, backendBindingKey, resumableBinding } from './project-task-sessions.js' +import { requireExecutionAuthority } from './project-execution-authority.js' +import { taskSession, backendBindingKey, resumableBinding, roleDefinitionKey } from './project-task-sessions.js' const refuse = message => Object.assign(new Error(message), { code: 'bot/harness' }) const sha = text => createHash('sha256').update(text).digest('hex') const VERSIONS = { codex: '0.166.0', 'claude-agent': '0.3.316' } +// Actual adapter support, independent of resource installation/authentication. +// Native continuation uses exact committed Task-role bindings and the existing +// Bot/parts lifecycle. Cross-Harness restoration is never inferred. +export function harnessCapabilities(harness) { + if (!['flybot', ...Object.keys(VERSIONS)].includes(harness)) throw refuse('Unknown execution Harness') + return { version: 'project-harness-capabilities/v1', harness, sdkVersion: VERSIONS[harness] ?? null, + taskRoleResume: true, input: harness === 'flybot' ? 'native-bot-turn' : 'between-sdk-turns', + cancellation: harness === 'flybot' ? 'native-bot-stop' : 'abort-and-worker-stop', + exitConfirmation: harness === 'flybot' ? 'native-agent-quiescence' : 'stop-ack-and-process-exit', crossHarnessResume: false } +} + export function handoffFrame(text, outputs) { if (typeof text !== 'string' || outputs?.length !== 1 || outputs[0] !== 'handoff' || /(?:^|\n)OUTPUT handoff:/.test(text)) return text try { @@ -73,8 +85,10 @@ export function createProjectHarness(rt) { const workerPath = selected.resource.runtimeDir ? join(selected.resource.runtimeDir, 'src/host/project-harness-worker.mjs') : resolve(import.meta.dirname, 'project-harness-worker.mjs') if (relative(selected.resource.root, workspace.path).startsWith('..')) throw refuse('SDK Workspace is outside the selected resource') const logical = !chief ? taskSession(rt.state, session) : null - const bindingKey = chief?.bindingKey ?? backendBindingKey({ harness: session.execution.harness, resourceId: session.resourceLease.resourceId, principalId: selected.principalId, cwd: workspace.path, model: session.execution.model ?? selected.selected.model, profile: selected.selected, definition: sha(spec.prompt ?? '') }) - const previous = logical ? resumableBinding(rt.state, logical, bindingKey) : null + const definitionKey = logical ? roleDefinitionKey(session) : null + const identity = { harness: session.execution.harness, resourceId: session.resourceLease.resourceId, principalId: selected.principalId, cwd: workspace.path, model: session.execution.model ?? selected.selected.model, profile: selected.selected } + const bindingKey = chief?.bindingKey ?? backendBindingKey({ ...identity, definition: definitionKey }) + const previous = logical ? resumableBinding(rt.state, logical, bindingKey, { legacyKey: backendBindingKey({ ...identity, definition: sha(spec.prompt ?? '') }), definitionKey }) : null const id = `sdk-worker-${randomUUID()}` // Verification may write fixtures, but never the sealed review checkout. const scratchDirectory = chief ? null : join(selected.resource.root, `.flybot-scratch-${id}`) @@ -83,11 +97,11 @@ export function createProjectHarness(rt) { Object.assign(selected.env, { TMPDIR: scratchDirectory, npm_config_cache: join(scratchDirectory, 'npm-cache'), npm_config_update_notifier: 'false', npm_config_offline: 'true' }) } const conversation = chief ? ownEntry(rt.state.projectAgents ?? {}, chief.agentId) : null - const row = { id, ...(conversation?.roomId ? { roomId: conversation.roomId } : {}), model: session.execution.model ?? selected.selected.model ?? null, chiefRunId: chief?.runId ?? null, visibility: conversation?.visibility ?? 'project', ...(conversation?.visibility === 'private' ? { memberId: conversation.memberId } : {}), projectId: session.projectId, taskId: session.taskId, attemptId: session.attemptId, role: session.role, sessionKey: session.id, + const row = { id, ...(conversation?.roomId ? { roomId: conversation.roomId } : {}), model: session.execution.model ?? selected.selected.model ?? null, chiefRunId: chief?.runId ?? null, visibility: conversation?.visibility ?? 'project', ...(conversation?.visibility === 'private' ? { memberId: conversation.memberId } : {}), projectId: session.projectId, taskId: session.taskId, attemptId: session.attemptId, role: session.role, policyRole: session.policyRole ?? session.definition?.policyRole ?? session.role, sessionKey: session.id, resourceId: session.resourceLease.resourceId, leaseId: session.resourceLease.id, generation: session.resourceLease.generation, harness: session.execution.harness, sdkVersion: null, expectedSdkVersion: VERSIONS[session.execution.harness], principalId: selected.principalId, - cwd: workspace.path, scratchDirectory, status: 'prepared', createdAt: Date.now(), eventCount: 0 } - if (logical) Object.assign(row, { logicalSessionId: logical.id, bindingKey, ...(previous ? { backendSessionId: previous.sessionId, continuedFrom: previous.workerId } : { rotationReason: logical.bindings.length ? 'Execution binding changed; retained logical Task history' : 'First Task-role execution' }) }) + cwd: workspace.path, scratchDirectory, status: 'prepared', executionStartedAt: null, createdAt: Date.now(), eventCount: 0 } + if (logical) Object.assign(row, { logicalSessionId: logical.id, bindingKey, bindingVersion: 'task-role-binding/v2', definitionKey, ...(previous ? { backendSessionId: previous.sessionId, continuedFrom: previous.workerId, ...(previous.key !== bindingKey ? { migratedBindingKey: previous.key } : {}) } : { rotationReason: 'First Task-role execution' }) }) else if (chief) Object.assign(row, { logicalSessionId: chief.agentId, bindingKey, ...(chief.resumeSessionId ? { backendSessionId: chief.resumeSessionId, continuedFrom: chief.previousWorkerId } : {}) }) setEntry(workers(), id, row) if (chief) await rt.projectResources.markStarted(row.leaseId, row.generation, { kind: 'sdk', handle: id }) @@ -150,7 +164,11 @@ export function createProjectHarness(rt) { if (claim?.id === session.poolClaimId && claim.status === 'active' && task.collaboration.generation === claim.generation) await rt.projectInputs?.transport(task.id, claim, control.inputBatch.batchId, { kind: 'sdk-output', sha256: row.lastEventHash }) } const backend = event.event.thread_id ?? event.event.session_id - if (backend) { if (row.backendSessionId && row.backendSessionId !== backend) throw refuse('SDK changed its bound Session'); row.backendSessionId = backend; session.backendSessionId = backend } + if (backend) { + if (row.backendSessionId && row.backendSessionId !== backend) throw refuse('SDK changed its bound Session') + row.backendSessionId = backend; session.backendSessionId = backend + if (logical && !logical.bindings.some(binding => binding.workerId === row.id)) logical.bindings.push({ version: 'task-role-binding/v2', definitionKey, key: bindingKey, sessionId: backend, workerId: row.id, policySha256: session.policy?.sha256 ?? null, attemptId: row.attemptId, at: Date.now(), observedBeforeCompletion: true }) + } } if (event.type === 'completed') { rt.projectResources.setting(row.projectId, row.resourceId) @@ -161,6 +179,7 @@ export function createProjectHarness(rt) { if (claim?.id === session.poolClaimId && claim.status === 'active' && task.collaboration.generation === claim.generation) { const next = await rt.projectInputs?.prepare(task.id, claim, { next: true }) if (next?.inputs.length) { + await requireExecutionAuthority(rt, session) row.backendSessionId = event.sessionId; session.backendSessionId = event.sessionId control.inputBatch = next; control.safeInputTurns = (control.safeInputTurns ?? 0) + 1 row.safeInputTurns = control.safeInputTurns @@ -189,7 +208,7 @@ export function createProjectHarness(rt) { if (event.text !== rawText) { row.rawResultHash = sha(rawText); row.outputFraming = 'json-handoff/v1' } row.backendSessionId = event.sessionId; session.backendSessionId = event.sessionId; row.usage = event.usage; row.resultHash = sha(event.text) if (logical) { - logical.bindings.push({ key: bindingKey, sessionId: event.sessionId, workerId: row.id, policySha256: session.policy?.sha256 ?? null, attemptId: row.attemptId, at: Date.now() }) + if (!logical.bindings.some(binding => binding.workerId === row.id)) logical.bindings.push({ version: 'task-role-binding/v2', definitionKey, key: bindingKey, sessionId: event.sessionId, workerId: row.id, policySha256: session.policy?.sha256 ?? null, attemptId: row.attemptId, at: Date.now() }) await rt.save() } if (control.outputs.every(name => new RegExp(`(?:^|\\n)OUTPUT ${name}:`).test(event.text))) { @@ -254,6 +273,7 @@ export function createProjectHarness(rt) { const next = await rt.projectInputs?.prepare(task.id, claim, { next: true }) if (!next?.inputs.length) return { woken: false } if (active.get(row.id) !== control || task.collaboration.claim !== claim || task.collaboration.generation !== claim.generation || claim.status !== 'active' || row.status !== 'waiting-input' || row.pending?.kind !== 'message' || lease.status === 'released') return { woken: false } + await requireExecutionAuthority(rt, control.session) rt.requireProjectAccess(actor, task.projectId, 'task-write', task) control.inputBatch = next; control.safeInputTurns = (control.safeInputTurns ?? 0) + 1 row.safeInputTurns = control.safeInputTurns; row.status = 'running'; delete row.pending @@ -272,14 +292,29 @@ export function createProjectHarness(rt) { if (!control || control.closed || !['prepared', 'idle'].includes(row.status)) throw refuse('SDK worker is unavailable or already dispatched') const lease = ownEntry(rt.state.projectResourceLeases ?? {}, row.leaseId) if (!lease || lease.generation !== row.generation || lease.status === 'released') throw refuse('SDK callback lease is stale') + if (control.session.poolClaimId) await requireExecutionAuthority(rt, control.session) control.outputs = (step.outputs ?? []).map(port => port.name) control.inputBatch = step.inputs?.taskInbox const prompt = `${roleExecutionPrompt(control.prompt, control.session, control.continuedPolicyHash)}${control.archivePath ? '\nExact scoped context archive: ' + control.archivePath + '. Read only needed entries with bounded output; do not print the whole archive. Archive SHA256: ' + control.session.context.archiveSha256 : ''}\n\nTask ${step.taskId}: ${step.brief}\nInputs: ${JSON.stringify(step.inputs ?? {})}\nFinish with OUTPUT : for ${control.outputs.join(', ')}.` - row.status = 'running' const workspace = rt.state.projectAttempts?.[row.attemptId]?.workspaces?.[row.role] - const writableRoots = row.role === 'developer' && row.harness === 'codex' && rt.projectResources.setting(row.projectId, row.resourceId).type === 'local' && workspace?.path === row.cwd ? await gitWritableRoots({ cwd: row.cwd, repository: workspace.repository, developer: true }) : [] + const writableRoots = (row.policyRole ?? row.role) === 'developer' && row.harness === 'codex' && rt.projectResources.setting(row.projectId, row.resourceId).type === 'local' && workspace?.path === row.cwd ? await gitWritableRoots({ cwd: row.cwd, repository: workspace.repository, developer: true }) : [] + row.status = 'running'; row.executionStartedAt = Date.now() + try { + const before = rt.savesQueued?.() + const saved = rt.save() + const queued = rt.savesQueued?.() + await saved + if (typeof before === 'number' && queued <= before) throw refuse('SDK dispatch did not commit a state snapshot') + const currentLease = ownEntry(rt.state.projectResourceLeases ?? {}, row.leaseId) + if (control.session.poolClaimId) await requireExecutionAuthority(rt, control.session) + if (active.get(row.id) !== control || control.closed || row.status !== 'running' || currentLease?.generation !== row.generation || currentLease.status === 'released') throw refuse('SDK dispatch was superseded before input delivery') + } catch (error) { + if (row.status === 'running') row.status = 'idle' + row.executionStartedAt = null; row.dispatchPreventedAt = Date.now() + throw error + } control.process.stdin.write(JSON.stringify({ type: 'start', settings: { harness: row.harness, cwd: row.cwd, sessionId: control.resumeSessionId, noTools: Boolean(control.chief), model: control.profile.model, env: control.env, - ...(control.profile.baseUrl ? { baseUrl: control.profile.baseUrl } : {}), readOnly: Boolean(control.chief) || row.role !== 'developer', writableRoots, scratchDirectory: row.scratchDirectory, maxBudgetUsd: control.profile.maxBudgetUsd ?? 1 }, prompt }) + '\n') + ...(control.profile.baseUrl ? { baseUrl: control.profile.baseUrl } : {}), readOnly: Boolean(control.chief) || (row.policyRole ?? row.role) !== 'developer', writableRoots, scratchDirectory: row.scratchDirectory, maxBudgetUsd: control.profile.maxBudgetUsd ?? 1 }, prompt }) + '\n') if (control.inputBatch?.inputs.length && control.session.poolClaimId) { const task = rt.state.tasks[row.taskId], claim = task?.collaboration?.claim if (claim?.id === control.session.poolClaimId) await rt.projectInputs?.transport(task.id, claim, control.inputBatch.batchId, { kind: 'adapter-written', sha256: sha(JSON.stringify(control.inputBatch)) }) @@ -354,7 +389,9 @@ export function createProjectHarness(rt) { if (!row || !control) throw refuse('Execution Session is unavailable') const task = ownEntry(rt.state.tasks, row.taskId) rt.requireProjectAccess(actor, row.projectId, 'task-write', task) - if (row.status !== 'waiting-input' || body.generation !== row.generation) throw refuse('Execution input is stale') + if (control.session.poolClaimId) await requireExecutionAuthority(rt, control.session) + rt.requireProjectAccess(actor, row.projectId, 'task-write', task) + if (row.status !== 'waiting-input' || control.closed || body.generation !== row.generation) throw refuse('Execution input is stale') if (row.pending.kind === 'permission') { rt.requireProjectAccess(actor, row.projectId, 'configure') if (body.requestId !== row.pending.id) throw refuse('Permission request is stale') @@ -366,6 +403,6 @@ export function createProjectHarness(rt) { delete row.pending; row.status = 'running'; await rt.save(); return { sent: true } } return { profile, spawn, dispatch, taskInputWake, release, input, chiefTurn, onResult: fn => { listeners.add(fn); return () => listeners.delete(fn) }, - owns: id => Boolean(ownEntry(workers(), id)), hasController: id => active.has(id) && !active.get(id).closed, versions: VERSIONS } + capabilities: harnessCapabilities, owns: id => Boolean(ownEntry(workers(), id)), hasController: id => active.has(id) && !active.get(id).closed, versions: VERSIONS } } export function install(rt) { rt.projectHarness = createProjectHarness(rt) } diff --git a/patches/ds-bot/files/src/host/project-native-sessions.js b/patches/ds-bot/files/src/host/project-native-sessions.js new file mode 100644 index 000000000..63d58f2c7 --- /dev/null +++ b/patches/ds-bot/files/src/host/project-native-sessions.js @@ -0,0 +1,193 @@ +// Native execution keeps exact Task-role context; Bot deletion is not exit proof. +import { createHash, randomUUID } from 'node:crypto' +import { captureProcessScope, priorScopeExited, supervisorProofMatches } from './project-process-proof.js' +import { ownEntry, setEntry } from './store.js' +import { requireExecutionAuthority } from './project-execution-authority.js' +import { taskSession, taskSessionId, roleDefinitionKey } from './project-task-sessions.js' +const digest = value => createHash('sha256').update(JSON.stringify(value)).digest('hex') +const refuse = message => Object.assign(new Error(message), { code: 'bot/context-resume' }) +const clone = value => structuredClone(value) +export function nativeExitConfirmed(state, row) { + const proof = row?.releaseProof + if (!row || row.version !== 'project-native-worker/v2' || row.status !== 'quiescent' || !['native-agent-quiescent', 'systemd-prior-invocation-exited'].includes(proof?.kind) || proof.workerId !== row.id || proof.sessionId !== row.sessionId || proof.generation !== row.generation || proof.bindingKey !== row.bindingKey) return false + if (proof.kind === 'native-agent-quiescent' && proof.registeredWorkEmpty !== true) return false + if (!row.leaseId) return proof.kind === 'native-agent-quiescent' || supervisorProofMatches(row.processScope, proof) + const lease = ownEntry(state.projectResourceLeases ?? {}, row.leaseId) + return Boolean(lease && lease.projectId === row.projectId && lease.attemptId === row.attemptId && lease.role === row.role && lease.sessionKey === row.sessionKey && lease.resourceId === row.resourceId && lease.generation === row.generation && lease.worker?.kind === 'native' && lease.worker.botId === row.id && (proof.kind !== 'systemd-prior-invocation-exited' || supervisorProofMatches(lease.processScope, proof))) +} +export function createProjectNativeSessions(rt, { priorExit = priorScopeExited, captureScope = captureProcessScope } = {}) { + const hostEpoch = randomUUID() + const rows = () => (rt.state.projectNativeWorkers ??= {}) + const stopping = new Map() + const binding = new Set() + rt.projectEffects?.register('native.activate', { run: effect => ({ workerId: effect.payload.workerId, status: 'durable-native-binding' }) }, { replace: true }) + const bindingKey = (session, workspace) => digest({ projectId: session.projectId, taskId: session.taskId, role: session.role, + harness: 'flybot', resourceId: session.resourceLease?.resourceId ?? null, cwd: workspace.path, definitionKey: roleDefinitionKey(session) }) + function previous(session, workspace) { + const logical = taskSession(rt.state, session), last = logical.bindings.at(-1), key = bindingKey(session, workspace) + if (!last) { + const historical = Object.values(rt.state.projectAttempts ?? {}).filter(attempt => attempt.projectId === session.projectId && attempt.taskId === session.taskId).flatMap(attempt => [attempt.roleSessions?.[session.role], ...(attempt.roleSessionHistory ?? [])]).some(prior => prior && prior.id !== session.id && prior.role === session.role && prior.sessionId && (prior.execution?.harness ?? 'flybot') === 'flybot') + if (historical) throw refuse('Legacy native Task-role context has no confirmed binding; explicit recovery is required instead of a fresh Session') + return { logical, key, row: null } + } + const row = ownEntry(rows(), last.workerId) + const lease = row?.leaseId && ownEntry(rt.state.projectResourceLeases ?? {}, row.leaseId) + if (last.harness !== 'flybot' || !row || row.logicalSessionId !== logical.id || row.projectId !== session.projectId || row.taskId !== session.taskId || row.role !== session.role || last.key !== key || row.bindingKey !== key || last.sessionId !== row.sessionId || !nativeExitConfirmed(rt.state, row) || row.leaseId && lease?.status !== 'released' || rt.botOf(row.id)) throw refuse('Latest native Task-role Session is incompatible or its execution has not confirmed release; a fresh Session is forbidden') + return { logical, key, row } + } + const authority = session => requireExecutionAuthority(rt, session) + async function bind(session, workspace, botId) { + await authority(session) + const bot = rt.botOf(botId) + if (!bot || bot.workspaceId !== workspace.id || bot.attemptId !== session.attemptId || session.status === 'released') throw refuse('Native carrier is unavailable for its registered Workspace') + const selected = previous(session, workspace), original = selected.row + let sessionId = rt.chatOf(botId) + if (original) { + if ((original.model ?? null) !== (bot.model ?? null)) throw refuse('Native Task-role model changed; explicit recovery is required') + const resumed = await rt.resolveAgent(original.sessionId) + if (resumed.session?.id !== original.sessionId || resumed.status !== 'idle' || typeof resumed.whenIdle !== 'function') throw refuse('Original native Session cannot be resumed safely') + await resumed.whenIdle() + await authority(session) + const current = previous(session, workspace) + if (current.row !== original || rt.botOf(botId) !== bot || session.status === 'released') throw refuse('Native resume binding changed while loading context') + await rt.ctx.workspaceRegistry.archiveSession(sessionId, { stopActivity: true }) + rt.state.retiredSessions ??= {} + setEntry(rt.state.retiredSessions, sessionId, botId) + sessionId = original.sessionId + bot.sessionId = sessionId + bot.segments = clone(original.segments ?? []) + rt.indexSessions?.() + } + const row = { id: botId, hostEpoch, version: 'project-native-worker/v2', processScope: await captureScope(), projectId: session.projectId, taskId: session.taskId, attemptId: session.attemptId, + role: session.role, model: bot.model ?? null, ...(original ? { continuedFrom: original.id } : {}), logicalSessionId: selected.logical.id, sessionKey: session.id, sessionId, cwd: workspace.path, bindingKey: selected.key, + definitionKey: roleDefinitionKey(session), generation: session.resourceLease?.generation ?? session.generation ?? 0, + ...(session.resourceLease ? { leaseId: session.resourceLease.id, resourceId: session.resourceLease.resourceId } : {}), status: 'bound', createdAt: Date.now() } + setEntry(rows(), botId, row) + selected.logical.bindings.push({ harness: 'flybot', version: 'native-session/v1', key: row.bindingKey, sessionId, workerId: botId, attemptId: session.attemptId }) + session.botId = botId; session.sessionId = sessionId + await rt.save() + return { id: botId, sessionId } + } + async function activate(session, botId) { + await authority(session) + const row = ownEntry(rows(), botId) + if (!row || row.hostEpoch !== hostEpoch || row.sessionKey !== session.id || row.attemptId !== session.attemptId || row.sessionId !== rt.chatOf(botId) || row.status !== 'bound') throw refuse('Native activation no longer matches its registered execution Session') + if (!rt.projectEffects) throw refuse('Native activation requires durable Effects proof') + const token = rt.projectEffects.stage({ kind: 'native.activate', key: botId, projectId: row.projectId, taskId: row.taskId, + payload: { workerId: botId, sessionId: row.sessionId, sessionKey: row.sessionKey, bindingKey: row.bindingKey, attemptId: row.attemptId, generation: row.generation } }) + row.activationEffect = token + await rt.projectEffects.persist(token) + if (!rt.projectEffects.intentCommitted(token)) throw refuse('Native activation binding is not committed') + await authority(session) + if (row.continuedFrom) { + if (typeof rt.ctx.workspaceRegistry.unarchiveSession !== 'function') throw refuse('Native Session unarchive is unavailable; explicit recovery is required') + await rt.ctx.workspaceRegistry.unarchiveSession(row.sessionId) + await authority(session) + } + return { workerId: row.id, sessionId: row.sessionId } + } + function ownedSessions(row, bot, sessionId) { + const ids = new Set([sessionId, row.sessionId, ...(bot.segments ?? []).map(value => value.sessionId), ...(row.knownSessionIds ?? [])].filter(Boolean)) + const headers = (rt.ctx.sessions?.list?.() ?? []).map(value => value.header) + let changed + do { + changed = false + for (const header of headers) if (header.origin === 'subagent' && ids.has(header.parentSession) && !ids.has(header.id)) { ids.add(header.id); changed = true } + } while (changed) + row.knownSessionIds = [...ids] + return row.knownSessionIds + } + function recordExit(row, bot, details) { + const sessionId = rt.chatOf(row.id), logical = taskSession(rt.state, row), last = logical.bindings.at(-1) + if (last?.workerId !== row.id || last.key !== row.bindingKey) throw refuse('Native worker is not the latest Task-role binding') + row.version = 'project-native-worker/v2' + row.sessionId = sessionId; row.model = bot.model ?? null; row.segments = clone(bot.segments ?? []) + last.sessionId = sessionId + const attempt = ownEntry(rt.state.projectAttempts ?? {}, row.attemptId) + const session = [attempt?.roleSessions?.[row.role], ...(attempt?.roleSessionHistory ?? [])].find(value => value?.id === row.sessionKey && value.botId === row.id) + if (session) session.sessionId = sessionId + row.releaseProof = { ...details, workerId: row.id, sessionId, bindingKey: row.bindingKey, generation: row.generation, at: Date.now() } + row.status = 'quiescent' + } + async function stop(botId) { + const row = ownEntry(rows(), botId) + if (!row) return false + if (nativeExitConfirmed(rt.state, row)) return clone(row.releaseProof) + const bot = rt.botOf(botId), sessionId = bot && rt.chatOf(botId) + row.status = 'stopping'; await rt.save() + try { + if (row.hostEpoch !== hostEpoch) { + const lease = ownEntry(rt.state.projectResourceLeases ?? {}, row.leaseId), scope = lease?.processScope ?? row.processScope + const proof = { kind: 'systemd-prior-invocation-exited', previousInvocationId: scope?.invocationId } + if (!bot || !supervisorProofMatches(scope, proof) || !await priorExit(scope)) throw refuse('Native execution belongs to an earlier host; actual prior process-group exit is unconfirmed') + if (ownEntry(rows(), botId) !== row || rt.botOf(botId) !== bot || rt.chatOf(botId) !== sessionId || row.leaseId && (!lease || lease.generation !== row.generation || lease.worker?.kind !== 'native' || lease.worker.botId !== botId || lease.projectId !== row.projectId || lease.attemptId !== row.attemptId || lease.role !== row.role || lease.sessionKey !== row.sessionKey || lease.resourceId !== row.resourceId || lease.processScope !== scope) || !row.leaseId && row.processScope !== scope) throw refuse('Native prior-host scope or binding changed during exit inspection') + recordExit(row, bot, proof) + await rt.save() + return clone(row.releaseProof) + } + const agent = sessionId && await rt.resolveAgent(sessionId) + if (!bot || !agent || agent.session?.id !== sessionId || typeof agent.cancel !== 'function' || typeof agent.whenIdle !== 'function') throw refuse('Native worker stop is unconfirmed; keep its execution/resource held') + agent.cancel({ kind: 'hook', reason: 'Project native execution release' }) + if (typeof rt.ctx.parallel !== 'function' || typeof rt.ctx.waterfall !== 'function') throw refuse('Native registered session work cannot be inspected; keep execution held') + const sessions = ownedSessions(row, bot, sessionId) + await Promise.all(sessions.map(sessionId => rt.ctx.parallel('workspace/session-stop', { sessionId }))) + await agent.whenIdle() + const reports = await Promise.all(sessions.map(sessionId => rt.ctx.waterfall('workspace/session-activity', { sessionId }, () => Promise.resolve([])))) + const activity = reports.flatMap(report => Array.isArray(report) ? report : [{ kind: 'unknown' }]) + if (activity.length) { + row.pendingWork = activity.map(value => ({ kind: value.kind, count: value.items?.length ?? 1 })) + throw refuse('Native registered session work is still stopping; keep execution and resources held') + } + delete row.pendingWork + if (ownEntry(rows(), botId) !== row || rt.botOf(botId) !== bot || rt.chatOf(botId) !== sessionId || agent.status !== 'idle') throw refuse('Native worker changed before quiescence confirmation') + recordExit(row, bot, { kind: 'native-agent-quiescent', registeredWorkEmpty: true }) + await rt.save() + return clone(row.releaseProof) + } catch (error) { row.status = 'stop-unconfirmed'; delete row.releaseProof; row.failure = String(error.message).slice(0, 1000); await rt.save(); throw error } + } + function admitsAgent(agent, track = false) { + let header = agent.session.header ?? { id: agent.session.id } + const visited = new Set() + while (!visited.has(header.id)) { + visited.add(header.id) + const botId = rt.selfOf(header.id), bot = botId && rt.botOf(botId) + const row = ownEntry(rt.state.projectNativeWorkers ?? {}, botId) + if (row) { + const admitted = api.admits(botId) + if (admitted && track) row.knownSessionIds = [...new Set([...(row.knownSessionIds ?? []), ...visited])] + return admitted + } + if (bot?.ephemeral && bot.workspaceId && bot.attemptId) return false + if (header.origin !== 'subagent' || !header.parentSession) return true + const parent = rt.ctx.sessions.get(header.parentSession) + header = parent?.header ?? { id: header.parentSession } + } + return false + } + const api = { previous, activate, admitsAgent, async validateStep(agent) { + if (!api.admitStep(agent)) return false + const row = Object.values(rows()).find(value => value.hostEpoch === hostEpoch && value.status === 'bound' && rt.botOf(value.id) && value.knownSessionIds?.includes(agent.session.id)) + if (!row) return true + const session = ownEntry(rt.state.projectAttempts ?? {}, row.attemptId)?.roleSessions?.[row.role] + if (!session?.poolClaimId) return true + try { await authority(session); return api.admits(row.id) } + catch (error) { + // Do not wait for this same Agent to become idle inside its pre-step hook. + void rt.projectPools?.revokeExecution(row.taskId, session.poolClaimId, String(error.message).slice(0, 1000)).catch(failure => rt.warn('Execution revocation: %s', failure.message)) + return false + } + }, admitStep: agent => admitsAgent(agent, true), currentHost: botId => Boolean(rt.botOf(botId) && ownEntry(rt.state.projectNativeWorkers ?? {}, botId)?.hostEpoch === hostEpoch), async bind(session, workspace, botId) { + const key = taskSessionId(session.projectId, session.taskId, session.role) + if (binding.has(key)) throw refuse('Native Task-role binding is already in progress') + binding.add(key) + try { return await bind(session, workspace, botId) } finally { binding.delete(key) } + }, admits(botId) { + const row = ownEntry(rt.state.projectNativeWorkers ?? {}, botId) + return !row || row.hostEpoch === hostEpoch && row.status === 'bound' && rt.readOnly?.() === undefined && Boolean(row.activationEffect && rt.projectEffects?.intentCommitted(row.activationEffect)) + }, + stop(botId) { + if (stopping.has(botId)) return stopping.get(botId) + const operation = stop(botId).finally(() => stopping.delete(botId)); stopping.set(botId, operation); return operation + } } + return api +} diff --git a/patches/ds-bot/files/src/host/project-pools.js b/patches/ds-bot/files/src/host/project-pools.js index 947f397a8..b38663262 100644 --- a/patches/ds-bot/files/src/host/project-pools.js +++ b/patches/ds-bot/files/src/host/project-pools.js @@ -19,6 +19,14 @@ export function createProjectPools(rt) { if (!target) throw refuse('Target logical Bot is not in this Project') return target } + const sourceOf = (task, botId) => { + const active = definitions(task.projectId).roles.find(row => row.id === botId) + if (active) return active + const claim = task.collaboration?.claim + const retired = Object.values(rt.state.projectRetiredRoles ?? {}).find(row => row.projectId === task.projectId && row.id === botId) + if (!retired || claim?.botId !== botId || !['active', 'completed', 'intervened'].includes(claim.status)) throw refuse('Source logical Bot has no retained execution claim') + return { ...retired, role: claim.role ?? retired.role, policyRole: claim.policyRole ?? retired.policyRole ?? retired.role } + } const dispatchIntent = (task, row) => rt.projectEffects?.stage({ kind: 'pool.dispatch', key: row.id, projectId: task.projectId, taskId: task.id, payload: { handoffId: row.id, botId: row.to, initiatedBy: row.initiatedBy } }) async function authority(task) { @@ -39,7 +47,7 @@ export function createProjectPools(rt) { async function checkAdmission(task, { from, to, attemptId, requestId, purpose }) { if (!adapter) throw refuse('Handoff execution adapter is unavailable') const c = task.collaboration, target = targetOf(task.projectId, to) - const fromRole = from ? targetOf(task.projectId, from).role : '' + const fromRole = from ? sourceOf(task, from).role : '' // A privileged, audited Chief decision resolves a routing loop. Keep global // usage/history, but bound autonomous repetitions after that intervention. const epoch = c.history.findLastIndex(row => row.purpose === 'chief-intervention') @@ -108,14 +116,16 @@ export function createProjectPools(rt) { async function claim(taskId, botId, handoffId) { const task = taskOf(taskId), c = task.collaboration if (!c || c.botId !== botId || c.status !== 'queued' || handoffId && c.currentHandoffId !== handoffId) return null - await authority(task); targetOf(task.projectId, botId) + await authority(task) + const definition = definitions(task.projectId).roles.find(row => row.id === botId) + if (!definition) { c.status = 'blocked'; c.block = { reason: 'Target role is retired; Chief must explicitly select an active role', at: Date.now() }; await rt.save(); return null } if (Object.values(rt.state.tasks).some(row => row.collaboration?.claim?.botId === botId && row.collaboration.claim.status === 'active')) return null - if (adapter.ready && !adapter.ready({ task, definition: targetOf(task.projectId, botId) })) return null + if (adapter.ready && !adapter.ready({ task, definition })) return null const receipt = c.history.find(row => row.id === c.currentHandoffId) if (!receipt || receipt.status !== 'pending') throw refuse('Task has no pending handoff receipt') const limit = rt.config.projectHandoff?.quota?.steps if (Number.isFinite(limit) && (c.usage?.steps ?? 0) >= limit) { c.status = 'blocked'; c.block = { reason: 'Handoff refused (quota): execution step quota is exhausted', at: Date.now() }; await rt.save(); throw refuse(c.block.reason) } - const claim = { id: `claim-${randomUUID()}`, botId, generation: ++c.generation, taskId: task.id, projectId: task.projectId, attemptId: c.attemptId, status: 'active', createdAt: Date.now() } + const claim = { id: `claim-${randomUUID()}`, botId, role: definition.role, policyRole: definition.policyRole ?? definition.role, generation: ++c.generation, taskId: task.id, projectId: task.projectId, attemptId: c.attemptId, status: 'active', createdAt: Date.now() } c.usage ??= {}; c.usage.steps = (c.usage.steps ?? 0) + 1 c.claim = claim; c.status = 'claimed'; receipt.status = 'accepted'; receipt.acceptedAt = Date.now(); receipt.claimId = claim.id await rt.save() @@ -133,7 +143,7 @@ export function createProjectPools(rt) { const claim = await current(task, token) const action = decision?.action, reason = reasonOf(decision?.reason) if (!['handoff', 'return', 'transfer', 'complete', 'arbitrate'].includes(action)) throw refuse('Bot must hand off, return, transfer or request Chief judgement') - const role = targetOf(task.projectId, claim.botId).role + const role = sourceOf(task, claim.botId).role const next = ['complete', 'arbitrate'].includes(action) ? definitions(task.projectId).roles.find(row => row.role === 'chief') : targetOf(task.projectId, decision.to) if (!next) throw refuse('Chief is not configured') // Stop confirmation precedes relinquishing the claim and capacity. @@ -150,7 +160,7 @@ export function createProjectPools(rt) { c.usage ??= {}; c.usage.ms = (c.usage.ms ?? 0) + Date.now() - claim.createdAt const usage = rt.state.projectHarnessWorkers?.[claim.workerId]?.usage c.usage.tokens = (c.usage.tokens ?? 0) + (usage?.input_tokens ?? 0) + (usage?.output_tokens ?? 0) - if (next.role === 'developer' && c.kind === 'code') c.attemptId = await adapter.rework?.({ task, claim, reason }) ?? c.attemptId + if ((next.policyRole ?? next.role) === 'developer' && c.kind === 'code') c.attemptId = await adapter.rework?.({ task, claim, reason }) ?? c.attemptId try { return await offer(task, { from: claim.botId, to: next.id, reason, requestId: decision.requestId ?? claim.id, attemptId: c.attemptId, evidence: decision.evidence ?? [], purpose: action, expectedGeneration: claim.generation }) } catch (error) { @@ -226,15 +236,23 @@ export function createProjectPools(rt) { } } } + async function revokeExecution(taskId, claimId, reason) { + const task = taskOf(taskId), c = task.collaboration + if (!c || c.claim?.id !== claimId || c.claim.status !== 'active') return false + const claim = c.claim + c.status = 'blocked'; ++c.generation; c.block = { reason, at: Date.now() } + claim.status = 'revoked' + await rt.save() + if (claim.workerId) { try { claim.exitConfirmed = await adapter.release(claim.workerId) } catch { claim.exitConfirmed = false } } + await rt.save() + return true + } async function revokePrincipal(projectId, memberId) { for (const task of Object.values(rt.state.tasks)) { const c = task.collaboration if (task.projectId !== projectId || c?.initiatedBy !== memberId || !['queued', 'claimed'].includes(c.status)) continue - c.status = 'blocked'; ++c.generation; c.block = { reason: 'Initiating human authority was revoked', at: Date.now() } - if (c.claim) c.claim.status = 'revoked' - await rt.save() - if (c.claim?.workerId) { try { c.claim.exitConfirmed = await adapter.release(c.claim.workerId) } catch { c.claim.exitConfirmed = false } } - await rt.save() + if (c.claim?.status === 'active') await revokeExecution(task.id, c.claim.id, 'Initiating human authority was revoked') + else { c.status = 'blocked'; ++c.generation; c.block = { reason: 'Initiating human authority was revoked', at: Date.now() }; await rt.save() } } } async function validateClosure(taskId, actor, evidence) { @@ -281,7 +299,7 @@ export function createProjectPools(rt) { ++c.generation if (oldClaim) oldClaim.status = 'intervened' c.status = 'stopping'; await rt.save() - if (oldClaim?.workerId && targetOf(task.projectId, oldClaim.botId).role !== 'chief') { + if (oldClaim?.workerId && sourceOf(task, oldClaim.botId).role !== 'chief') { let stopped = false try { stopped = await adapter.release(oldClaim.workerId) } catch { /* retain unknown worker capacity */ } oldClaim.exitConfirmed = stopped @@ -293,7 +311,7 @@ export function createProjectPools(rt) { if (currentReceipt) { currentReceipt.status = 'intervened'; currentReceipt.finishedAt = Date.now(); if (oldClaim) currentReceipt.execution = clone(oldClaim); currentReceipt.chiefDecisionId = body.decisionId } c.initiatedBy = actor.memberId; c.sourceSessionId = null const previous = rt.state.projectAttempts?.[c.attemptId] - if (target.role === 'developer') c.attemptId = await adapter.rework({ task, claim: oldClaim ?? { attemptId: c.attemptId, id: body.decisionId, botId: `${task.projectId}:chief` }, reason: body.reason, force: body.rework !== false, decisionId: body.decisionId }) + if ((target.policyRole ?? target.role) === 'developer') c.attemptId = await adapter.rework({ task, claim: oldClaim ?? { attemptId: c.attemptId, id: body.decisionId, botId: `${task.projectId}:chief` }, reason: body.reason, force: body.rework !== false, decisionId: body.decisionId }) else if (previous && previous.status !== 'running') throw refuse('Interrupted delivery must return to Developer in a new Attempt') return await offer(task, { from: `${task.projectId}:chief`, to: target.id, reason: body.reason, requestId: body.decisionId, attemptId: c.attemptId, purpose: 'chief-intervention' }) } catch (error) { @@ -325,7 +343,7 @@ export function createProjectPools(rt) { return changed } return { connect(value) { adapter = value; return () => { if (adapter === value) adapter = null } }, admit: (attempt, actor, body) => serial(() => admit(attempt, actor, body)), available, owns, - claim: (id, botId) => serial(() => claim(id, botId)), decide: (id, token, decision) => serial(() => decide(id, token, decision)), result: row => serial(() => result(row)), pump, recover, revokePrincipal, validateClosure, retryChief: (id, actor) => serial(() => retryChief(id, actor)), chiefClaim: (id, actor) => serial(() => chiefClaim(id, actor)), chiefRoute: (body, actor) => serial(() => chiefRoute(body, actor)) } + claim: (id, botId) => serial(() => claim(id, botId)), decide: (id, token, decision) => serial(() => decide(id, token, decision)), result: row => serial(() => result(row)), pump, recover, revokePrincipal: (projectId, memberId) => serial(() => revokePrincipal(projectId, memberId)), revokeExecution: (id, claimId, reason) => serial(() => revokeExecution(id, claimId, reason)), validateClosure, retryChief: (id, actor) => serial(() => retryChief(id, actor)), chiefClaim: (id, actor) => serial(() => chiefClaim(id, actor)), chiefRoute: (body, actor) => serial(() => chiefRoute(body, actor)) } } export function install(rt) { rt.projectPools = createProjectPools(rt) diff --git a/patches/ds-bot/files/src/host/project-process-proof.js b/patches/ds-bot/files/src/host/project-process-proof.js index 5da21cdf6..d641c943b 100644 --- a/patches/ds-bot/files/src/host/project-process-proof.js +++ b/patches/ds-bot/files/src/host/project-process-proof.js @@ -24,3 +24,7 @@ export async function priorScopeExited(scope, options = {}) { // Different machines/boots, unmanaged launchers or changed kill policy remain unknown. return current.kind === 'systemd-control-group' && current.bootId === scope.bootId && current.unit === scope.unit && current.controlGroup === scope.controlGroup && current.invocationId !== scope.invocationId } +export function supervisorProofMatches(scope, proof) { + return Boolean(scope?.kind === 'systemd-control-group' && scope.killMode === 'control-group' && /^[a-f0-9]{32}$/.test(scope.invocationId ?? '') && + proof?.kind === 'systemd-prior-invocation-exited' && proof.previousInvocationId === scope.invocationId) +} diff --git a/patches/ds-bot/files/src/host/project-resources.js b/patches/ds-bot/files/src/host/project-resources.js index 3602f9328..3541227b8 100644 --- a/patches/ds-bot/files/src/host/project-resources.js +++ b/patches/ds-bot/files/src/host/project-resources.js @@ -1,5 +1,6 @@ // Operator-provisioned execution capabilities; Project configuration only references IDs. -import { captureProcessScope, priorScopeExited } from './project-process-proof.js' +import { nativeExitConfirmed } from './project-native-sessions.js' +import { captureProcessScope, priorScopeExited, supervisorProofMatches } from './project-process-proof.js' import { execFile, spawn } from 'node:child_process' import { realpath, copyFile, stat } from 'node:fs/promises' import { randomUUID, createHash } from 'node:crypto' @@ -13,7 +14,7 @@ const active = lease => lease.status !== 'released' const contains = (root, path) => { const part = relative(root, path); return part === '' || part !== '..' && !part.startsWith(`..${sep}`) && !isAbsolute(part) } const safeEnv = () => ({ PATH: process.env.PATH ?? '/usr/local/bin:/usr/bin:/bin', LANG: 'C.UTF-8' }) -export function createProjectResources(rt) { +export function createProjectResources(rt, { priorExit = priorScopeExited } = {}) { const { state, config } = rt const leases = () => (state.projectResourceLeases ??= {}) const resources = () => config.projectResources ?? {} @@ -120,9 +121,14 @@ export function createProjectResources(rt) { const lease = requireLease(id, generation) if (lease.status === 'released') return const worker = lease.worker - const confirmed = !worker || worker.kind === 'native' && !ownEntry(state.bots ?? {}, worker.botId) || confirmation?.handle === worker.handle && confirmation?.exited === true + const native = worker?.kind === 'native' && ownEntry(state.projectNativeWorkers ?? {}, worker.botId) + const scope = lease.processScope + const supervisor = worker?.kind === 'native' && supervisorProofMatches(scope, confirmation?.proof) && await priorExit(scope) + if (requireLease(id, generation) !== lease || lease.worker !== worker || lease.processScope !== scope) throw failure('Execution resource binding changed during exit inspection') + if (lease.status === 'released') return + const confirmed = supervisor || !worker || worker.kind === 'native' && !ownEntry(state.bots ?? {}, worker.botId) && native?.leaseId === lease.id && nativeExitConfirmed(state, native) || worker.kind !== 'native' && typeof worker.handle === 'string' && worker.handle.length > 0 && confirmation?.handle === worker.handle && confirmation?.exited === true lease.status = confirmed ? 'released' : 'stopping' - if (confirmed) { lease.releasedAt = Date.now(); lease.releaseProof ??= confirmation?.proof ?? { kind: worker ? 'host-confirmed-worker-exit' : 'unbound-reservation-release', at: lease.releasedAt } } + if (confirmed) { lease.releasedAt = Date.now(); lease.releaseProof ??= confirmation?.proof ?? native?.releaseProof ?? { kind: worker ? 'host-confirmed-worker-exit' : 'unbound-reservation-release', at: lease.releasedAt } } await rt.save() if (confirmed) rt.projectScheduler?.wake() } @@ -139,8 +145,8 @@ export function createProjectResources(rt) { async function reconcile() { let changed = false for (const lease of Object.values(leases()).filter(active)) { - if (lease.processScope?.hostPid === process.pid && (lease.worker?.kind === 'native' || lease.worker?.kind === 'sdk' && rt.projectHarness?.hasController(lease.worker.handle) || lease.worker?.kind === 'native-turn' && rt.projectAgents?.hasExchange(lease.worker.handle))) continue - const priorExited = lease.processScope && await priorScopeExited(lease.processScope) + if (lease.processScope?.hostPid === process.pid && (lease.worker?.kind === 'native' && rt.projectDelivery?.nativeSessions?.currentHost(lease.worker.botId) || lease.worker?.kind === 'sdk' && rt.projectHarness?.hasController(lease.worker.handle) || lease.worker?.kind === 'native-turn' && rt.projectAgents?.hasExchange(lease.worker.handle))) continue + const priorExited = lease.processScope && await priorExit(lease.processScope) // The supervisor read yields: a confirmed concurrent release is final. if (!active(lease)) continue if (priorExited) { @@ -150,14 +156,15 @@ export function createProjectResources(rt) { if (!bot.ephemeral || !bot.attemptId) continue await rt.flybot.scope('flybot-workflow').bot.remove(bot.id) } - lease.releaseProof = { kind: 'systemd-prior-invocation-exited', previousInvocationId: lease.processScope.invocationId, at: Date.now() } - await release(lease.id, lease.generation, { handle: lease.worker?.handle, exited: true }) + if (!active(lease)) { changed = true; continue } + const proof = { kind: 'systemd-prior-invocation-exited', previousInvocationId: lease.processScope.invocationId, at: Date.now() } + await release(lease.id, lease.generation, { handle: lease.worker?.handle, exited: true, proof }) const row = rt.state.projectHarnessWorkers?.[lease.worker?.handle] if (row) { row.status = 'released'; row.releasedAt = Date.now(); row.releaseProof = lease.releaseProof; await rt.projectDelivery?.releaseSession(row.id); changed = true } continue } if (lease.processScope?.hostPid === process.pid) { - if (lease.worker?.kind === 'native' || lease.worker?.kind === 'sdk' && rt.projectHarness?.hasController(lease.worker.handle) || lease.worker?.kind === 'native-turn' && rt.projectAgents?.hasExchange(lease.worker.handle)) continue + if (lease.worker?.kind === 'native' && rt.projectDelivery?.nativeSessions?.currentHost(lease.worker.botId) || lease.worker?.kind === 'sdk' && rt.projectHarness?.hasController(lease.worker.handle) || lease.worker?.kind === 'native-turn' && rt.projectAgents?.hasExchange(lease.worker.handle)) continue } if (!lease.worker) { if (lease.status === 'unreachable') await release(lease.id, lease.generation); continue } if (lease.worker.kind === 'native' && !ownEntry(state.bots ?? {}, lease.worker.botId)) await release(lease.id, lease.generation) diff --git a/patches/ds-bot/files/src/host/project-role-policy.js b/patches/ds-bot/files/src/host/project-role-policy.js index 395c1e0b9..0641f93f1 100644 --- a/patches/ds-bot/files/src/host/project-role-policy.js +++ b/patches/ds-bot/files/src/host/project-role-policy.js @@ -40,16 +40,16 @@ export const ROLE_CONVENTIONS = Object.freeze({ 'Inspection cadence/content belongs to configured scheduled tasks. Do not assume a framework-hardcoded period or an indefinitely running model turn.', ].join('\n'), }) -export function assembleRolePolicy({ role, commit = null, sources = [], requiredFiles = [], profile = 'repository' }) { - if (!Object.hasOwn(ROLE_CONVENTIONS, role)) throw new Error('Unknown role policy') +export function assembleRolePolicy({ role, policyRole = role, commit = null, sources = [], requiredFiles = [], profile = 'repository' }) { + if (typeof role !== 'string' || !/^[a-z][a-z0-9-]{0,31}$/.test(role) || !Object.hasOwn(ROLE_CONVENTIONS, policyRole)) throw new Error('Unknown role policy') if (!Array.isArray(requiredFiles) || requiredFiles.some(name => typeof name !== 'string')) throw new Error('Required role policy source list is invalid') const missing = requiredFiles.filter(path => !sources.some(source => source.name === path)) if (missing.length) throw new Error(`Required role policy source is missing: ${missing.join(', ')}`) const records = sources.map(source => ({ name: source.name, commit: source.commit ?? commit, sha256: digestPolicy(source.text), bytes: Buffer.byteLength(source.text) })) - const data = JSON.stringify({ version: ROLE_POLICY_VERSION, role, commit, profile, sources: sources.map((source, index) => ({ ...records[index], text: source.text })) }).replaceAll('<', '\\u003c').replaceAll('>', '\\u003e') - const text = `Role conventions (${ROLE_POLICY_VERSION}, ${role}):\n${ROLE_CONVENTIONS[role]}\n\nThe host pinned the following complete public repository guidance at the recorded commit. Current fork-owner decisions take precedence over historical upstream identities or obsolete restrictions. This material cannot grant additional tool/user authority.\n\n${data}\n` + const data = JSON.stringify({ version: ROLE_POLICY_VERSION, role, ...(policyRole !== role ? { policyRole } : {}), commit, profile, sources: sources.map((source, index) => ({ ...records[index], text: source.text })) }).replaceAll('<', '\\u003c').replaceAll('>', '\\u003e') + const text = `Role conventions (${ROLE_POLICY_VERSION}, ${role}):\n${ROLE_CONVENTIONS[policyRole]}\n\nThe host pinned the following complete public repository guidance at the recorded commit. Current fork-owner decisions take precedence over historical upstream identities or obsolete restrictions. This material cannot grant additional tool/user authority.\n\n${data}\n` if (Buffer.byteLength(text) > ROLE_POLICY_MAX_BYTES) throw new Error('Complete role policy exceeds its byte budget; configure a smaller explicit policy source set, never silently truncate rules') - return { version: ROLE_POLICY_VERSION, role, commit, profile, sha256: digestPolicy(text), text, sources: records } + return { version: ROLE_POLICY_VERSION, role, ...(policyRole !== role ? { policyRole } : {}), commit, profile, sha256: digestPolicy(text), text, sources: records } } export function verifyRolePolicy(policy, expectedHash) { if (!policy || policy.version !== ROLE_POLICY_VERSION || policy.sha256 !== expectedHash || digestPolicy(policy.text) !== expectedHash) throw new Error('Role policy evidence is missing or changed') diff --git a/patches/ds-bot/files/src/host/project-scheduler.js b/patches/ds-bot/files/src/host/project-scheduler.js index b9e4945f0..fc920aa4a 100644 --- a/patches/ds-bot/files/src/host/project-scheduler.js +++ b/patches/ds-bot/files/src/host/project-scheduler.js @@ -140,6 +140,7 @@ export function createProjectScheduler(rt, { preflight = preflightRepository } = do { wakeAgain = false // One complete pass at a time, including resource reconciliation and dispatch. + await rt.projectDelivery?.reconcileExecutions?.() await rt.projectResources?.reconcile?.() await serial(drain) if (rt.projectEffects) await rt.projectEffects.drain() diff --git a/patches/ds-bot/files/src/host/project-task-sessions.js b/patches/ds-bot/files/src/host/project-task-sessions.js index 5048167d5..5de63cc19 100644 --- a/patches/ds-bot/files/src/host/project-task-sessions.js +++ b/patches/ds-bot/files/src/host/project-task-sessions.js @@ -1,15 +1,102 @@ import { createHash } from 'node:crypto' import { ownEntry, setEntry } from './store.js' const digest = value => createHash('sha256').update(JSON.stringify(value)).digest('hex') +const resumeError = message => Object.assign(new Error(message), { code: 'bot/context-resume' }) export const taskSessionId = (projectId, taskId, role) => `task-session-${digest([projectId, taskId, role]).slice(0, 32)}` +function contextDefinition(session) { + const text = session.context?.text + const start = '\n', end = '\n' + if (typeof text !== 'string' || !text.includes(start) || !text.endsWith(end)) throw resumeError('Original role context is unavailable; cannot infer a new Session') + if (!/^[a-f0-9]{64}$/.test(session.context.sha256 ?? '') || createHash('sha256').update(text).digest('hex') !== session.context.sha256) throw resumeError('Original role context hash is unavailable or changed') + let data + try { data = JSON.parse(text.slice(text.indexOf(start) + start.length, -end.length)) } catch { throw resumeError('Original role context cannot be decoded') } + if (data.version !== 'project-role-context/v1' || data.projectId !== session.projectId || data.role !== session.role || typeof data.persona !== 'string' || !Array.isArray(data.skills)) throw resumeError('Original role context has another scope or contract') + return { role: session.role, instructions: data.persona, resolvedSkills: data.skills, skills: data.skills.map(skill => skill.name), revision: data.revision } +} +export function roleDefinitionKey(session) { + if (session.definitionKey !== undefined) { + if (typeof session.definitionKey !== 'string' || !/^[a-f0-9]{64}$/.test(session.definitionKey)) throw resumeError('Role definition fingerprint is invalid') + return session.definitionKey + } + const definition = session.definition ?? contextDefinition(session) + return digest({ role: session.role, ...(definition.policyRole && definition.policyRole !== session.role ? { policyRole: definition.policyRole } : {}), persona: definition.instructions ?? '', skills: (definition.resolvedSkills ?? definition.skills ?? []).map(skill => typeof skill === 'string' ? skill : { name: skill.name, sha256: skill.sha256 }) }) +} +function workerSession(state, worker) { + const attempt = ownEntry(state.projectAttempts ?? {}, worker.attemptId) + if (!attempt || attempt.projectId !== worker.projectId || attempt.taskId !== worker.taskId) return null + return [attempt.roleSessions?.[worker.role], ...(attempt.roleSessionHistory ?? [])].find(session => session?.id === worker.sessionKey && session.projectId === worker.projectId && session.taskId === worker.taskId && session.role === worker.role) ?? null +} +function latestRoleWorker(state, record) { + return Object.entries(state.projectHarnessWorkers ?? {}).filter(([, worker]) => worker.logicalSessionId === record.id && worker.projectId === record.projectId && worker.taskId === record.taskId && worker.role === record.role) + .sort((a, b) => (a[1].createdAt ?? 0) - (b[1].createdAt ?? 0)).at(-1) +} +export function pinTaskRoleDefinition(state, scope, definition, revision = 0, catalog) { + catalog ??= ownEntry(state.projectAgentDefinitions ?? {}, scope.projectId) + const currentRevision = catalog?.revision ?? revision + revision = currentRevision + const logical = taskSession(state, scope, catalog) + state.projectRoleDefinitionSnapshots ??= {} + const snapshots = state.projectRoleDefinitionSnapshots + let frozen = logical.definitionRef && ownEntry(snapshots, logical.definitionRef) + if (logical.definitionRef && (!frozen || frozen.version !== 'project-role-definition/v1' || frozen.projectId !== scope.projectId || frozen.role !== scope.role || frozen.sha256 !== digest(frozen.definition))) throw resumeError('Pinned Task-role definition is unavailable or changed') + if (!frozen) { + const initial = Object.fromEntries(Object.entries(definition).filter(([key]) => key !== 'memory')) + let selected = initial, origin = 'initial-definition' + const previous = logical.bindings.at(-1) + const observed = latestRoleWorker(state, logical) + const worker = observed?.[1].backendSessionId ? observed[1] : previous && ownEntry(state.projectHarnessWorkers ?? {}, previous.workerId) + if (worker) { + const session = worker && workerSession(state, worker) + if (!session) throw resumeError('Original Task-role Session is unavailable for definition migration') + const old = session.definition ?? contextDefinition(session) + selected = { ...initial, ...Object.fromEntries(Object.entries(old).filter(([key]) => key !== 'revision')), execution: session.execution ?? initial.execution } + revision = session.definitionRevision ?? old.revision ?? revision + origin = 'legacy-latest-role-context' + } + const sha256 = digest(selected), id = `role-definition-${digest([scope.projectId, scope.role, sha256])}` + frozen = ownEntry(snapshots, id) + if (!frozen) { frozen = { version: 'project-role-definition/v1', id, projectId: scope.projectId, role: scope.role, revision, sha256, definition: structuredClone(selected), origin, createdAt: Date.now() }; setEntry(snapshots, id, frozen) } + logical.definitionRef = id + } + const registered = catalog?.roles.find(row => row.role === scope.role) + if (catalog && !registered || registered?.status === 'retired') throw resumeError('Task role was retired; explicit reassignment is required') + const live = registered ?? definition + const current = live.execution ?? {}, pinned = frozen.definition.execution ?? {} + const available = (pinned.resourceIds ?? []).filter(id => (current.resourceIds ?? []).includes(id)) + if ((live.policyRole ?? live.role) !== (frozen.definition.policyRole ?? frozen.definition.role) || (current.harness ?? 'flybot') !== (pinned.harness ?? 'flybot') || pinned.resourceIds?.length && !available.length || frozen.definition.toolPolicy?.shell === true && live.toolPolicy?.shell === false) throw resumeError('Task-role execution permission or resource binding was revoked; explicit recovery is required') + const retained = structuredClone(frozen.definition) + if (retained.execution) retained.execution.resourceIds = available + const observed = latestRoleWorker(state, logical) + const priorWorker = observed?.[1].backendSessionId ? observed[1] : ownEntry(state.projectHarnessWorkers ?? {}, logical.bindings.at(-1)?.workerId) + if (priorWorker && available.length) { + if (!available.includes(priorWorker.resourceId)) throw resumeError('The prior Session resource is no longer authorized') + retained.execution.resourceIds = [priorWorker.resourceId] + } + return { definition: retained, revision: frozen.revision, definitionRef: frozen.id, definitionKey: roleDefinitionKey({ role: scope.role, definition: frozen.definition }), + ...(frozen.sha256 !== digest(Object.fromEntries(Object.entries(definition).filter(([key]) => key !== 'memory'))) ? { change: { pinnedRevision: frozen.revision, currentRevision, retainedOriginalDefinition: true } } : {}) } +} +export function discardUnpreparedTaskDefinitions(state, taskId) { + let changed = false + for (const logical of Object.values(state.projectTaskSessions ?? {})) { + if (logical.taskId !== taskId || !logical.definitionRef || logical.bindings.length || latestRoleWorker(state, logical)) continue + const prepared = Object.values(state.projectAttempts ?? {}).some(attempt => attempt.taskId === taskId && attempt.projectId === logical.projectId && (attempt.roleSessions?.[logical.role] || attempt.roleSessionHistory?.some(session => session.role === logical.role))) + if (prepared) continue + delete logical.definitionRef + changed = true + } + return changed +} + // Logical Sessions outlive processes and Attempts. Backend bindings never cross cwd/profile. -export function taskSession(state, session) { +export function taskSession(state, session, currentCatalog) { const task = ownEntry(state.tasks ?? {}, session.taskId) - if (!task || task.projectId !== session.projectId || !['developer', 'tester', 'reviewer'].includes(session.role)) throw new Error('Task Session scope changed') - state.projectTaskSessions ??= {} const id = taskSessionId(session.projectId, session.taskId, session.role) - let record = ownEntry(state.projectTaskSessions, id) + let record = ownEntry(state.projectTaskSessions ?? {}, id) + const catalog = currentCatalog ?? ownEntry(state.projectAgentDefinitions ?? {}, session.projectId) + const registered = catalog?.roles.find(row => row.role === session.role && row.status !== 'retired') + if (!task || task.projectId !== session.projectId || typeof session.role !== 'string' || !/^[a-z][a-z0-9-]{0,31}$/.test(session.role) || session.role === 'chief' || !record && (catalog ? !registered : !['developer', 'tester', 'reviewer'].includes(session.role))) throw new Error('Task Session scope changed') + state.projectTaskSessions ??= {} if (!record) { record = { id, projectId: session.projectId, taskId: session.taskId, role: session.role, bindings: [], createdAt: Date.now() }; setEntry(state.projectTaskSessions, id, record) } if (record.projectId !== session.projectId || record.taskId !== session.taskId || record.role !== session.role) throw new Error('Task Session identity changed') return record @@ -35,11 +122,32 @@ export function confirmedWorkerExit(state, worker) { lease.worker?.kind === 'sdk' && lease.worker.handle === worker.id && lease.processScope?.kind === 'systemd-control-group' && lease.processScope.killMode === 'control-group' && lease.processScope.invocationId === proof.previousInvocationId && lease.releaseProof?.kind === proof.kind && lease.releaseProof.previousInvocationId === proof.previousInvocationId) } -export function resumableBinding(state, record, key) { - const previous = record.bindings.findLast(row => row.key === key) +export function resumableBinding(state, record, key, { legacyKey, definitionKey } = {}) { + let previous = record.bindings.at(-1) + if (previous && !ownEntry(state.projectHarnessWorkers ?? {}, previous.workerId)) throw resumeError('Latest Task Session binding is unconfirmed') + const latest = latestRoleWorker(state, record) + if (latest && latest[0] !== previous?.workerId) { + const [workerId, worker] = latest + const lease = ownEntry(state.projectResourceLeases ?? {}, worker.leaseId) + const released = lease?.status === 'released' && lease.projectId === record.projectId && lease.generation === worker.generation && lease.resourceId === worker.resourceId + const cold = !worker.backendSessionId && (worker.executionStartedAt === null || worker.executionStartedAt === undefined && worker.releaseProof?.kind === 'sdk-worker-spawn-failed') + if (cold && released && worker.status === 'released' && worker.releaseProof?.generation === worker.generation && worker.releaseProof?.workerId === workerId) { + // A persist-before-dispatch marker (or proven failed spawn) establishes that + // no model context was started. Keep any earlier real context authoritative. + } else { + if (!confirmedWorkerExit(state, worker) || !released) throw resumeError('Latest Task Session exit is unconfirmed') + if (!worker.backendSessionId) throw resumeError('Latest Task-role context has no recoverable backend Session') + previous = { version: worker.bindingVersion, definitionKey: worker.definitionKey, key: worker.bindingKey, sessionId: worker.backendSessionId, workerId, attemptId: worker.attemptId, policySha256: workerSession(state, worker)?.policy?.sha256 ?? null, recoveredFromWorker: true } + } + } if (!previous) return null + if (previous.key !== key) { + const worker = ownEntry(state.projectHarnessWorkers ?? {}, previous.workerId) + const session = worker && workerSession(state, worker) + if (previous.version || previous.key !== legacyKey || !session || roleDefinitionKey(session) !== definitionKey) throw resumeError('Latest Task-role context cannot resume with this execution binding; a fresh Session is forbidden') + } const worker = ownEntry(state.projectHarnessWorkers ?? {}, previous.workerId) const lease = worker && ownEntry(state.projectResourceLeases ?? {}, worker.leaseId) - if (!worker || worker.projectId !== record.projectId || worker.taskId !== record.taskId || worker.role !== record.role || worker.logicalSessionId !== record.id || worker.bindingKey !== key || !confirmedWorkerExit(state, worker) || lease?.status !== 'released' || lease.projectId !== record.projectId || lease.resourceId !== worker.resourceId || lease.generation !== worker.generation || worker.backendSessionId !== previous.sessionId) throw new Error('Previous Task Session exit or binding is unconfirmed') + if (!worker || worker.projectId !== record.projectId || worker.taskId !== record.taskId || worker.role !== record.role || worker.logicalSessionId !== record.id || worker.bindingKey !== previous.key || !confirmedWorkerExit(state, worker) || lease?.status !== 'released' || lease.projectId !== record.projectId || lease.resourceId !== worker.resourceId || lease.generation !== worker.generation || worker.backendSessionId !== previous.sessionId) throw resumeError('Previous Task Session exit or binding is unconfirmed') return previous } diff --git a/patches/ds-bot/files/test/project-delivery.test.js b/patches/ds-bot/files/test/project-delivery.test.js index ea094763b..0eff1cf8a 100644 --- a/patches/ds-bot/files/test/project-delivery.test.js +++ b/patches/ds-bot/files/test/project-delivery.test.js @@ -210,7 +210,7 @@ test('managed delivery definitions and role/model snapshots cannot be edited or const attempt = fixture.attempt() assert.equal(attempt.workflowDefinition.nodes[0].model, 'fixture/model') for (const operation of ['graph', 'set-model', 'concurrency', 'reset', 'delete']) await assert.rejects(fixture.rt.projectEndpoint('project-workflow', { projectId: 'p1', workflowId: attempt.workflowId, operation, nodes: [] }, alice), /immutable/) - assert.equal(attempt.workflowVersion, 'project-delivery/v6') + assert.equal(attempt.workflowVersion, 'project-delivery/v7') }) @@ -292,7 +292,9 @@ test('temporary role context freezes memory, repository instructions and session fixture.rt.bindProject({ kind: 'run', id: first.runId, projectId: 'p1', status: 'failed', taskId: fixture.task.id, attemptId: first.id }) await fixture.start('session-second') assert.notEqual(fixture.attempt().roleSessions.developer.id, descriptor.id) - assert.match(fixture.attempt().roleSessions.developer.context.text, /New persona/) + assert.doesNotMatch(fixture.attempt().roleSessions.developer.context.text, /New persona/) + assert.equal(fixture.attempt().roleSessions.developer.definitionRef, descriptor.definitionRef) + assert.equal(fixture.attempt().roleSessions.developer.definitionChange.retainedOriginalDefinition, true) assert.match(fixture.attempt().roleSessions.developer.context.text, /previousAttemptId/) }) @@ -790,7 +792,7 @@ test('failed context admission preserves an existing unsealed developer workspac const path=a.workspaces.developer.path writeFileSync(join(path,'code.mjs'),'export const preserved = true;\n') a.seedSha=a.baseSha - dev.instructions='x'.repeat(49000) // Required persona cannot be summarized away. + a.feedback='x'.repeat(49000) // Required current Task input cannot be summarized away. await assert.rejects(f.rt.projectDelivery.preparePoolRole(task,{id:'second',attemptId:a.id,generation:2},dev),/exceeds/) assert.equal(readFileSync(join(path,'code.mjs'),'utf8'),'export const preserved = true;\n') assert.equal(a.preparationFailures.at(-1).releasedBeforeExternalStart,false) @@ -1350,3 +1352,49 @@ test('Chief non-rework recovery retains a clean interrupted Attempt and its appr assert.equal(git(path,['rev-parse','HEAD']),before.head);assert.equal(git(path,['status','--porcelain']),before.status) assert.equal(Object.keys(f.rt.state.projectAttempts).length,1) }) + +test('approved custom role capabilities preserve logical identities while sealing testing and reviewing real code', async t => { + const f = await setup(t, undefined, { pool: true }) + f.rt.projectPools.connect({ validate: () => ({ ok: true }) }) + const original = f.rt.projectAgents.deliveryDefinitions('p1') + const names = { developer: 'implementer', tester: 'validator', reviewer: 'auditor' } + // Governance approval is tested through real host APIs separately. This fixture + // starts from an approved catalog and exercises the actual artifact controller. + for (const [policyRole, role] of Object.entries(names)) { + const definition = structuredClone(original.roles.find(row => row.role === policyRole)) + delete definition.memory; delete definition.resolvedSkills; delete definition.definitionRevision + f.rt.state.projectAgentDefinitions.p1.roles.push({ ...definition, id: `p1:${role}`, role, policyRole, name: role }) + } + ++f.rt.state.projectAgentDefinitions.p1.revision + await f.start('custom-role-artifacts') + const attempt = f.attempt(), task = f.rt.state.tasks[f.task.id] + const prepare = async (policyRole, generation) => { + const role = names[policyRole], definition = f.rt.projectAgents.deliveryDefinitions('p1').roles.find(row => row.role === role) + await f.rt.projectDelivery.preparePoolRole(task, { id: `custom-${role}`, attemptId: attempt.id, generation }, definition) + return role + } + const dev = await prepare('developer', 1) + const workspace = attempt.workspaces[dev] + assert.equal(workspace.role, 'implementer') + const backup = { ...structuredClone(f.rt.state.projectAgentDefinitions.p1.roles.find(row => row.role === 'implementer')), id: 'p1:backup', role: 'backup' } + f.rt.state.projectAgentDefinitions.p1.roles.push(backup); ++f.rt.state.projectAgentDefinitions.p1.revision + f.rt.config.projectRepositories['example/demo'].requiredRolePolicySources = { backup: ['MISSING.md'] } + await assert.rejects(f.rt.projectDelivery.preparePoolRole(task, { id: 'bad-backup', attemptId: attempt.id, generation: 2 }, f.rt.projectAgents.deliveryDefinitions('p1').roles.find(row => row.role === 'backup')), /Required role policy source is missing/) + assert.equal(attempt.roleAssignments.developer, 'implementer', 'Failed preparation cannot replace the actual artifact producer') + assert.match(attempt.roleSessions[dev].policy.text, /Never invent passing tests/) + writeFileSync(join(workspace.path, 'code.mjs'), 'export const add = (a,b) => a+b;\n') + await f.rt.projectDelivery.completePoolRole({ task, claim: { attemptId: attempt.id }, role: dev, outputs: { implementation: 'Correct addition' } }) + assert.equal(attempt.evidence.implementation.sha, attempt.candidateSha) + const tester = await prepare('tester', 2) + await f.rt.projectDelivery.completePoolRole({ task, claim: { attemptId: attempt.id }, role: tester, outputs: { tests: 'Request actual configured host checks' } }) + assert.equal(attempt.evidence.tests.passed, true) + const reviewer = await prepare('reviewer', 3) + await f.rt.projectDelivery.completePoolRole({ task, claim: { attemptId: attempt.id }, role: reviewer, outputs: { review: JSON.stringify({ commit: attempt.candidateSha, verdict: 'approve', summary: 'Fixture candidate recommendation, not signed forge review' }) } }) + assert.equal(attempt.evidence.review.sha, attempt.candidateSha) + assert.equal(attempt.pr.sha, attempt.candidateSha) + assert.deepEqual(attempt.roleAssignments, names) + assert.equal(attempt.evidence.implementation.logicalRole, dev) + assert.equal(attempt.evidence.tests.logicalRole, tester) + assert.equal(attempt.evidence.review.logicalRole, reviewer) + for (const role of Object.values(names)) assert.equal(attempt.roleSessions[role].logicalSessionId, Object.values(f.rt.state.projectTaskSessions).find(row => row.role === role).id) +}) diff --git a/patches/ds-bot/files/test/project-execution-authority.test.js b/patches/ds-bot/files/test/project-execution-authority.test.js new file mode 100644 index 000000000..f1da6f7d0 --- /dev/null +++ b/patches/ds-bot/files/test/project-execution-authority.test.js @@ -0,0 +1,78 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { pinTaskRoleDefinition } from '../src/host/project-task-sessions.js' +import { requireExecutionAuthority, reconcileExecutionAuthority } from '../src/host/project-execution-authority.js' +function fixture() { + const definition = { id: 'p:developer', role: 'developer', policyRole: 'developer', instructions: 'Retain original persona', toolPolicy: { shell: true }, execution: { harness: 'codex', resourceIds: ['local'] } } + const claim = { id: 'claim', status: 'active', role: 'developer', botId: definition.id, generation: 1, workerId: 'worker' } + const session = { id: 'role-session', projectId: 'p', taskId: 't', attemptId: 'a', role: 'developer', generation: 1, poolClaimId: claim.id, botId: claim.workerId, resourceLease: { id: 'lease', resourceId: 'local' }, status: 'running' } + const task = { id: 't', projectId: 'p', status: 'open', deliveryAttemptId: 'a', collaboration: { initiatedBy: 'human', generation: 1, claim } } + const catalog = { revision: 1, roles: [definition] } + const rt = { state: { tasks: { t: task }, projectAttempts: { a: { roleSessions: { developer: session } } } }, readOnly: () => undefined, projects: { requireActive() {} }, requireProjectAccess() {}, projectDelegations: { async background() {} }, projectAgents: { deliveryDefinitions: () => catalog }, projectResources: { requireBinding() {} } } + Object.assign(session, pinTaskRoleDefinition(rt.state, session, definition, 1, catalog)) + return { rt, definition, claim, session, task, catalog } +} +test('ongoing execution retains persona but refuses live role capability and resource revocation', async () => { + const f = fixture() + f.definition.instructions = 'New persona'; f.definition.model = 'new-model'; f.catalog.revision++ + await requireExecutionAuthority(f.rt, f.session) + assert.equal(f.session.definition.instructions, 'Retain original persona') + f.definition.toolPolicy.shell = false + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /revoked/) + f.definition.toolPolicy.shell = true; f.definition.execution.resourceIds = [] + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /revoked/) +}) +test('retired role may finish its retained active claim but cannot borrow another role identity', async () => { + const f = fixture(); f.catalog.roles = [] + f.rt.state.projectRetiredRoles = { 'p:developer': structuredClone(f.definition) } + await requireExecutionAuthority(f.rt, f.session) + f.rt.state.projectRetiredRoles['p:developer'].id = 'p:foreign' + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /retired/) +}) +test('live authority rechecks membership and claim after asynchronous delegation', async () => { + const f = fixture() + f.rt.projectDelegations.background = async () => { f.claim.status = 'revoked' } + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /revoked during/) + f.claim.status = 'active'; f.rt.projectDelegations.background = async () => {} + f.rt.requireProjectAccess = () => { throw Error('Member was removed') } + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /Member was removed/) +}) +test('current implicit native host cannot survive a newly explicit role resource restriction', async () => { + const f = fixture(); delete f.session.resourceLease + f.definition.execution = { harness: 'flybot', resourceIds: [] } + delete f.session.definitionRef + f.rt.state.projectTaskSessions = {}; f.rt.state.projectRoleDefinitionSnapshots = {} + Object.assign(f.session, pinTaskRoleDefinition(f.rt.state, f.session, f.definition, 1, f.catalog)) + await requireExecutionAuthority(f.rt, f.session) + f.definition.execution.resourceIds = ['local'] + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /Implicit host/) +}) +test('scheduler reconciliation stops only the exact current claim, preserving released sessions and other tasks', async () => { + const f = fixture(), stopped = [] + f.rt.projectPools = { async revokeExecution(...args) { stopped.push(args); f.claim.status = 'revoked' } } + await reconcileExecutionAuthority(f.rt); assert.equal(stopped.length, 0) + f.definition.execution.resourceIds = [] + await reconcileExecutionAuthority(f.rt) + assert.equal(stopped.length, 1); assert.deepEqual(stopped[0].slice(0, 2), ['t', 'claim']) + await reconcileExecutionAuthority(f.rt); assert.equal(stopped.length, 1) + f.claim.status = 'active'; f.session.status = 'released' + await reconcileExecutionAuthority(f.rt); assert.equal(stopped.length, 1) +}) + +test('missing initiator and Task identity replacement fail closed during authority checks', async () => { + const f = fixture(); delete f.task.collaboration.initiatedBy + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /initiating human/) + f.task.collaboration.initiatedBy = 'human' + f.rt.projectDelegations.background = async () => { f.rt.state.tasks.t = structuredClone(f.task) } + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /Task scope changed/) +}) + +test('operator disabled resources and removed Harness capability stop retained execution', async () => { + const f = fixture(), resource = { harnesses: ['codex'], type: 'local' } + f.rt.projectResources.requireBinding = () => resource + await requireExecutionAuthority(f.rt, f.session) + resource.disabled = true + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /resource.*revoked/) + resource.disabled = false; resource.harnesses = ['flybot'] + await assert.rejects(requireExecutionAuthority(f.rt, f.session), /resource.*revoked/) +}) diff --git a/patches/ds-bot/files/test/project-execution-exit.test.js b/patches/ds-bot/files/test/project-execution-exit.test.js new file mode 100644 index 000000000..7bc1a2f96 --- /dev/null +++ b/patches/ds-bot/files/test/project-execution-exit.test.js @@ -0,0 +1,38 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { mkdtemp, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { createRuntime } from '../index.js' +import { fakeContext } from './fake-ctx.js' + +test('the actual pool host adapter cannot confirm an unknown missing worker from Bot absence alone', async t => { + const home = await mkdtemp(join(tmpdir(), 'flybot-exit-adapter-')) + const ctx = fakeContext() + const rt = createRuntime(ctx, { home }) + t.after(async () => { await ctx.dispose(); await rm(home, { recursive: true, force: true }) }) + let adapter + rt.projectPools.connect = value => { adapter = value; return () => {} } + rt.flybot.scope('flybot-workflow').projects.execution.connectPools({ release: async () => {} }, () => ({ ok: true })) + assert.equal(await adapter.release('unknown-native-carrier'), false) +}) + +test('physical exit query accepts only matching persisted facts and never infers an unknown worker', async () => { + const { executionExitConfirmed } = await import('../src/host/project-execution-exit.js') + const rt = { state: {}, botOf: () => undefined } + assert.equal(executionExitConfirmed(rt, 'unknown'), false) + const scope = { kind: 'systemd-control-group', killMode: 'control-group', invocationId: 'a'.repeat(32) } + rt.state.projectResourceLeases = { legacy: { id: 'legacy', status: 'released', worker: { kind: 'native', botId: 'legacy-native' }, processScope: scope, releaseProof: { kind: 'systemd-prior-invocation-exited', previousInvocationId: scope.invocationId } } } + assert.equal(executionExitConfirmed(rt, 'legacy-native'), true) + rt.state.projectResourceLeases.legacy.releaseProof.previousInvocationId = 'b'.repeat(32) + assert.equal(executionExitConfirmed(rt, 'legacy-native'), false) + rt.state.projectHarnessWorkers = { sdk: { id: 'sdk', status: 'released', generation: 3, leaseId: 'sdk-lease', releaseProof: { kind: 'sdk-stop-ack-and-process-exit', generation: 3 } } } + rt.state.projectResourceLeases['sdk-lease'] = { status: 'running' } + assert.equal(executionExitConfirmed(rt, 'sdk'), false) + rt.state.projectResourceLeases['sdk-lease'].status = 'released' + assert.equal(executionExitConfirmed(rt, 'sdk'), true) + rt.state.projectNativeWorkers = { native: { id: 'native', version: 'project-native-worker/v2', status: 'quiescent', generation: 1, sessionId: 'native-session', bindingKey: 'key', releaseProof: { kind: 'native-agent-quiescent', registeredWorkEmpty: true, workerId: 'native', generation: 1, sessionId: 'native-session', bindingKey: 'key' } } } + assert.equal(executionExitConfirmed(rt, 'native'), true) + rt.botOf = id => id === 'native' ? { id } : undefined + assert.equal(executionExitConfirmed(rt, 'native'), false) +}) diff --git a/patches/ds-bot/files/test/project-harness.test.js b/patches/ds-bot/files/test/project-harness.test.js index 6dcda2c61..07fa9617c 100644 --- a/patches/ds-bot/files/test/project-harness.test.js +++ b/patches/ds-bot/files/test/project-harness.test.js @@ -6,7 +6,7 @@ import { join } from 'node:path' import { createProjectEffects } from '../src/host/project-effects.js' import { install as installProjectInputs } from '../src/host/project-inputs.js' import { createProjectResources } from '../src/host/project-resources.js' -import { createProjectHarness, handoffFrame } from '../src/host/project-harness.js' +import { createProjectHarness, handoffFrame, harnessCapabilities } from '../src/host/project-harness.js' async function fixture(t) { const root = await mkdtemp(join(tmpdir(), 'flybot-harness-contract-')) @@ -21,17 +21,17 @@ lines.on('line',line=>{const m=JSON.parse(line); if(m.type==='stop'){write({type:'stopped',released:true});setTimeout(()=>process.exit(0),20);} if(m.type==='start') { write({type:'sdk-event',event:{type:'thread.started',thread_id:'bound-thread'}}); - write({type:'sdk-event',event:{type:'debug',noTools:m.settings.noTools,readOnly:m.settings.readOnly,value:m.settings.env.OPENAI_API_KEY,taskId:'task-1234567890abcdef',unknownCredential:'sk-unitsecret123456789012345'}}); + write({type:'sdk-event',event:{type:'debug',noTools:m.settings.noTools,readOnly:m.settings.readOnly,resumeSessionId:m.settings.sessionId??null,value:m.settings.env.OPENAI_API_KEY,taskId:'task-1234567890abcdef',unknownCredential:'sk-unitsecret123456789012345'}}); write({type:'completed',sessionId:'bound-thread',text:'OUTPUT implementation: actual contract worker output',usage:{input_tokens:1}}); } });`) - const session = { id: 's', projectId: 'p', taskId: 't', attemptId: 'a', role: 'developer', execution: { harness: 'codex' }, context: { text: 'Frozen criteria and role memory' }, resourceLease: { id: 'l', resourceId: 'r', generation: 1 } } + const session = { id: 's', projectId: 'p', taskId: 't', attemptId: 'a', role: 'developer', definition: { role: 'developer', instructions: 'Stable developer convention', skills: [] }, execution: { harness: 'codex' }, context: { text: 'Frozen criteria and role memory' }, resourceLease: { id: 'l', resourceId: 'r', generation: 1 } } const workspace = { id: 'w', projectId: 'p', attemptId: 'a', role: 'developer', path: cwd } const rt = { home: root, state: { tasks: { t: { id: 't', projectId: 'p' } }, projectResourceLeases: { l: { id: 'l', resourceId: 'r', projectId: 'p', attemptId: 'a', generation: 1, status: 'reserved' } } }, - config: { projectResources: { r: { type: 'local', root, capacity: 1, runtimeDir, allowedProjectOwnerIds: ['owner'], profiles: { owner: { codex: { home: join(root, 'profile'), configDir: join(root, 'profile/codex'), model: 'test-model', envRefs: { OPENAI_API_KEY: 'FLYBOT_TEST_HARNESS_SECRET' } } } } } } }, + config: { projectResources: { r: { type: 'local', root, capacity: 1, runtimeDir, harnesses: ['codex'], allowedProjectOwnerIds: ['owner'], profiles: { owner: { codex: { home: join(root, 'profile'), configDir: join(root, 'profile/codex'), model: 'test-model', envRefs: { OPENAI_API_KEY: 'FLYBOT_TEST_HARNESS_SECRET' } } } } } } }, save: async () => {}, warn() {}, readProject: () => ({ id: 'p', ownerUserId: 'owner' }), requireProjectAccess(actor) { if (actor.memberId !== 'owner') throw new Error('Access denied') }, projectDelivery: { ownsWorkspace: (projectId, path) => projectId === 'p' && path === cwd ? workspace : null, sessionFor: () => session, - bindSession: async (_workspace, bot) => rt.projectResources.markStarted('l', 1, { kind: 'sdk', handle: bot.id }), releaseSession: async () => {} } } + bindSession: async (_workspace, bot) => rt.projectResources.markStarted(session.resourceLease.id, session.resourceLease.generation, { kind: 'sdk', handle: bot.id }), releaseSession: async () => {} } } process.env.FLYBOT_TEST_HARNESS_SECRET = 'unit-credential-opaque-value' rt.projectResources = createProjectResources(rt) const harness = createProjectHarness(rt) @@ -178,7 +178,7 @@ test('busy SDK queues feedback until a safe next turn in the same worker, thread const session = rt.projectDelivery.sessionFor() session.poolClaimId = 'claim' const task = rt.state.tasks.t - task.status = 'working' + task.status = 'working'; task.deliveryAttemptId = session.attemptId; session.generation = 2 task.collaboration = { status: 'claimed', generation: 2, botId: 'p:developer', currentHandoffId: 'h', initiatedBy: 'owner', claim: { id: 'claim', generation: 2, botId: 'p:developer', status: 'active' } } rt.projects = { requireActive() {} } rt.projectEffects = createProjectEffects(rt); installProjectInputs(rt) @@ -234,7 +234,7 @@ createInterface({input:process.stdin}).on('line',line=>{const m=JSON.parse(line) test('safe SDK input turns are bounded and excess feedback remains for the next responsible Bot', async t => { const {rt,harness,cwd}=await fixture(t),session=rt.projectDelivery.sessionFor(),task=rt.state.tasks.t - session.poolClaimId='bounded-claim';task.status='working' + session.poolClaimId='bounded-claim';task.status='working';task.deliveryAttemptId=session.attemptId;session.generation=2 task.collaboration={status:'claimed',generation:2,botId:'p:developer',initiatedBy:'owner',claim:{id:session.poolClaimId,generation:2,botId:'p:developer',status:'active'}} rt.projects={requireActive(){}};rt.projectEffects=createProjectEffects(rt);installProjectInputs(rt) rt.save=async()=>{rt.projectEffects.saved(rt.projectEffects.capture())} @@ -278,7 +278,7 @@ write({type:'completed',sessionId:'bounded-thread',text:'OUTPUT handoff: '+JSON. async function taskInputFixture(t, workerSource) { const f=await fixture(t),{rt,harness}=f,session=rt.projectDelivery.sessionFor(),task=rt.state.tasks.t - session.poolClaimId='idle-claim';task.status='working' + session.poolClaimId='idle-claim';task.status='working';task.deliveryAttemptId=session.attemptId;session.generation=2 task.collaboration={status:'claimed',generation:2,botId:'p:developer',initiatedBy:'owner',claim:{id:session.poolClaimId,generation:2,botId:'p:developer',status:'active'}} rt.projects={requireActive(){}};rt.projectEffects=createProjectEffects(rt);installProjectInputs(rt);rt.projectHarness=harness rt.save=async()=>{const proof=rt.projectEffects.capture();await writeFile(join(rt.home,'inbox.json'),JSON.stringify(rt.state));rt.projectEffects.saved(proof)} @@ -349,3 +349,147 @@ test('a result callback failing after confirmed release cannot poison the worker assert.equal(rt.projectResources.occupied('r'),0) assert.equal(row.failure,undefined) }) + + +test('changed caller prompt resumes the Task-role SDK thread when its role definition is unchanged', async t => { + const { rt, harness, cwd } = await fixture(t) + const first = await harness.spawn({ executionProjectId: 'p', executionWorkDir: cwd, prompt: 'Initial Task requirement' }) + const completed = new Promise(resolve => harness.onResult(resolve)) + await harness.dispatch({ botId: first, taskId: 't', brief: 'Initial work', outputs: [{ name: 'implementation' }] }) + await completed + await harness.release(first) + const session = rt.projectDelivery.sessionFor() + session.id = 'returned-role-session' + session.attemptId = 'second-attempt' + rt.state.projectResourceHealth = { r: { status: 'ready' } } + rt.state.projectHarnessHealth = { 'p:r:codex': { status: 'ready' } } + session.resourceLease = await rt.projectResources.reserve({ projectId: 'p', attemptId: session.attemptId, role: 'developer', sessionKey: session.id, harness: 'codex', resourceIds: ['r'] }) + const next = await harness.spawn({ executionProjectId: 'p', executionWorkDir: cwd, prompt: 'Tester returned the Task with new feedback and changed candidate facts' }) + assert.equal(rt.state.projectHarnessWorkers[next].continuedFrom, first) + assert.equal(rt.state.projectHarnessWorkers[next].backendSessionId, 'bound-thread') + const returned = new Promise(resolve => harness.onResult(resolve)) + await harness.dispatch({ botId: next, taskId: 't', brief: 'Continue previous work', outputs: [{ name: 'implementation' }] }) + await returned + assert.match(await readFile(rt.state.projectHarnessWorkers[next].eventLog, 'utf8'), /"resumeSessionId":"bound-thread"/) +}) + +test('a no-op state save cannot authorize SDK input, and the unstarted process remains releasable', async t => { + const { rt, harness, cwd } = await fixture(t) + const id = await harness.spawn({ executionProjectId: 'p', executionWorkDir: cwd, prompt: 'No uncommitted input' }) + rt.savesQueued = () => 0 + await assert.rejects(harness.dispatch({ botId: id, taskId: 't', brief: 'Must not be sent', outputs: [{ name: 'implementation' }] }), /commit a state snapshot/) + assert.equal(rt.state.projectHarnessWorkers[id].backendSessionId, undefined) + assert.equal(rt.state.projectHarnessWorkers[id].executionStartedAt, null) + await harness.release(id) + assert.equal(rt.projectResources.occupied('r'), 0) +}) + + +test('Harness capability inspection distinguishes SDK support, native limitations and unknown adapters', () => { + for (const harness of ['codex', 'claude-agent']) { + const capabilities = harnessCapabilities(harness) + assert.equal(capabilities.taskRoleResume, true) + assert.equal(capabilities.crossHarnessResume, false) + assert.equal(capabilities.input, 'between-sdk-turns') + capabilities.taskRoleResume = false + assert.equal(harnessCapabilities(harness).taskRoleResume, true) + } + assert.equal(harnessCapabilities('flybot').taskRoleResume, true) + assert.equal(harnessCapabilities('flybot').exitConfirmation, 'native-agent-quiescence') + assert.equal(harnessCapabilities('flybot').sdkVersion, null) + assert.throws(() => harnessCapabilities('unknown'), /Unknown/) +}) + +test('SDK sandbox policy follows pinned capability while custom logical role identity remains distinct', async t => { + for (const [role, policyRole, readOnly] of [['implementer', 'developer', false], ['auditor', 'reviewer', true]]) { + await t.test(role, async t => { + const f = await fixture(t), session = f.rt.projectDelivery.sessionFor() + session.role = role; session.policyRole = policyRole + session.definition = { ...session.definition, role, policyRole } + f.rt.state.projectAgentDefinitions = { p: { revision: 1, roles: [session.definition] } } + f.rt.state.projectResourceLeases.l.role = role + f.rt.projectDelivery.ownsWorkspace('p', f.cwd).role = role + let resolve; const result = new Promise(done => { resolve = done }) + f.harness.onResult(resolve) + const id = await f.harness.spawn({ executionProjectId: 'p', executionWorkDir: f.cwd, prompt: 'Execute the pinned capability contract' }) + await f.harness.dispatch({ botId: id, brief: 'Inspect assigned work', inputs: {} }) + await result; await f.harness.release(id) + const row = f.rt.state.projectHarnessWorkers[id] + const events = (await readFile(row.sealedLog, 'utf8')).split('\n').filter(Boolean).map(line => JSON.parse(line)) + const debug = events.find(item => item.event?.type === 'debug') + assert.equal(debug.event.readOnly, readOnly) + assert.equal(row.role, role); assert.equal(row.policyRole, policyRole) + assert.equal(f.rt.state.projectTaskSessions[row.logicalSessionId].role, role) + }) + } +}) + +test('an actual prepared SDK subprocess receives no start after its role resource is revoked', async t => { + const { pinTaskRoleDefinition } = await import('../src/host/project-task-sessions.js') + const { rt, harness, cwd } = await fixture(t), session = rt.projectDelivery.sessionFor() + const task = rt.state.tasks.t + session.poolClaimId = 'revoked-start'; session.generation = 2; task.deliveryAttemptId = session.attemptId + task.collaboration = { initiatedBy: 'owner', generation: 2, claim: { id: session.poolClaimId, generation: 2, status: 'active', botId: 'p:developer', role: 'developer' } } + const role = { id: 'p:developer', ...session.definition, execution: { harness: 'codex', resourceIds: ['r'] } } + const catalog = { revision: 1, roles: [role] } + rt.projectAgents = { deliveryDefinitions: () => catalog } + Object.assign(session, pinTaskRoleDefinition(rt.state, session, role, 1, catalog)) + const id = await harness.spawn({ executionProjectId: 'p', executionWorkDir: cwd, prompt: 'Do not execute after revocation' }) + task.collaboration.claim.workerId = id + role.execution.resourceIds = [] + await assert.rejects(harness.dispatch({ botId: id, taskId: task.id, brief: 'Current Task', outputs: [{ name: 'handoff' }] }), /revoked/) + const row = rt.state.projectHarnessWorkers[id] + assert.equal(row.executionStartedAt, null) + assert.equal(row.status, 'prepared') + await harness.release(id) + assert.equal(row.releaseProof.kind, 'sdk-stop-ack-and-process-exit') + assert.equal(rt.state.projectResourceLeases.l.status, 'released') +}) + +test('existing pool revocation stops an actual live SDK worker before resource release after role rights change', async t => { + for (const revocation of ['role-resource', 'operator-disable']) await t.test(revocation, async t => { + const { pinTaskRoleDefinition } = await import('../src/host/project-task-sessions.js') + const { reconcileExecutionAuthority } = await import('../src/host/project-execution-authority.js') + const { createProjectPools } = await import('../src/host/project-pools.js') + const { rt, harness, cwd } = await fixture(t), session = rt.projectDelivery.sessionFor(), task = rt.state.tasks.t + session.poolClaimId = 'live-revoke'; session.generation = 2; task.deliveryAttemptId = session.attemptId + task.collaboration = { status: 'claimed', initiatedBy: 'owner', generation: 2, claim: { id: session.poolClaimId, generation: 2, status: 'active', botId: 'p:developer', role: 'developer' } } + const role = { id: 'p:developer', ...session.definition, execution: { harness: 'codex', resourceIds: ['r'] } } + const catalog = { revision: 1, roles: [role] } + rt.projectAgents = { deliveryDefinitions: () => catalog } + rt.state.projectAttempts = { [session.attemptId]: { roleSessions: { developer: session } } } + Object.assign(session, pinTaskRoleDefinition(rt.state, session, role, 1, catalog)) + rt.projectPools = createProjectPools(rt) + rt.projectPools.connect({ release: id => harness.release(id) }) + const id = await harness.spawn({ executionProjectId: 'p', executionWorkDir: cwd, prompt: 'Current Task' }) + session.botId = id; task.collaboration.claim.workerId = id + await harness.dispatch({ botId: id, taskId: task.id, brief: 'Current Task', outputs: [{ name: 'handoff' }] }) + for (let n = 0; n < 200 && rt.state.projectHarnessWorkers[id].status !== 'waiting-input'; n++) await new Promise(resolve => setTimeout(resolve, 5)) + assert.equal(rt.state.projectHarnessWorkers[id].status, 'waiting-input') + if (revocation === 'role-resource') role.execution.resourceIds = [] + else rt.config.projectResources.r.disabled = true + await reconcileExecutionAuthority(rt) + assert.equal(task.collaboration.status, 'blocked') + assert.equal(task.collaboration.claim.status, 'revoked') + assert.equal(task.collaboration.claim.exitConfirmed, true) + assert.equal(rt.state.projectHarnessWorkers[id].releaseProof.kind, 'sdk-stop-ack-and-process-exit') + assert.equal(rt.state.projectResourceLeases.l.status, 'released') + assert.equal(harness.hasController(id), false) + }) +}) + +test('SDK human input rechecks the sending actor after awaited initiator authorization', async t => { + const { rt, harness, task, id } = await taskInputFixture(t, ` +import {createInterface} from 'node:readline';const write=v=>process.stdout.write(JSON.stringify(v)+'\\n');write({type:'ready',sdk:{codex:'0.166.0'}}); +createInterface({input:process.stdin}).on('line',line=>{const m=JSON.parse(line);if(m.type==='stop'){write({type:'stopped',released:true});setTimeout(()=>process.exit(0),20);return;} +if(m.type==='start')write({type:'completed',sessionId:'input-actor-thread',text:'Need clarification'}); +});`) + await harness.dispatch({ botId: id, taskId: task.id, brief: 'Current Task', outputs: [{ name: 'handoff' }] }) + for (let n = 0; n < 200 && rt.state.projectHarnessWorkers[id].status !== 'waiting-input'; n++) await new Promise(resolve => setTimeout(resolve, 5)) + assert.equal(rt.state.projectHarnessWorkers[id].status, 'waiting-input') + let senderAllowed = true + rt.requireProjectAccess = actor => { if (actor.memberId !== 'owner' && !senderAllowed) throw Error('Sending member was removed') } + rt.projectDelegations = { async background() { senderAllowed = false } } + await assert.rejects(harness.input({ workerId: id, generation: rt.state.projectHarnessWorkers[id].generation, text: 'Late unauthorized input' }, { memberId: 'sender' }), /Sending member was removed/) + assert.equal(rt.state.projectHarnessWorkers[id].status, 'waiting-input') +}) diff --git a/patches/ds-bot/files/test/project-native-exit.test.js b/patches/ds-bot/files/test/project-native-exit.test.js new file mode 100644 index 000000000..04ac66ca2 --- /dev/null +++ b/patches/ds-bot/files/test/project-native-exit.test.js @@ -0,0 +1,24 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { createProjectResources } from '../src/host/project-resources.js' +async function fixture(botExists) { + const rt = { state: { bots: botExists ? { worker: { id: 'worker' } } : {}, projectResourceHealth: { local: { status: 'ready' } } }, + config: { projectResources: { local: { type: 'local', root: '/tmp/native-exit-test', capacity: 1, allowedProjectOwnerIds: ['owner'], harnesses: ['flybot'] } } }, + readProject: () => ({ id: 'p', ownerUserId: 'owner' }), save: async () => {} } + const service = createProjectResources(rt) + const lease = await service.reserve({ projectId: 'p', attemptId: 'a', role: 'developer', sessionKey: 's', harness: 'flybot', resourceIds: ['local'] }) + await service.markStarted(lease.id, lease.generation, { kind: 'native', botId: 'worker' }) + return { rt, service, lease } +} +test('a missing native Bot keeps its resource held until actual quiescence proof', async () => { + const f = await fixture(false) + await f.service.release(f.lease.id, f.lease.generation) + assert.equal(f.service.occupied('local'), 1) + assert.equal(f.rt.state.projectResourceLeases[f.lease.id].status, 'stopping') +}) +test('an undefined handle plus exited flag cannot manufacture native exit acknowledgement', async () => { + const f = await fixture(true) + await f.service.release(f.lease.id, f.lease.generation, { exited: true }) + assert.equal(f.service.occupied('local'), 1) + assert.equal(f.rt.state.projectResourceLeases[f.lease.id].releaseProof, undefined) +}) diff --git a/patches/ds-bot/files/test/project-native-sessions.test.js b/patches/ds-bot/files/test/project-native-sessions.test.js new file mode 100644 index 000000000..12cf5463a --- /dev/null +++ b/patches/ds-bot/files/test/project-native-sessions.test.js @@ -0,0 +1,299 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { createProjectEffects } from '../src/host/project-effects.js' +import { pinTaskRoleDefinition } from '../src/host/project-task-sessions.js' +import { createProjectNativeSessions, nativeExitConfirmed } from '../src/host/project-native-sessions.js' +function fixture() { + let cancelled = 0, archived = [], unarchived = [], hold, failure = false + const session = { id: 'role-1', projectId: 'p', taskId: 't', attemptId: 'a', role: 'developer', generation: 1, definition: { role: 'developer', instructions: 'Original convention' }, status: 'prepared' } + const workspace = { id: 'workspace-1', path: '/registered/task', attemptId: 'a' } + const agent = { session: { id: 'physical-1' }, status: 'idle', cancel() { cancelled++ }, async whenIdle() { if (hold) await hold; if (failure) throw Error('native drain failed') } } + const rt = { state: { tasks: { t: { projectId: 'p' } }, bots: { bot1: { id: 'bot1', sessionId: 'physical-1', workspaceId: workspace.id, attemptId: 'a' } } }, + ctx: { async parallel() {}, async waterfall(_event, _payload, next) { return next() }, sessions: { get: () => undefined }, agents: { get: id => id === 'physical-1' ? agent : undefined }, workspaceRegistry: { async archiveSession(id) { archived.push(id) }, async unarchiveSession(id) { unarchived.push(id) } } }, + botOf(id) { return rt.state.bots[id] }, chatOf(id) { return rt.state.bots[id]?.sessionId ?? id }, indexSessions() {}, resolveAgent: async () => agent, async save() { const captured = rt.projectEffects.capture(); rt.projectEffects.saved(captured) } } + rt.projectEffects = createProjectEffects(rt) + return { rt, session, workspace, agent, service: createProjectNativeSessions(rt), cancelled: () => cancelled, archived: () => archived, unarchived: () => unarchived, hold(value) { hold = value }, fail(value) { failure = value } } +} +test('native deletion or cancellation acknowledgement alone does not prove quiescence', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1') + let release; f.hold(new Promise(resolve => { release = resolve })) + const stopping = f.service.stop('bot1') + await new Promise(resolve => setImmediate(resolve)) + assert.equal(f.cancelled(), 1); assert.equal(f.service.admits('bot1'), false) + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), false) + release(); await stopping + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), true) + assert.equal((await f.service.stop('bot1')).workerId, 'bot1'); assert.equal(f.cancelled(), 1) +}) +test('a fresh native carrier resumes only the exact latest released Task-role physical Session', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.stop('bot1') + delete f.rt.state.bots.bot1 + f.rt.state.bots.bot2 = { id: 'bot2', workspaceId: 'workspace-2', attemptId: 'a2' } + const session = { ...f.session, id: 'role-2', attemptId: 'a2', status: 'prepared' }, workspace = { ...f.workspace, id: 'workspace-2', attemptId: 'a2' } + const result = await f.service.bind(session, workspace, 'bot2') + await f.service.activate(session, 'bot2') + assert.equal(result.sessionId, 'physical-1'); assert.deepEqual(f.archived(), ['bot2']) + assert.deepEqual(f.unarchived(), ['physical-1'], 'The original archived Session must be admitted by DSH only after exact binding') + assert.equal(f.rt.state.retiredSessions.bot2, 'bot2', 'Unused carrier Session stays managed and cannot become an unowned plain Session') + assert.equal(f.rt.state.bots.bot1, undefined) + assert.equal(f.rt.state.projectTaskSessions[Object.keys(f.rt.state.projectTaskSessions)[0]].bindings.length, 2) +}) +test('native incompatible cwd, definition, unknown exit and foreign bindings refuse silent fresh sessions', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1') + assert.throws(() => f.service.previous(f.session, f.workspace), /forbidden/) + await f.service.stop('bot1'); delete f.rt.state.bots.bot1 + for (const [session, workspace] of [[f.session, { ...f.workspace, path: '/other' }], [{ ...f.session, definition: { role: 'developer', instructions: 'Changed convention' } }, f.workspace], [{ ...f.session, role: 'reviewer' }, f.workspace]]) { + if (session.role === 'reviewer') assert.equal(f.service.previous(session, workspace).row, null) + else assert.throws(() => f.service.previous(session, workspace), /forbidden/) + } + f.rt.state.projectNativeWorkers.bot1.releaseProof.generation++ + assert.throws(() => f.service.previous(f.session, f.workspace), /forbidden/) +}) +test('failed native drain keeps execution held and denies later input', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1'); f.fail(true) + await assert.rejects(f.service.stop('bot1'), /drain failed/) + assert.equal(f.rt.state.projectNativeWorkers.bot1.status, 'stop-unconfirmed') + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), false) + assert.equal(f.service.admits('bot1'), false) +}) + + +test('native restoration preserves current context parts and refuses a changed execution model', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1') + f.rt.state.bots.bot1.sessionId = 'physical-part-2' + f.rt.state.bots.bot1.segments = [{ sessionId: 'physical-1', endedAt: 1 }] + f.agent.session.id = 'physical-part-2' + await f.service.stop('bot1'); delete f.rt.state.bots.bot1 + f.rt.state.bots.bot2 = { id: 'bot2', workspaceId: 'workspace-2', attemptId: 'a2', model: 'other-model' } + const session = { ...f.session, id: 'role-2', attemptId: 'a2', status: 'prepared' }, workspace = { ...f.workspace, id: 'workspace-2', attemptId: 'a2' } + await assert.rejects(f.service.bind(session, workspace, 'bot2'), /model changed/) + delete f.rt.state.bots.bot2.model + const result = await f.service.bind(session, workspace, 'bot2') + await f.service.activate(session, 'bot2') + assert.equal(result.sessionId, 'physical-part-2') + assert.deepEqual(f.rt.state.bots.bot2.segments, [{ sessionId: 'physical-1', endedAt: 1 }]) +}) +test('unknown prior host and legacy native context remain explicitly fenced without reopening a Session', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1') + const restarted = createProjectNativeSessions(f.rt) + await assert.rejects(restarted.stop('bot1'), /earlier host/) + assert.equal(restarted.admits('bot1'), false); assert.equal(f.cancelled(), 0) + const g = fixture() + g.rt.state.projectAttempts = { old: { projectId: 'p', taskId: 't', roleSessions: { developer: { id: 'legacy-role', role: 'developer', sessionId: 'legacy-session', execution: { harness: 'flybot' } } } } } + assert.throws(() => g.service.previous(g.session, g.workspace), /Legacy native/) +}) +test('current pool claim authority is required before any native carrier binding', async () => { + const f = fixture() + f.session.poolClaimId = 'claim'; f.rt.state.tasks.t.deliveryAttemptId = 'a' + f.rt.state.tasks.t.collaboration = { initiatedBy: 'owner', generation: 1, claim: { id: 'other', generation: 1, status: 'active' } } + f.rt.requireProjectAccess = () => {} + await assert.rejects(f.service.bind(f.session, f.workspace, 'bot1'), /claim changed/) + assert.deepEqual(f.rt.state.projectNativeWorkers, undefined) +}) +test('overlapping native binds cannot attach two carriers to the same restored physical Session', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.stop('bot1'); delete f.rt.state.bots.bot1 + let release; const gate = new Promise(resolve => { release = resolve }) + f.rt.resolveAgent = async () => { await gate; return f.agent } + f.rt.state.bots.bot2 = { id: 'bot2', workspaceId: 'workspace-2', attemptId: 'a2' } + f.rt.state.bots.bot3 = { id: 'bot3', workspaceId: 'workspace-2', attemptId: 'a2' } + const session = { ...f.session, id: 'role-2', attemptId: 'a2', status: 'prepared' }, workspace = { ...f.workspace, id: 'workspace-2', attemptId: 'a2' } + const first = f.service.bind(session, workspace, 'bot2') + await new Promise(resolve => setImmediate(resolve)) + await assert.rejects(f.service.bind({ ...session }, workspace, 'bot3'), /already in progress/) + release(); await first + const logical = Object.values(f.rt.state.projectTaskSessions)[0] + assert.equal(logical.bindings.length, 2); assert.equal(logical.bindings.at(-1).workerId, 'bot2') +}) + + +test('native bind rechecks pinned current permissions and rejects a claim revoked during authorization', async () => { + const f = fixture() + f.rt.state.projectAgentDefinitions = { p: { revision: 1, roles: [f.session.definition] } } + const pinned = pinTaskRoleDefinition(f.rt.state, f.session, f.session.definition, 1) + f.session.definitionRef = pinned.definitionRef + f.session.poolClaimId = 'claim'; f.rt.state.tasks.t.deliveryAttemptId = 'a' + f.rt.state.tasks.t.collaboration = { initiatedBy: 'owner', generation: 1, claim: { id: 'claim', generation: 1, status: 'active' } } + let checks = 0; f.rt.requireProjectAccess = () => { checks++ } + f.rt.projectDelegations = { background: async () => {} } + await f.service.bind(f.session, f.workspace, 'bot1') + assert.equal(checks, 2) + const g = fixture() + g.session.poolClaimId = 'claim'; g.rt.state.tasks.t.deliveryAttemptId = 'a' + g.rt.state.tasks.t.collaboration = { initiatedBy: 'owner', generation: 1, claim: { id: 'claim', generation: 1, status: 'active' } } + g.rt.requireProjectAccess = () => {} + g.rt.projectDelegations = { background: async () => { g.rt.state.tasks.t.collaboration.claim.status = 'revoked' } } + await assert.rejects(g.service.bind(g.session, g.workspace, 'bot1'), /revoked during/) + assert.equal(g.rt.state.projectNativeWorkers, undefined) + const h = fixture(); h.session.definitionRef = 'missing-definition' + await assert.rejects(h.service.bind(h.session, h.workspace, 'bot1'), /definition is unavailable/) +}) + + +test('a noncommitting native binding cannot lift archive admission or authorize new model input', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.stop('bot1'); delete f.rt.state.bots.bot1 + f.rt.state.bots.bot2 = { id: 'bot2', workspaceId: 'workspace-2', attemptId: 'a2' } + const session = { ...f.session, id: 'role-2', attemptId: 'a2', status: 'prepared' }, workspace = { ...f.workspace, id: 'workspace-2', attemptId: 'a2' } + f.rt.save = async () => {} // Resolving is not evidence that the binding reached disk. + await f.service.bind(session, workspace, 'bot2') + assert.deepEqual(f.unarchived(), [], 'Uncommitted restoration must leave the original DSH archive gate intact') + assert.equal(f.service.admits('bot2'), false) + await assert.rejects(f.service.activate(session, 'bot2'), /commit/) + assert.deepEqual(f.unarchived(), []) +}) + + +test('native activation needs committed binding and current writable epoch before input admission', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1') + assert.equal(f.service.admits('bot1'), false) + await f.service.activate(f.session, 'bot1') + assert.equal(f.service.admits('bot1'), true) + const restarted = createProjectNativeSessions(f.rt) + assert.equal(restarted.admits('bot1'), false, 'A previous host epoch cannot silently restart native input') + f.rt.readOnly = () => 'Another writer holds the home' + assert.equal(f.service.admits('bot1'), false) + await assert.rejects(f.service.activate(f.session, 'bot1'), /writable host/) + assert.equal(f.service.admits('unrelated-personal-chief'), true) +}) + +function supervisedLease(f) { + const lease = { id: 'native-lease', projectId: 'p', taskId: 't', attemptId: 'a', role: 'developer', sessionKey: f.session.id, resourceId: 'native-resource', generation: 7, status: 'running', + worker: { kind: 'native', botId: 'bot1' }, processScope: { kind: 'systemd-control-group', killMode: 'control-group', invocationId: 'a'.repeat(32), bootId: 'same-boot', unit: 'flybot.service', controlGroup: '/system.slice/flybot.service', hostPid: 12345 } } + f.session.resourceLease = { ...lease } + f.rt.state.projectResourceLeases = { [lease.id]: lease } + f.rt.state.projectAttempts = { a: { id: 'a', projectId: 'p', taskId: 't', roleSessions: { developer: f.session } } } + return lease +} +test('supervisor-confirmed old host exit preserves the latest native context without resuming a stale worker', async () => { + const f = fixture(), lease = supervisedLease(f) + await f.service.bind(f.session, f.workspace, 'bot1') + f.rt.state.bots.bot1.sessionId = 'latest-native-part'; f.rt.state.bots.bot1.segments = [{ sessionId: 'physical-1', endedAt: 1 }] + f.rt.resolveAgent = async () => { throw Error('Must not resume old activity to manufacture exit proof') } + let checks = 0 + const restarted = createProjectNativeSessions(f.rt, { priorExit: async scope => { checks++; return scope.invocationId === 'a'.repeat(32) } }) + const proof = await restarted.stop('bot1') + assert.equal(checks, 1); assert.equal(f.cancelled(), 0) + assert.equal(proof.kind, 'systemd-prior-invocation-exited'); assert.equal(proof.previousInvocationId, lease.processScope.invocationId) + assert.equal(proof.sessionId, 'latest-native-part'); assert.equal(f.session.sessionId, 'latest-native-part') + const row = f.rt.state.projectNativeWorkers.bot1 + assert.equal(nativeExitConfirmed(f.rt.state, row), true) + lease.status = 'released'; lease.releaseProof = proof; delete f.rt.state.bots.bot1 + assert.equal(restarted.previous({ ...f.session, id: 'next-role' }, f.workspace).row, row) +}) +test('native old-host recovery rejects missing process-group proof and scope changes during inspection', async () => { + for (const variant of ['unconfirmed', 'unmanaged', 'changed']) { + const f = fixture(), lease = supervisedLease(f) + await f.service.bind(f.session, f.workspace, 'bot1') + if (variant === 'unmanaged') lease.processScope.kind = 'unmanaged-process' + const restarted = createProjectNativeSessions(f.rt, { priorExit: async () => { + if (variant === 'changed') lease.generation++ + return variant !== 'unconfirmed' + } }) + await assert.rejects(restarted.stop('bot1'), /host|scope|binding|generation/) + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), false) + assert.equal(lease.status, 'running') + } +}) + +test('native exit remains unconfirmed while registered jobs or subagents are still stopping', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1') + let requested = 0 + f.rt.ctx.parallel = async (event, body) => { assert.equal(event, 'workspace/session-stop'); assert.equal(body.sessionId, 'physical-1'); requested++ } + f.rt.ctx.waterfall = async () => [{ kind: 'job', items: [{ id: 'still-stopping-job' }] }, { kind: 'subagent', items: [{ id: 'child' }] }] + await assert.rejects(f.service.stop('bot1'), /registered session work/) + const row = f.rt.state.projectNativeWorkers.bot1 + assert.equal(nativeExitConfirmed(f.rt.state, row), false) + assert.deepEqual(row.pendingWork, [{ kind: 'job', count: 1 }, { kind: 'subagent', count: 1 }]) + f.rt.ctx.waterfall = async () => [] + await f.service.stop('bot1') + assert.equal(requested, 2); assert.equal(nativeExitConfirmed(f.rt.state, row), true) + assert.equal(row.releaseProof.registeredWorkEmpty, true) +}) +test('native admission fences subagent lineage and missing legacy execution bindings while preserving ordinary Bots', async () => { + const f = fixture() + f.rt.selfOf = id => id === 'physical-1' ? 'bot1' : id === 'legacy-session' ? 'legacy' : id === 'personal-session' ? 'personal' : undefined + f.rt.state.bots.legacy = { id: 'legacy', ephemeral: { plugin: 'flybot-workflow' }, workspaceId: 'w', attemptId: 'old' } + f.rt.state.bots.personal = { id: 'personal' } + await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.activate(f.session, 'bot1') + const child = { session: { id: 'child', header: { id: 'child', origin: 'subagent', parentSession: 'physical-1' } } } + assert.equal(f.service.admitsAgent(child), true) + await f.service.stop('bot1') + assert.equal(f.service.admitsAgent(child), false) + assert.equal(f.service.admitsAgent({ session: { id: 'legacy-session' } }), false) + assert.equal(f.service.admitsAgent({ session: { id: 'personal-session' } }), true) +}) +test('native host scope is captured without a resource and supports only actual supervisor exit recovery', async () => { + const f = fixture() + const scope = { kind: 'systemd-control-group', killMode: 'control-group', invocationId: 'b'.repeat(32), unit: 'flybot.service', controlGroup: '/system.slice/flybot.service', bootId: 'same-boot' } + const original = createProjectNativeSessions(f.rt, { captureScope: async () => scope }) + await original.bind(f.session, f.workspace, 'bot1') + const restarted = createProjectNativeSessions(f.rt, { priorExit: async value => value === scope }) + const proof = await restarted.stop('bot1') + assert.equal(proof.previousInvocationId, scope.invocationId) + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), true) +}) + + +test('native drain includes idle subagent jobs and earlier context parts before proving exit', async () => { + const f = fixture(); await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.activate(f.session, 'bot1') + f.rt.selfOf = id => id === 'physical-1' ? 'bot1' : undefined + const child = { session: { id: 'child-session', header: { id: 'child-session', origin: 'subagent', parentSession: 'physical-1' } } } + assert.equal(f.service.admitStep(child), true) + f.rt.state.bots.bot1.segments = [{ sessionId: 'earlier-part' }] + const stopped = [] + f.rt.ctx.parallel = async (_event, body) => { stopped.push(body.sessionId) } + f.rt.ctx.waterfall = async (_event, body) => body.sessionId === 'child-session' ? [{ kind: 'job', items: [{ id: 'idle-child-job' }] }] : [] + await assert.rejects(f.service.stop('bot1'), /registered session work/) + assert.ok(stopped.includes('child-session')); assert.ok(stopped.includes('earlier-part')) + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), false) + f.rt.ctx.waterfall = async () => [] + await f.service.stop('bot1') + assert.equal(nativeExitConfirmed(f.rt.state, f.rt.state.projectNativeWorkers.bot1), true) +}) + +test('actual native pre-step refuses revoked role rights while retaining the original role context', async () => { + const f = fixture(), revoked = [] + const role = { id: 'p:developer', ...structuredClone(f.session.definition), toolPolicy: { shell: true }, execution: { harness: 'flybot', resourceIds: [] } } + const task = f.rt.state.tasks.t + task.deliveryAttemptId = 'a'; task.collaboration = { initiatedBy: 'owner', generation: 1, claim: { id: 'claim', botId: role.id, role: 'developer', generation: 1, status: 'active' } } + f.session.poolClaimId = 'claim' + f.rt.requireProjectAccess = () => {} + f.rt.projectAgents = { deliveryDefinitions: () => ({ revision: 1, roles: [role] }) } + f.rt.projectPools = { async revokeExecution(...args) { revoked.push(args) } } + f.rt.state.projectAttempts = { a: { roleSessions: { developer: f.session } } } + f.rt.selfOf = id => id === f.agent.session.id ? 'bot1' : undefined + Object.assign(f.session, pinTaskRoleDefinition(f.rt.state, f.session, role, 1, f.rt.projectAgents.deliveryDefinitions())) + await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.activate(f.session, 'bot1') + const check = agent => f.service.validateStep ? f.service.validateStep(agent) : f.service.admitStep(agent) + assert.equal(await check(f.agent), true) + role.toolPolicy.shell = false + assert.equal(await check(f.agent), false) + assert.equal(revoked.length, 1) + assert.deepEqual(revoked[0].slice(0, 2), ['t', 'claim']) + assert.match(revoked[0][2], /revoked/) + assert.equal(f.session.definition.instructions, 'Original convention') +}) + +test('native ongoing authority follows the live carrier when a resumed physical Session also appears in history', async () => { + const f = fixture() + f.rt.selfOf = id => id === f.agent.session.id ? (f.rt.state.bots.bot2 ? 'bot2' : 'bot1') : undefined + const role = { id: 'p:developer', ...structuredClone(f.session.definition), execution: { harness: 'flybot', resourceIds: [] } } + f.rt.projectAgents = { deliveryDefinitions: () => ({ revision: 1, roles: [role] }) } + Object.assign(f.session, pinTaskRoleDefinition(f.rt.state, f.session, role, 1, f.rt.projectAgents.deliveryDefinitions())) + f.session.poolClaimId = 'old-claim' + f.rt.state.tasks.t.deliveryAttemptId = 'a'; f.rt.state.tasks.t.collaboration = { initiatedBy: 'owner', generation: 1, claim: { id: 'old-claim', botId: role.id, role: 'developer', generation: 1, status: 'active' } } + f.rt.requireProjectAccess = () => {} + f.rt.state.projectAttempts = { a: { roleSessions: { developer: f.session } } } + await f.service.bind(f.session, f.workspace, 'bot1'); await f.service.activate(f.session, 'bot1') + assert.equal(await f.service.validateStep(f.agent), true) + await f.service.stop('bot1'); delete f.rt.state.bots.bot1 + const session = { ...f.session, id: 'role-2', attemptId: 'a2', generation: 2, poolClaimId: 'new-claim', status: 'prepared' } + const task = f.rt.state.tasks.t + task.deliveryAttemptId = 'a2'; task.collaboration = { initiatedBy: 'owner', generation: 2, claim: { id: session.poolClaimId, botId: role.id, role: 'developer', generation: 2, status: 'active' } } + f.rt.requireProjectAccess = () => {} + f.rt.state.projectAttempts.a2 = { roleSessions: { developer: session } } + f.rt.state.bots.bot2 = { id: 'bot2', workspaceId: 'w2', attemptId: 'a2' } + await f.service.bind(session, { ...f.workspace, id: 'w2', attemptId: 'a2' }, 'bot2') + await f.service.activate(session, 'bot2') + assert.equal(f.rt.state.projectNativeWorkers.bot1.sessionId, f.rt.state.projectNativeWorkers.bot2.sessionId) + assert.equal(await f.service.validateStep(f.agent), true) +}) diff --git a/patches/ds-bot/files/test/project-pools.test.js b/patches/ds-bot/files/test/project-pools.test.js index 52ab2109e..3cd70ec3b 100644 --- a/patches/ds-bot/files/test/project-pools.test.js +++ b/patches/ds-bot/files/test/project-pools.test.js @@ -222,3 +222,41 @@ test('Chief intervention does not reset global usage or bypass quota', async () await assert.rejects(f.pool.chiefRoute({ taskId: 'task', targetBotId: 'p:developer', expectedGeneration: f.task.collaboration.generation, reason: 'No quota remains', decisionId: 'quota-chief' }, { memberId: 'owner' }), /quota/) assert.deepEqual(f.task, before) }) + + +test('a retired source can finish its frozen active claim but cannot receive new offers', async () => { + const f = setup(); await f.admit(); const claim = await f.claim() + const roles = f.rt.projectAgents.deliveryDefinitions().roles + const index = roles.findIndex(row => row.role === 'developer'), retired = roles.splice(index, 1)[0] + f.rt.state.projectRetiredRoles = { 'p:developer': { ...retired, projectId: 'p', retiredAt: 1 } } + await f.pool.decide('task', claim, { action: 'handoff', to: 'p:tester', reason: 'Complete original responsibility and request actual testing' }) + assert.equal(f.completed[0].role, 'developer') + assert.equal(f.task.collaboration.botId, 'p:tester') + const tester = await f.pool.claim('task', 'p:tester') + await assert.rejects(f.pool.decide('task', tester, { action: 'return', to: 'p:developer', reason: 'Cannot hand off to an inactive role' }), /Target/) +}) +test('queued work for a retired role becomes visibly blocked without reassignment or a new worker', async () => { + const f = setup(); await f.admit() + const roles = f.rt.projectAgents.deliveryDefinitions().roles + roles.splice(roles.findIndex(row => row.role === 'developer'), 1) + assert.equal(await f.claim(), null) + assert.equal(f.task.collaboration.status, 'blocked') + assert.equal(f.task.collaboration.botId, 'p:developer') + assert.match(f.task.collaboration.block.reason, /Chief must explicitly/) + assert.equal(f.task.collaboration.claim, undefined) +}) + +test('host execution revocation fences the exact claim before stopping and never infers exit or reassigns', async () => { + const f = setup(); await f.admit(); const token = await f.claim() + f.task.collaboration.claim.workerId = 'worker' + const events = [] + f.rt.save = async () => { events.push('saved') } + f.adapter.release = async workerId => { events.push('stop'); assert.equal(workerId, 'worker'); assert.equal(f.task.collaboration.claim.status, 'revoked'); assert.equal(events[0], 'saved'); return false } + assert.equal(await f.pool.revokeExecution('task', 'stale-claim', 'Wrong worker'), false) + assert.equal(await f.pool.revokeExecution('task', token.id, 'Role resource permission revoked'), true) + assert.equal(f.task.collaboration.status, 'blocked') + assert.equal(f.task.collaboration.claim.exitConfirmed, false) + assert.equal(f.task.collaboration.history.length, 1) + assert.equal(await f.pool.revokeExecution('task', token.id, 'Repeated observation'), false) + await assert.rejects(f.pool.decide('task', token, { action: 'handoff', to: 'p:tester', reason: 'Stale callback' }), /stale/) +}) diff --git a/patches/ds-bot/files/test/project-resources.test.js b/patches/ds-bot/files/test/project-resources.test.js index b078d6a4f..8fe3e03ec 100644 --- a/patches/ds-bot/files/test/project-resources.test.js +++ b/patches/ds-bot/files/test/project-resources.test.js @@ -4,6 +4,7 @@ import { tmpdir } from 'node:os' import { join } from 'node:path' import { once } from 'node:events' import { test } from 'node:test' +import { createProjectNativeSessions } from '../src/host/project-native-sessions.js' import { createProjectResources } from '../src/host/project-resources.js' function fixture() { @@ -29,6 +30,19 @@ test('resource leases reserve real capacity once and never release an unconfirme assert.equal(resources.occupied('local'), 1) delete rt.state.bots.worker await resources.release(first.id, first.generation) + assert.equal(resources.occupied('local'), 1, 'Bot absence alone is not exit proof') + await resources.release(first.id, first.generation, { exited: true }) + assert.equal(resources.occupied('local'), 1, 'Missing handles cannot manufacture native exit') + rt.state.bots.worker = { id: 'worker', workspaceId: 'workspace', attemptId: 'a1' } + rt.state.tasks = { task: { projectId: 'p1' } } + rt.botOf = id => rt.state.bots[id]; rt.chatOf = id => id + rt.resolveAgent = async () => ({ session: { id: 'worker' }, status: 'idle', cancel() {}, async whenIdle() {} }) + rt.ctx = { async parallel() {}, async waterfall(_event, _payload, next) { return next() } } + const native = createProjectNativeSessions(rt) + await native.bind({ id: 's1', projectId: 'p1', taskId: 'task', attemptId: 'a1', role: 'developer', status: 'prepared', definition: { role: 'developer' }, resourceLease: first }, { id: 'workspace', path: '/registered' }, 'worker') + await native.stop('worker') + delete rt.state.bots.worker + await resources.release(first.id, first.generation) await resources.release(first.id, first.generation) assert.equal(resources.occupied('local'), 0) await assert.rejects(resources.markStarted(first.id, first.generation, { kind: 'native', botId: 'late' }), /released/) @@ -118,3 +132,32 @@ test('reconciliation cannot resurrect a concurrently confirmed released SDK leas assert.equal(rt.state.projectResourceLeases[lease.id].status, 'released') assert.equal(resources.occupied('local'), 0) }) + +test('verified supervisor exit can free a legacy native resource without claiming its context is recoverable', async () => { + const f = fixture(), resources = createProjectResources(f.rt, { priorExit: async () => true }) + const lease = await resources.reserve({ projectId: 'p1', attemptId: 'a', role: 'developer', sessionKey: 's', harness: 'flybot', resourceIds: ['local'] }) + await resources.markStarted(lease.id, lease.generation, { kind: 'native', botId: 'missing-legacy-bot' }) + const stored = f.rt.state.projectResourceLeases[lease.id] + stored.processScope = { kind: 'systemd-control-group', killMode: 'control-group', invocationId: 'a'.repeat(32), hostPid: 12345 } + await resources.reconcile() + assert.equal(stored.status, 'released') + assert.equal(stored.releaseProof.kind, 'systemd-prior-invocation-exited') + assert.equal(f.rt.state.projectNativeWorkers, undefined, 'Capacity release does not manufacture a Session binding') +}) + +test('supervisor inspection cannot release a binding changed while the manager yields', async () => { + const f = fixture() + let inspect, inspected + const started = new Promise(resolve => { inspected = resolve }) + const resources = createProjectResources(f.rt, { priorExit: () => { inspected(); return new Promise(resolve => { inspect = resolve }) } }) + const lease = await resources.reserve({ projectId: 'p1', attemptId: 'a', role: 'developer', sessionKey: 's', harness: 'flybot', resourceIds: ['local'] }) + await resources.markStarted(lease.id, lease.generation, { kind: 'native', botId: 'old-worker' }) + const stored = f.rt.state.projectResourceLeases[lease.id] + stored.processScope = { kind: 'systemd-control-group', killMode: 'control-group', invocationId: 'a'.repeat(32) } + const releasing = resources.release(lease.id, lease.generation, { proof: { kind: 'systemd-prior-invocation-exited', previousInvocationId: stored.processScope.invocationId } }) + await started + stored.worker = { kind: 'native', botId: 'replacement' } + inspect(true) + await assert.rejects(releasing, /binding changed/) + assert.equal(stored.status, 'running') +}) diff --git a/patches/ds-bot/files/test/project-role-policy.test.js b/patches/ds-bot/files/test/project-role-policy.test.js index b518246c4..69c961700 100644 --- a/patches/ds-bot/files/test/project-role-policy.test.js +++ b/patches/ds-bot/files/test/project-role-policy.test.js @@ -45,3 +45,15 @@ test('complete current FlyBot repository and subsystem rules fit every developme assert.ok(policy.sources.some(source => source.name === 'docs/subsystems/project-access.md')) } }) + + +test('custom logical roles inherit the exact approved capability conventions while default policy bytes stay compatible', () => { + const original = assembleRolePolicy({ role: 'developer' }) + assert.deepEqual(assembleRolePolicy({ role: 'developer', policyRole: 'developer' }), original) + const custom = assembleRolePolicy({ role: 'analyst', policyRole: 'reviewer' }) + assert.equal(custom.role, 'analyst'); assert.equal(custom.policyRole, 'reviewer') + assert.ok(custom.text.includes(ROLE_CONVENTIONS.reviewer)) + assert.equal(verifyRolePolicy(custom, custom.sha256), true) + assert.throws(() => assembleRolePolicy({ role: 'analyst', policyRole: 'unapproved' }), /Unknown role/) + assert.throws(() => assembleRolePolicy({ role: '', policyRole: 'reviewer' }), /Unknown role/) +}) diff --git a/patches/ds-bot/files/test/project-scheduler.test.js b/patches/ds-bot/files/test/project-scheduler.test.js index c2365bf19..45bea4881 100644 --- a/patches/ds-bot/files/test/project-scheduler.test.js +++ b/patches/ds-bot/files/test/project-scheduler.test.js @@ -186,3 +186,12 @@ test('queue recovery rechecks original requester authority and refuses terminal state.projectAttempts.active = { id: 'active', taskId: 'one', status: 'running' } await assert.rejects(scheduler.retry({ queueId: 'q' }, { memberId: 'owner-1' }), /active Attempt/) }) + +test('the existing scheduler checks in-flight execution authority before resource release and new dispatch', async () => { + const { rt, scheduler } = fixture(), calls = [] + rt.projectDelivery.reconcileExecutions = async () => { calls.push('authority') } + rt.projectResources = { async reconcile() { calls.push('resources') } } + rt.projectPools = { async pump() { calls.push('dispatch') } } + await scheduler.wake() + assert.deepEqual(calls, ['authority', 'resources', 'dispatch']) +}) diff --git a/patches/ds-bot/files/test/project-task-sessions.test.js b/patches/ds-bot/files/test/project-task-sessions.test.js index d4ea2cb16..59ae57aab 100644 --- a/patches/ds-bot/files/test/project-task-sessions.test.js +++ b/patches/ds-bot/files/test/project-task-sessions.test.js @@ -1,6 +1,6 @@ import assert from 'node:assert/strict' import {test} from 'node:test' -import {taskSession,taskSessionId,backendBindingKey,resumableBinding,confirmedWorkerExit} from '../src/host/project-task-sessions.js' +import {taskSession,taskSessionId,backendBindingKey,resumableBinding,confirmedWorkerExit,pinTaskRoleDefinition,discardUnpreparedTaskDefinitions,roleDefinitionKey} from '../src/host/project-task-sessions.js' test('Task-role identity persists across Attempts and separates projects, tasks and roles', () => { const state={tasks:{t:{projectId:'p'},t2:{projectId:'p'}}} @@ -16,7 +16,7 @@ test('Task-role identity persists across Attempts and separates projects, tasks assert.throws(()=>taskSession(state,session),/identity/) }) -test('SDK bindings rotate on cwd, principal, resource, profile, model or persona changes', () => { +test('SDK binding fingerprints distinguish cwd, principal, resource, profile, model and persona changes', () => { const input={harness:'codex',resourceId:'r',principalId:'owner',cwd:'/task',model:'model',profile:{home:'/home',configDir:'/config'},definition:'persona'} const key=backendBindingKey(input) assert.equal(backendBindingKey(structuredClone(input)),key) @@ -29,7 +29,7 @@ test('resumption requires same-scope backend identity and confirmed worker and l const record={id:'logical',projectId:'p',taskId:'t',role:'developer',bindings:[{key:'key',workerId:'w',sessionId:'thread'}]} const original={projectHarnessWorkers:{w:{projectId:'p',taskId:'t',role:'developer',logicalSessionId:'logical',bindingKey:'key',leaseId:'l',resourceId:'r',generation:2,status:'released',backendSessionId:'thread',releaseProof:{kind:'sdk-stop-ack-and-process-exit',generation:2}}},projectResourceLeases:{l:{projectId:'p',resourceId:'r',generation:2,status:'released'}}} assert.equal(resumableBinding(original,record,'key').sessionId,'thread') - assert.equal(resumableBinding(original,record,'different'),null) + assert.throws(()=>resumableBinding(original,record,'different'),/resume|binding/) for(const [target,key,value] of [['worker','projectId','foreign'],['worker','taskId','other'],['worker','role','reviewer'],['worker','logicalSessionId','other'],['worker','bindingKey','other'],['worker','status','unreachable'],['worker','backendSessionId','foreign'],['worker','releaseProof',null],['lease','status','reserved'],['lease','generation',3],['lease','projectId','other'],['lease','resourceId','other']]) { const state=structuredClone(original) ;(target==='worker'?state.projectHarnessWorkers.w:state.projectResourceLeases.l)[key]=value @@ -86,3 +86,163 @@ test('late callback failure cannot erase a fully fenced SDK stop and lease exit const l=structuredClone(lease);mutate(l);assert.equal(confirmedWorkerExit({projectResourceLeases:{l}},worker),false) } }) + +test('an incompatible latest Task-role binding cannot silently start a fresh Session or select older context', () => { + const record = { id: 'logical', projectId: 'p', taskId: 't', role: 'developer', bindings: [{ key: 'old', workerId: 'w', sessionId: 'original-thread' }] } + assert.throws(() => resumableBinding({}, record, 'new'), /resume|binding|context/i) + const state = { projectHarnessWorkers: { w: { projectId: 'p', taskId: 't', role: 'developer', logicalSessionId: 'logical', bindingKey: 'old', leaseId: 'l', resourceId: 'r', generation: 2, status: 'released', backendSessionId: 'original-thread', releaseProof: { kind: 'sdk-stop-ack-and-process-exit', generation: 2 } } }, projectResourceLeases: { l: { projectId: 'p', resourceId: 'r', generation: 2, status: 'released' } } } + record.bindings.push({ key: 'new', workerId: 'later', sessionId: 'latest-thread' }) + assert.throws(() => resumableBinding(state, record, 'old'), /resume|binding|context/i) +}) + + +function pinnedFixture() { + const scope = { projectId: 'p', taskId: 't', role: 'developer' } + const definition = { role: 'developer', instructions: 'Original persona', resolvedSkills: [{ name: 'verify', sha256: 'a'.repeat(64) }], memory: ['Current fact'], toolPolicy: { shell: true }, execution: { harness: 'codex', resourceIds: ['r', 'backup'] } } + const state = { tasks: { t: { projectId: 'p' }, other: { projectId: 'p' } }, projectAgentDefinitions: { p: { revision: 1, roles: [structuredClone(definition)] } } } + return { state, scope, definition } +} + +test('a Task pins its definition once, shares immutable snapshots, and keeps current memory outside them', () => { + const { state, scope, definition } = pinnedFixture() + const first = pinTaskRoleDefinition(state, scope, definition) + assert.equal(first.definition.memory, undefined) + assert.equal(first.revision, 1) + const same = pinTaskRoleDefinition(state, { ...scope, taskId: 'other' }, definition) + assert.equal(same.definitionRef, first.definitionRef) + state.projectAgentDefinitions.p.revision = 2 + definition.instructions = 'Changed persona' + definition.memory = ['New fact'] + const retry = pinTaskRoleDefinition(state, scope, definition) + assert.equal(retry.definition.instructions, 'Original persona') + assert.equal(retry.definitionKey, first.definitionKey) + assert.deepEqual(retry.change, { pinnedRevision: 1, currentRevision: 2, retainedOriginalDefinition: true }) + state.tasks.fresh = { projectId: 'p' } + const next = pinTaskRoleDefinition(state, { ...scope, taskId: 'fresh' }, definition) + assert.notEqual(next.definitionRef, first.definitionRef) + assert.equal(next.definition.instructions, 'Changed persona') + assert.equal(next.revision, 2) + retry.definition.instructions = 'Mutating a returned copy' + assert.equal(pinTaskRoleDefinition(state, scope, definition).definition.instructions, 'Original persona') +}) + +test('pinned role definitions do not preserve revoked live permissions or tolerate changed snapshots', () => { + const fixture = pinnedFixture() + pinTaskRoleDefinition(fixture.state, fixture.scope, fixture.definition) + for (const mutate of [ + role => { role.status = 'retired' }, + role => { role.execution.harness = 'claude-agent' }, + role => { role.execution.resourceIds = [] }, + role => { role.toolPolicy.shell = false }, + ]) { + const state = structuredClone(fixture.state) + mutate(state.projectAgentDefinitions.p.roles[0]) + assert.throws(() => pinTaskRoleDefinition(state, fixture.scope, fixture.definition), { code: 'bot/context-resume' }) + } + const snapshot = Object.values(fixture.state.projectRoleDefinitionSnapshots)[0] + snapshot.definition.instructions = 'Changed stored persona' + assert.throws(() => pinTaskRoleDefinition(fixture.state, fixture.scope, fixture.definition), /changed/) +}) + +test('failed preparation discards only unused definition pins and preserves established Session history', () => { + const { state, scope, definition } = pinnedFixture() + const first = pinTaskRoleDefinition(state, scope, definition) + assert.equal(discardUnpreparedTaskDefinitions(state, 'other'), false) + assert.equal(discardUnpreparedTaskDefinitions(state, 't'), true) + assert.equal(Object.keys(state.projectRoleDefinitionSnapshots).length, 1) + definition.instructions = 'Repaired initial definition' + assert.notEqual(pinTaskRoleDefinition(state, scope, definition).definitionRef, first.definitionRef) + state.projectAttempts = { attempt: { projectId: 'p', taskId: 't', roleSessionHistory: [{ role: 'developer' }] } } + assert.equal(discardUnpreparedTaskDefinitions(state, 't'), false) +}) + +function observedFixture() { + const scope = { projectId: 'p', taskId: 't', role: 'developer' } + const logical = { ...scope, id: 'logical', bindings: [] } + const worker = { ...scope, id: 'w', logicalSessionId: 'logical', attemptId: 'a', sessionKey: 's', bindingKey: 'key', backendSessionId: 'partial-thread', leaseId: 'l', resourceId: 'r', generation: 2, status: 'released', releaseProof: { kind: 'sdk-stop-ack-and-process-exit', workerId: 'w', generation: 2 } } + const state = { projectHarnessWorkers: { w: worker }, projectResourceLeases: { l: { ...scope, attemptId: 'a', status: 'released', resourceId: 'r', generation: 2 } } } + return { state, logical, worker, scope } +} + +test('the latest partially observed SDK Session supersedes old completed bindings, but unknown context never resets', () => { + const { state, logical, worker } = observedFixture() + assert.equal(resumableBinding(state, logical, 'key').sessionId, 'partial-thread') + logical.bindings.push({ key: 'old-key', workerId: 'old', sessionId: 'old-thread' }) + state.projectHarnessWorkers.old = { ...worker, id: 'old', bindingKey: 'old-key', backendSessionId: 'old-thread', createdAt: -1 } + assert.equal(resumableBinding(state, logical, 'key').sessionId, 'partial-thread') + worker.backendSessionId = null + assert.throws(() => resumableBinding(state, logical, 'key'), /no recoverable/) + worker.executionStartedAt = null + assert.throws(() => resumableBinding(state, logical, 'key'), /binding|resume/) + logical.bindings = [] + delete state.projectHarnessWorkers.old + assert.equal(resumableBinding(state, logical, 'key'), null) + worker.status = 'unreachable' + assert.throws(() => resumableBinding(state, logical, 'key'), /unconfirmed/) +}) + +test('legacy definition migration verifies original typed context and preserves its binding proof', async () => { + const { roleContext } = await import('../src/host/project-role-context.js') + const { state, scope, definition } = pinnedFixture() + const logical = taskSession(state, scope) + const context = roleContext({ ...scope, definition, revision: 1 }) + const session = { ...scope, id: 's', context, execution: definition.execution } + const proof = { kind: 'sdk-stop-ack-and-process-exit', workerId: 'w', generation: 2 } + state.projectAttempts = { a: { projectId: 'p', taskId: 't', roleSessionHistory: [session] } } + state.projectHarnessWorkers = { w: { ...scope, id: 'w', attemptId: 'a', sessionKey: 's', logicalSessionId: logical.id, backendSessionId: 'legacy-thread', bindingKey: 'legacy-key', resourceId: 'r', leaseId: 'l', generation: 2, status: 'released', releaseProof: proof } } + state.projectResourceLeases = { l: { projectId: 'p', resourceId: 'r', generation: 2, status: 'released' } } + logical.bindings.push({ key: 'legacy-key', workerId: 'w', sessionId: 'legacy-thread' }) + definition.instructions = 'New definition after legacy execution' + const pinned = pinTaskRoleDefinition(state, scope, definition) + assert.equal(pinned.definition.instructions, 'Original persona') + assert.deepEqual(pinned.definition.execution.resourceIds, ['r']) + const oldProof = structuredClone(logical.bindings) + assert.equal(resumableBinding(state, logical, 'v2-key', { legacyKey: 'legacy-key', definitionKey: pinned.definitionKey }).sessionId, 'legacy-thread') + assert.deepEqual(logical.bindings, oldProof) + session.context.sha256 = '0'.repeat(64) + assert.throws(() => roleDefinitionKey(session), /hash/) + assert.throws(() => resumableBinding(state, logical, 'v2-key', { legacyKey: 'legacy-key', definitionKey: pinned.definitionKey }), /hash/) +}) + +test('approved removal or capability reclassification cannot restart a pinned role with its old authority', () => { + const f = pinnedFixture() + pinTaskRoleDefinition(f.state, f.scope, f.definition) + const removed = structuredClone(f.state) + removed.projectAgentDefinitions.p.roles = [] + removed.projectRetiredRoles = { 'p:developer': { ...f.definition, retiredAt: 1 } } + assert.throws(() => pinTaskRoleDefinition(removed, f.scope, f.definition), { code: 'bot/context-resume' }) + const changed = structuredClone(f.state) + changed.projectAgentDefinitions.p.roles[0].policyRole = 'reviewer' + assert.throws(() => pinTaskRoleDefinition(changed, f.scope, f.definition), { code: 'bot/context-resume' }) + assert.equal(f.state.projectRoleDefinitionSnapshots[Object.keys(f.state.projectRoleDefinitionSnapshots)[0]].definition.instructions, 'Original persona') +}) + +test('custom logical roles fingerprint their capability class without invalidating original default-role keys', () => { + const base = { role: 'developer', instructions: 'Convention', resolvedSkills: [] } + const original = roleDefinitionKey({ role: 'developer', definition: base }) + assert.equal(roleDefinitionKey({ role: 'developer', definition: { ...base, policyRole: 'developer' } }), original) + const reviewer = roleDefinitionKey({ role: 'analyst', definition: { ...base, role: 'analyst', policyRole: 'reviewer' } }) + const developer = roleDefinitionKey({ role: 'analyst', definition: { ...base, role: 'analyst', policyRole: 'developer' } }) + assert.notEqual(reviewer, developer) +}) + + +test('approved custom roles have distinct Task-role Sessions while retired Sessions remain stoppable', () => { + const f = pinnedFixture(), analyst = { ...f.definition, role: 'analyst', policyRole: 'reviewer' } + f.state.projectAgentDefinitions.p.roles.push(analyst) + const logical = taskSession(f.state, { ...f.scope, role: 'analyst' }) + assert.notEqual(logical.id, taskSession(f.state, f.scope).id) + f.state.projectAgentDefinitions.p.roles = [] + assert.equal(taskSession(f.state, { ...f.scope, role: 'analyst' }), logical) + assert.throws(() => taskSession(f.state, { ...f.scope, role: 'unapproved' }), /scope changed/) +}) + + +test('staged governance mutations cannot replace the execution-visible committed catalog', () => { + const f = pinnedFixture(), committed = structuredClone(f.state.projectAgentDefinitions.p) + f.state.projectAgentDefinitions.p = { revision: 2, roles: [] } + const pinned = pinTaskRoleDefinition(f.state, f.scope, f.definition, 1, committed) + assert.equal(pinned.revision, 1) + assert.equal(pinned.definition.instructions, 'Original persona') + assert.throws(() => pinTaskRoleDefinition(f.state, f.scope, f.definition, 2, f.state.projectAgentDefinitions.p), /retired/) +}) diff --git a/plugins/workflow/index.js b/plugins/workflow/index.js index 09baf7445..054269489 100644 --- a/plugins/workflow/index.js +++ b/plugins/workflow/index.js @@ -91,7 +91,7 @@ export function apply(ctx, config = {}) { if (!from || typeof fb.bot.send !== 'function') return { ok: false, code: 'no-sender' } const lines = [`Task ${step.taskId}: ${step.brief}`] for (const [name, value] of Object.entries(step.inputs ?? {})) { - if (value) lines.push(`Input ${name}: ${value}`) + if (value !== undefined && value !== null) lines.push(`Input ${name}: ${typeof value === 'string' ? value : JSON.stringify(value)}`) } const outputs = (step.outputs ?? []).map(port => port.name).filter(Boolean) if (outputs.length > 0) lines.push(`When you finish, add one line per output: OUTPUT : (${outputs.join(', ')}).`) diff --git a/plugins/workflow/test/native-inputs.test.js b/plugins/workflow/test/native-inputs.test.js new file mode 100644 index 000000000..931e5d359 --- /dev/null +++ b/plugins/workflow/test/native-inputs.test.js @@ -0,0 +1,22 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' +import { apply } from '../index.js' + +test('native dispatch preserves structured handoff identity instead of collapsing different inputs into the same text', async t => { + let lifecycle; const messages = [] + const fb = { bot: { create() {}, list: () => [{ id: 'chief' }], async send(message) { messages.push(message); return { ok: true } } }, + api: { handle: () => () => {} }, tasks: { onWrite: () => () => {} }, + projects: { context: () => ({ projectId: 'p' }), read: () => ({ chief: { botId: 'chief' } }), execution: { connectPools(value) { lifecycle = value; return () => {} } } } } + const plugin = apply({ flybot: { scope: () => fb }, effect() {} }, { persist: false }) + t.after(() => plugin.dispose()) + const base = { taskId: 'task', botId: 'current-carrier', brief: 'Same Task and acceptance', outputs: [{ name: 'handoff' }] } + const inputs = { handoff: { id: 'first', from: 'p:chief', evidence: [{ sha: 'candidate' }] }, note: 'Exact text', count: 0, required: false } + await lifecycle.dispatch({ ...base, inputs }) + await lifecycle.dispatch({ ...base, inputs: { ...inputs, handoff: { ...inputs.handoff, id: 'returned', from: 'p:tester' } } }) + assert.notEqual(messages[0].text, messages[1].text, 'A return to the same physical Session must not be deduplicated as its previous dispatch') + assert.deepEqual(JSON.parse(messages[0].text.split('\n').find(line => line.startsWith('Input handoff: ')).slice('Input handoff: '.length)), inputs.handoff) + assert.ok(messages[0].text.includes('Input count: 0')); assert.ok(messages[0].text.includes('Input required: false')) + assert.ok(messages[0].text.includes('Input note: Exact text')) + await lifecycle.dispatch({ ...base, inputs }) + assert.equal(messages[2].text, messages[0].text, 'The same dispatch still has stable content for ordinary duplicate protection') +})