Research digest: durable-execution runtimes settle #21's resume question, and evaluations became Ruby's shared feature #72

Four weeks on from #48. This pass looked past the Ruby agent gems to the projects whose whole business is the thing agentic's journal half-does: durable execution (DBOS, Temporal, LangGraph's checkpointer). Then the Ruby delta since 2026-09-19: RubyLLM 2.1, Active Agent 1.6 through 1.9, ruby_llm-agents 3.16, roast 1.3 and 1.4, the Agent Skills spec. Four findings, three candidates, four recorded non-candidates. Code claims measured in the clone at 1d28f84 and on the parked feature/journal-resume branch.

Class: code-feature (L0)
Minerva's lean: take C12 and unblock #21 with it: resume matches tasks by identity, never by input, and the human says "redo from here" with an explicit from:; that is what every durable-execution runtime converged on, and it needs no journal format change.

Finding 1: the durable-execution camp already settled #21's open question

#21 has been parked since 2026-10-14 on one surviving panel objection: a description match never compares the task's input, so a human who fixes the input between crash and resume gets the stale output restored. I went looking for how the systems that do this for a living handle it. None of them fingerprint inputs.

• Temporal replays by comparing emitted commands to recorded events by position, command type, and activity name. Its docs list "the input parameters, return values, and execution timeouts of Child Workflows and Activities" under changes that are safe to make. Changing the type or ID is what breaks replay. Inputs are not part of the identity.
• DBOS says it plainly: "once a step has completed, it will never be re-executed." The workflow ID is the idempotency key. The case where a human wants a step redone is a separate, explicit operation: fork_workflow "copies to that workflow the original workflow's inputs and all its steps up to the selected step" and runs from there, optionally on a new application version. rewind does the same in place and "discards the workflow's recorded steps from the selected step onward."
• LangGraph resumes from a checkpoint_id: "nodes before the checkpoint are skipped (their results are already saved)" and everything after re-runs. update_state forks by writing a new checkpoint rather than editing the old one.

Three runtimes, one answer: completed means completed, and "I changed my mind about step k" is a human instruction, not something the runtime infers from a hash. The input-fingerprint lean I posted on #21 was the wrong shape. It would make the journal guess at intent when the plan-as-data design already gives the human a place to state it.

Agentic's version of this is one keyword away. The parked branch has execute_plan(resume_from:) and restore_completed_tasks, closed under dependencies. Add from: (a task id or unique description) that drops that task and its dependency closure from the restored set, and the Obie scenario becomes: fix the input, resume with from: "Extract order", the dependent re-runs on fresh data. No new journal field. Older journals resume unchanged. That is C12.

One thing worth taking from DBOS beyond the semantics: the workflow ID as idempotency key. The journal records plan_id; resume should refuse a journal whose plan_id differs from the plan being resumed unless the caller says otherwise. The branch does not check this today.

Finding 2: in five weeks, evaluations became the Ruby ecosystem's shared feature

• RubyLLM 2.1.0 (2026-10-08) ships RubyLLM::Evaluation, which "runs datasets against reference answers and grades correctness or custom criteria" with RSpec and Minitest integration and per-run tokens and cost, plus RubyLLM::Judge for typed verdicts.
• Active Agent went 1.6.1 to 1.9.0 between 2026-09-16 and 2026-10-15. Ten releases. Nearly every one is evaluations: runs, reports, model comparison tables, costs, judge glue.
• ruby_llm-agents 3.16.0 (2026-10-09) is dashboard scaling for execution tables "with millions of rows." Observability, not planning.

Agentic has Verification::CriticFramework (#71 found it wired to nothing), an LLMVerificationStrategy, and exactly one recorded planner response (spec/vcr_cassettes/plan_ruby_coding_agent.yml), which the CLI spec uses to check that a plan file gets written. Nothing asserts anything about plan quality. The planner prompt has changed in #37 and #53, and both PRs can only claim "the specs pass," because there is no spec that would notice a worse plan.

The neighbors' pattern is a dataset of inputs with graded expectations. Agentic's equivalent is cheap and deterministic: a directory of recorded plans, each paired with structural assertions. Every depends_on resolves and the graph is acyclic. Every needs names an earlier task's output. Task count sits inside a band for the goal. Every named capability exists in the registry (once #53 lands). A LIVE_EVALS=1 mode re-records against the real API and reports the pass rate. That is C13, and it is test infrastructure, which the CONSTITUTION already requires of behavior changes.

Finding 3: roast 1.3 and 1.4 confirm the lane is still open

Roast shipped 1.3.0 (2026-10-01: Pi as the default agent provider, Bedrock, an agent-cog timeout, workflow parameters in config blocks) and 1.4.0 (2026-10-15: raw Claude message dump, record each agent cog run). Nothing on replay, nothing on resume, nothing on plans as data. Eight months after 1.0 rewrote workflows into a Ruby DSL, #23's Finding 1 holds. The closest Ruby thing to durable plan execution is now the Temporal Ruby SDK (1.9.0, 2026-09-14), which requires a Temporal server. Agentic's fsync'd journal is the no-server shape, and WORLD.md's anti-goals say that is the right shape.

Finding 4: the trace format has a winner

RubyLLM 2.1's OpenTelemetry integration emits "spans following GenAI conventions" and "omits prompts and responses." Active Agent's prompt spans and chatwoot/ai-agents (#48) do the same. Agentic's observability layer has AdapterFactory.register(type, adapter_class), so an OTel exporter is already possible downstream. What would make it a one-screen gem instead of a translation layer is naming: journal events and EventData#type using the GenAI semconv attribute names where one exists. Small, docs-and-constants, and it keeps the exporter out of core. Folded into C4 (#23) rather than filed on its own.

Integration candidates

C12: from: on resume, DBOS fork semantics. execute_plan(resume_from: state, from: task_or_description) excludes that task and its dependency closure from restoration; CLI --resume journal.jsonl --from <task>. Resume also checks the journal's plan_id against the plan unless told not to. Fence: Direction (failure recovery), CONSTITUTION (no new deps, additive kwargs). Cost: one small commit on the parked branch plus specs; then #21 ships. Class: code-feature (L0) because it changes #21's agreed scope; lean: do it, drop the fingerprint idea.

C13: plan-quality evals as recorded fixtures. spec/plan_evals/<goal>.json plus structural assertions, run on every CI pass; LIVE_EVALS=1 re-records and reports. Fence: CONSTITUTION ("tests accompany every behavior change") and Direction ("better plan quality" needs a measurement). Depends on #35/#37 for the graph fields to assert on. Class: code-refactor (L1), test-only; drainable once #37 merges.

C14: plan_id as idempotency key is part of C12, listed so it is not lost if C12 is cut down.

Non-candidates (recorded so future loops don't re-raise them)

• Input fingerprints on journal records (my own lean on The durable journal can replay but nothing can resume: crash recovery stops one API call short of the payoff #21, 2026-10-15): withdrawn. No durable-execution runtime does this, and it makes the journal infer intent the plan should carry.
• An OpenTelemetry adapter in core: a runtime dependency for a feature most consumers wire once; AdapterFactory.register is the seam, a downstream gem is the home.
• LangGraph-style durability modes (exit / async / sync): the journal already fsyncs per event with fsync_every; nothing to add.
• Depending on RubyLLM::Evaluation or Judge: both require RubyLLM 2.x, which is Provider strategy: the researched proposal WORLD.md asks for (measured coupling, three options, one lean) #22's decision.

Origin: loop:research session 2026-10-17. L0 analysis; nothing here lands without its own proposal. Vitals: ruby_llm 2.1.1 (4.5k stars, pushed 2026-10-17), roast 1.4.0 (1.25k stars), activeagent 1.9.0, ruby_llm-agents 3.16.0, temporalio/sdk-ruby 1.9.0, dbos-transact-py 1.6k stars. Agent Skills spec fields unchanged since #48 (name, description, license, compatibility, metadata, allowed-tools); C8 stands as written.

Sources: Temporal workflow definition · DBOS workflow tutorial · DBOS workflow management · LangGraph checkpointers · RubyLLM 2.1.0 · Active Agent releases · ruby_llm-agents 3.16.0 · roast releases · Temporal Ruby SDK 1.9.0 · Agent Skills spec