Repository navigation
feat: a refuted-then-verified goal now teaches the next one - #212
Merged
Merged
Conversation
Harness²-style harnessing-level recursion (google-research/harness2, arXiv 2610.14224), scaled to one agent with a wallet. The verifier panel refuting a /goal and then verifying it on a later round is a behavioral contrast between two attempts at the same objective. That contrast was saved under ~/.blockrun/goals/<id>/verdicts/ and never read. Now a reflector (free extraction models) turns it into at most one lesson per goal: a cue (shape of situation), a lesson naming its component, the closed gap as evidence. A round-1 pass writes nothing — adoption alone is not evidence. A round-capped goal writes a confounded row that is never rendered. Lessons are scoped to the wallet (trading goals) or the workspace (everything else), never global; rendered after the learnings block under a 1,200-char budget; lose a confirmation when a later goal in the same scope is refuted on round 1 while they were in the prompt, and flip to a ⛔ rule after two. Injection patterns, addresses, amounts and the objective's tickers are rejected before anything is saved. Lessons are prose only: the reflector never writes hooks, skills or scripts. FRANKLIN_NO_EXPERIENCE=1 removes the component (ADR-0003); the bench gains a `no-experience` config and the audit a row. The prompt injection scanner moves to src/agent/injection-scan.ts so the lesson path and the project-config path share it. `/learnings` shows the verified lessons; `/learnings clear` leaves them alone and `/learnings clear lessons` removes them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
First slice of the Harness² (google-research/harness2) analysis: the goal verifier panel's refuted→verified rounds become evidence-backed lessons that the next goal in the same wallet/workspace scope reads from its system prompt.
src/learnings/experience.ts(new): experience library (~/.blockrun/experience.jsonl), reflector prompt ported from harness2'sexperience()stage, parse/reject rules, merge/decay/retire, prompt rendering.src/goal/runtime.ts: fire-and-forget reflection after a verified claim when at least one round was refuted; confounded row on the round cap; first-round refutation counts against rendered lessons.src/agent/context.ts: renders the scope's lessons after the learnings block; injection scanner extracted tosrc/agent/injection-scan.ts.src/agent/loop.ts: hands the transcript to the goal engine; records skills hinted while a goal is active./learningsshows verified lessons;FRANKLIN_NO_EXPERIENCE=1disables the component;no-experiencebench config; audit row chore: use package version in Franklin user agents #18.Safety
0x…/base58 addresses,$amounts and the objective's tickers are rejected.Tests
test/experience.local.mjs(18 tests, fake client, TMP_HOME): store round-trip, merge/cap/decay, scope keying, rendering + budget, refute-feedback, parse + reject, reflector end to end, runtime trigger (round-1 skip, refuted→verified, round cap), and theFRANKLIN_NO_EXPERIENCEflag. Full local suite: 918/918.🤖 Generated with Claude Code