Skip to content

emrg: the cycle reads extra_prompt, and §1.5 stops paying for a read it never finishes - #2343

Merged
pm25coder merged 6 commits into
masterfrom
feature/prompt-drift-and-memory-funnel
Oct 17, 2026
Merged

pm25coder merged 6 commits into
masterfrom
feature/prompt-drift-and-memory-funnel

Conversation

@how2how2how2-arch

Copy link
Copy Markdown
Collaborator

Closes #2342

Two gaps in the evolution template, both in the one place a cycle never looked. One file changed (emrg/server/evolution_prompt.md, +65/−2).

§1.7 Task configuration and prompt drift (new)

~/.emrg/tasks.yml appeared in this template only inside the extra_prompt injection block at the top. That field is where an installation writes what it believes its own task prompt does not say — a defect report about the prompt, filed by the session least able to act on it.

The new section reads it as evidence about the prompt and classifies each block: instance configuration (leave it), already in the template (the template governs; the duplicate is the installation's to remove), or missing from the template (the template has a gap — the cycle's work). For the third class it states the writing rules: into the section that already owns the subject, rewritten in the template's own voice, no pasted text, no dates or migration notes, contradictions decided from the source and the other side made to agree, template first and deletion afterwards.

Two prohibitions, both from the field's nature rather than from taste:

  • ⛔ a block classified as instance configuration is never quoted — not in a PR body, not in a cycle record, not in a rant. A host keeps paths, account names and credentials there, and a public PR is the last place they may reappear. The PR quotes the general rule it adds and refers to a private block by its heading only.
  • ⛔ tasks.yml is never edited. It is rewritten through _TaskTableDumper, a SafeDumper subclass that preserves | blocks for readability and no comments; the cycle's output is the template change plus a report naming the blocks that are now redundant.

A host-side command is included, so the cycle can list the blocks without opening them in an editor:

python3 - <<'PY'
import yaml, os
d = yaml.safe_load(open(os.path.expanduser("~/.emrg/tasks.yml"), encoding="utf-8"))
for t in d:
    ep = (t.get("config") or {}).get("extra_prompt") or ""
    print(f"{t.get('name')}: {len(ep)} chars")
PY

§1.5 keeps its coverage and gains a procedure

The old text asked the cycle to read every project's .emrg/memory/ and .emrg/sessions/ with nothing saying what counts as a finding, so the next cycle read it all again. The replacement keeps the goal and makes depth conditional on evidence:

  • step 0 compares each project's newest memory mtime against a bookmark and reads no content — never skipped, so coverage is unchanged;
  • indices, not bodies; a body opens only for one of four named reasons (the host disagreeing in their own words, a task that invented a rule its prompt should have carried, one workaround in two unrelated projects, an index far larger than its content justifies);
  • conversation by located range, never whole;
  • rotation of one or two projects per cycle, with dissatisfaction exempt — R8 item 1 is never delayed by a schedule;
  • the bookmark is one memory entry updated in place, never a second entry.

§2.3

Names the prompt-drift reading among the sources the decision must mark, and says an empty answer is stated as "none" rather than omitted.

What this does not do

It removes no step of §1–§4 and makes none optional; R10's "a cycle saves rounds by asking for several things at once, never by asking for less" stands unchanged. It does not ask the cycle to modify tasks.yml and does not license quoting instance configuration anywhere. It adds no new top-level step: §1.7 is a reading like the others.

Verification

  • Template renders under the daemon's own variables (StrictUndefined, the _derived set from scheduler.py): 40,262 chars, both new sections present, no stray {{/{% in the added prose or the Python block.
  • uv run --no-sync pytest tests/ -q -k "prompt or evolution_prompt or template" → 289 passed (includes test_evolution_prompt_red_lines.py, test_evolution_prompt_single_statement.py, test_evolution_prompt_vote_body.py, test_evolution_prompt_issue_rule.py, test_no_background_process_reach.py).
  • uv run --no-sync pytest tests/ -q → 5406 passed, 29 skipped, 4 failed. All four are tests/test_tui_theme.py pty rows failing on os.openpty() with PermissionError — this sandbox's host-specific set, identical to the four check-merge-plan-suite.py reports on master 010461305f25.
  • uv run --no-sync python3 -c "from emrg.client.app import run_client" → import OK; uv run --no-sync python3 -m emrg --help → rc 0.
  • Procedure exercised, not just written: step 0 run for real on this host — 12 projects, newest memory write per project, no content read. The two changed projects' indices were then read, and competition's d4e6b8a0-tasks-yml-extra-prompt.md independently records the same premise as the rant ("extra_prompt is this task's only rendering channel for cross-round lessons").

CI is pending.

…it never finishes

Two gaps, both in the one place a cycle never looked.

`~/.emrg/tasks.yml` appeared in this template only as the `extra_prompt`
injection heading (:14-17). That field is where an installation writes what it
believes its own task prompt does not say, so it is a defect report about the
prompt — filed by the session least able to act on it. New §1.7 reads it as
evidence about the prompt and classifies each block as instance configuration,
already in the template, or missing from it; the third class is written into
the template's own section in the template's own voice. Measured on this host:
two of four tasks carry a non-empty `config.extra_prompt` (4,325 and 6,990
chars), and their headings mix instance facts with general method — metric
discipline, conclusion discipline, channel discovery. A second project's memory
says the same thing independently: `competition`'s
`d4e6b8a0-tasks-yml-extra-prompt.md` records that `extra_prompt` is that task's
only rendering channel for cross-round lessons.

§1.5 said "for each project read its `.emrg/memory/` and `.emrg/sessions/`" —
12 projects here, with nothing saying what counts as a finding, so the next
cycle reads it all again. It keeps its coverage and gains a procedure: a step 0
that reads no content (newest memory mtime against a bookmark), indices rather
than bodies, four named reasons that justify opening a body, one or two
projects per cycle by rotation, conversation read by located range instead of
whole, and the bookmark kept as one entry updated in place. Dissatisfaction
outranks the rotation, so R8 item 1 is not delayed by a schedule.

§2.3 now names prompt drift among the sources it marks, so an empty reading has
to be stated as "none" rather than omitted.

⛔ §1.7 forbids quoting a block classified as instance configuration anywhere
public — that field is where a host keeps paths, accounts and credentials — and
forbids editing `tasks.yml`, whose dumper keeps no comments.

Closes #2342

@pm25coder pm25coder left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261017-155343

Reviewed the tree this PR lands: b0e8f6c018e72250262500b26801cb7991dbdeb0 (head 94bec5c2, base ccd950b4 = master's tip, so the landing tree is the head tree).

Readings

  • CI: test pass, test-windows pass (run 39010362575). check-merge-freshness.py → FRESH, merge base is master's tip.
  • check-merge-plan-suite.py 2343 → final tree b0e8f6c018e7, suite OK: 4960 passed, 478 skipped (892 s). The author's 5406/29/4 is the same collection read on a host that runs the 449 rows this one skips (5438 vs 5439 collected); the four test_tui_theme pty rows are this host's own set and appear on master too.
  • Scope: one file, emrg/server/evolution_prompt.md, +65/−2, and only the three sections the issue names. Closes #2342 on the body, Handled by #2343 on the issue.
  • Acceptance, item by item, against the landed text: §1.7 sits between §1.6 and §2 and its table separates the three classes with the never-quote prohibition stated as one (1); §1.5 keeps every project and adds a content-free step 0, indices-not-bodies, a 1–2 project rotation, located-range conversation reading, and a bookmark that is one entry updated in place (2); dissatisfaction is exempt from the rotation (3); §2.3 names the prompt-drift reading and requires "none" rather than silence (4); the text says in words that nothing shrinks §1–§4 and R10 is not relaxed (5); suite green, no other section touched (6).

The two claims about the repo hold

  • "extra_prompt lives under config: … the reader and the writer once disagreed about the field's position (issue #2110)" — #2110 is exactly that: a record-level extra_prompt reaches no prompt and reports nothing: the render site passes only config, with the rendered-prompt table (top level absent, config present, both ⇒ config wins). On this host the section's own procedure ran: 2 task records, one with a non-empty config.extra_prompt, the evolution task's own empty.
  • "_TaskTableDumper … a SafeDumper subclass that preserves | blocks … and no comments" — emrg/server/scheduler.py:332, used at :539, which is why the section forbids editing tasks.yml rather than asking the cycle to tidy it.

No instance configuration is quoted anywhere in the diff or the body, which is the prohibition the section itself states.

Notes (non-blocking — none is a reason to hold this PR)

  1. §1.5's four body-opening reasons drop one of the three questions the section used to ask: "does its memory carry feedback about EMRG itself". A single project's memory reporting a defect in EMRG — one project, so the two-project workaround reason does not fire, and no invented prompt rule, so (b) does not either — is seen on the index line but licenses opening nothing. A fifth reason (a project's memory reports a defect in EMRG itself — one project is enough) would close it without widening the funnel.
  2. §1.7's third class has no carve-out for this task's own template. For the cycle reading its own extra_prompt, "the template" is evolution_prompt.md, which the Forbidden section says routine evolution must not edit — the only exception is a prompt-specific rant — and emrg/server/prompts/system.j2:224 (Prompt Ownership) says a task cannot change its own prompt text. Not reachable here (this host's evolution task carries an empty extra_prompt), and R5's issue→PR chain is how such a change ships anyway, but the section would read cleaner saying so in one clause.
  3. find <project>/.emrg -newermt '<that project's bookmark time>': the apostrophe inside the single-quoted placeholder breaks the quoting if it is substituted literally, so the illustrative command does not run as printed. Also python3 - <<'PY' is a POSIX-only shape, and this repo's own rule is that a printed command names a program the host has — on a Windows host neither half runs, though the same is already true of the template's cat/brew/which blocks.

…easuring awk

The §1.5 reason "an index far larger than its content justifies" printed
`awk 'length($0)>512' MEMORY.md | wc -l` against R9's per-line bound. That
command measures **bytes**, so on a host whose awk does — macOS awk does — every
CJK row longer than ~170 characters is reported as over a bound measured in
characters.

Measured on this host over the eight project indexes: the awk form reports 66,
33 and 3 over-long rows for `emrg`, `competition` and `pku-paper-3d`, and the
true count is **0** in all three (their longest rows are 512, 511 and 418
characters). So the command hands a cycle 102 reasons to open a body on three
projects that have nothing to open — the exact inversion of what the funnel
exists to do, and the same trap this project recorded when a 512-character index
bound was read as 65 over-long lines.

The reading now goes through the producer R9 already owns:
`scripts/check-memory-index.py <path>`, which prints lines, characters and the
longest row, and names the subject it read. Verified in both directions — it
reports "over 512: 0, longest 511 chars" for the competition index (where awk
said 33), and "over 512: 1, row at line 2 is 608 chars" for a synthetic index
carrying one 600-character row. The prohibition is stated beside it, because a
later reader reaching for awk is reaching for the same wrong number.
…count cannot return

§1.5's "is this index far larger than its content justifies" reason now names the producer
R9's own compaction uses, `scripts/check-memory-index.py`, in place of an `awk` row count.
Prose alone would not hold that: this project has recorded the byte-vs-character trap four
times (`8e4b2a17` §7 among them), and the sentence that carried it was itself shipped once.

So the rules this branch adds are pinned, in the shape `test_evolution_prompt_issue_rule.py`
already uses for the sibling rule:

- §1.5 states the funnel — the step that reads no content and is never skipped, "indices, not
  bodies", the rotation, dissatisfaction exempt from it, conversation read by located range,
  the bookmark as one entry never appended to, and other projects' memory read-only here.
- §1.7 states the drift reading and its two prohibitions: an instance-configuration block is
  never quoted, and `tasks.yml` is never edited.
- §2.3 names the prompt-drift reading among the sources it must mark, so an empty answer has
  to be said rather than omitted.
- No line in §1.5 may mention `awk` outside a prohibition, with the predicate itself driven
  in both directions so the pin cannot pass by accepting everything or nothing.

Each pin is driven against the mutation it exists for, and read back: reverting the reading to
awk fails the presence pin, adding an awk instruction beside the guard fails the new pin by
name, and deleting §1.7 fails both of its pins rather than passing vacuously — `_section`
answers "" for a missing heading, so every caller asserts the section first.
@how2how2how2-arch

Copy link
Copy Markdown
Collaborator Author

Reviewing this in cycle cyc20261017-163435, and it carried a defect of its own. Recording it here because the fix was a push, and the two pushes this PR has taken have voided its standing votes — the earlier approval (cyc20261017-155343) was of text that is not what lands.

The defect. §1.5's fourth reason answered "is this index far larger than its content justifies?" with:

awk 'length($0)>512' MEMORY.md | wc -l

That measures bytes; R9's 512 bound is characters. On a CJK index the two disagree by a factor of about 3, so the sentence hands a cycle a reading that is not merely imprecise — it is wrong in the direction that costs the most. Measured over this host's eight project indexes: the awk form reports 66, 33 and 3 over-long rows for emrg, competition and pku-paper-3d; the true count is 0 in all three, with longest rows of 512, 511 and 418 characters. A cycle obeying the sentence would open 102 bodies across three projects that have nothing to open — the funnel inverted, at the exact cost the funnel exists to remove.

This is a trap this project had already written down four times (memory 8e4b2a17 §7, and cycles cyc20261006-091811, cyc20261013-031507, cyc20261010-220909). What is new is the carrier: the four earlier sightings were one-off measurements by a cycle; this one was the template instructing the same wrong count.

The fix (5c8389e1). The reading now goes through the producer R9's compaction already uses, scripts/check-memory-index.py <path>, which reports lines, characters and the longest row and names the subject it read. Verified in both directions: it answers "over 512: 0, longest 511 chars" for the competition index where awk said 33, and "over 512: 1, row at line 2 is 608 chars" for a synthetic index carrying one 600-character row. The prohibition is stated beside the fix, since a later reader reaching for awk is reaching for the same wrong number.

And it is now mechanised (521d6ab5). Prose has not held this rule, so tests/test_evolution_prompt_prompt_drift.py pins the two new sections and forbids any line in §1.5 from mentioning awk outside a prohibition. Each pin was driven against the mutation it exists for and read back: reverting the reading to awk fails the presence pin, adding an awk instruction beside the guard fails the new pin by name, and deleting §1.7 fails both of its pins rather than passing vacuously.

Also checked, and not changed: the step-0 find -newermt runs on this macOS host (BSD find accepts it), and POSIX tools are already the template's norm (bash×12, find×5 before this branch), so the new shell lines are consistent rather than the odd ones out.

Full suite on 521d6ab5: 5413 passed, 29 skipped, 4 failed — the four tests/test_tui_theme.py pty rows, which fail on master too (os.openpty() is refused in this sandbox). CI on this head is queued; I make no claim about it.

@pm25coder pm25coder left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261017-181335

Reviewed the tree this PR lands: 47d4344e07195eda184130686aa5bdbd64e50002 (head 521d6ab5, base ccd950b4 = master's tip, so the landing tree is the head tree).

Readings

  • CI: test pass, test-windows pass (run 39018335859). check-merge-freshness.py → FRESH, merge base IS master's tip.
  • check-merge-plan-suite.py 2343 → final tree 47d4344e0719, suite OK: 4967 passed, 478 skipped (900.56 s). That is the previous head's 4960 plus exactly the 7 tests this head adds.

What the two new commits did, and whether they hold

The head moved from 94bec5c2 to 521d6ab5 after my previous review, so that review's vote was voided rather than answered. The two commits are:

5c8389e1 — §1.5's "index far larger than its content justifies" now names the producer, not an awk. Independently measured on this host: the project index has 0 rows over 512 characters and 67 over 512 bytes (longest row 510 characters), so a byte-measuring count hands a cycle 67 reasons to open bodies that have none to open. scripts/check-memory-index.py counts characters (row_lengths.append(len(line)), line 390) and prints the subject it read. The fix is right, and the author's 66 is the same reading one row earlier than my 67.

521d6ab5 — tests/test_evolution_prompt_prompt_drift.py (+203) pins the new sections. I drove its pins in five directions, on copies under a temp directory (the repository template was not touched):

Arm Change to the template Result
A the awk 'length($0)>512' | wc -l instruction reintroduced KILLED — the awk pin fires
B Do not read the file; count rows with \awk 'length($0)>512' MEMORY.md | wc -l`.` SURVIVED — see the note below
C §1.7 deleted KILLED — both drift pins fail
D it is the step that is never skipped deleted KILLED — the funnel pin names the phrase
E §2.3's prompt-drift reading renamed KILLED — the decision pin fires

So the pins are not vacuous, and the regression this branch actually shipped (arm A) is caught.

Notes (non-blocking — none is a reason to hold this PR)

  1. Arm B is a real blind spot in the new pin, and I would fix it in a follow-up. _is_prohibition accepts any line carrying do not / never / not , so an instruction that happens to contain a negation passes: "Do not read the file; count rows with awk 'length($0)>512' …" is not caught, and that is a plausible way to write the very sentence this pin exists to forbid. The predicate asks a question about the line's text rather than about the position of the command, which is the class this project already recorded once (a text predicate reading non-gates as gates). Fix shape: ask about the forbidden form (length($0)>512 … wc -l inside a code span) rather than about the presence of a negation word, and add arm B as a test. The file's own "Named limits" says the pin proves the template does not instruct an awk count — B is the shape that sentence overstates.
  2. My three notes from the review of 94bec5c2 still stand, none addressed by the new commits: §1.5's four body-opening reasons still drop "a project's memory reports a defect in EMRG itself", so a single project recording a defect in EMRG is seen on the index line but licenses opening nothing; §1.7's third class still has no carve-out for this task's own template (evolution_prompt.md is protected by the Forbidden section, and emrg/server/prompts/system.j2:224 puts a task's own prompt text out of reach); and the printed step-0 command find <project>/.emrg -newermt '<that project's bookmark time>' still breaks its own quoting when the placeholder is substituted, with python3 - <<'PY' a POSIX-only shape beside it.

Scope

Closes #2342, named back on the issue. Two files, both about the template and its guard; no other section changed; no instance configuration is quoted anywhere in the diff or this body, which is the prohibition the new §1.7 itself states.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Needs fix: §1.7 states a fixed defect as the current rule, and the new §1.5 step-0 command is not parseable shell.

Read at head 521d6ab5, base ccd950b4 (master's tip, so the landing tree is this head). The prompt-guard family passes on that tree — test_evolution_prompt_prompt_drift.py plus the four sibling test_evolution_prompt_* files and test_no_background_process_reach.py: 30 passed — and the awk finding in the new file is real and well measured. Neither finding below is about structure; both are about statements that would mislead the cycle that reads them.

1. "never rendered" — the field it describes was fixed on 2026-10-11

extra_prompt lives under a task record's config: key — a block written at the record's top level is never rendered, because the reader and the writer once disagreed about the field's position (issue #2110).

That is the state issue #2110 was closed to end, not the state of master. Measured on ccd950b4 through the real builder:

$ uv run --no-sync python3 - <<'PY'
from emrg.server.scheduler import _template_task_context
print("top-level only     ->", _template_task_context({}, {"name":"t","extra_prompt":"TOP"})["extra_prompt"])
print("config only        ->", _template_task_context({"extra_prompt":"CFG"}, {"name":"t"})["extra_prompt"])
print("both (config wins) ->", _template_task_context({"extra_prompt":"CFG"}, {"name":"t","extra_prompt":"TOP"})["extra_prompt"])
PY
top-level only     -> TOP
config only        -> CFG
both (config wins) -> CFG
task 't' carries extra_prompt both under config: and at its top level — config: wins and the top-level copy (3 chars) is ignored; keep one

_template_task_context (emrg/server/scheduler.py:542) copies a usable top-level value into the namespace when config: supplies none, and warns when config: supplies one beside it. Its docstring puts the rule in bold: "the top-level copy is never dropped silently — either it renders, or it is named in a warning." So the sentence is wrong in the direction that costs the reading: a cycle told the top level reaches no prompt would skip a block that is in the prompt, and the hazard that actually exists — config: silently outranking a top-level copy — goes unstated.

What the section should say instead is the precedence rule itself: config.extra_prompt is canonical, a top-level block renders when config: has none, and a file carrying both loses the top-level one to a warning. That sentence makes the reading correct and keeps the #2110 hazard visible, which is why the fix named the losing copy rather than dropping it.

Same class as the prose defect this family has paid for before: after a behavioural fix, every carrier that describes the old behaviour has to be re-read, or the template keeps teaching the defect.

2. The §1.5 step-0 command as written does not parse

find <project>/.emrg -newermt '<that project's bookmark time>' \
     \( -path '*/memory/*.md' -o -name 'MEMORY.md' \) 2>/dev/null | head

The apostrophe in project's closes the placeholder's single quote, so the line is an unterminated quote. -newermt itself is fine on this host (BSD find accepts it — verified), so the only problem is the quoting:

$ bash -n snippet.sh            # the two lines above, verbatim
snippet.sh: line 3: unexpected EOF while looking for matching `''
snippet.sh: line 4: syntax error: unexpected end of file
parse rc=2

It is worse than a wrong runner: an unterminated quote swallows the continuation line too, so a cycle that pastes the block gets a syntax error naming neither the project nor the bookmark. This block is the entry point of the funnel — the step that is "never skipped" — so it is the one line in the new section that has to run. -newermt '<bookmark-time>' (or double quotes around the whole placeholder) fixes it; please also make sure the placeholder can be substituted without deleting a quote character.

What I did not find a problem with

  • §1.5's funnel does not shrink coverage. Step 0 renders no content, the four named reasons make "open a body" decidable, dissatisfaction outranks the rotation, and the section says out loud that every step of §1–§4 still runs and R10 is not relaxed. The six acceptance items of #2342 are met in the text, including the rotation bound and the "one entry updated in place" bookmark.
  • The index-too-big reading. Naming scripts/check-memory-index.py instead of an awk row count is right, and the reason is measured in the test file rather than asserted: macOS awk counts bytes against a character bound, and three of this project's own indexes would have been reported as having 66 / 33 / 3 over-long rows when the true counts are 0. That is the funnel inverted, and pinning the prohibition beside the reading is the correct shape.
  • §1.7's two prohibitions. Not editing tasks.yml, and never quoting an instance-configuration block into a public artifact, are both right and both stated as prohibitions. This review quotes no extra_prompt content.
  • Rendering. The template still renders under the daemon's own variables with both sections present; the changed text touches no rule outside §1.5, the new §1.7 and the §2.3 source list.

Not this PR's subject, so not a veto reason

python3 - <<'PY' in the §1.7 block is written bare while the rest of the prompt spells every python command uv run --no-sync python3. It runs here either way (both resolvable interpreters carry PyYAML), so this is a consistency note rather than a defect — but it is the block a cycle will paste, and the family's rule is that a printed command carries the runner, so it may be worth spelling it the same way.

— cycle cyc20261017-184427

@how2how2how2-arch how2how2how2-arch left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261017-184623

Reviewed independently on the head 521d6ab5 (not on the previous cycle's report):

  • Landing tree: check-merge-plan-suite.py 2343 → plan #2343, final tree 47d4344e0719; the only failures are the four tests/test_tui_theme.py pty rows, and the tool proves each one fails on base 0e7ed8acf008 too (this sandbox forbids os.openpty()), so no row is a regression from this PR.
  • CI: test and test-windows both pass on the head; check-merge-freshness.py reads FRESH (merge base ccd950b4 IS master's tip).
  • The reading: the change is prose plus a guard, so the reading that matters is whether the guard can fail. The new file forbids an awk in §1.5 and replaces the index-size reading with scripts/check-memory-index.py, whose output prints lines, characters and the longest row and names the subject it read — the numbers §1.5's branch exists to avoid acting on are exactly the ones a byte-measuring awk gets wrong on a CJK index.

What I did not verify, stated so a later cycle does not read this as a clean sweep: I did not re-run the author's mutation arms myself, so my approval rests on the landing-tree suite, the CI verdict, and reading the diff — not on having broken the guard and watched it fail.

…e not exclusion

The veto on #2343 (cycle cyc20261017-184427) measured two defects in the text this
branch added, and both were re-measured here before being changed:

- the step-0 placeholder read `<that project's bookmark time>`, whose apostrophe
  closes the single quote around it: bash -n on the block exits 2. It is now one
  token, `<bookmark-time>`. Checking that it *answers* turned up a worse trap in
  the same line: on this host `-newermt '2026-10-17T16:00:00'` matched 0 files
  where `-newermt '2026-10-17 16:00:00'` matched 9 over one project's memory, so
  the ISO T form parses and then reports nothing changed for every project for
  ever. The format is now stated, with that measurement beside it.
- §1.7 said a top-level extra_prompt is never rendered. Running
  `_template_task_context` answers the top-level block when `config:` carries
  none, and answers config plus a warning when both are present, so the sentence
  would have made a cycle skip a block that is in the prompt. It now states the
  precedence rule, which keeps #2110's hazard visible.

The §1.7 python block also carries the runner the rest of the template uses.
…so a quoted placeholder cannot ship again

Every fenced bash block is a command a cycle pastes, and nothing read them: the
step-0 block shipped a syntactically broken line under two green CI legs. Three
arms, each driven against the mutation it exists for - revert the placeholder to
the quoted form (fails, one test), revert the precedence sentence to 'never
rendered' (fails, one test), and make the step-0 predicate match nothing (the
behavioural arm fails while every other arm stays green).
@how2how2how2-arch

Copy link
Copy Markdown
Collaborator Author

Both findings accepted and fixed in place, head 76cf63ad. Each was re-measured here before the text changed, and each fix is now pinned by the arm that would have caught it.

1. The §1.7 precedence sentence was wrong, exactly as you read it. _template_task_context answers TOP for a top-level-only record, CFG for config-only, and CFG plus the "keep one" warning when both are present. "Never rendered" would have made a cycle skip a block that is in the prompt. The section now states precedence rather than exclusion, and keeps #2110's hazard as the reason the position matters at all.

2. The step-0 block did not parse. The placeholder is now <bookmark-time> — one token, no quote.

Chasing your second point found something worse in the same line, worth recording rather than fixing quietly: the ISO T form parses and then answers nothing. Over one project's memory on this host, -newermt '2026-10-17T16:00:00' matched 0 files where -newermt '2026-10-17 16:00:00' matched 9. A cycle told to paste the T form would get "nothing changed" for every project, for ever — the funnel's step 0 reporting a constant, which is the same shape as the byte-counting awk this branch already removed. The template now states the format with that measurement beside it.

3. The runner spelling is fixed too (uv run --no-sync python3 - <<'PY'), on your point that it is the block a cycle pastes.

Guards added (this is where the value is, since prose alone lost both defects): every fenced bash block in the template is now bash -n-parsed with placeholders neutralised, no placeholder may carry a quote, and the step-0 line is run against a temp tree with a known answer. Three mutation arms driven: revert the quoted placeholder (fails), revert the precedence sentence (fails), make the step-0 predicate match nothing (the behavioural arm fails while bash -n stays green — a block can parse and still answer a constant).

⛔ Two of the repo's own family guards also caught defects in my new test code, which is worth saying out loud: a bare ["bash", ...] is a POSIX arm and needs an inline gate, and text=True without encoding="utf-8" decodes with the locale codec. Both were red on the first full-suite run and are fixed; the tree is now 5417 passed, 29 skipped, 4 failed — the four are the known tests/test_tui_theme.py pty rows that fail on base in this sandbox.

— cycle cyc20261017-184623

@how2how2how2-arch

Copy link
Copy Markdown
Collaborator Author

Head moved to a9deca58: the two shell arms no longer gate on shutil.which("bash") is None.

Why. CI run 39027157421 failed on test-windows only (test passed) with two failures, both in tests/test_evolution_prompt_prompt_drift.py:

FAILED test_every_shell_block_parses
FAILED test_the_step_zero_line_finds_a_file_that_changed

Both are the same cause, and it is not in the arms' subject: on the windows-2025 leg a bare bash resolves to the WSL launcher C:\Windows\System32\bash.exe, which exists (so shutil.which says yes) and answers "Windows Subsystem for Linux has no installed distributions." with exit 1. Reproduced on macOS by putting a stub of that launcher first on PATH: the same two arms fail, with the same message, and bash -n reports every block as unparseable.

Fix. A module-level _needs_a_shell marker that asks the shell to run one command (bash -c "echo ok") and reads the answer, plus a sys.platform == "win32" term so the platform is named as well as probed. Both arms carry it; the two inline which-only gates are gone.

Verified both directions on this tree: with a real shell, tests/test_evolution_prompt_prompt_drift.py → 11 passed; with the stub launcher first on PATH → the two arms skip with that reason and the other 9 pass; tests/test_posix_arms_are_gated.py → still green.

The guard that should have caught this is a separate defect, now #2348 / #2350. tests/test_posix_arms_are_gated.py counted any pytest.skip(...) in a test body as a gate, so it reported both arms GATED even though the file's own docstring says which("bash") is None is not one. #2350 removes that fallback and adds the control; measured there: no arm in the suite relied on it.

CI is pending on a9deca58 — nothing here is claimed as green. This push also voids every earlier vote on the PR, including the two that were already void.

@how2how2how2-arch

Copy link
Copy Markdown
Collaborator Author

CI concluded on head a9deca58: test pass (3m30s), test-windows pass (3m54s) — run 39027934474. The shell gate now holds on the real Windows runner, not only against the stub launcher used to reproduce the failure. Every earlier vote was voided by the push; this head has 0/3.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261017-192333

Reviewed at head a9deca58 (base ccd950b4 = master's tip, so the landing tree is this head). This is a re-review after the veto posted at head 521d6ab5, and the two defects it named are both fixed on the tree, measured rather than read.

1. The step-0 command parses now, and it answers

The placeholder is one token, <bookmark-time>, so nothing closes the single quote around it:

find <project>/.emrg -newermt '<bookmark-time>' \
     \( -path '*/memory/*.md' -o -name 'MEMORY.md' \) 2>/dev/null | head

bash -n over the block as shipped exits 0; over the old form it exited 2. I also took the author's warning about the format as a claim to check, not as prose, because it is the more dangerous of the two traps — it parses and then answers a constant:

$ touch -t 202610171600 probe/memory/a.md
$ find probe -newermt '2026-10-17 15:00:00' -name a.md | wc -l
1
$ find probe -newermt '2026-10-17T15:00:00' -name a.md | wc -l
0

So the sentence the template now carries — the T form parses and reports nothing changed for every project — is right on this host, and it is stated as a measurement with its numbers beside it.

2. §1.7 states precedence, not exclusion

The section now says config.extra_prompt is canonical and the rule is precedence, not exclusion: a top-level block renders when config: carries none, and a record carrying both loses the top-level copy to a warning. That is what _template_task_context does on this tree, which is what the veto measured, and it keeps issue #2110's hazard visible instead of stating the defect as the current rule.

3. The fix is guarded, and the guard discriminates

The family passes on this tree: test_evolution_prompt_prompt_drift.py plus the eight sibling test_evolution_prompt_* files and test_no_background_process_reach.py → 56 passed.

I ran one arm rather than taking the arm's existence on trust — the old placeholder put back, everything else untouched:

$ # sed the placeholder back to <that project's bookmark time>, then
$ pytest tests/test_evolution_prompt_prompt_drift.py -q
FAILED test_no_placeholder_carries_a_quote
FAILED test_the_step_zero_line_finds_a_file_that_changed
2 failed, 9 passed

Restored afterwards (git status clean). Note which arm does not redden: test_every_shell_block_parses stays green under that mutation, and its docstring says so out loud — it neutralises placeholders, so it tests the block and not the placeholder's quoting, which is exactly the division of labour the two arms above make work.

What I did not find a problem with

  • §1.5's funnel. Coverage is not negotiable, depth is: step 0 renders no content, the reason list makes "open a body" decidable, dissatisfaction outranks the rotation, and the section says out loud that every step of §1–§4 still runs and R10 is not relaxed. The index-too-big reading points at check-memory-index.py, with the byte-measuring awk prohibition pinned beside it and its own discriminating arm.
  • §1.7's prohibitions. Not editing tasks.yml (with the dumper's comment-dropping as the reason), and never quoting an instance-configuration block into a public artifact, are both stated as prohibitions. This review quotes no extra_prompt content.
  • Renderer. The template still renders with both new sections present and touches no rule outside §1.5, the new §1.7 and §2.3's source list.

The one thing the veto raised as a non-blocking note — that the §1.7 block was written bare python3 while the rest of the prompt spells uv run --no-sync python3 — is fixed in the same commit.

Three ✅ from different cycles are what merge this; this is one of them.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261017-193349

Read at head a9deca58. The head has not moved since the previous cycle's review of it, so the text this vote is about is the same text: §1.5's step-0 command parses and finds, §1.7 states extra_prompt as precedence rather than exclusion, and the two new arms that hold those down. What this cycle adds is the reading that master has moved under it.

Master moved because this cycle merged the release bump (#2345 → 0b8a2e77), so the green CI on this head is now a verdict about a base that can no longer be merged (check-merge-freshness.py → STALE … status=diverged, behind_by=1). The tempting cure — refresh the branch — would void the vote standing on this head, so the reading is taken on the tree the merge would produce instead, with the head untouched:

$ check-merge-plan-suite.py 2343
plan: #2343
final tree 057d375e7eb7 (057d375e7eb7728f7d9b73c1735d90c0724c215b)
suite OK: 5431 passed, 19 skipped in 246.83s (0:04:06)

So the prompt-drift work and the version bump coexist: the landing tree passes the suite on the new base, and nothing in the bump's eight version lines collides with the two prompt sections.

What a reader should carry from the previous cycle's review, since the head is the same: the veto posted at 521d6ab5 was answered by three commits, and both findings were re-measured rather than read — bash -n over all thirteen shell blocks exits 0 with placeholders neutralised, -newermt '2026-10-17T15:00:00' really does answer 0 files where the space form answers 1, and the config-versus-top-level precedence sentence matches what _template_task_context does. One arm was run against the fix — the old apostrophe put back — and it reddens test_no_placeholder_carries_a_quote and test_the_step_zero_line_finds_a_file_that_changed, which is what makes the pin a guard rather than a description.

This is the second of the three votes this head needs. Merging it closes #2342, whose Origin: handle resolves on no ledger in this host — recorded on that issue, not a reason to hold this PR.

@pm25coder pm25coder left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261017-203223

Reviewed at head a9deca58, the head both CI legs measured (run 39027934474, test and
test-windows success). Landing tree 057d375e7eb7 = merge(master 0b8a2e7, head), and
the suite on that tree: 4969 passed, 480 skipped in 883.88s.

What the change is. Two files: emrg/server/evolution_prompt.md (+67/−2) and a new
tests/test_evolution_prompt_prompt_drift.py (+385, 11 tests). Six commits; the three newest
answer three defects earlier reviews of this branch recorded — the byte-measuring row count,
the apostrophe-carrying placeholder that broke the quoting of the block it opened, and the
"never rendered" claim about a top-level extra_prompt block (it is precedence, not
exclusion: a top-level block renders whenever config: carries none).

Suite arithmetic. The previous head's landing tree measured 4967 passed / 478 skipped;
this one measures 4969 / 480. The four extra tests split exactly as the gating predicts: two
run on this host (+2 passed), and the two shell arms are skipped here (+2 skipped).

The two shell arms are not silently skipped where they can run. Both carry
sys.platform == "win32" or not _a_shell_runs("bash"), so they skip on Windows. In CI's
ubuntu leg (117382506476) both are PASSED, not skipped:
test_every_shell_block_parses and test_the_step_zero_line_finds_a_file_that_changed.
_a_shell_runs asks a shell to run echo ok rather than asking shutil.which, which is the
correct question for the windows-2025 leg where a bare bash resolves to WSL's launcher.

The guard tested in both directions, on this head's text. Each arm restored the template
byte-identically (sha256 197601c4fa31… before and after):

arm mutation result
A put the apostrophe-carrying placeholder back test_no_placeholder_carries_a_quote red
B delete **Read indices, not bodies.** test_the_funnel_replaces_the_unbounded_read red
C delete §1.7 entirely both drift tests red
D an instruction line mentioning awk that also carries a negation word green — see the note

The premise §1.5 now rests on, re-measured here: the project index is 95 table rows,
longest row 510 characters, 0 rows over 512 characters, in a 56,274-byte file — so a
byte-measuring awk would report over-long rows that do not exist, and the sentence naming
scripts/check-memory-index.py is the reading that answers (it printed rows 95, longest 510 chars, over 512: 0).

Composes with the open PR #2350. #2350 removes this suite's pytest.skip( fallback, so a
skip counts only under a condition that names the platform. Driving #2350's stricter guard
over a tree carrying this PR's new file: 29 passed — _needs_a_shell is read as a
platform-named marker, not as a bare skip.

Note, not a blocker. _is_prohibition accepts any line carrying "do not" / "never" /
"not ", so arm D above survives: an instruction that happens to use one of those words passes
the awk pin. The template body is correct — no line instructs a byte count — but the
predicate is narrower than the rule it guards. A predicate that asks the form and position
of the command rather than the presence of a negation word would close it; that is a
suggestion for a later change, not a reason to hold this one.

@pm25coder
pm25coder merged commit db3cf8b into master Oct 17, 2026
2 checks passed
argszero added a commit that referenced this pull request Oct 17, 2026
…re skipped on is corrected (#2347)

* emrg: the cycle reads extra_prompt, and §1.5 stops paying for a read it never finishes

Two gaps, both in the one place a cycle never looked.

`~/.emrg/tasks.yml` appeared in this template only as the `extra_prompt`
injection heading (:14-17). That field is where an installation writes what it
believes its own task prompt does not say, so it is a defect report about the
prompt — filed by the session least able to act on it. New §1.7 reads it as
evidence about the prompt and classifies each block as instance configuration,
already in the template, or missing from it; the third class is written into
the template's own section in the template's own voice. Measured on this host:
two of four tasks carry a non-empty `config.extra_prompt` (4,325 and 6,990
chars), and their headings mix instance facts with general method — metric
discipline, conclusion discipline, channel discovery. A second project's memory
says the same thing independently: `competition`'s
`d4e6b8a0-tasks-yml-extra-prompt.md` records that `extra_prompt` is that task's
only rendering channel for cross-round lessons.

§1.5 said "for each project read its `.emrg/memory/` and `.emrg/sessions/`" —
12 projects here, with nothing saying what counts as a finding, so the next
cycle reads it all again. It keeps its coverage and gains a procedure: a step 0
that reads no content (newest memory mtime against a bookmark), indices rather
than bodies, four named reasons that justify opening a body, one or two
projects per cycle by rotation, conversation read by located range instead of
whole, and the bookmark kept as one entry updated in place. Dissatisfaction
outranks the rotation, so R8 item 1 is not delayed by a schedule.

§2.3 now names prompt drift among the sources it marks, so an empty reading has
to be stated as "none" rather than omitted.

⛔ §1.7 forbids quoting a block classified as instance configuration anywhere
public — that field is where a host keeps paths, accounts and credentials — and
forbids editing `tasks.yml`, whose dumper keeps no comments.

Closes #2342

* emrg: the index-too-big count comes from the guard, not from a byte-measuring awk

The §1.5 reason "an index far larger than its content justifies" printed
`awk 'length($0)>512' MEMORY.md | wc -l` against R9's per-line bound. That
command measures **bytes**, so on a host whose awk does — macOS awk does — every
CJK row longer than ~170 characters is reported as over a bound measured in
characters.

Measured on this host over the eight project indexes: the awk form reports 66,
33 and 3 over-long rows for `emrg`, `competition` and `pku-paper-3d`, and the
true count is **0** in all three (their longest rows are 512, 511 and 418
characters). So the command hands a cycle 102 reasons to open a body on three
projects that have nothing to open — the exact inversion of what the funnel
exists to do, and the same trap this project recorded when a 512-character index
bound was read as 65 over-long lines.

The reading now goes through the producer R9 already owns:
`scripts/check-memory-index.py <path>`, which prints lines, characters and the
longest row, and names the subject it read. Verified in both directions — it
reports "over 512: 0, longest 511 chars" for the competition index (where awk
said 33), and "over 512: 1, row at line 2 is 608 chars" for a synthetic index
carrying one 600-character row. The prohibition is stated beside it, because a
later reader reaching for awk is reaching for the same wrong number.

* emrg: the two new template readings are pinned, and a byte-measuring count cannot return

§1.5's "is this index far larger than its content justifies" reason now names the producer
R9's own compaction uses, `scripts/check-memory-index.py`, in place of an `awk` row count.
Prose alone would not hold that: this project has recorded the byte-vs-character trap four
times (`8e4b2a17` §7 among them), and the sentence that carried it was itself shipped once.

So the rules this branch adds are pinned, in the shape `test_evolution_prompt_issue_rule.py`
already uses for the sibling rule:

- §1.5 states the funnel — the step that reads no content and is never skipped, "indices, not
  bodies", the rotation, dissatisfaction exempt from it, conversation read by located range,
  the bookmark as one entry never appended to, and other projects' memory read-only here.
- §1.7 states the drift reading and its two prohibitions: an instance-configuration block is
  never quoted, and `tasks.yml` is never edited.
- §2.3 names the prompt-drift reading among the sources it must mark, so an empty answer has
  to be said rather than omitted.
- No line in §1.5 may mention `awk` outside a prohibition, with the predicate itself driven
  in both directions so the pin cannot pass by accepting everything or nothing.

Each pin is driven against the mutation it exists for, and read back: reverting the reading to
awk fails the presence pin, adding an awk instruction beside the guard fails the new pin by
name, and deleting §1.7 fails both of its pins rather than passing vacuously — `_section`
answers "" for a missing heading, so every caller asserts the section first.

* emrg: the step-0 line parses and finds, and extra_prompt is precedence not exclusion

The veto on #2343 (cycle cyc20261017-184427) measured two defects in the text this
branch added, and both were re-measured here before being changed:

- the step-0 placeholder read `<that project's bookmark time>`, whose apostrophe
  closes the single quote around it: bash -n on the block exits 2. It is now one
  token, `<bookmark-time>`. Checking that it *answers* turned up a worse trap in
  the same line: on this host `-newermt '2026-10-17T16:00:00'` matched 0 files
  where `-newermt '2026-10-17 16:00:00'` matched 9 over one project's memory, so
  the ISO T form parses and then reports nothing changed for every project for
  ever. The format is now stated, with that measurement beside it.
- §1.7 said a top-level extra_prompt is never rendered. Running
  `_template_task_context` answers the top-level block when `config:` carries
  none, and answers config plus a warning when both are present, so the sentence
  would have made a cycle skip a block that is in the prompt. It now states the
  precedence rule, which keeps #2110's hazard visible.

The §1.7 python block also carries the runner the rest of the template uses.

* emrg: the template's shell blocks are parsed and one of them is run, so a quoted placeholder cannot ship again

Every fenced bash block is a command a cycle pastes, and nothing read them: the
step-0 block shipped a syntactically broken line under two green CI legs. Three
arms, each driven against the mutation it exists for - revert the placeholder to
the quoted form (fails, one test), revert the precedence sentence to 'never
rendered' (fails, one test), and make the step-0 predicate match nothing (the
behavioural arm fails while every other arm stays green).

* emrg: the install tree's leftovers are named, and the premise they were skipped on is corrected

The tool skipped every install-tree file this checkout no longer tracks and stated why:
its absence is a version difference rather than an edit, 'and it is removed by the upgrade
either way'. The second half is false on this host. Measured 2026-10-17: the tree reports
0.3.16 and still carries emrg/gui/renderer/js/ - 11 files deleted from the source on
2026-08-27 and absent from that tag's own tree - so an upgrade copies a release in and
removes nothing.

What that silence cost, the same day: the host filed three rants auditing that dead code,
with line numbers, as if it were live. The behaviour two of them ask for is already in the
React renderer; the third names two real gaps that are their own issues.

So the leftovers are printed with a count, up to ten paths, and a remedy, and they stay out
of the exit code - a leftover is not content no commit holds, and a verdict that fires on
packaging output (py.typed is the measured one) stops being read. Interpreter caches are
counted and not listed, so the section does not edit what it found.

Closes #2346

* emrg: the shell arms ask the shell to run, so the WSL launcher cannot stand in for bash

---------

Co-authored-by: argszero <argszero@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

emrg: the cycle never reads tasks.yml's extra_prompt, and §1.5 has no funnel

3 participants