Skip to content

Continual Harness and Harness² declare the adapter parts they write - #123

Merged
haizhongzheng merged 2 commits into
mainfrom
method-needs
Nov 21, 2026
Merged

haizhongzheng merged 2 commits into
mainfrom
method-needs

Conversation

@haizhongzheng

@haizhongzheng haizhongzheng commented Nov 21, 2026 •

Copy link
Copy Markdown
Member

Change

Continual Harness and Harness² write skills and sub-agents (Harness² writes hooks too), but neither method declared them in its evolver.yaml needs:

Method Before After
continual_harness needs: [llm] needs: [llm, skills, agents]
harness2_upstream needs: [] needs: [skills, agents, hooks]

The Trainer refuses a run whose acting agent does not take what the method needs (adapter_needs_errors, supports against needs). Because these parts were not declared, a run with react (which takes only context, context_tool and tools) passed that check, and trained skills and sub-agents that no session ran. Now such a run is refused before it starts, as AHE Adapter's (needs: [skills, agents, hooks]) already is.

The parts come from each method's template exports: continual-harness exports skills and agents; harness2-codex exports skills, agents and hooks.

What does not change:

  • The CLI agents (Codex, Claude Code, Pi, ...) take every export (CliAgent.supports = ALL), so every example config of these methods passes the check as before. They all use Codex or Claude Code (examples/officeqa/continual_harness/*.yaml, examples/officeqa/harness2_stream/run_*.yaml).
  • No code changes, only the two evolver.yaml files. The new test is in tests/core/test_environment.py, and there is a CHANGELOG line under Fixed.

Validation

  • New test test_a_method_whose_harness_has_skills_and_roles_is_refused_react, parametrized over AHE Adapter, Continual Harness and Harness². For each method it checks that react is refused with the missing parts named and that codex is accepted. Run against origin/main (the test file copied into a clean worktree), the Continual Harness and Harness² cases fail and AHE Adapter passes. With the change all 3 pass, about 0.1 s each.
  • tests/core/test_environment.py, tests/methods/harness2, tests/methods/continual_harness: 47 passed, 9 skipped, 2 failed. The 2 failures are in test_harness2_on_pi.py::test_what_a_pi_session_spent_is_kept_when_it_times_out_or_is_stopped. They fail the same way on main locally, because the upstream harness2 package is not installed in this environment (ModuleNotFoundError in channel.py).
  • python -m pytest --changed --base origin/main at 8a3a8b7, which selected 1287 of 1942 tests: 1167 passed, 117 skipped, 7 failed. All 7 fail the same way on a clean main worktree in this environment, so they come from the local install, not from this change:
    • test_the_moved_modules_old_paths_are_gone ×4: an older hermit installed in site-packages still provides the old paths;
    • test_harness2_on_pi ×2: the upstream harness2 package is missing;
    • test_rrsi_sees_a_harbor_rollouts_failure_and_a_cli_model_is_refused_for_harbor ×1.
      The follow-up commit changes only CHANGELOG wording. CI is the authority.

Review

  • Tests and documentation cover the affected behavior, where applicable: the test above and the CHANGELOG entry. No document quotes either method's needs.
  • Commands, defaults, and public interfaces are consistent: only the two needs values change. Every value is in ADAPTER_NEEDS.
  • Agents follow the agent layers: no agent changes. This is the needs/supports rule of the agent layers, applied to two methods that were missing it.
  • Shared logs and examples contain no credentials or private data.
  • Documentation-only: not applicable.
  • Core changes: none in code. Two method declarations change.
  • Real execution paths and tests: the test goes through Config.load and the Trainer's adapter_needs_errors with each method's real evolver.yaml (find(method).needs), and with the built-in react and codex agents.
  • Test checklist: the test is in tests/core/ beside the existing needs test and reuses its _config helper. Its name states the behaviour. It writes only to tmp_path and uses no network or CLI.
  • This description: read back against the diff.
  • Independent review: 2 rounds. Each round was this session plus a fresh, read-only Claude Opus 5.5 session.
    • Round 1: one fix, the CHANGELOG wording (3d81658). It said "training skills and sub-agents", which is wrong for Harness², whose adapter never changes. Two minor points were left.
    • Round 2: no new finding.
    • Left as minor:
      • With react, Continual Harness is now refused even when components writes no skills or sub-agents. react could not run this method anyway: the template and the memory handoff need a CLI agent.
      • The test's ahe_adapter case repeats the existing needs test.

🤖 Generated with Claude Code

haizhongzheng and others added 2 commits November 20, 2026 20:57
Both methods write skills and sub-agents (Harness² hooks too), but their
evolver.yaml declared `needs: [llm]` and `needs: []`, so a run with an acting
agent that drops those parts (react) passed the supports check and trained
parts no session ran. They now declare `[llm, skills, agents]` and
`[skills, agents, hooks]`, as AHE Adapter does; the CLI agents take every
part, so the examples (Codex, Claude Code) are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@haizhongzheng
haizhongzheng merged commit c22db4d into main Nov 21, 2026
1 check passed
@haizhongzheng
haizhongzheng deleted the method-needs branch November 21, 2026 03:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant