Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 17 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,20 @@ Changes are grouped by software version. Entries under Unreleased describe work

### Breaking

- The old import paths of moved modules are gone, with no forward: 0.1's, which 0.2 forwarded (`hermit.agents`,
`hermit.evolver`, `hermit.llm`, `hermit.backends`, `hermit.trajectory`), and 0.2's `hermit.agent.base.direct` and
`hermit.agent.base.container`, and `hermit.agent.environment` (never released). Import instead:

| Old | New |
|---|---|
| `hermit.agents` (`.base`, `.cli`, `.react`, `.tools`, `.container`, `.login`) | `hermit.agent.acting` (`.agent`, `.workspace`, `.react`, `.tools`), `hermit.environment.docker`, `hermit.agent.base.login` |
| `hermit.evolver` (`.base`, `.agent`, `.loader`, `.dependencies`) | `hermit.agent.evolver` (the same submodules) |
| `hermit.llm`, `hermit.backends` | `hermit.agent.model.llm`, `hermit.agent.model.backends` |
| `hermit.trajectory` | `hermit.agent.base.trajectory` |
| `hermit.agent.base.direct` | `hermit.agent.model.direct` |
| `hermit.agent.base.container` | `hermit.environment.docker` |
| `hermit.agent.environment` (`.base`, `.docker`) | `hermit.environment` (`.base`, `.docker`) |

- Harbor gives a task set its data, its environments and its verifier; every agent is Hermit's
([Benchmarks](docs/framework/task.md#benchmarks)). `agent.name: harbor` is gone, with `hermit/harbor/runner.py`,
`hermit/harbor/harborside/` (the HermitXXX classes, the scripted agent, the stream driver) and the runner's parts
Expand Down Expand Up @@ -69,7 +83,7 @@ Changes are grouped by software version. Entries under Unreleased describe work
`tasks/`.

- Tasks and environments are packages of their own beside the agents: `hermit/environment/` (moved from
`hermit/agent/environment/`, which forwards for one release) and `hermit/task/` (from `hermit/task.py`: `Task` with
`hermit/agent/environment/`) and `hermit/task/` (from `hermit/task.py`: `Task` with
a public half the agent is handed and a private half only the judge reads, `Benchmark`, the verifier). `RunOutput`
and `load_callable` live in `hermit/common.py`; every name importable from `hermit.task` before still is. Tasks and
agents never import each other (`tests/core/test_layers.py`).
Expand All @@ -79,8 +93,8 @@ Changes are grouped by software version. Entries under Unreleased describe work
roles=)` replaces `run(prompt, workspace=, limits=)` (a path is a `Local` environment), and a CLI Base Agent
refuses `isolation`, `sandbox` or `container` given as settings (use `Sandboxed(path, sandbox=...)` or
`Container(path, settings=...)`). A single model call is Layer 1: `Direct(settings, model)` in
`hermit.agent.model.direct` returns a `Reply`, and `make("direct")` is refused. `hermit.agent.base.direct` and
`hermit.agent.base.container` (now `hermit.environment.docker`) forward for one release. A run whose acting
`hermit.agent.model.direct` returns a `Reply`, and `make("direct")` is refused. `hermit.agent.base.container` is
now `hermit.environment.docker`. A run whose acting
agent does not take what its method needs of an adapter (`supports` against `evolver.yaml` `needs`) is refused
before it starts.

Expand Down
4 changes: 2 additions & 2 deletions docker/claude/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# The Claude Code agent's container (hermit/agent/base/container.py): Ubuntu 24.04, python3 (for the session's
# The Claude Code agent's container (hermit/environment/docker.py): Ubuntu 24.04, python3 (for the session's
# scripts), ripgrep, bubblewrap and socat (Claude Code's own sandbox, which runs the session's commands, needs both), and
# the machine's own Claude Code, its native build (one file), copied here first:
#
# cp ~/.local/share/claude/versions/2.1.318 docker/claude/claude
# docker build --build-context codex=docker/codex -t officeqa-claude:2.1.318 docker/claude
#
# A run names the tag it built (environment.image, or a CLI model endpoint's container.image); a session given none
# takes HERMIT_CLAUDE_IMAGE, else hermit-claude:2.1.318 (container.py's default).
# takes HERMIT_CLAUDE_IMAGE, else hermit-claude:2.1.318 (docker.py's default).
#
# The egress proxy's script is the Codex images' (docker/codex/egress_proxy.py, the `codex` build context), so an
# egress container can run this image too. The CLI does not update itself here (DISABLE_AUTOUPDATER): the image is
Expand Down
2 changes: 1 addition & 1 deletion docker/codex/Dockerfile.standalone
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# The Codex agent's container (hermit/agent/base/container.py) with the standalone Codex a machine installs for
# The Codex agent's container (hermit/environment/docker.py) with the standalone Codex a machine installs for
# itself instead of the npm package (the Codex a ChatGPT login runs: the OfficeQA GPT-5.6-Luna configs):
# `codex` keeps it under ~/.codex/packages/standalone/releases/<version>-<target>/ (bin/codex, codex-path/rg,
# codex-resources/bwrap). Copy that directory into ./codex-standalone first:
Expand Down
2 changes: 1 addition & 1 deletion docker/offline.sh
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ EGRESS=hermit-egress
PROXY_URL=http://$EGRESS:3128
USER_IDS=$(id -u):$(id -g)
STAMP=$(date +%Y%m%d-%H%M%S)-$$
# Docker's default capabilities less three the runs do not need (hermit/agent/base/container.py's list); label=disable keeps an
# Docker's default capabilities less three the runs do not need (hermit/environment/docker.py's list); label=disable keeps an
# SELinux host from refusing the mounts. SANDBOX adds what hermit's bubblewrap sandboxes need: SYS_ADMIN with seccomp
# and AppArmor off to make namespaces, and Docker's masks over /proc lifted (systempaths), without which a fresh
# /proc cannot be mounted in a namespace ("Can't mount proc on /newroot/proc"); the user is unprivileged, so what the
Expand Down
10 changes: 5 additions & 5 deletions docs/framework/agent-layers.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,9 +319,9 @@ The layers moved `hermit/backends.py`, `hermit/llm.py`, `hermit/trajectory.py`,
`hermit/evolver/` into `hermit/agent/`, the learning methods from `hermit/algorithms/evolvers/` to
`hermit/agent/evolver/methods/`, and the adapter templates from `hermit/algorithms/templates/` to
`hermit/adapter/templates/`. The old import paths (`hermit.backends`, `hermit.llm`, `hermit.trajectory`,
`hermit.agents`, `hermit.evolver`) forward to the new ones for one release and warn when used; `hermit.algorithms`
is gone, and so are `hermit.tools_mcp` and the tool gate `hermit.agent.evolver.gate` (an agent Evolver's tools are
served over MCP and JSON by the call's tool server).
`hermit.agents`, `hermit.evolver`) forwarded to the new ones in 0.2 only; `hermit.algorithms` is gone, and so are
`hermit.tools_mcp` and the tool gate `hermit.agent.evolver.gate` (an agent Evolver's tools are served over MCP and
JSON by the call's tool server).
Configurations that name an agent, an Evolver or a template (`agent.name: codex`, `evolver.name: gepa`,
`template: prompt-only@1.0`) are unchanged.

Expand All @@ -331,8 +331,8 @@ Configurations that name an agent, an Evolver or a template (`agent.name: codex`
|---|---|
| `agent.run(prompt, workspace=path, limits=Limits(...))` | `agent.run(prompt, environment=Local(path, Limits(...)))` (a path alone is a `Local`) |
| `isolation`, `sandbox`, `container` in a Base Agent's settings | the environment: `Sandboxed(path, sandbox=box)`, `Container(path, settings={...})`; given as settings, refused |
| `make("direct", settings, model)`, `hermit.agent.base.direct` | `Direct(settings, model)`, `hermit.agent.model.direct` (Layer 1); `make("direct")` is refused; the old module path forwards for one release |
| `hermit.agent.base.container` | `hermit.environment.docker` (the old path forwards for one release) |
| `make("direct", settings, model)`, `hermit.agent.base.direct` | `Direct(settings, model)`, `hermit.agent.model.direct` (Layer 1); `make("direct")` is refused |
| `hermit.agent.base.container` | `hermit.environment.docker` |
| `agent.name: harbor` (Hermit's agents injected into `harbor run`) | Codex, Claude Code, `answer`, `react` or a custom agent that takes the `environment` argument, in the task's Harbor environment (`environment.harbor`; Pi, OpenCode, dsh, mini-swe-agent and Hermes refuse it), verified by Harbor's verifier (`judge: {path: hermit.task.verify, name: environment}`) |
| `Mount`, `Limits` from `hermit.agent.base.interface` | `hermit.environment` (still importable from the interface) |
| a benchmark's own copy of the tool loop | a `ReactAgent` subclass with `@tool` methods and hooks |
Expand Down
25 changes: 0 additions & 25 deletions hermit/_moved.py

This file was deleted.

4 changes: 0 additions & 4 deletions hermit/agent/base/container.py

This file was deleted.

4 changes: 0 additions & 4 deletions hermit/agent/base/direct.py

This file was deleted.

5 changes: 0 additions & 5 deletions hermit/agent/environment/__init__.py

This file was deleted.

7 changes: 0 additions & 7 deletions hermit/agents/__init__.py

This file was deleted.

8 changes: 0 additions & 8 deletions hermit/backends.py

This file was deleted.

5 changes: 0 additions & 5 deletions hermit/evolver/__init__.py

This file was deleted.

4 changes: 0 additions & 4 deletions hermit/llm.py

This file was deleted.

4 changes: 0 additions & 4 deletions hermit/trajectory.py

This file was deleted.

19 changes: 7 additions & 12 deletions tests/core/test_task.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
"""Tasks (hermit.task): the public half an agent is handed, the private half only the judge reads, the task set and
its splits, and the run that brings a task and an agent together (hermit.trainer.rollout)."""
import json
import warnings

import pytest

Expand Down Expand Up @@ -68,18 +67,14 @@ def score(row, output):
assert wrapped(as_task({"id": "c", "q": 2}), RunOutput(output=4)).score == 1.0


def test_the_environments_moved_to_the_top_level_and_the_old_path_forwards():
@pytest.mark.parametrize("old", ["hermit.agent.environment", "hermit.agent.base.direct",
"hermit.agent.base.container", "hermit.agents", "hermit.evolver", "hermit.llm",
"hermit.backends", "hermit.trajectory"])
def test_the_moved_modules_old_paths_are_gone(old):
import importlib
import hermit.agent.environment as forwarded
from hermit.environment import Local
with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("always")
importlib.reload(forwarded) # the warning comes with the import, once
from hermit.agent.environment import Local as old
from hermit.agent.environment.base import Sandboxed as old_sandboxed
from hermit.environment.base import Sandboxed
assert old is Local and old_sandboxed is Sandboxed
assert any("hermit.environment" in str(w.message) for w in caught)
with pytest.raises(ModuleNotFoundError) as missing:
importlib.import_module(old)
assert missing.value.name == old # the old module itself, not a module a leftover forward imports


def test_the_names_imported_from_hermit_task_before_the_package_still_import():
Expand Down
4 changes: 2 additions & 2 deletions tests/selection.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,14 +58,14 @@ def fnmatch(path: str, pattern: str) -> bool:
"harness2": {"continual_harness"}} # Continual Harness deploys with the harness2_codex template
# what every method is built on: a change here runs every group (tests/core/test_selection.py checks that every
# framework module these import, or a method's tests import, is here too)
SHARED = ["hermit/agent/__init__.py", "hermit/agent/base/*", "hermit/environment/*", "hermit/agent/environment/*",
SHARED = ["hermit/agent/__init__.py", "hermit/agent/base/*", "hermit/environment/*",
"hermit/agent/acting/*",
"hermit/harbor/*",
"hermit/agent/evolver/*.py",
"hermit/agent/evolver/methods/__init__.py",
"hermit/agent/model/*", "hermit/trainer/*", "hermit/tree.py", "hermit/evaluate.py",
"hermit/sandbox.py", "hermit/sandbox_relay.py", "hermit/isolate.py", "hermit/worker.py", "hermit/spec.py",
"hermit/redact.py", "hermit/_moved.py", # the moved modules' forwards in hermit/agent/base
"hermit/redact.py",
"hermit/hosts/*",
"hermit/runtime/*", "hermit/adapter/*.py", "hermit/config.py", "hermit/defaults.yaml", "hermit/tools.py",
"hermit/task/*", "hermit/common.py", "hermit/__init__.py", "hermit/adapter/templates/prompt_only/*", # toy
Expand Down
Loading