Skip to content

feat(claude-code): add headless in-pod runner, multi-runtime Dockerfile, and metrics exporter - #144

Open
stingram wants to merge 10 commits into
feature/configurable-model-namefrom
feature/claude-code-in-pod-runner
Open

stingram wants to merge 10 commits into
feature/configurable-model-namefrom
feature/claude-code-in-pod-runner

Conversation

@stingram

Copy link
Copy Markdown
Collaborator

Summary

Adds headless in-pod runner support, a multi-runtime container image specification, and a metrics exporter for Claude Code optimization runs on Cloud TPUs.

Key Changes

  1. Headless In-Pod Runner (run_in_pod.sh):
    • Manages non-interactive background execution of local TPU server (server/tpu_server.py) and Claude Code CLI (claude -p).
    • Polls TPU server /health endpoint before launching optimization.
    • Configures headless environment (ANTHROPIC_HEADLESS=1, non-interactive flags).
    • Traps exit signals to ensure background TPU server processes are terminated cleanly.
  2. Multi-Runtime Container Image (Dockerfile):
    • Ubuntu 22.04 base with Node.js 20.x, @anthropic-ai/claude-code, Python 3.12, JAX with Cloud TPU support, and Anthropic Vertex AI SDK.
  3. Run Metrics Exporter (export_run_metrics.py):
    • Parses state.json and claude_stream.json to emit structured summary_metrics.json containing kernel speedup, baseline and best execution latency, candidate iterations, and prompt/candidate token breakdowns.
  4. Unit Tests:
    • Unit tests covering runner dry-run, environment detection, and stream/state metric extraction.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a Dockerfile, a headless execution script (run_in_pod.sh), and a metrics exporter (export_run_metrics.py) with corresponding unit tests for running MaxKernel Claude Code on TPU containers. The feedback highlights a missing ANTHROPIC_HEADLESS=1 environment variable in the execution script, a potential process leak during cleanup where running run_code.py subprocesses are not terminated, and a falsy value bug in the metrics exporter where using or for token fallbacks fails when the token count is 0.

Comment on lines +9 to +11
OUTPUT_DIR="${OUTPUT_DIR:-${MK_ROOT}/workspace/_pod_run}"

DRY_RUN=false

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The PR description mentions that the script configures a headless environment (ANTHROPIC_HEADLESS=1), but this environment variable is never exported or set in run_in_pod.sh. Without setting ANTHROPIC_HEADLESS=1, Claude Code CLI may attempt to run interactively or open a browser for authentication, causing the headless execution to hang or fail.

Suggested change
OUTPUT_DIR="${OUTPUT_DIR:-${MK_ROOT}/workspace/_pod_run}"
DRY_RUN=false
OUTPUT_DIR="${OUTPUT_DIR:-${MK_ROOT}/workspace/_pod_run}"
export ANTHROPIC_HEADLESS=1
DRY_RUN=false

Comment on lines +167 to +175
cleanup() {
local exit_code=$?
if [ -n "$TPU_SERVER_PID" ] && kill -0 "$TPU_SERVER_PID" 2>/dev/null; then
echo "Stopping background TPU server (PID: $TPU_SERVER_PID)..."
kill "$TPU_SERVER_PID" 2>/dev/null || true
wait "$TPU_SERVER_PID" 2>/dev/null || true
fi
exit "$exit_code"
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When run_in_pod.sh is terminated or times out, it stops the background TPU server process. However, the TPU server runs jobs in separate process groups (using preexec_fn=os.setsid in tpu_server.py). If a job is currently running, killing the TPU server process will not terminate the running run_code.py subprocess, leading to leaked orphan processes that keep the TPU locked. Adding a cleanup command to kill any leaked run_code.py processes ensures the TPU is freed up.

Suggested change
cleanup() {
local exit_code=$?
if [ -n "$TPU_SERVER_PID" ] && kill -0 "$TPU_SERVER_PID" 2>/dev/null; then
echo "Stopping background TPU server (PID: $TPU_SERVER_PID)..."
kill "$TPU_SERVER_PID" 2>/dev/null || true
wait "$TPU_SERVER_PID" 2>/dev/null || true
fi
exit "$exit_code"
}
cleanup() {
local exit_code=$?
if [ -n "$TPU_SERVER_PID" ] && kill -0 "$TPU_SERVER_PID" 2>/dev/null; then
echo "Stopping background TPU server (PID: $TPU_SERVER_PID)..."
kill "$TPU_SERVER_PID" 2>/dev/null || true
wait "$TPU_SERVER_PID" 2>/dev/null || true
fi
pkill -9 -f run_code.py || true
exit "$exit_code"
}

Comment on lines +104 to +108
if "cache_read_input_tokens" in u or "cache_read_tokens" in u:
val = u.get("cache_read_input_tokens") or u.get("cache_read_tokens")
if val is not None:
cache_read_tokens += int(val)
found_tokens = True

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using the or operator to fall back from cache_read_input_tokens to cache_read_tokens introduces a bug when cache_read_input_tokens is 0. Since 0 is falsy in Python, the expression will incorrectly evaluate to cache_read_tokens (which might be None or a non-zero value), or if both are 0 or None, it will skip setting found_tokens to True. Use a conditional expression checking for None explicitly to avoid this falsy value bug.

Suggested change
if "cache_read_input_tokens" in u or "cache_read_tokens" in u:
val = u.get("cache_read_input_tokens") or u.get("cache_read_tokens")
if val is not None:
cache_read_tokens += int(val)
found_tokens = True
if "cache_read_input_tokens" in u or "cache_read_tokens" in u:
val = u.get("cache_read_input_tokens") if u.get("cache_read_input_tokens") is not None else u.get("cache_read_tokens")
if val is not None:
cache_read_tokens += int(val)
found_tokens = True

Comment on lines +118 to +121
if "cache_read_input_tokens" in u or "cache_read_tokens" in u:
val = u.get("cache_read_input_tokens") or u.get("cache_read_tokens")
if val is not None:
cache_read_tokens += int(val)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the previous finding, using or to fall back from cache_read_input_tokens to cache_read_tokens will incorrectly use the fallback value when cache_read_input_tokens is 0. Explicitly check for None instead.

          val = u.get("cache_read_input_tokens") if u.get("cache_read_input_tokens") is not None else u.get("cache_read_tokens")
          if val is not None:
            cache_read_tokens += int(val)
            found_tokens = True

if result_usage is not None:
in_tok = result_usage.get("input_tokens")
out_tok = result_usage.get("output_tokens")
cache_tok = result_usage.get("cache_read_input_tokens") or result_usage.get("cache_read_tokens")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using or to fall back from cache_read_input_tokens to cache_read_tokens here also suffers from the falsy 0 bug. If cache_read_input_tokens is 0, it will incorrectly fall back to cache_read_tokens.

Suggested change
cache_tok = result_usage.get("cache_read_input_tokens") or result_usage.get("cache_read_tokens")
cache_tok = result_usage.get("cache_read_input_tokens") if result_usage.get("cache_read_input_tokens") is not None else result_usage.get("cache_read_tokens")

@stingram
stingram force-pushed the feature/configurable-model-name branch from cb1f0f6 to cb10413 Compare November 16, 2026 23:48
@stingram
stingram force-pushed the feature/claude-code-in-pod-runner branch from d083082 to d18da3e Compare November 16, 2026 23:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant