Skip to content

Latest commit

 

History

History
448 lines (426 loc) · 31.5 KB

CHECKLIST.md

File metadata and controls

448 lines (426 loc) · 31.5 KB

Post-Generation Checklist — Harmon DevKit

Work through this after generating the repo from harmon-init. Delete items that don't apply, then keep this file as a record of what was configured.

Run task status:setup at any point to audit setup completeness — local credentials (gh, Codex), GitHub config, toolchain, devcontainer, and dev environment — against the items below (✓ done · ✗ missing · ? unknown · – n/a).

1. Local setup

  • task install — Brewfile deps, and lefthook git hooks
  • task verify passes locally
  • Verify harmon-devkit.code-workspace opens the repo's folder in VS Code and has a unique VS Code Workspace color. Then add any other related repos (e.g. other org repos) to the folders list in the workspace file so you have quick access to those repos
  • Extend .gitignore for your stack — the template ships a base; add stack-specific entries via gitignore.io
  • macOS: add a Raycast quicklink/alias that opens the harmon-devkit.code-workspace
  • macOS (Bunch): scaffold the launcher with task util:bunch-add (if not generated at copier time), then task util:bunch-install to move it to iCloud and leave a .meta/*.bunch symlink (re-run install if missing)

2. GitHub repo settings

  • Confirm draft pull requests are available on this repo — every agent PR opens as a draft and is promoted only by the readiness gate (AGENTS.md, "Dev Loop"). GitHub restricts draft PRs on private repositories to its paid plans, so on a private free-plan repo gh pr create --draft fails outright and the whole lifecycle stops at its first step. Check the current plan matrix in GitHub's draft-PR docs; if drafts are unavailable, make the repo public, upgrade the plan, or decide deliberately not to run agent workflows here — do not "fix" it by dropping --draft and letting automations surprise human reviewers.

  • Automated settings — run task setup:github (idempotent, safe to re-run): enables Dependabot alerts and private vulnerability reporting when public. Routing uses CI_RUNS_ON, BROWSER_RUNS_ON, IMAGE_BUILD_RUNS_ON, MULTICORE_RUNS_ON and ISOLATED_RUNS_ON. Public repositories always create or replace all five repository overrides as "ubuntu-latest", even without flags, so inherited persistent runners cannot reach jobs. On private/internal repositories, the script creates missing values only with --ci-runs-on, --browser-runs-on, --multicore-runs-on, --image-build-runs-on or --isolated-runs-on; intentional replacement also requires the matching --replace-* flag. task setup:github supplies the rendered CI answer; pass other class flags after --, for example task setup:github -- --browser-runs-on '"ubuntu-latest"'. Repository values shadow organization values. Isolated selectors may be strings, label arrays, or objects with usable labels; self-hosted is refused case-insensitively, and unsupported shapes fail closed. Requested invalid isolated values fail before any GitHub access or writes. Repair an existing unsafe repository value with a safe --isolated-runs-on plus --replace-isolated-runs-on (public repositories repair unsafe and unsupported values automatically before organization checks). Private/internal unsupported shapes fail closed. Unsafe or unsupported organization values fail only when inherited, with no repository isolated override. A safe repository value or an explicit safe isolated value plus replace flag shadows them and succeeds with a warning. Otherwise ask the organization's routing owner (whoever manages organization Actions variables) to repair them, or create a safe override. Unreadable organization values report unknown without failing setup; task status:setup reports known violations. Do not add dependabot.yml: Renovate owns routine and vulnerability-remediation PRs; Dependabot owns advisory alerts.

  • Import the branch ruleset (see architecture/branch-protection.md) — do this once build.yml, devcontainer-build.yml are on main so the required verify/security/devcontainer-verify checks resolve. Use the UI import: Settings → Rules → Rulesets → New ruleset ▸ Import a ruleset → select .github/Branch Protection Ruleset - Protect Main.json. (Prefer the UI over gh api … rulesets: the API POST is not idempotent — re-running creates a duplicate ruleset — and currently rejects the merge_queue rule. To later change the ruleset, edit the existing one in the UI rather than re-importing.) Import it before provisioning the bot and agent PATs below: those postures carry no local merge guard, so this ruleset's code-owner approval and required checks are the only thing between their PAT and an unreviewed merge into main.

  • Bot PAT — the agent's GH_TOKEN. If a fine-grained PAT already covers evanharmon1, just add this repo to its selected repositories; a token is scoped to one resource owner, so a new owner needs a new PAT. Both layers are required — the collaborator grant above sets the ceiling, the PAT's repo list reaches it. Procedure: guides/bot-account.md.

  • [human-only] Agent PAT — only if this repo allows unattended remote lanes (the agent devcontainer, .devcontainer/agent/). A separate fine-grained PAT on the bot account, same permissions as the bot's, expiry of 180 days at most, its own repository list, one per resource owner; supplied as AGENT_GH_TOKEN, never as GH_TOKEN. Procedure: guides/devcontainers.md (agent posture).

  • [human-only] Add closing-keywords to the live branch ruleset — required, not optional, and no longer deferrable. Until harmon-init#1328 the guard was enforced transitively: the job sat in build.yml and fed the aggregate verify check, so a failing guard failed a check the ruleset already required. Splitting it into closing-keywords.yml (so it can keep the pull_request.edited trigger the build matrix must not have) removes that path — needs: cannot cross workflows. A live ruleset that does not name closing-keywords therefore enforces nothing, and a copier update cannot fix that for you, because it does not mutate live rulesets. After the job has reported once, edit the existing main-branch ruleset in Settings → Rules → Rulesets and add that exact required status check. Do not re-import the JSON solely for this change: GitHub creates a duplicate ruleset rather than updating the live one. task audit:ruleset names a context the live ruleset is missing — run it to confirm. (evanharmon1/harmon-init#1333 will automate this as task setup:ruleset; until it lands the edit is by hand.)

  • [human-only] Enable unattributed-changes approval in the live branch ruleset — after a copier update adds this parameter, edit the existing main-branch ruleset in Settings → Rules → Rulesets and enable the matching additional-approval setting. Do not re-import the JSON: GitHub creates a duplicate ruleset rather than updating the live one.

  • Install and activate Renovate — install the Renovate app for Only select repositories and select this repo. In the Mend Developer Portal choose the Renovate product and Scan and Alert mode. Do not choose Scan Only: it puts Renovate in silent mode, which scans without creating checks, issues (including the Dependency Dashboard), or update/remediation PRs. This repo already has renovate.json; keep that configuration rather than replacing it with a generic onboarding config.

  • Install the CodeRabbit app on the repo (.coderabbit.yaml is pre-configured, including reviews.auto_review.drafts: true so it reviews the draft workbench rather than firing on promotion)

  • Install the Greptile app on the repo (.greptile/config.json turns automatic reviews off; the dev loop requests them)

  • [human-only] Connect Gemini Code Assist on GitHub — in Google Cloud Console (under your foundation project), navigate to Gemini Code Assist → Agents & Tools → Code Assist Source Code Management, create/select the connection in us-east1, install the Gemini Code Assist GitHub App for this repository, and complete the Cloud-side repository link.

  • [human-only] Connect Codex cloud review — connect this repository in ChatGPT Codex settings, grant private-repository access if applicable, and confirm review activity is authored by GitHub actor ID 199175422 (chatgpt-codex-connector[bot], type Bot).

  • [human-only] Disable Codex Automatic reviews — turn personal Auto review off and set this repository's Auto code review preference to Follow personal — and its review Trigger to Follow personal too, since an "On every push" trigger sits dormant while Auto review is off and arms across every Follow-personal repo at once the moment the personal toggle changes. The draft-workbench lifecycle drives Codex with explicit @codex review requests while the PR is draft; left on, gh pr ready starts a new asynchronous review after the readiness gate, and non-draft stops truthfully meaning "ready for a human". No API exposes this setting, so it is a human-configured prerequisite the gate trusts on the strength of this record — never report it as mechanically verified, and re-check it here if Codex's settings change. Codex Auto-review attestation (confirmed 2026-08-13): personal Auto review off; this repository's Auto code review preference Follow personal; its review Trigger Follow personal. Agents rely on this entry silently and re-raise only on drift evidence (an unsolicited Codex review nobody triggered); re-confirm and re-date it if that happens.

  • [human-only] Any other automatic reviewer must review drafts — if you enable one (GitHub Copilot code review, for example), turn on its draft-review option. A reviewer that skips drafts first reports after promotion, so the readiness gate would hand a human a PR it had not actually reviewed. Leave it off rather than run it blind to the workbench.

  • Actions secret: CLAUDE_CODE_OAUTH_TOKEN (claude-* workflows) — generate with claude setup-token; the value must start sk-ant-oat01- (an OAuth token, billed to your Claude subscription), not sk-ant-api03- (a raw API key, billed at pay-as-you-go API rates). Then gh secret set CLAUDE_CODE_OAUTH_TOKEN

  • Foreman operator setup — provision the separate READ-ONLY PAT that foreman hands to dispatched agents: export/store it as FOREMAN_AGENT_GH_TOKEN where the bot devcontainer's init-env.sh can inject it (1Password → devcontainer.env). Run task setup:github-labels so the foreman:* arming labels exist. Import the two tag rulesets (.github/Tag Protection Ruleset - Version Tag Creation.json / … Immutability.json, same UI import as the branch ruleset), then add the CI GitHub App to the Creation ruleset's bypass list (always) — release-please tags via that App, and bypass-actor App IDs are per-installation so the JSON cannot ship them. (Immutability keeps an empty bypass list on purpose: a moved v* tag is code execution in every consumer, so nobody bypasses it.) Create the standing probe tag on an orphan commit, so it is reachable from no branch: git tag v0.0.0-probe "$(git commit-tree "$(git hash-object -t tree /dev/null)" -m 'foreman tag-immutability probe target (orphan; keep unreachable from any branch)')" && git push origin v0.0.0-probe. Do not tag HEAD or any commit on main: git describe considers only tags reachable from HEAD and prefers the nearest, so a probe tag on a release commit outranks the release tag, and everything deriving a version from git describe — release tooling, image tags, package builds — then reports 0.0.0-probe instead of the release. The probe only needs the remote tag's sha to differ from main's, so an orphan target satisfies it permanently. Preflight empirically asserts v* tags are immutable and fails until both rulesets and the tag exist. Then task foreman:preflight (inside the bot devcontainer — foreman refuses to start anywhere else) to assert the security controls before any dispatch.

  • [human-only] Foreman reviewer-gate check — .foreman.toml's [reviewer] table is foreman's current-head review gate for the PRs it shepherds. Before the first dispatch (and again after any Foreman bump), confirm the configured login still matches the live Codex connector identity (actor ID 199175422), that its terminal results — an APPROVED review at the head, or a 👍 from that login on foreman's own request comment — still mean what the readiness gate assumes, and that required checks run on draft PRs (foreman promotes only after they conclude).

  • SAST coverage — this profile has no CodeQL workflow, so Semgrep CE runs in build.yml for public and private repositories. Add CodeQL later if the repo gains supported first-party source: turn off GitHub's CodeQL Default setup (Settings → Advanced Security → CodeQL analysis, or gh api --method PATCH repos/OWNER/REPO/code-scanning/default-setup -f state=not-configured), re-apply the template with use_codeql=true and the selected codeql_languages (then, if Default setup had disabled an existing CodeQL workflow, re-enable it via Actions → CodeQL → "Enable workflow" or gh workflow enable codeql.yml), and ensure the repository is public (free) or has paid GitHub Code Security (private/internal).

  • Choose the Snyk posture — the default is manual/local only via task security:sast:snyk and task security:sca:snyk; it is not part of task security or required PR CI. Free private-repository tests share the Snyk Organization's monthly quota, including local CLI tests. Leave the Snyk GitHub App off unless deliberately adopting its PR integration; its checks are not required by the default branch ruleset.

  • Optional scheduled Snyk — leave this off for ordinary and free private repos. For a selected important public repo, re-render with snyk_scan_schedule=weekly (conservative) or daily (public or accepted unlimited OSS), set the generated workflow's SNYK_TOKEN Actions secret, and verify one manual run. Confirm Snyk classifies the public Git remote correctly. The workflow is advisory and never a required PR check.

  • Create the CI GitHub App evanharmon1-ci by hand (one App per org; Settings → Developer settings → GitHub Apps), or reuse the org's existing one.

  • [human-only] Confirm CI GitHub App permissions — the App must grant Checks: Read-only, Commit statuses: Read-only, and Actions: Read-only for claude-implement's gh pr checks. For an existing App, update its Permissions & events and accept the new permissions on every installation; until then, create-github-app-token fails and claude-implement cannot start. See architecture/security.md.

  • Install the App on this repo — Install App → Only select repositories (the harmon-init repos that run release-please / claude-* / project-automation), not "All". Creating the App is not enough: an App whose credentials are set but which is not installed on the repo makes actions/create-github-app-token fail at runtime with a 404 (Not Found — "not installed on this repository"). This is the single easiest step to miss.

  • Set CI_APP_CLIENT_ID (Actions variable) + CI_APP_PRIVATE_KEY (Actions secret) — pipe the .pem in (never paste it; flattened newlines break the key), and scope both to those same repos (least privilege — the key can act as the App: commits, PRs, releases, workflow edits):

    ```bash
    gh secret set CI_APP_PRIVATE_KEY --org evanharmon1 \
      --visibility selected --repos <repo-a>,<repo-b> < evanharmon1-ci.private-key.pem
    gh variable set CI_APP_CLIENT_ID --org evanharmon1 \
      --visibility selected --repos <repo-a>,<repo-b> --body "<client-id>"  # Iv…-style, not the numeric App ID
    ```
    
    Personal account: use `--repo evanharmon1/harmon-devkit` instead of
    `--org`/`--visibility`/`--repos`. Re-running `--repos` **replaces** the list —
    re-run with the full list to add a repo. Drives release-please, the claude-*
    workflows, and project-automation; blast-radius + rotation in
    docs/architecture/security.md.
    
  • GHCR: ensure the org/user allows publishing packages; the first devcontainer prebuild populates ghcr.io/evanharmon1/harmon-devkit-devcontainer on merge to main

  • GitHub Project: run task setup:github-project (needs gh auth refresh -s project) to create the owner's default project (titled evanharmon1 Project) and idempotently sync its Status pipeline — see project-management.md. On a personal account it also creates Product as a project field (issue fields are org-only; there is deliberately no Priority, Size, Domain, or Layer field — see project-management.md, "Fields" and "Label or field?"); status automation is a separate follow-up — the board is set up, but issue/PR status isn't auto-synced yet. Re-runs append any starter option the Status field is missing (so a value added by a later harmon-init release lands on the next run) and never touch, reorder, or delete the options you added.

  • Upgrading from a release before harmon-init#875? If this repo's board still carries Domain/Layer fields from an earlier release, they are not deleted automatically. Retiring them is a deliberate, irreversible operator step — see project-management.md, "Migrating a board that still has one."

  • [human-only] Delete the Priority and Size project fields on an existing board — only if this repo's board was set up by a release before harmon-init#1451; they are not deleted automatically, and deleting a field destroys every value on it, unrecoverably. Do it after the classification labels exist (on a personal account, task setup:github-labels provisions the priority:* labels that replace Priority, plus priority-ai:* for the AI's separate suggestion beside it) and after enumerating every board and view that references each field (harmon-init#910), as project-management.md, "Migrating a board that still has one" (the Priority / Size entry) lays out. On a personal account both fields go; on an organization only Size was ever a project field — its Priority is GitHub's built-in issue field and stays.

  • Upgrading from a release before harmon-init#1047 (method:* → strategy:*)? Run task setup:github-labels first so the strategy:* destinations exist, then use the read-only report and guarded maintenance flow below. For each live value among oneshot, plan, plan-approved, orchestrate, council, and human-led, pass one --migrate method:<v>=strategy:<v>; the guarded flow attempts to move associations for matching issues, PRs, and discussions found in its paginated snapshots, re-reads them, and only then allows --prune to remove zero-association retired labels.

  • Labels: run task setup:github-labels to seed this repo's starter label families from label-registry.json (see the generated taxonomy table in project-management.md) — grow domain: values there as the product's own problem-space vocabulary and area: values as its solution-space subsystems; both starter lists are a floor. layer: is product-independent and normally needs no edits. Labels are per-repo, so run it in each repo; org default labels (org Settings → Repository, UI-only) only seed new repos.

  • [human-only] Pin any hand-applied Tier before the first classification write — on an existing repository an unqualified tier:<value> label used to be a human's implementer override; it is now the cache automation rewrites. On an unpinned issue with Risk and Complexity set, a writer of those axes (such as the triage skill, once its pin carries harmon-devkit#1250) adds the derived Tier and removes every other tier label. The classification reconciler does the same on its schedule (daily on a personal-account repository, monthly on an organization one). Before that first write, add tier:pinned to each open issue whose hand-set tier label you want to keep.

  • After a copier update that adds label families (e.g. impact:* — a pure addition), re-run task setup:github-labels to provision the new labels here — it is additive and never deletes, so existing labels and the issues they sit on are untouched — then classify open issues with the added families. A RENAMED family (like method:* → strategy:* above) is not a pure addition — provision the new destinations first, then migrate the old associations before retiring their labels.

  • [human-only] Inspect live label drift before retiring anything — keep the default setup path additive, then run ./scripts/setup-github-labels.sh --repo <owner/repo> --report-unregistered with the same --foreman / --release-please profile flags used for setup when you want to mirror provisioning. Maintenance protection still includes every non-retired registered family, including gated tool labels, when those flags are omitted. This is read-only: it pages through the live labels, all-state issues and pull requests, and repository discussions, and reports separate association counts. An indeterminate read is a stop, not permission to clean up.

  • [human-only] Retire any legacy agent:* claim labels or pre-2026 codex/copilot labels — use the guarded maintenance flow, never a direct gh label delete or a hand-written association list. Perform its write path in a quiescent maintenance window: pause claim/release, Foreman, release-please, and other human/API label writers for the whole run. --report-unregistered is read-only, but its counts are a snapshot; run it again immediately before --prune.

    `suggest:*` (superseded by the derived Tier) and `tier:adaptive` (no rung
    on the Tier scale) have no `--migrate` destination, and the command
    refuses one: remove the label from every issue, pull request and
    discussion that carries it (an issue then resolves through its derived
    Tier), then run `--prune`.
    
    For fixed-family sources, these mappings are authoritative:
    `agent:claude-code` → `claim:claude`, `agent:codex` → `claim:gpt`,
    `agent:gemini-cli` → `claim:gemini`, `agent:kimi-k2` → `claim:kimi`, `agent:qwen-code` →
    `claim:qwen`, and `claim:codex` → `claim:gpt`. Pass one repeatable
    `--migrate OLD=NEW` per exact live
    source, together with `--prune`; for example,
    `./scripts/setup-github-labels.sh --repo <owner/repo> --prune
    --migrate agent:gemini-cli=claim:gemini`. `--migrate` does not match a
    prefix, so each live model-level source needs its own mapping. The
    `OLD=NEW` form contains exactly one `=`; a label name containing `=` must
    be relabeled per record instead of passed to bulk migration. Move only
    the family segment for fixed mappings and preserve the recorded model
    suffix, e.g. `claim:codex:sol` → `claim:gpt:sol`; model-level labels
    refine rather
    than replace their family-level label, and the command retains or adds
    both associations. If a recognized model-level
    destination is absent, the command creates it after confirmation by
    copying metadata from the live family label; missing family labels stop
    maintenance and require setup first.
    
    Use this association-migration path, not `gh label edit` or a hand-written
    create-then-delete sequence: the command validates a live registry
    destination or creates a recognized on-demand model destination as
    described above, attempts the association move for each matching issue,
    PR, and discussion found in its paginated snapshots, then permits
    `--prune` only when a fresh snapshot
    shows the source has zero associations. Enumerate
    model-level names explicitly with `gh label list --repo <owner/repo>
    --limit 1000 --json name --jq '.[].name' | grep -E
    '^claim:(codex|copilot):'`; for each source, inspect all-state
    `gh issue list --label <old> --state all --limit 1000` **and**
    `gh pr list --label <old> --state all --limit 1000`. An exactly-full
    manual result is capped; increase the limit and rerun before writes. The
    maintenance path itself uses `gh api --paginate` and refuses an
    indeterminate read.
    
    Copilot is a broker, not a fixed family: `mai` is only the picker default
    and is never a guessed destination. Do **not** pass
    `agent:github-copilot*` or `claim:copilot*` to bulk
    `--migrate` — the command rejects broker-derived sources because one
    destination cannot represent mixed runtime records. For `claim:copilot`,
    inspect each issue/PR's claim/session record and relabel that record to
    `claim:<actual-family>`; use `claim:mai` only when the record confirms
    MAI. Apply the same per-record distinction to
    `claim:copilot:<model>` and preserve a model
    suffix only after the actual family is known. Include Discussions in that
    per-record inventory: the read-only report gives their association count,
    and the Discussions UI or GraphQL API identifies the records to relabel.
    If a live claim's record is
    missing, settle it with its owner or leave the label untouched rather
    than guess. Add and verify the per-record destination before removing the
    old association; after every record is handled, a fresh zero-association
    snapshot may permit guarded `--prune` to attempt retiring the source.
    
    Before moving any in-flight `claim:*`/legacy `agent:*` marker, settle the
    claim or amend its durable record in the same sitting: its release path
    names the exact label it will remove, and moving only the issue/PR
    association strands the replacement marker. Interactive runs confirm on
    the TTY; automation must state destructive intent again with separate
    `--yes` (piped stdin is refused).
    
    The command verifies each migration around source removal, then takes one
    complete, bounded post-migration association snapshot before the deletion
    batch and fails closed on read/verification errors, but GitHub has no transaction or
    compare-and-swap that binds the final read to the following edit/DELETE.
    A concurrent writer can still change labels after that read and before
    the request, and the command cannot undo a successful concurrent
    mutation. If writers were not paused or any verification drifts, treat
    the operation as incomplete, reconcile live associations, and rerun in a
    new quiet window; do not infer association preservation from a successful
    exit alone; this is a guarded best-effort operation at that API boundary.
    
  • Project views: create the starter views that the Views section of project-management.md specifies — including Needs review and Human queue — in the Project UI. Projects V2 has no view API, so this is a one-time manual step; the filters and layouts are in that section.

  • GitHub Project auto-add (adds every issue to the board): in the Project's Settings → Workflows, turn on "Auto-add to project" and point it at this repo (filter is:issue, is:pr) so every new issue and PR lands on the board automatically, however it's created. GitHub's native built-in — no Actions or tokens, and it's the reliable way to guarantee coverage (the issue-form projects: key only covers form-created issues and needs a static project number). See project-management.md.

3. Framework scaffolding (conventions-only template)

  • Add the project's primary toolchain; extend Taskfile build/test accordingly

4. Secrets & environment

  • For local .env needs, use 1Password Environments (mounts a virtual .env; secrets never hit disk or git) or op run/op inject. Commit only .env.example-style files
  • Devcontainer secrets: create a 1Password environment that mounts .devcontainer/devcontainer.env (and .devcontainer/dev/devcontainer.env) with CLAUDE_CODE_OAUTH_TOKEN and AGENT_DECK_TELEGRAM_KEY, plus GH_TOKEN for the bot profile and TS_AUTHKEY for the dev one — the dev profile carries no GH_TOKEN and runs gh auth login instead. init-env.sh enforces the per-profile allow-list; on Coder the values come from workspace parameters. The agent posture's .devcontainer/agent/devcontainer.env takes only AGENT_GH_TOKEN and CLAUDE_CODE_OAUTH_TOKEN (plus opted-in provider keys) and refuses to build with anything else. See guides/devcontainers.md

5. Docs & meta

  • Fill in the TODO: markers in README.md and docs/ (architecture diagram first)
  • Confirm README badges render (Actions URLs are correct once CI runs)
  • Initial release when ready: task release:init (v0.1.0) — releases stay manual
  • Stay current with harmon-init: periodically use the standardize-repo skill's update mode to pull template improvements (a three-way merge — your own edits are preserved). Follow its guarded source/ref/answer flow end to end; do not substitute a bare copier update --trust.