docs(deploy): two-phase deploy_component, revert_component, stage/activate CLI (draft) - #599
docs(deploy): two-phase deploy_component, revert_component, stage/activate CLI (draft)#599dawsontoth wants to merge 7 commits into
Conversation
…ivate CLI Document the two-phase deploy behavior and the new surface from HarperFast/harper#1849: - deploy_component: cluster-wide stage -> barrier -> activate; new properties activate:false (stage-and-stop, returns a staged deployment_id), deployment_id (activate a previously-staged deployment), revert_on_failure, two_phase, and the ignore_replication_errors / deployment_timeout knobs. - revert_component: fast cluster-wide rollback to the retained previous version. - Deployment lifecycle: stage/activate phase names, staged/staging/activating/ rolled_back statuses, and staged-build retention (deployment_stagingRetention_maxCount). - CLI: harper stage / activate / revert verbs (aliases + examples). Companion to HarperFast/harper#1849 (still a draft). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the documentation for the CLI and Operations API to reflect the new two-phase deployment process (staging and activation) and the rollback mechanism (revert_component). It introduces new CLI commands, aliases, API parameters, and examples. The review feedback suggests improving consistency by pointing the new CLI operation category links to the main #components section, and enhancing readability in the API documentation by adding clear labels to the multi-step "Stage now, activate later" JSON examples.
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-599 This preview will update automatically when you push new commits. |
…tage/activate example Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-599 This preview will update automatically when you push new commits. |
Conflict in reference/operations-api/operations.md was additive: main added the `credentials` parameter plus its "Deploy credentials" section (#581) where this branch adds the two-phase parameters. Kept both — the parameter list now carries activate/deployment_id/revert_on_failure/ignore_replication_errors/ deployment_timeout/two_phase AND credentials, followed by main's credentials section. main's get_deployment_payload / delete_deployment_payload docs (#600) are untouched.
… values Documents harper-pro#594 and syncs with harper#1849's latest behavior. add_ssh_key `generate: true` (harper-pro#594): - Harper mints the ed25519 keypair on the node and returns only `public_key`, so the private key is never carried in a request body, shell history, or CI log. Includes the response shape and the `harper:<name>` comment. - `key` and `generate` are mutually exclusive; ssh-keygen must be on PATH. - Notes that `public_key` is returned ONLY on the generating call — Harper does not retain it, and update_ssh_key requires a key you supply (it cannot mint one), so the recovery path is delete_ssh_key + add_ssh_key generate again. - CLI example added alongside the deploy/stage/activate/revert examples. Payload retention (harper#1849): - Splits the old one-line aside into its own subsection with a table covering both bounds: payloadRetention_maxSize (10 MiB, bounds a single payload) and the new payloadRetention_maxCount (default 1, bounds how many are kept per project). Explains why the default is conservative, that rows are never deleted, and that a reclaimed payload reports payload_blob_present: false. Also: - revert_on_failure / ignore_replication_errors now note they apply to a `deployment_id` activate too, matching the peer-failure gate added to that path. - list_deployments `status` listed only 5 of 13 real values; replaced with terminal vs in-flight groupings. - New note on restartRequired: deploying a brand-new component without a restart marks one as required (per node), while redeploying a live component does not.
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-599 This preview will update automatically when you push new commits. |
…cument them The SSH rows linked to #certificate-management, but the SSH operations are documented under ## Components — so every one of those seven links landed the reader in the wrong section. Repointed them to #components (the convention the other 15 component rows already use). Auditing the same block turned up the identical defect on the six user/role rows (list_users, add_user, alter_user, drop_user, list_roles, drop_role), which are documented under ## Users & Roles. Repointed those to #users--roles, matching the existing link to that section elsewhere in the docs. The five genuine certificate rows (create_csr, sign_certificate, list_certificates, add_certificate, remove_certificate) were already correct and are unchanged. Category labels updated to match their new targets. Verified every operations.md anchor referenced from this table against the real headings — all 11 resolve.
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-599 This preview will update automatically when you push new commits. |
Builds on the two-phase deploy docs (#599) to cover the rest of the deploy-by-reference workflow. reference/components/applications.md - "Deploying by Reference" — `harper deploy by_ref=true`, `ref=`, `credential=`. Explains why a reference pins to a resolved SHA rather than the tag or branch name typed: peers resolve the package independently, so a tag that moves mid-deploy could otherwise leave nodes running different code. Notes that the cluster clones the pushed remote, and that by-reference means the cluster builds from source — so an app whose build can't run on the node should stay on payload deploys. - "Provisioning a Deploy Credential" — `harper deploy setup=true`, the client-side sealing flow, and that reverting to the previous version needs no credential at all. reference/cli/authentication.md - Documents HARPER_CLI_REFRESH_TOKEN / HARPER_CLI_OPERATION_TOKEN and adds "Token credentials for CI/CD", so the CI guidance is no longer "put an admin password in your pipeline". Covers precedence, in-memory-only refresh, and token lifetimes (1d / 30d defaults). - Warns that a user holds only ONE valid refresh token at a time: issuing a new one invalidates the last, so a routine local `harper login` silently breaks a pipeline using the same account. Hence the recommendation to give CI its own user. (Verified against security/tokenAuthentication.ts, which stores a single hashed refresh_token per user record.) reference/cli/commands.md - `harper login --for-ci`, including the stdout/stderr split that makes `| gh secret set --env-file -` work without displaying the token. reference/security/secrets.md - Points the existing private-source deploy credentials section at the CLI flow that automates it. Documents HarperFast/harper#1850, #1851, and #1876. Verified every anchor link in the changed files resolves (16 checked). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Opened #616 stacked on this branch: docs(deploy): by-reference deploys, sealed credentials, CI token auth. It targets Where the two meet: this PR documents the two-phase deploy and |
…vert caveat Addresses cb1kenobi's review on #599. - "Terminal" wrongly included `staged`. The implementation's TERMINAL_STATUSES is {success, failed, rolled_back} and that set gates get_deployment_payload and payload-pruning eligibility, so listing `staged` as terminal contradicted the get_deployment_payload section. Split into Terminal / Resting / In flight, and noted why a resting payload is deliberately still held. - get_deployment's `status` row listed 7 of 13 values; now lists all, pointing at the grouped explanation. - The `restart: "rolling"` example's response showed the no-restart message. Corrected to the rolling shape (restartJobId + ", restarting Harper") and noted what the no-restart response looks like instead. - revert_component: added a caution that reverting swaps live directories and does NOT rewrite the stored `package:` reference, so a node provisioned after a revert installs the reverted-away version. Verified in code: revertComponent never calls writeComponentRootConfig, and installApplications() installs from root config. - deployment_id activate: documents that the staged package identifier and credential references are recovered and persisted at activation (fixed in harper#1849), so `package` need not be repeated on the activate call.
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-599 This preview will update automatically when you push new commits. |
kriszyp
left a comment
kriszyp
left a comment
There was a problem hiding this comment.
This is looking good, but some comments worth addressing.
🤖 Reviewed with GPT 5.6
|
|
||
| Deploys a component. The `package` option accepts any valid NPM reference including GitHub repos (`HarperDB/app#semver:v1.0.0`), tarballs, or NPM packages. The `payload` option accepts a base64-encoded tar string from `package_component`. Supports `"replicated": true` and `"restart": true` or `"restart": "rolling"`. | ||
|
|
||
| Across a cluster, `deploy_component` runs in two phases so a deploy is all-or-nothing at go-live: |
There was a problem hiding this comment.
deploy_component is not actually all-or-nothing at go-live. The stage barrier prevents fetch/install failures from touching live directories, but activation still occurs per node; if one swap fails, other nodes can already be live, and revert_on_failure is off by default. Please describe this as an all-nodes staging barrier (or reduced-risk activation), and explicitly say that an activation failure can split versions unless automatic rollback is enabled.
|
|
||
| - `urlPath` — override the HTTP URL path the component is mounted at (e.g. `"/api/v2"`) | ||
| - `install_allow_scripts` — set to `true` to allow npm pre/post install scripts (disabled by default) | ||
| - `activate` — set to `false` to **stage only** and stop before go-live. The build is prepared and verified on every node and the response returns a `deployment_id` in a `staged` state; nothing goes live. Activate it later by calling `deploy_component` again with that `deployment_id` (see below). Useful for pre-staging a release and flipping it live in a separate, fast step. |
There was a problem hiding this comment.
Please define and enforce the deployment modes before presenting these as independent knobs. The companion core validation currently permits deployment_id with activate: false or with a fresh package/payload; dispatch prioritizes deployment_id, so the request activates despite activate: false and ignores fresh build input. Likewise, two_phase: false plus activate: false—or any staged request when system is excluded from replication—takes the legacy one-shot path and goes live. Please reject conflicting/unsupported combinations in core, then document the valid matrix; a request that says stage-only should never activate.
| harper deploy ref=9f8c2a1 restart=true replicated=true | ||
| ``` | ||
|
|
||
| **A reference is pinned to a SHA, not to the name you typed.** Tags and branches are resolved locally and the full commit SHA is what ships. This matters on a cluster: peers resolve the package independently, so a tag that moves mid-deploy — or a branch that advances — could otherwise leave nodes running different code. |
There was a problem hiding this comment.
The companion CLI does not currently provide this guarantee for an explicit ref: resolveGitCommittish returns the supplied string unchanged, so ref=v1.2.0 produces a package ending in #v1.2.0 rather than a full SHA. Each peer can therefore resolve a mutable tag or branch independently, recreating the mixed-version risk this paragraph says is eliminated. Please resolve explicit refs locally to a commit (for example, git rev-parse <ref>^{commit}) before building the package, or require a full SHA and qualify the docs accordingly.
|
|
||
| ### `revert_component` | ||
|
|
||
| Swaps a component's live version back to its **retained previous version** across the cluster, then restarts. Every `deploy_component` activation retains the version it replaced (one previous version is kept per component), so `revert_component` is a fast rollback that does not re-fetch or re-install. The swap is bidirectional — reverting a revert rolls forward again. |
There was a problem hiding this comment.
This is narrower than documented. The retained .deploy-previous directory is created by two-phase activation; the legacy one-shot path used by two_phase: false (and by the no-system-replication fallback) does not retain it, so a later revert_component fails even after repeated deploys. Restart is also optional in the implementation. Please scope rollback to versions activated through the two-phase path and say it restarts only with restart: true/"rolling", or change core so these guarantees hold on every path.
| Adds an SSH key (must be ed25519) for authenticating deployments from private repositories. | ||
| Adds an SSH key (must be ed25519) for authenticating deployments from private repositories. Supply the private key with `key`, or omit it and pass `generate: true` to have Harper mint the keypair itself. | ||
|
|
||
| The stored private key is encrypted at rest and only ever crosses the cluster as ciphertext; `list_ssh_keys` and the logs never return key material. |
There was a problem hiding this comment.
This absolute security guarantee does not match the Pro implementation. sealSSHKey deliberately falls back to storing and replicating the private key in plaintext when no secret custody is registered; it logs a warning but does not fail. Please either make add_ssh_key (including generate: true) reject that configuration, or qualify this text to say encryption/ciphertext requires configured custody and explain the fallback, so operators do not assume a guarantee they may not have.
| HARPER_CLI_REFRESH_TOKEN: ${{ secrets.HARPER_CLI_REFRESH_TOKEN }} | ||
| ``` | ||
|
|
||
| **Precedence**: an explicitly supplied username (arguments or `HARPER_CLI_USERNAME`/`HARPER_CLI_PASSWORD`) wins; otherwise env-var tokens are used; otherwise the token saved by `harper login`. Env-var tokens deliberately outrank the saved credentials file so a runner that has both behaves predictably. A token refreshed from an env var is held in memory for that invocation only — nothing is written to `~/.harperdb/credentials.json`. |
There was a problem hiding this comment.
Please name the argument form precisely here. In the companion CLI, only auth_username=/auth_password=, target-URL userinfo, or a complete username/password environment pair beat env/saved bearer tokens. Ordinary username=/password= arguments are a legacy fallback after those tokens, so a reader following Method 3 can explicitly name one account while the saved account still executes the operation. Document the dedicated auth_* override and clarify where the ordinary payload fields sit in precedence.
|
|
||
| ### Deploying by Reference | ||
|
|
||
| Available since: v5.2.0 |
There was a problem hiding this comment.
Current reference docs use <VersionBadge version="v5.2.0" /> directly below headings for newly added minor-version surface. Please replace the plain Available since lines under both new sections with badges, and add these user-facing workflows—by-reference deploys, sealed deploy credentials, and CI token authentication/--for-ci—to release-notes/v5-lincoln/5.2.md so users can discover them from the release.
| To roll back durably for a `package` deploy, deploy the older version explicitly (`deploy_component` with the previous `package` reference) instead of, or after, reverting. `revert_component` is the fast live-instance swap; an explicit deploy is what changes what a future node will install. | ||
| ::: | ||
|
|
||
| ### Deployment Operations |
There was a problem hiding this comment.
Please finish the v5.2 discoverability/versioning work for this new surface: add a changed <VersionBadge> under deploy_component, added badges under revert_component and the generate subsection, add revert_component to the Components operation summary table, and cover these user-facing features in release-notes/v5-lincoln/5.2.md. The CLI table records v5.2 availability, but the API reference currently reads as though these behaviors exist throughout v5.
Ethan-Arrowood
left a comment
Ethan-Arrowood
left a comment
There was a problem hiding this comment.
Holding rather than stacking another CHANGES_REQUESTED on top of @kriszyp's — his eight findings already cover the prose, all are still open, and I reproduced 1, 2, 4 and 5 in local harper/harper-pro checkouts with no disagreement. My additions are structural, and they're the actual reason for the hold.
1. Merging this publishes four unmerged upstream features. There is no "next" tree for reference docs — docusaurus.config.ts sets the reference plugin to lastVersion: 'current', includeCurrentVersion: false, versions: { current: { label: 'v5', path: 'v5' } }. reference/ is the published /reference/v5 site, so merging is publishing. Upstream state as of this pass:
| Documented surface | Implementation | State |
|---|---|---|
two-phase, revert_component, staging retention |
harper#1849 | open |
deploy by_ref=true / ref= / credential= |
harper#1850 | open |
deploy setup=true |
harper#1851 | open |
add_ssh_key generate: true |
harper-pro#594 | open |
login --for-ci, HARPER_CLI_REFRESH_TOKEN |
harper main |
shipped (5.2.0) |
One of five has landed. @cb1kenobi raised harper-pro#594 and the answer was that sequencing is handled outside the PR — that doesn't scale to four dependencies, and the merge gate in the body still names only #1849.
2. The conflict with main can silently drop content. git merge-tree origin/main 94152378 conflicts in reference/cli/authentication.md and reference/cli/operations-api-commands.md. (operations.md and applications.md auto-merge cleanly — I checked the merged blob and main's new host badge survives.) Two hazards:
- The ops table is rewritten wholesale here (column re-padding plus the Users/Roles and SSH recategorisation).
mainhas since added seven rows this branch has never seen:create_backup,list_backups,verify_backup,restore_backup,delete_backup,purge_backups,get_backup. Resolving by taking "ours" drops all seven — this needs a hand merge and re-pad, not picking a side. (The recategorisation itself is right; I confirmed#users--rolesand#componentsboth exist and the old#certificate-managementtargets for user ops were wrong.) - Authentication precedence is about to be stated twice.
mainlanded a canonical numbered "Authentication Precedence" section inauthentication.mdafter this branch forked. This PR adds a second, prose precedence statement in the same file, so post-merge the file gives the order twice, differently — andHARPER_CLI_REFRESH_TOKENnever gets slotted into the canonical list at all. Folding the token rules into the numbered section instead of adding a competing paragraph also resolves @kriszyp'sauth_*vs. ordinaryusername=point in one place. Same file re-adds the(or CLI_TARGET_*)bullet style thatmainrewrote away.
3. Smaller, all new:
- The PR body is stale. It describes two files; #616 merged into this branch on 07-29, so the diff is six. Someone reading the description alone won't know they're approving CI-credential and private-source-deploy docs.
deployment_stagingRetention_maxCountanddeployment_payloadRetention_*are documented only inline inoperations.md.grep -rn "payloadRetention\|stagingRetention" reference/onmainreturns nothing, so the body's "matching howdeployment_payloadRetention_maxSizeis currently surfaced" doesn't hold — it isn't surfaced anywhere. Three operator-facing config keys with no entry inreference/configuration/options.md, which is where operators look.- Title still says "(draft)" but
isDraft: false. Same on #616. If the marker is load-bearing use GitHub's draft state; otherwise drop it.
Two questions before I'd move off the hold:
- Sequencing. Merge only once all four upstream PRs ship in the same release, or split so the
--for-ci/token half (already on harpermainat 5.2.0) can land now? - The
system-replication precondition. Independent of whether core tightens the flag matrix per @kriszyp, does the prose get this caveat now?if (req.two_phase === false || isReplicatedExecution || !isSystemDatabaseReplicated())takes the one-shot path, so a cluster withsystemexcluded from replication silently gets one-shot deploys and no revert target while reading docs that promise both as unconditional cluster behavior.
Verified correct and not worth re-litigating: the status buckets against TERMINAL_STATUSES, deployment_timeout default 120000 vs. DEFAULT_AWAIT_ROW_TIMEOUT_MS, retention defaults (5 / 10 MiB / 1), the rolling-restart response carrying restartJobId and the , restarting Harper suffix, and every new internal anchor. @cb1kenobi's five findings are genuinely fixed — two at the source rather than papered over with a caveat, which was the right call.
sent with Claude Opus 5
4 participants
What this documents
The
deploy_componentoperation now runs as a two-phase, cluster-wide deploy, and there are new operator-facing knobs plus a rollback operation. This PR updates the reference docs to match.reference/operations-api/operations.mddeploy_component— documents the internal stage → cluster-wide barrier → activate flow (all-or-nothing at go-live), and the new properties:activate: false— stage-and-stop; returns astageddeployment_id, nothing goes live.deployment_id— activate a previously-staged deployment (no re-fetch/install).revert_on_failure,two_phase,ignore_replication_errors,deployment_timeout.revert_component(new section) — fast cluster-wide rollback to the retained previous version; bidirectional; customer-driven-rollback use case.stage/activate, legacyprepare/replicate), the new statuses (staging/staged/activating/rolled_back), and staged-build retention (deployment_stagingRetention_maxCount, default 5) with the "aged-out deployment_id" caveat.reference/cli/operations-api-commands.mdharper stage,harper activate, andharper revertto the alias table and the aliases list, with examples in Component Operations.Notes for review
v5.2.0as a placeholder — confirm against the actual shipping release before merge.reference_versioned_docs/version-v4/copies (this is a v5 feature).deployment_stagingRetention_maxCount; it's described inline where deployments are documented, matching howdeployment_payloadRetention_maxSizeis currently surfaced. If you'd prefer a dedicated configuration entry, happy to add one.🤖 Generated with Claude Code