Skip to content

feat(custom-blocks): log and bill child runs in the publisher's workspace - #6023

Open
TheodoreSpeaks wants to merge 14 commits into
stagingfrom
feat/custom-block-log
Open

feat(custom-blocks): log and bill child runs in the publisher's workspace#6023
TheodoreSpeaks wants to merge 14 commits into
stagingfrom
feat/custom-block-log

Conversation

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator

Summary

  • A custom block's child run is now its own execution: own executionId, own logging session against the source (publisher's) workspace, own ledger rows. The publisher sees a full run with the trace waterfall in their own Logs; the consumer's view stays as opaque as before.
  • Billing follows the invocation boundary. The consumer no longer absorbs the child's model spend — they pay only their own base execution fee, and the child adds no second fee.
  • Fixes a privacy leak: SSE callbacks were forwarded into the source run, streaming the publisher's block names, inputs, outputs, workflow name, and raw agent tokens to the consumer's browser. Invisible in the UI because the terminal drops them for custom_block_*, so it went unnoticed.
  • Fixes failed custom blocks billing $0 — buildCostCarrierSpans wrote to blockLog.childTraceSpans, but its only reader is gated on isWorkflowBlockType, which excludes custom blocks. Dissolves now that the child bills itself; both roll-ups deleted.
  • Fixes the depth-limit throw leaking the source workflow's UUID into the consumer's output (childWorkflowName still holds the workflow id before it's resolved).
  • Errors: adds BoundarySafeError so consumer visibility is a property decided at the throw site (fail-closed) instead of depending on whether a throw sits before or after the try. Consumers now get a machine-readable errorType plus an opaque errorRef (the child's execution id) they can quote in support.
  • Deletes the regex round-trip (parseNestedWorkflowError / buildNestedWorkflowErrorMessage) in favour of a structured workflowChain. That also fixes Workflow chain: Child → Child | err, which happened whenever a child returned success: false rather than throwing.
  • Fixes 429/503 surfacing to API callers as 500 — readStatusCode now walks .cause and reads HttpError/status/statusCode, with a narrow 503-only 5xx whitelist so an upstream 502/504 never becomes ours. buildBlockExecutionError now preserves cause. Deletes dead buildHTTPError.
  • Unifies four copies of the "did this run fail?" span walk into hasUnhandledError / traceSpansIndicateFailure.

Behaviour changes worth flagging

  • Runs where a mothership block recovered from a failed child span were previously logged failed by logger.ts / logging-session.ts, which recursed past the error boundary that trace-spans.ts honours. They now log as succeeded. Existing rows are not backfilled, so expect a step-change in dashboard failure rates.
  • Rate-limited / hosted-key-exhausted runs now return 429/503 instead of 500. Anything retrying on status === 500 should be checked.
  • A custom block inside an N-iteration loop writes N log rows into the publisher's workspace.

Type of Change

  • New feature (also fixes several bugs found along the way)

Testing

Tested manually end-to-end across two workspaces in the same org: child run lands in the publisher's workspace with the full trace, consumer's row shows only its own 0.005 base fee and no model cost, and the child resolves credentials against the source workspace (verified via BYOK — a publisher key correctly zeroes the charge, and removing it bills the source workspace).

Full suite green: type-check 0 errors, lint:check, and all 11 CI audits (check:boundaries, check:api-validation:strict, check:utils, check:zustand-v5, check:react-query, check:client-boundary, check:bare-icons, check:icon-paths, check:realtime-prune, skills:check, agent-stream-docs:check). 15107/15108 tests pass — the one failure is a pre-existing local-env issue (cloud-review-tools shells out to a rg binary that isn't installed).

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Jul 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 29, 2026 11:25pm

Request Review

@cursor

cursor Bot commented Jul 28, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Touches billing attribution, cross-workspace execution/logging, and privacy redaction at the custom-block boundary; behavior changes include log success/failure classification and 429/503 API responses instead of 500.

Overview
Custom block invocations now run the publisher workflow as a separate execution: fresh executionId, custom_block logging session in the source workspace (zero base fee on the child), usage admitted against the source payer, correlation back to the consumer run, parent cancellation bridged to the child, and bounded waitForChildRuns at the end of the invoking run so abandoned children are less likely to stick in running.

Billing and visibility shift across the boundary: model spend is no longer rolled onto the consumer block or failed tool results; the consumer sees only curated outputs plus success / error / errorType / errorRef. Publishing and updates require at least one exposed output—the whole-child result fallback is removed so raw agent/workflow internals cannot leak. Consumer failures use BoundarySafeError and redacted ChildWorkflowError (consumerFacing); regular workflow blocks still get child names and trace spans. SSE callbacks are not forwarded into custom-block children (fixes a publisher-data stream leak).

Agent-as-tool paths adopt the invoker’s executionId and abort signal. HTTP mapping for workflow API errors now walks .cause, forwards Sim-owned 4xx/503 via HttpError/statusCode on tool responses, and avoids treating upstream status as the API status. Failure detection is centralized in hasUnhandledError / traceSpansIndicateFailure (including mothership recovery semantics). Nested workflow errors use structured workflowChain instead of message parsing.

Reviewed by Cursor Bugbot for commit b345750. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread apps/sim/executor/handlers/workflow/workflow-handler.ts
Comment thread apps/sim/executor/handlers/workflow/workflow-handler.ts
Comment thread apps/sim/executor/handlers/workflow/workflow-handler.ts
Comment thread apps/sim/lib/workflows/custom-blocks/child-execution.ts
@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Follow-up review of custom-block boundary error classification and child-run billing/logging.

  • Prior unavailable-path fix routes getCustomBlockAuthority null through buildBoundaryFailure with errorType: 'unavailable'.
  • Prior admission-denial fix makes CustomBlockAdmissionError a BoundarySafeError with errorType: 'usage_limit' so consumers get a distinct signal.
  • No new incomplete or unsafe residual tied to those threads remains at HEAD.

Confidence Score: 5/5

This PR appears safe to merge with respect to the previously reported boundary-classification issues; those failures are resolved at HEAD.

The unavailable and usage-limit consumer signals are wired through buildBoundaryFailure and BoundarySafeError as intended, and no remaining blocking failure from those threads is present.

Important Files Changed

Filename Overview
apps/sim/executor/handlers/workflow/workflow-handler.ts Unavailable failures go through buildBoundaryFailure with consumerFacing errorType; admission and other boundary paths share the same classification path.
apps/sim/lib/workflows/custom-blocks/child-execution.ts CustomBlockAdmissionError extends BoundarySafeError as usage_limit; payer vs actor messaging is sanitized before throw.
apps/sim/executor/execution/block-executor.ts Propagates ChildWorkflowError.consumerFacing into errorType and errorRef on block output.

Reviews (10): Last reviewed commit: "fix(agent): forward the execution id thr..." | Re-trigger Greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/executor/handlers/workflow/workflow-handler.ts
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/executor/handlers/workflow/workflow-handler.ts
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/lib/workflows/custom-blocks/child-execution.ts
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/tools/index.ts
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

Comment thread apps/sim/lib/workflows/custom-blocks/child-execution.ts Outdated
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/executor/handlers/workflow/custom-block-tool-runner.ts
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/tools/index.ts Outdated
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/executor/handlers/agent/agent-handler.ts
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cursor review

// source workflow would bill if run directly, exactly as if it ran the key.
const childCost = aggregateChildCost(childTraceSpans)
return this.projectCustomBlockOutput(executionResult, exposedOutputs, childCost)
return this.projectCustomBlockOutput(executionResult, exposedOutputs)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paused child treated as success

Medium Severity

Cancelled custom-block children are classified with a BoundarySafeError after the child session is finalized, but paused children (status: 'paused', success: true) are not. They go through safeComplete, then projectCustomBlockOutput, so a source HITL pause becomes a consumer success with incomplete outputs and a completed publisher log. CustomBlockErrorType has no paused member either.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 67805cb. Configure here.

Comment thread apps/sim/tools/index.ts

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 264bc74. Configure here.

Comment thread apps/sim/executor/handlers/workflow/workflow-handler.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant