Add the Cloud plugin and AI Gateway

[ymichael]

Summary

• expand the existing getbb.app connection into one user-facing Cloud plugin while preserving the compatibility id connect and the bb connect command
• keep the existing Remote access experience and make the sidebar phone a shortcut to that section of the Cloud plugin settings
• add an experiment-gated AI Gateway section with one preference: “Enable for thread titles, commit messages and voice transcription”
• add authenticated Cloudflare endpoints for structured inference and voice transcription, including input limits and per-account daily budget enforcement
• route thread-title generation, commit-message generation, and voice transcription through Cloud first, then fall back to locally configured providers

Responsibility boundaries

server cloudAi experiment policy
  -> server inference / transcription
  -> experimental plugin provider slot
  -> builtin Cloud plugin (compatibility id: connect)
  -> @bb/connect-client
  -> Connect worker
  -> OpenAI

Connect worker -> D1 ai_usage metering

• Server: owns experiment policy, provider ordering, timeout behavior, final validation, and local fallback
• Connect worker: owns authentication, cloud model/cost policy, input caps, upstream calls, quota reservation/refunds, and sanitized errors
• Connect client: owns request encoding, credential headers, response validation, and stable transport errors
• Cloud plugin: owns pairing-aware availability, the AI Gateway preference, credential/quota backoff, UI, CLI, and RPC integration
• App host: owns plugin identity, lifecycle switch, experiment-gated settings sections, and deep-linked sidebar navigation

Contract changes

• add cloudAi to the server-backed experiment key contract; it defaults to false
• add POST /api/connect/ai/inference and POST /api/connect/ai/transcription
• add typed inference/transcription calls and errors to @bb/connect-client
• add experimental Cloud AI provider registration to the plugin SDK
• allow sidebar actions to open a specific plugin settings section
• add cloudAiEnabled to Connect status, setCloudAi RPC, and bb connect ai [on|off]
• preserve the internal plugin id, RPC paths, storage namespace, and bb connect compatibility
• no host-daemon wire/protocol change

Data model

• persist cloudAi through the existing key/value system_experiments table; no local SQLite schema migration is needed
• add D1 ai_usage, keyed by user and UTC day, storing estimated aggregate spend in micro-USD
• reserve usage atomically before the upstream call and refund failed upstream calls best-effort
• store no prompts, schemas, responses, diffs, or audio
• retain the plugin KV preference cloudAiEnabled, defaulting to true when absent, as the inner routing switch

Lifecycle and fallback

• effective Cloud routing requires the outer experiment, the inner plugin preference, a live pairing, no quota cooldown, and an accepted credential
• disabling the experiment hides AI Gateway and prevents server routing without affecting Remote access
• Cloud failures and provider bugs fall back to local providers; timeouts preserve existing host timeout semantics
• rejected credentials remain latched until re-pairing supplies a different credential; quota exhaustion pauses attempts for five minutes
• disabling the Cloud plugin cuts off Remote access and AI Gateway together

Verification

• affected-package tests and typechecks run through Turbo after rebasing onto main
• app: 2,472 tests; server: 1,402; host daemon: 512; CLI: 383; database: 374; Connect plugin: 89; templates: 23
• Connect worker/client/database, domain, plugin SDK, desktop, web, shared UI, and plugin registry suites also pass