Recover transient terminal SSE failures before downstream commitment

[jroth1111]

Problem

Codex/OpenAI Responses can return a generic terminal server_error inside an otherwise successful HTTP 200 SSE stream. CLIProxyAPI currently translates early lifecycle events such as response.created immediately, so the downstream stream is considered committed before the request reaches terminal success.

When a later terminal failure is normalized to 502:

1. the partial failure reaches Claude Code or another streaming client;
2. the selected credential/model receives the legacy transient cooldown;
3. with a single credential and max-retry-interval below that cooldown, immediate client retries become 503 auth_unavailable;
4. client retry loops can repeatedly replay large request bodies.

Existing bootstrap recovery only covers failures before the first deliverable payload. It cannot safely replay once downstream bytes have been emitted because regenerated text, reasoning, and tool calls may differ.

Desired behavior

Provide an opt-in, bounded recovery mode that:

• withholds the translated stream until a valid terminal success;
• silently retries eligible transient failures on the same selected route before applying cooldown or fallback;
• discards every losing attempt completely;
• releases only the winning attempt's chunks and headers;
• never retries after downstream commitment;
• remains bounded by attempts, retry-start window, buffered bytes, and concurrent recovery requests;
• honors cancellation without cooling a healthy credential;
• preserves request-scoped incomplete-stream behavior and existing authentication refresh semantics.

Configuration

Proposed configuration:

streaming:
  recovery:
    attempts: 2
    max-buffer-bytes: 8388608
    max-retry-window-seconds: 20
    max-concurrent: 16
    initial-backoff-milliseconds: 250
    max-backoff-milliseconds: 2000

Recovery remains disabled by default. If the concurrency limit is saturated or the response exceeds the byte limit, the request fails open to ordinary streaming rather than blocking or using unbounded memory.

Scope

This is in-request, pre-downstream-commit recovery. It complements but does not replace:

• Add provider/model transient-error circuit breaker to prevent retry amplification #4634, which tracks cross-request provider/model circuit breaking;
• fix(streaming): retry Responses after SSE prefix failure #4580, which buffers only syntactic Responses SSE prefixes;
• fix(auth): retry transient upstream overload (503/529) #4584, which covers selected HTTP 503/529 overload retries;
• fix(codex): retry transient capacity errors #3229, which covers capacity-specific Codex errors;
• fix(codex): preserve upstream error headers #4635, which preserves headers for HTTP non-2xx Codex failures.

A complete, tested implementation is ready and the closing PR will be opened immediately after this issue.