Skip to content

Add Batch API support (Anthropic/OpenAI) for ~50% cheaper async translation #1

Description

@jaylann

Motivation

Provider Batch APIs (Anthropic Message Batches, OpenAI Batch) process requests asynchronously at roughly 50% of the standard per-token cost, which is ideal for large localization jobs where a 24h turnaround is acceptable. This was requested alongside the OpenRouter work but deferred.

Why it's non-trivial

pydantic-ai (now on v1.x in this repo) has no batch-API abstraction. The current translation path uses pydantic_ai.Agent(...).run_sync() with structured output (output_type=TranslationResult). Batch mode cannot reuse that and would need a separate code path:

  • Build raw request payloads per provider (native anthropic / openai SDKs), replicating the structured-output contract (tool/response_format) that pydantic-ai currently handles.
  • Submit the batch, then poll for completion (minutes to ~24h) and retrieve/parse results.
  • Persist job state (batch IDs ↔ file/lang/keys) so a run can be resumed after the process exits, since results aren't available synchronously.
  • Map results back into the xcstrings entries and reconcile partial failures.

Proposed shape

  • Opt-in --batch flag on translate.
  • Implement Anthropic first, then OpenAI. OpenRouter/Gemini batch are out of scope initially.
  • Sub-commands or flags to submit vs. poll/collect a previously submitted batch (job-state file in the working dir).

References

  • Anthropic Message Batches API
  • OpenAI Batch API

Deferred from the deps-update / nested-files / OpenRouter change.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions