YakiFlow

简体中文 | English

YakiFlow turns a local video, audio file, web video, or live stream into an ASS subtitle file. It transcribes with whisper.cpp or the ElevenLabs Scribe API, translates with Codex or Claude Code, and lets you review the result with an interactive Agent before it is saved. With a diarizing backend, each subtitle carries its speaker, and two speakers talking at once become two overlapping subtitles.

Jobs are resumable: if a run is interrupted, YakiFlow keeps its completed work and prints the command needed to continue.

Quick start

1. Install the required tools

You need:

Python 3.12 or newer;
ffmpeg;
either the codex or claude CLI, installed and signed in;
for the default local transcription, a recent whisper.cpp build whose whisper-cli executable is on PATH.

Install yt-dlp as well if you want to process URLs. tmux and mpv are optional conveniences for subtitle review and video preview. The ElevenLabs backends need no local transcription tools at all — see Transcription backends.

2. Install YakiFlow

From this repository:

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install -e .

Add .[elevenlabs] to use the ElevenLabs backends and .[keyring] to store the API key in the OS keyring:

python -m pip install -e '.[elevenlabs,keyring]'

3. Save your usual settings

Create yakiflow.toml in the directory where you will run YakiFlow:

source-language = "auto"
target-language = "zh-CN"
output-dir = "./subtitles"
output-mode = "bilingual"

[agent]
backend = "codex"

Use backend = "claude" if you signed in with Claude Code. You can also omit this file and pass the equivalent options on every run.

4. Download the default model and check the setup

yakiflow models fetch
yakiflow doctor -c agent.backend=codex

Replace codex with claude when appropriate. Fix every reported FAIL that applies to your input; yt-dlp is only needed for URLs, and the whisper.cpp checks only apply to the Whisper backends. models fetch is unnecessary for the ElevenLabs backends.

5. Create subtitles

yakiflow run video.mp4

Without yakiflow.toml, provide the three required settings explicitly:

yakiflow run video.mp4 \
  --source-language auto \
  --target-language zh-CN \
  -c agent.backend=codex \
  --output-dir ./subtitles

YakiFlow shows transcription and translation progress. In an interactive terminal, it then opens the selected Agent for final review. Tell the Agent any changes you want, then end the review session when you are satisfied. The final subtitle path is printed when the job finishes.

Command-line options

A few high-frequency settings have dedicated options:

Option	Setting
`-s`, `--source-language`	Source language code, or `auto`
`-t`, `--target-language`	Translation language or locale
`--transcription-backend`	One of the four transcription backends
`--stream` / `--no-stream`	Treat a URL as a live stream
`--output-dir`	Directory for final subtitle files
`--output-mode`	`source`, `translated`, `bilingual`, or `all`
`--context-file`	Copy a reference file into the work directory; repeatable
`--work-dir`	Place resumable job state in a chosen directory
`--keep-workdir`	Keep intermediate files after success
`--config-file`	Use this project configuration file
`--profile`	Select a `[profiles.NAME]` section

Every other setting is changed with the repeatable -c KEY=VALUE option, where KEY is the setting's dotted TOML key:

yakiflow run video.mp4 \
  -c transcription.backend=elevenlabs \
  -c agent.draft.effort=medium \
  -c 'commands.yt-dlp-options=["--cookies-from-browser", "chrome"]'

Values are parsed as TOML (booleans, numbers, arrays); anything that does not parse as TOML is taken as a plain string. Dedicated options override -c, and -c overrides every configuration file.

Transcription backends

Choose with transcription.backend (or --transcription-backend):

Backend	What it is	Needs
`whisper-cli`	Local whisper.cpp batch transcription (default)	`whisper-cli`, a model file
`whisper-server`	A whisper.cpp server — local, or an already-running one via `whisper.server-url`	`whisper-server`, or a reachable server
`elevenlabs`	ElevenLabs Scribe batch API, with speaker diarization	`yakiflow[elevenlabs]`, an API key
`elevenlabs-stream`	ElevenLabs Scribe realtime API over WebSocket	`yakiflow[elevenlabs]`, an API key

All four backends work for both normal runs and --stream live capture. For live streams, whisper-server and elevenlabs-stream give the fastest live preview; whisper-cli also works but reloads its model for every chunk, which makes the preview lag well behind the stream.

The elevenlabs backend recognizes speakers (elevenlabs.diarize, on by default): each subtitle's speaker label lands in the ASS Name field, and overlapping speech becomes overlapping subtitles. The realtime API does not diarize. With the ElevenLabs backends, subtitle line breaks and translations are produced together by the draft Agent from the word-level transcript.

To use an already-running whisper-server instead of starting one:

[transcription]
backend = "whisper-server"

[whisper]
server-url = "http://127.0.0.1:8080"

The ElevenLabs API key

The key is looked up in this order:

an elevenlabs.api-key* setting given with -c on the command line;
the ELEVENLABS_API_KEY environment variable;
an elevenlabs.api-key* setting from a configuration file;
the OS keyring (requires yakiflow[keyring]).

In a configuration file (or with -c), pick exactly one of three spellings:

[elevenlabs]
api-key = "sk-…"                  # the key itself
# api-key-file = "~/.secrets/elevenlabs"   # read the key from a file
# api-key-command = "pass show elevenlabs" # run a command; stdout is the key

To store the key in the OS keyring instead of any file:

yakiflow secret set elevenlabs
yakiflow secret unset elevenlabs

yakiflow doctor reports where the key would come from without printing it.

Common tasks

Process a local file

Any audio or video format supported by ffmpeg can be used:

yakiflow run ./recordings/interview.mkv \
  --source-language en \
  --target-language zh-CN \
  --output-dir ./subtitles

Process a web video

yakiflow run 'https://example.com/watch?v=123' \
  --source-language ja \
  --target-language en \
  -c download-dir=./downloads \
  --output-dir ./subtitles

download-dir keeps the downloaded media; omit it to use a temporary copy. Set --output-dir for URL jobs so the subtitle location is predictable. Playlists are not downloaded.

Process a live stream

yakiflow run 'https://example.com/live' \
  --stream \
  --source-language auto \
  --target-language zh-CN \
  --output-dir ./subtitles

Live subtitles are provisional. After capture ends, YakiFlow transcribes the complete recording again before publishing the final subtitles. In the TUI, press s to stop capture and finalize what has been received; press Ctrl-C once in a non-TUI run. While the stream is still running, press o to open the recording so far with the current live subtitles in the media player.

The player loads the subtitle file only once. To make mpv pick up live subtitle updates automatically, copy contrib/mpv/sub-autoreload.lua from this repository into mpv's scripts directory (usually ~/.config/mpv/scripts/), or add --script=/path/to/contrib/mpv/sub-autoreload.lua to review.video-open-command. Setting that option replaces the default command entirely, so start from it:

[review]
video-open-command = "mpv --script=/path/to/contrib/mpv/sub-autoreload.lua --no-sub-auto --no-resume-playback --sub-file={subtitle} --sid=auto --sub-visibility=yes {file}"

Choose the subtitle format

Use --output-mode or set output-mode in TOML:

Value	Result for an input named `video.mp4`
`bilingual`	`video.en-zh-cn.ass`, with translation above source text (default)
`source`	`video.source.ass`
`translated`	`video.translated.ass`
`all`	All three variants

With automatic language detection, the bilingual filename uses the detected language, for example video.ja-zh-cn.ass. Players such as mpv load ASS files directly (mpv --sub-file=video.en-zh-cn.ass video.mp4).

Give each speaker its own style in Aegisub

With a diarizing backend, every subtitle carries its speaker in the ASS Name (actor) field, while all lines share the Default style. To give speakers different colors or positions in Aegisub:

Open the subtitle file in Aegisub and create one style per speaker in Subtitle → Styles Manager.
For each speaker, select any one of their lines and pick the new style in the edit box's style dropdown.
Load contrib/aegisub/yakiflow-actor-styles.lua from this repository via Automation → Automation... → Add, then run Automation → Fill actor styles.

Every remaining Default line takes the style you assigned to that speaker's line. Lines you already restyled are left untouched, and one Ctrl-Z undoes the whole run.

Resume an interrupted job

In the TUI, press s to stop the current job cleanly and Ctrl-Q to leave the interface. In a non-TUI run, the first Ctrl-C stops acquisition cleanly and a second Ctrl-C forces exit. After an interruption or failure, YakiFlow prints the preserved work directory. Resume it with:

yakiflow resume /path/to/workdir

Keep the entire work directory intact. resume uses the original input and settings; single settings can still be adjusted with -c when needed. A work directory created by an older YakiFlow with a different configuration schema must be finished by the version that created it. To choose the work directory location in advance, start the job with --work-dir ./work/video-job.

Configuration guide

Settings are read in this order, from highest to lowest priority:

dedicated command-line options;
-c KEY=VALUE overrides;
the selected profile in the file passed to --config-file, or in yakiflow.toml in the current directory;
the selected profile in the user configuration file;
base settings in the project configuration file;
base settings in the user configuration file;
built-in defaults.

On Linux, the user configuration file is ~/.config/yakiflow/config.toml (or $XDG_CONFIG_HOME/yakiflow/config.toml). Relative paths are resolved from the directory in which you start the job.

Settings are grouped into TOML tables. A full example with the most useful keys:

# Top-level settings
source-language = "auto"
target-language = "zh-CN"
output-mode = "bilingual"        # source | translated | bilingual | all
output-dir = "./subtitles"
# download-dir = "./downloads"   # keep media downloaded from URLs
# memory = "./memory.md"         # persistent terminology/style memory file
# context-files = ["notes.txt"]  # read-only references for interactive review
# work-dir = "./work"            # resumable job state location
# keep-workdir = false

[transcription]
backend = "whisper-cli"   # whisper-cli | whisper-server | elevenlabs | elevenlabs-stream

[whisper]                 # used by whisper-cli and whisper-server
# model = "/path/to/ggml-large-v3-turbo-q5_0.bin"
# vad-model = "/path/to/ggml-silero-v6.2.0.bin"
# vad = true              # set false to ignore the configured VAD model
# cli = "whisper-cli"     # executable name or path
# server = "whisper-server"
# server-url = "http://127.0.0.1:8080"  # connect instead of launching

[elevenlabs]              # used by elevenlabs and elevenlabs-stream
# api-key-command = "pass show elevenlabs"
# model = "scribe_v2"
# realtime-model = "scribe_v2_realtime"
# diarize = true          # speaker recognition (batch API only)
# num-speakers = 3        # optional speaker-count hint
# use-speaker-library = true  # match speakers against the workspace library

[alignment]
# backend = "vad"         # vad | whisperx | none
# device = "auto"         # auto | cpu | cuda (WhisperX)
# model = "custom/model"  # override the WhisperX alignment model

[stream]
# enabled = false         # same as --stream
# chunk-seconds = 15      # live transcription chunk length (unused by elevenlabs-stream)
# context-seconds = 5     # audio overlap kept before each chunk (unused by elevenlabs-stream)

[subtitles]
# max-cue-seconds = 8.0   # longest single subtitle (ElevenLabs backends)
# max-cue-chars = 84      # longest subtitle text per language (CJK counts double)

[agent]                   # shared by both translation stages
backend = "codex"         # codex | claude
# model = "…"             # rarely needed
# effort = "…"            # minimal | low | medium | high | xhigh
# extra-options = ["…"]   # extra argv tokens for the agent CLI

[agent.draft]             # batch draft translation; overrides [agent]
# backend = "…"
# model = "…"             # default: codex → gpt-5.6-terra, claude → sonnet
# effort = "low"
# extra-options = ["…"]
# workers = 4             # parallel draft requests
# batch-size = 20         # cues per request (Whisper backends)
# preceding-context = 10  # earlier cues shown for context
# following-context = 5   # later cues shown for context
# word-batch-size = 400   # words per request (ElevenLabs backends)
# word-following-context = 40
# timeout-seconds = 600
# max-attempts = 3
# retry-delay-seconds = 1

[agent.final]             # interactive review; overrides [agent]
# backend = "…"
# model = "…"             # default: codex → gpt-5.6-sol, claude → opus
# effort = "high"
# extra-options = ["…"]

[review]
# display-mode = "split"  # split | open | both
# open-command = "code --reuse-window {workdir} {subtitle} {memory}"
# auto-open-video = false
# video-open-command = "vlc --sub-file={subtitle} {file}"

[commands]
# ffmpeg = "ffmpeg"
# yt-dlp = "yt-dlp"
# yt-dlp-options = ["--cookies-from-browser", "chrome"]

The default Codex models are gpt-5.6-terra for draft translation and gpt-5.6-sol for review. The Claude defaults are sonnet and opus. Override them if those names are unavailable to your account. Settings in [agent.draft] and [agent.final] win over the shared [agent] values, so the two stages can use different backends.

extra-options and yt-dlp-options must be TOML arrays containing only strings. YakiFlow passes each string directly as one argv token; options with a value use two array items.

Configuration profiles

Put reusable variants under [profiles.NAME] and select one with --profile on run, resume, doctor, or models fetch:

source-language = "auto"
target-language = "zh-CN"

[agent]
backend = "codex"

[profiles.stream]
stream.enabled = true
transcription.backend = "whisper-server"
agent.draft.batch-size = 5
agent.draft.extra-options = ["-c", "service_tier=fast"]

yakiflow run 'https://example.com/live' --profile stream
yakiflow doctor --profile stream

Profiles may use dotted keys or nested tables and merge setting by setting: a profile that changes agent.draft.batch-size keeps every other [agent.draft] value from the base configuration. Arrays replace earlier arrays rather than being appended. If both user and project files define the selected name, the project profile wins; a profile is applied after both base files, and -c plus dedicated options still win over profiles. Selecting an unknown profile is an error, profiles cannot inherit from other profiles, and unselected profiles have no effect. YakiFlow saves the fully resolved settings with each job, so a resume does not depend on the profile or configuration files still existing.

Reference files for review

Pass a danmaku export and a speaker note to the interactive review Agent as read-only references:

yakiflow run video.mp4 --context-file danmaku.xml --context-file speakers.txt

YakiFlow snapshots each file under context/ in the job's work directory, so a resumed job does not depend on the original reference files.

Review and translation memory

When run from a terminal, YakiFlow starts an interactive review after batch translation. The Agent checks the complete subtitle file, applies agreed edits, and can propose reusable names, terminology, or style preferences. Memory is only updated after you approve a proposal.

The review Agent writes to you in the target language and switches if you write in another one. When the target language cannot keep one sentence split the way the source was, it may merge those cues, mirroring the merge across every configured artifact. Overlapping subtitles from different speakers are normal with a diarizing backend and are kept as they are.

By default YakiFlow uses tmux to show a read-only subtitle pane next to the Agent when possible. In that pane, press o to open the media with the current subtitles in mpv. To use another player:

[review]
video-open-command = "vlc --sub-file={subtitle} {file}"

To open the source video automatically as the interactive review starts, enable review.auto-open-video. The configured video-open-command is used, and the staged subtitles are attached when the player supports them.

To open the staged subtitle file in an editor instead of using the split preview:

[review]
display-mode = "open"
open-command = "code --reuse-window {workdir} {subtitle} {memory}"

Use display-mode = "both" to open the editor and retain the split preview at the same time. open-command supports the {subtitle}, {workdir}, and {memory} placeholders. Leave placeholders unquoted; YakiFlow safely keeps substituted paths, including paths containing spaces, as single arguments. The template uses shell-style argument parsing, but the resulting command is launched directly without a shell.

In CI or with redirected input/output, interactive review is skipped and the batch result is finalized directly.

Optional: improve subtitle timing

These options apply to the Whisper backends. The ElevenLabs backends deliver word-accurate timing already and default to alignment.backend = "none".

VAD

A Silero VAD model helps Whisper skip silence and improves cue starts. Download a model supported by your whisper.cpp build, then configure its path:

[whisper]
vad-model = "/absolute/path/to/ggml-silero-v6.2.0.bin"

yakiflow models fetch does not download VAD models. The whisper.cpp VAD guide has the model download commands.

Pass -c whisper.vad=false to run without VAD while keeping vad-model in the config file. VAD is off whenever no model is configured.

WhisperX forced alignment

WhisperX is optional and only adjusts timing; transcription still uses whisper.cpp. Python 3.12 or 3.13 is required for the current extra. Install exactly one variant with uv:

# CPU
uv sync --extra whisperx-cpu

# NVIDIA CUDA 12.8
uv sync --extra whisperx-cuda

Then enable and verify it:

[alignment]
backend = "whisperx"
device = "auto" # auto, cpu, or cuda

yakiflow doctor -c alignment.backend=whisperx

The first run may download NLTK data and a language-specific alignment model. If alignment setup fails interactively, YakiFlow lets you retry or fall back to VAD; a non-interactive run preserves the job for later resumption.

Troubleshooting

Start with yakiflow doctor using the same options as your run.
If the default Whisper model is missing, run yakiflow models fetch.
If a custom model path is configured, YakiFlow expects that file to already exist and will not replace it.
yt-dlp failures only affect URL inputs; whisper.cpp failures only affect the Whisper backends.
If an ElevenLabs run stops with a key error, yakiflow doctor shows which source the key would come from.
If review has no subtitle pane, install tmux or configure review.display-mode = "open".
Keep the printed work directory after a failure. It contains the job journal and process logs needed to diagnose or resume the run.

Development

uv sync --extra dev
uv run --extra dev pytest

FilesExpand file tree

README.md

Latest commit

History