繁體中文

從 3 小時到 30 分鐘：我的中文字幕自動化 pipeline

我幫一位財經教育創作者做影片字幕。一支兩小時的影片，人工校正要 2-3 小時；現在同樣的工作，pipeline 自動跑完加上我抽查，15-30 分鐘。這篇講這條生產線怎麼搭起來，以及三個讓它從「能跑」變成「可靠」的關鍵教訓。

生產線長什麼樣

整條 pipeline 在 Claude Code 裡用一個 skill 觸發，依序跑六個階段：

下載（yt-dlp）
  → 畫面術語抽取（OCR / VLM caption，每 60 秒一幀）
  → ASR（Breeze ASR 25，Apple Silicon 本地推理）
  → 預處理（斷句清理、詞中逗號修復）
  → LLM 校正（分段派 subagent，注入術語表＋畫面上下文）
  → 後處理（超長字幕切分、品質 gate、SRT 輸出）

全程本地跑 ASR，只有校正階段用雲端模型。財經內容的難點不是語音辨識本身，是專有名詞：股票代號、ETF、指標縮寫、講者的口頭禪。ASR 把「美光」聽成別的詞、把「MA30」拆成碎片，這類錯誤靠通用模型修不好，要靠領域知識。

術語表是活的

系統的核心不是任何單一模型，是一個會學習的術語迴圈：

每支影片校正完，跑一次 term learning，把「ASR 錯法 → 正確詞」寫成機械替換規則
下一支影片的 ASR 輸出，先過這批規則再進 LLM 校正
畫面上的投影片文字（OCR 抽取）補充當集的新術語

量化過一次效果：連續處理 8 支影片，第一輪學習後新增 16 條替換規則，同一支影片的自動修正數從 0 跳到 58。第二輪之後，LLM 校正 subagent 回報的修正量明顯下降——因為便宜的錯誤已經在上游被機械規則吃掉了。

教訓一：驗證要看值，不是看有沒有錯誤

有一次，一個校正 subagent 把 300 條字幕「過度合併」成 110 條，出現 10-29 秒的超長字幕。當時的驗證只檢查「檔案存在＋格式正確」，照樣放行。

修法是把驗證改成檢查數值：

ratio = corrected_count / original_count
if ratio < 0.55:
    fail()          # 條數掉太多，必是過度合併
if ratio < 0.80 and longest_entry_sec > 15:
    fail()          # 條數比例配合時長症狀雙重確認

閾值不是拍腦袋，是拿 10 部歷史影片的 88 個分段校準出來的（誤報率 0）。單看條數比例不夠——講者講碎句的段落天然低 ratio，要配上「最長條目時長」這個症狀一起判。

教訓二：subagent 的輸出永遠要防禦性解析

LLM subagent 偶爾會把判斷過程寫進輸出（「[通順，不改]」），或洩漏工具呼叫的 XML 標籤。更麻煩的是，後處理的切分邏輯會把單點污染放大成多處擴散。

防呆分三層：prompt 明列禁止輸出的模式、合併前先 strip、後處理完再 strip 一次。原則只有一句：清理必須在切片之前，不能信任 subagent 自律。

教訓三：資源衝突要編排，不是祈禱

32GB 的 M1 Max 跑不動兩個大模型同時推理。20GB 的視覺模型跑完之後，框架還會把模型在記憶體裡保留幾分鐘——這段時間啟動 ASR 就直接 OOM。修法很土但有效：caption 一跑完就明確下 ollama stop，再啟動下一個階段。小模型（1.5GB 的 ASR ＋ 5GB 的輔助 ASR）反而可以放心並行，頂多互搶 GPU 變慢，不會掛。

值得抄的三個設計

如果你要搭類似的東西，我覺得最值得抄的是：

保留所有中間產物。每個階段的輸出都留檔。品質 gate 的閾值校準、事故後的重建，靠的都是這批歷史資料。
機械規則吃便宜錯誤，LLM 吃難錯誤。分層之後，貴的模型只處理真正需要理解的部分。
gate 用真實資料校準。防呆閾值拿歷史產出算出來，不是猜出來。

這條 pipeline 的 skill 已經開源，跟它的姊妹技能（把錄音整理成忠於原話的文章）都在我的 GitHub 上。

English

From 3 Hours to 30 Minutes: My Automated Chinese Subtitle Pipeline

I make video subtitles for a finance-education creator. A two-hour video used to take 2-3 hours of manual correction; the same work now takes 15-30 minutes — the pipeline runs on its own and I spot-check the result. This post is how I built that production line, and the three lessons that took it from “it runs” to “it’s reliable.”

What the Production Line Looks Like

The whole pipeline is triggered by a single skill inside Claude Code, running six stages in sequence:

Download (yt-dlp)
  → On-screen term extraction (OCR / VLM caption, one frame every 60s)
  → ASR (Breeze ASR 25, local inference on Apple Silicon)
  → Preprocessing (sentence-boundary cleanup, mid-word comma repair)
  → LLM correction (dispatch subagents per segment, inject the glossary + on-screen context)
  → Postprocessing (split overlong captions, quality gate, SRT output)

ASR runs locally the whole way; only the correction stage uses a cloud model. The hard part of finance content isn’t speech recognition itself — it’s the proper nouns: ticker symbols, ETFs, indicator abbreviations, the speaker’s pet phrases. When ASR mishears a company name or shatters “MA30” into fragments, a general-purpose model can’t fix those well. It takes domain knowledge.

The Glossary Is Alive

The core of the system isn’t any single model — it’s a glossary that learns:

After each video is corrected, a term-learning pass runs, turning “ASR mistake → correct term” into mechanical substitution rules
The next video’s ASR output passes through this batch of rules before it goes into LLM correction
On-screen slide text (extracted by OCR) adds the new terms for that episode

I quantified the effect once: processing 8 videos in a row, the first learning round added 16 substitution rules, and the automatic corrections on the same video jumped from 0 to 58. After the second round, the correction volume reported by the LLM correction subagent dropped noticeably — because the cheap mistakes were already being eaten upstream by the mechanical rules.

Lesson 1: Verify the Values, Not Just the Absence of Errors

Once, a correction subagent “over-merged” 300 captions down to 110, producing overlong captions of 10-29 seconds each. The verification at the time only checked “the file exists and the format is valid,” so it let them straight through.

The fix was to change verification to check the numbers:

ratio = corrected_count / original_count
if ratio < 0.55:
    fail()          # too few entries left — must be over-merging
if ratio < 0.80 and longest_entry_sec > 15:
    fail()          # entry-ratio plus duration symptom, double confirmation

The thresholds aren’t guesses — they were calibrated against 88 segments from 10 historical videos (zero false positives). The entry ratio alone isn’t enough: passages where the speaker talks in fragments are naturally low-ratio, so you have to pair it with the “longest entry duration” symptom to judge.

Lesson 2: Always Parse Subagent Output Defensively

An LLM subagent will occasionally write its reasoning into the output (“[reads fine, no change]”), or leak the XML tags of a tool call. Worse, the postprocessing split logic will amplify a single point of contamination into multiple spread-out ones.

The fail-safe has three layers: the prompt explicitly lists the forbidden output patterns, a strip before merging, and another strip after postprocessing. The principle is a single sentence: cleanup must happen before slicing — you can’t trust the subagent to police itself.

Lesson 3: Resource Conflicts Need Orchestration, Not Prayer

A 32GB M1 Max can’t run two large models inferring at the same time. After the 20GB vision model finishes, the framework keeps the model resident in memory for a few minutes — start ASR during that window and you OOM outright. The fix is crude but effective: as soon as the caption stage finishes, explicitly issue ollama stop, then start the next stage. Small models (the 1.5GB ASR plus the 5GB auxiliary ASR) can actually run in parallel without worry — at worst they contend for the GPU and slow down, but they won’t crash.

Three Designs Worth Stealing

If you’re going to build something similar, the parts I think are most worth stealing:

Keep every intermediate artifact. Save the output of every stage. Calibrating the quality gate’s thresholds, and rebuilding after an incident, both rely on this stash of historical data.
Mechanical rules eat the cheap errors, the LLM eats the hard ones. Once you layer it this way, the expensive model only handles the parts that genuinely need understanding.
Calibrate the gate on real data. Compute the fail-safe thresholds from historical output, don’t guess them.

This pipeline’s skill is already open-sourced, alongside its sister skill (turning a recording into an article that stays faithful to the original words) — both are on my GitHub.