Methodology
41 tools (League A 17 · League B 24) measured with the same inputs, settings and scorer. Rubric version 2026.10.
Independence
This is an independent lab. We accept no paid placement from vendors. Rankings and scores follow the published methodology: price rankings are estimated costs calculated from official and third-party price data with sources and check dates, and measured scores appear only for tools we have measured ourselves. Affiliate commissions never change ranking, scores or display order.
No paid placement, no sponsored listings. Affiliate links exist only for approved programs and always carry a disclosure right above the link. Commissions never change ranking, scores or display order.
Measurement protocol
- League A (clippers & captions): same sources (3 Korean and 3 Japanese pieces, 3-5 minutes each, fast speech, slang and proper nouns, script = ground truth). First round is our own recordings (with consent forms), second round uses KOGL type 1 material.
- League B (generative): the same 10 standard 8-Beat prompts × the same settings (5 s · 720p · 9:16).
- Up to 5 generations per tool per case. Success criteria (broken image, gibberish text, sync over 300 ms = failure) are written down beforehand and every attempt is logged. Tools that deduct credits for failures are flagged "Charged on failure".
- Paid plans only (free tiers carry watermark and commercial-rights problems). Every figure shows check date, plan and source; unmeasured values are "—".
- Input material is limited to CC BY (Big Buck Bunny, Sintel, Tears of Steel), Pexels, Pixabay and KOGL type 1. There is no download button for input clips.
- Where a vendor's terms forbid publishing outputs (for example PixVerse outputs on hold) we publish neither video nor score, only a note. 🔒 Luma's Terms of Service 3.4(g) prohibit publishing benchmark or performance information. Quantitative scores stay locked until written permission; only a qualitative review is published. Source
KO/JA localization scorecard, 9 items (0-100)
| # | Item | Method |
|---|---|---|
| 1 | Character error rate (CER) | Tool captions vs script, own implementation with published normalization rules |
| 2 | Proper-noun errors | Checked against a published per-case list |
| 3 | Slang errors | Checked against a published per-case list |
| 4 | Hallucinated sentences | Sentences absent from the script |
| 5 | Sync drift (ms) | Reference = Azure Speech word timestamps; median and maximum |
| 6 | Line-break rule violations | Checker for forbidden line-start characters, mid-word cuts and more than 2 lines |
| 7 | CJK font breakage | Tofu (□) detection in rendered frames plus fallback-font check |
| 8 | Watermark removal conditions | Per-plan conditions (terms and measurement) |
| 9 | Cancellation and refund friction | Terms clause score plus Trustpilot 1-star share |
If any of the 9 items is empty, no total is computed and the tool is excluded from ranking. Korean starts at a disadvantage at the engine level (FLEURS WER Japanese 7.8% vs Korean 14.5%), so it is the slot that will stay empty longest.
TCO formula
clips_per_short = ceil(short_sec / clip_sec) gens = N_shorts × clips_per_short × R (R = retry multiplier 1-5) billed_gens = gens (charged on failure) | N_shorts × clips_per_short (successes only) credits_needed = billed_gens × credits_per_clip cost = min( ceil(credits_needed / plan_credits) × plan_price, pay-as-you-go path ) per_short = cost / N_shorts → USD · KRW · JPY (FX date shown)
The web calculator and the server-side formula are cross-checked with the same fixed vectors. Clippers are unified as one 60-minute source → 5 finished shorts.