Evaluation suites for the MimicScribe meeting transcription pipeline.
| Benchmark | What it measures | Results |
|---|---|---|
| Diarization | Speaker attribution accuracy on public corpora (AMI, Earnings-21, VoxConverse, SCOTUS) | Results |
| Live Transcription | Streaming window-stitch integrity — every stitch-point disagreement counted, words never silently dropped, content-word recall vs human transcripts, sentence punctuation | Results |
| ASR Model Bake-off | Dated September 2026 comparison of Parakeet Ultra, Parakeet v3/v2, Parakeet Unified, Cohere Transcribe and Granite Speech on the same corpus: word error, number fidelity, and two blind rubric judges, with a build-free reproduction | Results |
| Multilingual | Word error rate in all 25 supported languages on FLEURS read speech, measured through the app; MimicScribe's spoken-form fine-tune vs stock Parakeet Ultra | Results |
| Meeting Assistant | Real-time briefing quality — talking points, action items, question detection, interpersonal awareness, long-meeting compaction | Results |
| Context Retrieval | Reference document retrieval + end-to-end hallucination safety across 15 document types — CRM, HTML, PDF, contracts, competitive intel, strategic plans | Results |
| Meeting Search | Semantic search across months of meetings — buried sub-topics, disambiguation, ASR noise tolerance, conversational vs formal phrasing | Results |
| Screen Capture | Presentation pictures from shared windows: how many repeat a screen already captured, and how many faces are blurred before saving | Results |
| Transform | Selection-based text transformation quality — number/date/URL preservation, anti-injection from Reference Documents, voice matching, collision resilience | Results |