ElevenLabs tuned transcription for messy rooms. Publish the error by speaker

THE SIGNAL IN ONE SENTENCE

The tidy demo has always been the easy part of transcription. One person, one microphone, a quiet room and a sentence designed to behave. Real audio is the opposite. People overlap. Someone joins on a bad phone line. A nurse says a drug name. A customer switches from Hindi to English halfway through a thought. The person farthest from the microphone gets turned into punctuation. ElevenLabs launched Scribe v3 for recorded audio and Scribe v3 Live for real-time audio on November 10. The company says the recorded model makes 27 percent fewer word errors than Scribe v2 across 102 languages. It says the live model reduces semantic word error by 43 percent and standard word error by 31 percent compared with Scribe v2 Realtime. Speaker labeling is part of the release. ElevenLabs reports 12 percent fewer diarization errors for Scribe v3 overall and 17 percent fewer on recordings with three or more speakers than Scribe v2. Scribe v3 Live adds speaker diarization to the company's real-time model and uses conversational context, not only silence, to decide whether a person has finished a turn. The models also add controls for language hints, number formatting, key terms and transcript edits. The recorded model accepts prompts containing up to 1,000 likely terms. The live model can consult a larger key-term library and handle conversations that switch languages within a sentence. Both are available through ElevenAPI, and the live model is available in ElevenAgents. For enterprise customers, ElevenLabs says the models can run in isolated environments in the European Union, India and Singapore, with private deployment also available. A medical version of Scribe v3 Live is being released for ElevenAgents. Those are the company-announced facts. The comparisons need a large asterisk, because ElevenLabs ran them. Multilingual accuracy uses an internal version of the FLEURS benchmark, and the company says it also built a 16-language dataset of meetings, agent calls and voice notes. ElevenLabs says it will open-source that real-world dataset, but it was not public with the announcement. The post does not provide a complete independent benchmark packet or subgroup results for every claimed condition. That does not make the improvements meaningless. It means buyers should not stop at the average. Publish an error card by speaker. A single word-error number can hide the person who matters most. Imagine a six-person meeting where five people use good microphones and one joins from a noisy train. The average may look excellent while every decision attributed to the sixth person is wrong. The transcript reads smoothly. The record is still broken. The error card should start with the exact evaluation unit. Name the model version, API mode, region, language settings, key-term list, formatting rules, audio codec and whether the transcript was batch or live. Record microphone type, distance, background noise, packet loss and room acoustics where those are known. Then separate the speakers. For each person, report word error, semantic error, missed speech, false speech, speaker-attribution error and the share of overlapping turns. Show how often names, numbers, dates, addresses, product codes and specialist terms were wrong. If demographic details are collected with consent for evaluation, report performance by accent, dialect, age range, speech difference and first-language status without turning the test set into an identity database. Do not reduce language to one dropdown. A conversation can contain code switching, borrowed terms, local place names and a person speaking a second language. Report each segment's language and the transition points. A model that performs well on clean read speech may behave differently when two languages share one sentence in a busy call center. Speaker diarization deserves its own ledger. Which words were placed under the wrong name? Did the model merge two speakers, split one person into several labels or create a speaker who was only background audio? How long did the mistake persist before correction? The company explains that diarization scores can be calculated differently. Its real-time evaluation aligned transcripts to a reference before measuring whether words were assigned to the right speaker, which lowers error rates and prevents a direct comparison with its batch results. That methodological note is useful. Every published error card should preserve it, because a percentage without the scoring recipe invites a comparison that the test does not support. Next, grade consequences, not just tokens. Hearing fifteen as fifty can alter a financial model. Assigning a symptom to the wrong patient can contaminate a clinical note. Putting a manager's approval under an employee's name can change an employment record. Missing a postal code can break a customer check. A transcript for captions can exclude the only speaker whose microphone was poor. Create consequence classes before testing. Harmless formatting differences belong in one bucket. Meaning-changing errors belong in another. Identity, dosage, price, account, consent, deadline and legal-obligation errors need their own red flags. Report how many required human correction and how many escaped into summaries, actions or records. That last part matters because transcripts rarely stay transcripts. They become meeting summaries, customer records, clinical documentation, subtitles, search indexes, training data and instructions for an agent. A small error can be rewritten into a confident sentence and then copied into several systems. The downstream model may remove the hesitation that would have warned a human reader. Test the whole chain. Run the same audio through transcription, speaker labeling, summarization and any automated action. Preserve the raw audio, the first transcript, every correction, the final artifact and the model versions involved. Measure whether a transcription error changed the summary, the task list or the action taken. High-consequence uses need a human checkpoint designed around uncertainty. Do not give the reviewer a polished paragraph and ask for a rubber stamp. Highlight low-confidence spans, overlapping speech, speaker changes, names, numbers and key terms. Let the reviewer play the exact audio segment, compare alternatives and correct the speaker without hunting through an hour-long recording. Review queues should follow risk. A podcast typo and a clinical dosage do not deserve the same process. Route consequential fields to an authorized reviewer. Block automatic filing or action until those fields are confirmed. Record who reviewed what, which audio they heard and what changed. Key-term prompting is useful, but it also needs testing. A large list can improve recognition of drug names, products and acronyms. It can also pull the model toward a listed term when the speaker said something else. Add decoy terms and near matches. Measure false substitutions as well as successful recovery. Formatting instructions deserve the same skepticism. Turning dates into one format can improve consistency. It can also hide ambiguity if the original speech did not establish the month and day clearly. Keep the spoken form beside any normalized value when the field matters. Live transcription has an extra constraint: time. A real-time system may commit text before it hears the rest of the sentence. Report first-token latency, finalization delay, revision frequency and the time until a stable speaker label appears. A caption that is corrected ten seconds later may still have failed the person who needed it in the moment. A voice agent that interrupts less often may feel better, but the test should also count long awkward waits and cases where background speech keeps a turn open. Deployment controls do not replace evaluation. Regional processing, zero-retention options and private deployment can reduce some privacy risks. They do not prove that the transcript is accurate, fair or appropriate for a record. Buyers still need to test their own rooms, languages, microphones, terminology and consequences. Build a challenge set from real failures, with permission and careful handling. Include overlapping speakers, distant microphones, poor networks, quiet voices, speech differences, code switching, local names, similar numbers and the jargon the organization actually uses. Keep a held-out portion that vendors and operators cannot tune against. Invite the people whose speech is most often lost to help design the test. That includes multilingual staff, caption users, call-center workers, clinicians, field teams and people with speech differences. Pay them for the work. Let them define what a harmful error looks like in their context. Publish the denominator. If a chart says the model improved in every language, readers should be able to see the amount and type of speech tested for each language, the number of speakers, the recording conditions and the confidence interval. A language represented by clean short clips is not equivalent to one represented by hours of overlapping conversation. Publish failures that look bad. The point of an error card is not to turn every deployment into a leaderboard. It is to tell operators when the transcript is trustworthy, when it needs review and when it should not become a record at all. Scribe v3 may be a meaningful improvement. ElevenLabs has at least described several methodological choices and promised to release its real-world dataset. Now the evaluation needs to travel with the product. Give each speaker a row. Give each important term a consequence. Give each correction a trail. The room is messy. The error report should be honest enough to show exactly who got lost in it.