Accuracy methodology (high level)
In short: Recorded fixture runs through the real pipeline; curated references gate quality; lag sampled at time checkpoints; your room still dominates.
Last updated 5 October 2026
Why this page exists
Churches ask “How accurate is it?” Marketing percentages are easy to invent and hard to trust. This page explains what we measure internally when we change speech, translation, or audio delivery—so you can align expectations with process, not slogans.
We do not treat these lab metrics as a promise for your sanctuary. Your mixer, mic technique, and network dominate Sunday outcomes.
End-to-end, not cherry-picked demos
Evaluation runs the same path listeners use:
- Fixture audio (sermon-style recordings) enters live speech recognition
- Text is translated and synthesized (including Flash or clip paths where enabled)
- Output audio is captured and re-transcribed to approximate what a listener heard
- Results are compared to curated reference transcripts for that fixture—not to whatever the model happened to print that day
That closes the loop from pulpit feed → ear, not “translation API on perfect text.”
Word error rate (WER) and coverage
For each fixture we maintain reference source and target text.
| Signal | Meaning (simplified) |
|---|---|
| Source WER | How close streaming STT is to the curated source transcript |
| Target WER / coverage | How close heard translation (via re-STT of TTS output) is to the curated target |
High WER or low coverage triggers investigation in grouping, timing, and TTS continuity—common causes of “choppy” worship audio—not automatic model swaps.
We compare against expected.json-style references, not against the pipeline’s own best guess as ground truth.
Lag and continuity checkpoints
Accuracy is not only words—it is when words arrive.
Smoke and eval runs record checkpoints at fixed times into a segment (for example 10s, 30s) for:
- How far translation text trails live speech
- How far played audio trails live speech
Separately, continuity heuristics flag mid-sentence clip breaks and unnatural gaps between spoken chunks—failure modes that feel worse in a sermon than raw WER suggests.
See listener-facing symptoms: Captions lag, Fragmentary captions.
What we do not do
- Publish a single “X% accurate” banner for all languages and rooms
- Replace your pilot in the building with a lab score
- Treat translation text in the database alone as “heard quality” when audio delivery diverges
What your church should do
Read the pastoral framing in How accurate is AI sermon translation?, then run a pre-service test and bilingual feedback loop—Pre-service test, Quick guide for speakers.
If you need deeper technical detail for partnership or research pilots, contact Sanalive through your account channel.
Was this helpful?
Related articles
- How accurate is AI sermon translation for churches?Can AI translate a church sermon accurately? It can keep up with a fast preacher and cover languages a volunteer cannot, but it can miss Bible names, jokes, and words with two meanings. A church glossary and giving AI about one extra second to do the translation fixes many of those issues.
- Quick guide for speakersHow the microphone and short pauses help captions and listeners.
- TroubleshootingWhat to try when captions, audio, or the connection misbehave, and how to read the messages Sanalive shows.