Hayamimi Delivers Real-Time Multilingual Speech Recognition on CPU Without GPU or Cloud

Summary

Hayamimi is an open-source, real-time multilingual speech-to-text system that runs exclusively on CPU. It delivers live subtitles, speaker labels, and translation with under 2 GB of RAM, requiring no GPU or cloud APIs. The system routes each sentence to language-specific INT8 ONNX models via sherpa-onnx. On a 6-core desktop CPU, it achieves 10–50x real-time speed and a 3.8% Character Error Rate (CER) on Japanese TV audio.

Why it matters

By eliminating dependencies on GPUs and the cloud, it enables production-quality live subtitling on modest hardware. This is crucial for independent broadcasters, accessibility tools, and offline/edge deployments. Its sentence-level language routing architecture and INT8 ONNX pipeline offer a practical, reproducible template for multilingual ASR without the overhead of heavy frameworks.

Highlights

• 3.8% CER on clean Japanese broadcast audio; a two-stage correction reduces CER from 15.5% to 12.0%
• Sentence-level language detection routing to specialized INT8 ONNX models (no PyTorch/CUDA); Meta Omnilingual ASR serves as a fallback for ~1,600 languages
• Full feature set: live draft subtitles (~0.5s updates), ~100ms final confirmation, CAM++/pyannote speaker diarization, real-time translation, OBS overlay, WebSocket audio input, and LRU model eviction to stay under 2 GB

Critical take

The 3.8% CER cited in the title was measured on clean TV broadcast audio, not in noisy real-world conditions. The system also fails to reliably transcribe mixed-language sentences, overlapping speakers, or short sentences following background music/sound effects. Translation quality is limited by small models; errors in numbers and currencies are acknowledged for Chinese and Korean. This restricts the system's use in any task where numerical accuracy is critical.