Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models
AI Summary
A lightweight method is proposed to detect code-switching directly from speech without requiring full multilingual transcriptions. By identifying language-selective hidden-state dimensions in pretrained speech encoders like Whisper and aggregating their frame-level activations, the approach effectively finds code-switched utterances to build more representative multilingual training corpora.
Key Takeaways
- Uses frame-level language information in pretrained speech encoders to detect code-switching directly from speech.
- Identifies language-selective hidden-state dimensions using monolingual recordings.
- Whisper-small encoder achieves a mean F1 of 0.798 across 14 language pairs in CS-FLEURS.
Why it matters
Automatic speech recognition often struggles with code-switching due to sparse training data. This lightweight detection method enables engineers to efficiently curate multilingual corpora from large speech collections, improving downstream speech recognition performance.
Conversation 0
Be the first to share your perspective.