Speech & Audio Processing
Conformer
Key
innovation
Combining self-attention and convolution in a single block so the model captures both global and local dependencies in the speech signal at once.
Category
Speech & Audio Processing
Abstraction level
Use cases
Automatic speech recognition (ASR)Speech-to-text transcriptionOn-device speech models
How it works
A Conformer block consists of feed-forward modules surrounding a self-attention module and a convolution module (a "macaron"-style structure). Self-attention provides global context while depthwise convolution captures local patterns. Log-mel feature sequences are processed through a stack of such blocks, often with temporal subsampling.
Strengths & limitations
Strengths
✓Combines global and local context
✓High accuracy in ASR
✓Good computational efficiency
Limitations
✗More implementation complexity than a plain Transformer
✗Self-attention cost grows with sequence length