X-AuT unveils progressive audio‑encoder compression for speech LLMs using cross‑scale distillation
embeddings inference speech
| Source: HF Papers | Original article
Researchers present X-AuT, a progressive audio‑encoder compression framework that reduces inference cost for speech LLMs while mitigating embedding perturbations and sequence errors.
A new research paper released on 10 September 2026 introduces **X‑AuT**, a progressive compression framework for the audio encoders that sit in front of speech‑large language models (LLMs). The authors observe that trimming the depth of an audio encoder cuts inference cost, but outright removal of whole transformer blocks often distorts the embeddings fed to the decoder, leading to token deletions and premature end‑of‑sequence errors. X‑AuT tackles this by selecting and recombining layer groups across scales, effectively “cross‑scale distillation” that preserves the semantic richness of the original representation while shrinking the model.
The technique is demonstrated on the Qwen3‑ASR‑0.6B model. By reducing the encoder from 18 to 16 layers, the macro‑average error on ten public Chinese‑English speech benchmarks fell from 5.61 % to 5.27 %, showing that the compression does not merely maintain accuracy but can improve it. Lower‑depth encoders also mean fewer matrix multiplications per inference, translating into faster response times and reduced hardware demand—critical factors for deploying speech‑LLMs on edge devices or in cost‑sensitive cloud services.
X‑AuT arrives amid growing interest in making multimodal LLMs more efficient, echoing recent work on distillation and KV‑cache compression. The next steps to watch include broader evaluations on additional language pairs and real‑world workloads, open‑source releases of the training pipeline, and potential integration into commercial speech assistants. If the community can replicate the reported gains, X‑AuT could become a standard tool for scaling end‑to‑end speech models without sacrificing quality.
Sources
Back to AIPULSEN