Enhancing Pathological Speech through Articulatory Bottlenecks and Phonetic-Aware Losses

Abstract. Dysarthric speech reconstruction (DSR) typically relies on linguistic or phonetic representations extracted from impaired speech to generate a more intelligible waveform. We investigate a complementary approach that instead intervenes in a representation related to speech production. Starting from a neural analysis-synthesis framework, we introduce a residual mapper that modifies an articulatory-aligned latent space while preserving speaker and prosodic information. The mapper is pretrained on parallel synthetic healthy and artificially dysarthric speech, then adapted to natural dysarthric speech using phoneme-guided and adversarial objectives. We further compare the articulatory bottleneck with a dimension-matched unsupervised representation to assess the benefit of explicit articulatory supervision. Objective and perceptual evaluations show that the proposed approach successfully improves artificially generated dysarthric speech. However, these improvements transfer only in a limited number of cases to real pathological speech, highlighting the challenge of bridging synthetic articulatory degradations and natural dysarthria.