German text-to-speech · on-device

LenoTTS

A German voice model I trained from scratch on StyleTTS2 and shrank until it runs fully offline on an iPhone.

~150 hGerman studio recordings
216M → 82Mparameters, training vs. on-device
0.85×real-time on iPhone (faster than speech)
Listen to samples Try your own text Code on GitHub

iPhone test app (sherpa-onnx). Tap “Generate”: 2.3 s compute for 2.7 s of speech, no network. Sound on.

Listen

How it sounds

LenoTTS speaks German. All samples were generated with the sherpa-onnx runtime the iPhone uses, in a single fixed voice. Loudness is normalised; nothing else is edited.

Compare

Next to the original

Each model in its own language: the public StyleTTS2 LJSpeech checkpoint reads the English sentence, LenoTTS the German one. Same architecture, different training: LenoTTS was trained from scratch on German audio and compressed to run on a phone.

How it works

From studio recordings to an iPhone

  1. CollectAbout 150 hours of professionally narrated German studio recordings, 3 speakers.
  2. Align & cutWhisper transcripts, MMS forced alignment, cuts only at sentence ends in real silence, quality filters.
  3. TrainStyleTTS2 from scratch (stage 1 + 2) on 3× A40, then fine-tuned on 8 h of conversational lines.
  4. ShrinkDrop training-only modules, bake in the voice, rewrite untraceable parts, export to ONNX (300 MB).
  5. Run on devicePackaged for sherpa-onnx; speaks offline on iPhone at 0.85× real time.
What I built

The parts that were mine

Data pipeline

  • Whisper large-v3-turbo transcription on multiple GPUs
  • Word timestamps via MMS forced alignment (two passes)
  • Cuts only at sentence ends where the signal is actually silent; long sentences split at the longest pause
  • Filters for alignment score, loudness, abrupt endings and token length

German StyleTTS2

  • German IPA symbol set and my own German text aligner
  • Multilingual PL-BERT, iSTFTNet decoder
  • Multi-GPU training from scratch (stage 1 and 2), then a low-learning-rate fine-tune for a conversational voice
  • Tried and dropped: xLSTM prosody predictor, length bucketing

On-device export

  • Pruned to inference modules: 216M → 82M parameters
  • Loop-free duration alignment, real-valued iSTFT, hand-written AdaIN and weight norm so the model traces to ONNX
  • Packaged to an interface sherpa-onnx already supports, no runtime changes needed
  • Fixed three runtime issues (phoneme mapping, output gain, end-of-sentence token)

Model weights and training data are not released because of training-data licensing. The code is open source (MIT). You can try the model with your own German text; it runs server-side with the same runtime as the iPhone app, and the first request after a pause takes ~20 s.