phonon-en

High-quality Text-to-Speech, on your device.

phonon-31m is a text-to-speech model from the Phonon family. It turns text into natural 24 kHz speech in English. It streams audio while it is still generating, and it is small enough to run in a browser tab, on a phone, or on a laptop CPU.

Phonon builds on Pocket TTS and the Mimi codec, developed by Kyutai.

Try it in your browser · All Phonon models · Runtime on GitHub · Python · Browser · Discord

Model details

• Model type: text-to-speech. A flow-matching transformer predicts audio latents from the text one 80 ms frame at a time, and the Mimi codec's decoder turns them into audio.
• Parameters: 31M (excluding the audio tokenizer)
• Weights file: model.q8.gguf, 64.5 MB (8-bit quantized)
• Languages: English
• Output: mono, 24 kHz
• License: Apache 2.0

The Phonon family

All of them run on the same runtime and use the same file layout, so switching models means changing the repo ID.

Quickstart

Every runtime names this model as hf://gradium/phonon-31m-en, and takes lang, which selects how text is normalized (numbers, symbols, dates). This model speaks English, so use en.

Command line

uvx ptts --lang en --quant q8 \
  --voice Easton "Hello from Phonon." -o speech.wav

uvx runs the Python package's command without installing it. The standalone ptts binary from GitHub Releases or Homebrew (brew install gradium-ai/tap/ptts) takes the same flags.

Python

pip install ptts

import ptts

tts = ptts.TTS("hf://gradium/phonon-31m-en", lang="en", quant="q8")
tts.save("speech.wav", "Hello from Phonon.", voice="Easton")

# Or stream: chunks arrive while the rest is still being generated.
with tts.stream("Speech can start playing right away.", voice="Easton") as audio:
    for pcm in audio:
        ...  # float32 chunks at audio.sample_rate

Transformers usage 🤗

Install 🤗 Transformers from source:

pip install git+https://github.com/huggingface/transformers

from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from transformers import AutoProcessor, GradiumPhononTTSForConditionalGeneration

model_id = "gradium/phonon-31m-en"
processor = AutoProcessor.from_pretrained(model_id, device_map="auto")
model = GradiumPhononTTSForConditionalGeneration.from_pretrained(model_id, device_map="auto")

# Easton (en-us); every voice and its ID: https://huggingface.co/gradium/phonon-voices
voice_path = hf_hub_download("gradium/phonon-voices", "voices/LMDIYE0Oa377Qh6Z.safetensors")
voice = load_file(voice_path)["speaker_wavs"][0].T.float()  # (num_frames, 512)

conversation = [
    {"role": "user", "content": [{"type": "audio"}, {"type": "text", "text": "Hello from Phonon."}]}
]
inputs = processor.apply_chat_template(
    conversation,
    tokenize=True,
    return_dict=True,
    add_generation_prompt=True,
    processor_kwargs={"audio_prompt_latents": [voice]},
).to(model.device)

outputs = model.generate(**inputs)
audio = processor.decode(outputs.audio_latents, outputs.audio_latents_mask)
processor.save_audio(audio, "speech.wav")

This repo's default-voice.safetensors loads the same way. Unlike the runtime, the tokenizer does not spell out numbers, amounts, symbols or addresses, and long text is split with processor.split_text: see the Transformers docs.

Other platforms