Model reference · open weights
Paraformer is an open-weight audio or speech model from FunAudioLLM. Paraformer-GGUF (GGUF) weighs 435 MB; the smallest configuration that runs it is RTX 3060 12 GB.
For self-hosting, compare this build's requirements below with the GPU you plan to rent.
Summary of the FunAudioLLM/Paraformer-GGUF model card, 2026-10-01
What it is
| Released by | FunAudioLLM |
|---|---|
| Released | 2026-06-20 |
| VRAM | 435 MB for the weights |
What it runs on
| Card | One stream | Memory |
|---|---|---|
| RTX 3060 12 GB | fits | 11.6 GB |
| RTX 4060 Ti 16 GB | fits | 15.4 GB |
| RTX 3090 24 GB | fits | 23.4 GB |
| RTX 4090 24 GB | fits | 23.4 GB |
| RTX 5090 32 GB | fits | 31.0 GB |
| L40S 48 GB | fits | 44.0 GB |
| A100 80 GB | fits | 78.2 GB |
| H100 80 GB | fits | 78.1 GB |
| RTX PRO 6000 Blackwell 96 GB | fits | 93.8 GB |
| DGX Spark (GB10) 128 GB unified | fits | 107 GB |
| H200 141 GB | fits | 138 GB |
| B200 180 GB | fits | 176 GB |
From the model card
GGUF build of FunASR's Paraformer-zh (SAN-M encoder + CIF predictor + SAN-M decoder, non-autoregressive) for the zero-Python, CPU/edge FunASR llama.cpp runtime — fast Mandarin ASR, ~21× real-time on CPU.
These are GGUF weights for the FunASR llama.cpp runtime — a whisper.cpp-style, single self-contained binary for CPU / edge. Grab a prebuilt binary, then fetch this model and run:
runtime-llamacpp-v*)bash download-funasr-model.sh paraformer ./gguf
llama-funasr-paraformer -m ./gguf/paraformer-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav
| file | size | notes |
|---|---|---|
paraformer-f16.gguf | 435 MB | recommended (f16 matmul weights) |
paraformer-q8.gguf | ~217 MB | recommended — half of f16, same accuracy |
paraformer.gguf | 863 MB | f32 reference |
The binary prints transcription text directly (no Python detok). --ids for raw ids.
llama-funasr-paraformer -m paraformer-f16.gguf -a audio.wav --vad fsmn-vad.gguf
On CPU (8 threads): 9.85 % CER on the 184-clip Mandarin benchmark (vs whisper.cpp 22–31 %).
Quoted from the model card on Hugging Face. The full card is behind the Hugging Face link above.
How it works