FunASR – modelscope

(简体中文|English|日本語|한국어)

Industrial speech recognition toolkit for offline, streaming, and edge deployment.

ASR · VAD · punctuation · speaker pipelines · emotion and audio-event models · OpenAI-compatible serving

Quick Start · Colab · Benchmark · Model selection · Migration guide · Use cases · Community integrations · Deployment matrix · Deployment hub · Troubleshooting · Models · Agent Integration · OpenClaw · Docs · Contribute

Quick Start

<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab" class="ra0-md-img" loading="lazy" />

No local setup? Open the Colab quickstart to transcribe a public sample or upload your own audio in a browser.

# CPU-only installs can use the default PyPI wheels.
pip install torch torchaudio
pip install funasr

For GPU quickstarts, install the PyTorch and torchaudio wheels that match your
NVIDIA driver from pytorch.org
before installing FunASR. After installation, confirm the GPU is visible:

python - <<'PY'
import torch
print(torch.cuda.is_available())
PY

Only use device="cuda" when this prints True; otherwise use device="cpu"
or reinstall PyTorch with the correct CUDA wheel.

Flagship model — Fun-ASR-Nano (LLM-ASR for Chinese, English, and Japanese, plus Chinese dialect groups and regional accents; needs a GPU):

from funasr import AutoModel

model = AutoModel(model="FunAudioLLM/Fun-ASR-Nano-2512", device="cuda")
result = model.generate(input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav")
print(result[0]["text"])
# 欢迎大家来体验达摩院推出的语音识别模型。

For the separate 31-language checkpoint, use
Fun-ASR-MLT-Nano-2512.
Language coverage is checkpoint-specific, so Nano and MLT-Nano should be treated as distinct model choices.

On CPU (or for five-language ASR plus emotion and audio-event tags), use
SenseVoiceSmall. The pipeline below composes SenseVoiceSmall with FSMN-VAD
and CAM++; diarization is provided by the separate CAM++ model, not by the
SenseVoiceSmall checkpoint:
See the SenseVoice paper,
Hugging Face checkpoint,
and GGUF edge checkpoint.

from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess

model = AutoModel(model="iic/SenseVoiceSmall", vad_model="fsmn-vad", spk_model="cam++", device="cuda")  # use device="cpu" if you don't have a GPU
result = model.generate(
    input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav",
    batch_size_s=300,
)

# The AutoModel pipeline returns VAD segments with speaker ids and timestamps:
for seg in result[0]["sentence_info"]:
    print(f"[{seg['start']/1000:.1f}s] Speaker {seg['spk']}: {rich_transcription_postprocess(seg['sentence'])}")

Output — structured text with speaker labels, timestamps, and punctuation:

[0.6s] Speaker 0: 欢迎大家来体验达摩院推出的语音识别模型

One AutoModel pipeline call coordinates the configured ASR, VAD, and speaker
models and returns the combined result.

Scale & deploy the flagship

At scale, accelerate Fun-ASR-Nano with vLLM (batch processing):

from funasr.auto.auto_model_vllm import AutoModelVLLM

model = AutoModelVLLM(model="FunAudioLLM/Fun-ASR-Nano-2512", tensor_parallel_size=1)
results = model.generate(["audio1.wav", "audio2.wav"], language="auto")

> Deploy as API server: funasr-server --device cuda → OpenAI-compatible endpoint at localhost:8000
>
> Use with AI agents: MCP Server for Claude/Cursor · OpenAI API for LangChain/Dify/AutoGen
>
> Use with voice agents: OpenClaw realtime plugin for self-hosted Talk and Voice Call transcription

Why FunASR?

Whisper is a single model; FunASR is a toolkit — you pick the right model
per job: Fun-ASR-Nano (Chinese, English, Japanese, and Chinese dialects;
GPU), Fun-ASR-MLT-Nano (31 languages), SenseVoiceSmall (five-language
ASR plus emotion and audio events), and Paraformer (low-latency streaming).
The table shows toolkit-level capabilities and names the model or pipeline that
provides each one:

Trying FunASR for the first time? Use the Colab quickstart before setting up a local environment. Choosing a first model? Start with the model selection guide. Planning a switch from Whisper or a cloud ASR provider? Use the migration guide and benchmark example to test representative audio, map features, and roll out safely.

Installation

pip install funasr

git clone https://github.com/modelscope/FunASR.git && cd FunASR
pip install -e ./

Model Zoo

Usage

> Full examples with parameter docs: Tutorial →

from funasr import AutoModel

# Chinese production (VAD + ASR + punctuation + speaker)
model = AutoModel(model="paraformer-zh", vad_model="fsmn-vad", punc_model="ct-punc", spk_model="cam++", device="cuda")
result = model.generate(input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav", hotword="关键词 20")

# Optional Silero VAD (install first: python -m pip install "funasr[silero]")
model = AutoModel(
    model="paraformer-zh", vad_model="silero-vad", device="cuda",
    vad_kwargs={"silero_threshold": 0.5, "silero_min_silence_duration_ms": 100},
)
result = model.generate(input="audio.wav")

# Streaming real-time (feed audio chunk by chunk)
import soundfile as sf
model = AutoModel(model="paraformer-zh-streaming", device="cuda")
audio, sr = sf.read("speech.wav", dtype="float32")   # 16 kHz mono
chunk_size = [0, 10, 5]                               # 600 ms chunks
chunk_stride = chunk_size[1] * 960
cache = {}
n_chunks = (len(audio) - 1) // chunk_stride + 1
for i in range(n_chunks):
    chunk = audio[i * chunk_stride : (i + 1) * chunk_stride]
    res = model.generate(input=chunk, cache=cache, is_final=(i == n_chunks - 1),
                         chunk_size=chunk_size, encoder_chunk_look_back=4, decoder_chunk_look_back=1)
    if res[0]["text"]:
        print(res[0]["text"], end="", flush=True)

# Emotion recognition
model = AutoModel(model="emotion2vec_plus_large", device="cuda")
result = model.generate(input="audio.wav", granularity="utterance")

CLI (Agent-Friendly)

# Transcribe audio (simplest)
funasr audio.wav

# JSON output (for AI agents)
funasr audio.wav --output-format json

# SRT subtitles
funasr audio.wav --output-format srt --output-dir ./subs

# Speaker diarization + timestamps
funasr audio.wav --spk --timestamps -f json

# Choose model and language
funasr audio.wav --model paraformer --language zh

# Batch transcribe
funasr *.wav --output-format srt --output-dir ./output

Available models: sensevoice (default), paraformer, paraformer-en, fun-asr-nano

Deploy

# OpenAI-compatible API (recommended)
pip install torch torchaudio
pip install funasr vllm fastapi uvicorn python-multipart
funasr-server --device cuda
# → POST /v1/audio/transcriptions at localhost:8000

Verify it with a public sample:

curl -L https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav -o sample.wav
curl http://localhost:8000/v1/audio/transcriptions 
  -F file=@sample.wav 
  -F model=sensevoice 
  -F response_format=verbose_json

# Docker streaming service
docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12

CPU / Edge — llama.cpp / GGUF (no GPU, no Python)

Run SenseVoice / Paraformer / Fun-ASR-Nano as a single self-contained binary on CPU and edge devices — this is to FunASR what whisper.cpp is to Whisper, but with ~3× lower CER than whisper.cpp on Chinese. Built-in FSMN-VAD, no Python at runtime.

# Linux / macOS: run from the extracted release directory
bash download-funasr-model.sh sensevoice ./gguf        # or: paraformer | nano
./llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav
# → 欢迎大家来体验达摩院推出的语音识别模型

# Windows PowerShell: run from the extracted archive root (with the `hf` CLI installed)
hf download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir .gguf
hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .gguf
.llama-funasr-sensevoice.exe -m .ggufsensevoice-small-q8.gguf --vad .gguffsmn-vad.gguf -a audio.wav
# Use the windows-x64-vulkan package with a current AMD, Intel, or NVIDIA Vulkan driver:
.llama-funasr-sensevoice.exe -m .ggufsensevoice-small-q8.gguf --vad .gguffsmn-vad.gguf -a audio.wav --backend vulkan
# Use the windows-x64-cuda package on RTX 30-class GPUs:
.llama-funasr-sensevoice.exe -m .ggufsensevoice-small-q8.gguf --vad .gguffsmn-vad.gguf -a audio.wav --backend cuda

Use funasr-llamacpp-linux-x64-vulkan.tar.gz on Linux GPU systems with a
working Vulkan driver/ICD:

./llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav --backend vulkan

The Windows Vulkan ZIP uses the system Vulkan loader supplied by the GPU driver;
installing the Vulkan SDK is only necessary when building from source. Both
Vulkan packages currently accelerate SenseVoiceSmall.

The current Windows CUDA package targets CUDA architecture 86. RTX 50 / Blackwell
GPUs report compute capability 12.0 (sm_120) and should use the CPU package or
build from source with -DCMAKE_CUDA_ARCHITECTURES=120 until a dedicated CUDA
asset is published.

Prebuilt binaries: Releases · v0.2.0 · Linux Vulkan tarball · Windows Vulkan zip · Windows CUDA zip · Download & quickstart: funasr.com/deploy/llama-cpp · GGUF models: Hugging Face · Docs & benchmarks: runtime/llama.cpp/

OpenAI API example → · Gradio demo → · Client recipes → · JavaScript/TypeScript recipes → · Kubernetes template → · Workflow recipes → · Postman collection → · OpenAPI spec → · Security guide → · Deployment matrix → · Deployment docs → · Agent integration →

Benchmark

> 184 long-form audio files (192 min). Full report → · RTFx and reproducibility notes →

> Key takeaway: FunASR models run on CPU faster than Whisper runs on GPU.

What's new

• 2026/08/14: v1.4.2 on PyPI — punctuation-aware sentence alignment now preserves subtitle segmentation when a punctuation token boundary splits a timestamped ASR word. Distributed training now synchronizes DDP/FSDP gradients on the final microbatch of each accumulation window and initializes DeepSpeed/FSDP mode from the resolved configuration. The tagged source also includes llama.cpp SRT output and the v0.2.0 AMD Vulkan submission updates. Install with python -m pip install -U "funasr==1.4.2". Release ->
• 2026/08/11: llama.cpp runtime v0.2.0 — pins upstream llama.cpp at 803b7fca and publishes nine SHA-256-listed Linux, macOS, and Windows archives from one tested workflow. Fun-ASR-Nano, SenseVoice, and Paraformer CLIs can now write SRT subtitles; Vulkan startup reports actionable AMD diagnostics and a CPU fallback. The AMD Windows Vulkan crash fix still awaits confirmation on the reporter's hardware. Download matrix & quickstart -> · Release ->
• 2026/08/04: v1.4.1 on PyPI — the Hugging Face paraformer-en alias now resolves to the official English checkpoint instead of silently downloading the Chinese model. This patch also includes Fun-ASR-Nano LoRA fine-tuning and safer checkpoint handling; the tagged source adds JSONL timestamp output, SenseVoice TensorRT deployment, and the OpenClaw realtime transcription integration. Install with python -m pip install -U "funasr==1.4.1". Release ->
• 2026/08/04: OpenClaw realtime transcription integration — the new <code class="ra0-md-code">openclaw-funasr</code> source package connects self-hosted FunASR online, offline, and 2pass WebSocket recognition to OpenClaw Talk and Voice Call. Its 8 kHz G.711 mu-law conversion, 60 ms framing, partial/final transcripts, reconnect limits, package installation, and runtime registration have been verified against OpenClaw 2026.7.2; npm and ClawHub publication will follow the required upstream SDK change.
• 2026/07/31: v1.4.0 on PyPI — AutoModel now rejects the common vda_model misspelling before model download and points to vad_model, so VAD-dependent segmentation, speaker processing, and sentence_info are not silently disabled. The GitHub source release also updates the legacy WebSocket file runtime: clients wait for an explicit end-of-input acknowledgement while the server flushes pending offline, online, and 2pass audio and reports finalization failures. Install the Python package with python -m pip install -U "funasr==1.4.0". Release ->
• 2026/07/27: v1.3.30 on PyPI — container-formatted WAV, MP3, FLAC, OGG, MP4/M4A, and WebM audio bytes are now decoded through their codecs instead of being misread as raw PCM. OpenAI-compatible responses preserve speaker labels, VAD sentence timing survives punctuation mismatch, trusted browser clients can opt in to CORS, and vLLM VAD chunks are capped at 30 seconds. The GitHub release also includes the current prebuilt llama.cpp runtime for nine desktop and server targets. Install with python -m pip install -U "funasr==1.3.30". Release ->
• 2026/07/24: v1.3.29 hotfix on PyPI — SenseVoice long-audio inference now returns each VAD speech region through sentence_info when token timestamps and a punctuation model are unavailable. Subtitle clients receive the recognized text with real millisecond start/end bounds instead of one zero-length or full-media cue. Install with python -m pip install -U "funasr==1.3.29". Release ->
• 2026/07/24: v1.3.28 hotfix on PyPI — realtime WebSocket finalization now preserves clean continuous partial transcripts when a VAD-locked decode truncates to a short prefix, repeats a hallucinated phrase, or raises; short STOP tails, VAD finalization, and speaker completion now share the same reliable path. SenseVoice subtitle segmentation also aligns rich tags, punctuation, and word/BPE timestamps without collapsing Chinese into one cue or damaging English surface text. Install with python -m pip install -U "funasr==1.3.28". Release ->
• 2026/07/24: v1.3.27 on PyPI — the OpenAI-compatible server now reports detected SenseVoice language metadata in verbose_json and reuses the cached Fun-ASR-Nano AutoModel after vLLM fallback. When vLLM/VAD setup and its fallback both fail, half-initialized engine state is cleared so a later request can retry. Install with python -m pip install -U "funasr==1.3.27". Release ->
• 2026/07/23: llama.cpp runtime v0.1.9 — adds funasr-llamacpp-windows-x64-vulkan.zip for standalone SenseVoiceSmall Vulkan inference on Windows with AMD, Intel, or NVIDIA drivers. Linux Vulkan, Windows CUDA, CPU/AVX2, Linux arm64, and macOS arm64 assets remain available. Release ->
• 2026/07/23: v1.3.26 on PyPI — funasr-server --model fun-asr-nano --hub ms now honors the requested ModelScope hub for the default Fun-ASR-Nano model in both the vLLM path and the AutoModel fallback, avoiding unintended Hugging Face downloads when users choose ModelScope. Install with python -m pip install -U "funasr==1.3.26". Release ->
• 2026/07/23: v1.3.25 on PyPI — realtime WebSocket users can now use deterministic final-text hotword corrections with POSTPROCESS_HOTWORDS:wrong=>right or --postprocess-hotword-file, keeping fixed-name cleanup separate from model-level HOTWORDS: decoding bias. The source-tree realtime entrypoint also works without preinstalling the package. Install with python -m pip install -U "funasr==1.3.25". Release ->
• 2026/07/23: v1.3.24 on PyPI — OpenAI-compatible server deployments now support custom model paths and hub selection, the llama.cpp/GGUF runtime docs include the HTTP transcription wrapper and Linux Vulkan package, and public docs links were refreshed for cleaner onboarding. Install with python -m pip install -U "funasr==1.3.24". Release ->
• 2026/07/22: v1.3.23 on PyPI — packaging and onboarding refresh for this week's community integrations: the PyPI long description now highlights the current OpenAI-compatible server path, llama.cpp/GGUF runtime notes, Windows CUDA architecture guidance, and browser quickstart links shipped in the repository docs. Runtime code is unchanged from v1.3.22. Install with python -m pip install -U "funasr==1.3.23". Release ->
• 2026/07/22: llama.cpp runtime v0.1.8 — adds funasr-llamacpp-linux-x64-vulkan.tar.gz for SenseVoiceSmall on Linux Vulkan GPUs. Run llama-funasr-sensevoice ... --backend vulkan; CPU, AVX2, macOS arm64, Windows CPU/AVX2, and Windows CUDA packages remain available. Release ->
• 2026/07/19: v1.3.22 on PyPI — funasr-server now fills OpenAI-compatible verbose_json.segments for text-only SenseVoice/Paraformer fallback responses, so subtitle clients no longer see an empty segments array when text is populated. Install with python -m pip install -U "funasr==1.3.22". Release ->
• 2026/07/19: v1.3.21 on PyPI — fixes first-import onboarding in fresh environments where users install funasr before choosing a platform-specific PyTorch build. import funasr and funasr.__version__ now work without torch; accessing AutoModel still requires PyTorch and raises a clear install hint. Install with python -m pip install -U "funasr==1.3.21". Release ->
• 2026/07/19: v1.3.20 on PyPI — PyPI metadata and install guidance now point at the current FunASR docs, community integrations, and quoted python -m pip install -U "funasr>=1.3.19" commands for Fun-ASR-Nano deployment paths. This is a documentation/packaging sync; runtime code remains unchanged from v1.3.19. Install with python -m pip install -U "funasr==1.3.20". Release ->
• 2026/07/19: v1.3.19 on PyPI — realtime WebSocket long-session troubleshooting docs are now shipped with the package. Run the server with --enable-spk --log-session-stats-interval 30 and attach the emitted Session stats: lines when reporting disconnects or memory growth. Install with python -m pip install -U "funasr==1.3.19". Long-session guide -> · Release ->
• 2026/07/19: v1.3.18 on PyPI — CLI SRT/TSV subtitle output now requests sentence timestamps and loads punctuation when needed, so funasr audio.wav --output-format srt --output-dir ./subs writes segmented subtitle cues instead of one full-text block. Install with python -m pip install -U "funasr==1.3.18". Release ->
• 2026/07/18: v1.3.16 on PyPI — client-driven realtime endpoints for Fun-ASR-Nano. Start one WebSocket session, stream PCM, and send COMMIT for each utterance without loading server-side VAD; short utterances finalize and timestamps remain monotonic across commits. Install with pip install --upgrade funasr, then run funasr-realtime-server --endpoint-mode client. Guide →
• 2026/07/18: llama.cpp runtime v0.1.7 — prebuilt Windows CUDA package for SenseVoiceSmall (funasr-llamacpp-windows-x64-cuda.zip) plus Linux / macOS / Windows CPU packages. Download the GGUF model, then run llama-funasr-sensevoice ... --backend cuda on supported NVIDIA GPUs. Release →
• 2026/06/20: llama.cpp / GGUF runtime — run SenseVoice / Paraformer / Fun-ASR-Nano on CPU & edge as a single self-contained binary (a whisper.cpp-style alternative), built-in FSMN-VAD, no Python at runtime. Prebuilt binaries for Linux / macOS / Windows + q8 quantized models (~half the size, same accuracy). runtime/llama.cpp/ · Releases
• 2026/06/21: v1.3.12 on PyPI — rolling fixes (qwen3-asr language codes, glm_asr, vLLM repetition_penalty). pip install --upgrade funasr
• 2026/05/24: vLLM Inference Engine — 2-3x faster LLM decoding for Fun-ASR-Nano. Streaming WebSocket service with VAD + Speaker Diarization. Guide → · Realtime WS tuning → · API stability checklist →
• 2026/05/24: Dynamic VAD — adaptive silence threshold (default on). Short sentences stay intact, long segments get auto-split. Details →
• 2026/05/24: v1.3.3 — funasr-server CLI, OpenAI-compatible API, MCP Server for AI agents. pip install --upgrade funasr
• 2026/05/20: Added Qwen3-ASR (0.6B/1.7B) — 52 languages, auto detection. usage
• 2026/05/20: Added GLM-ASR-Nano (1.5B) — 17 languages, dialect support. usage
• 2026/05/19: Fun-ASR-Nano and SenseVoice can be combined with VAD and CAM++ for speaker diarization.
• 2025/12/15: Fun-ASR-Nano-2512 — Chinese, English, Japanese, and Chinese dialect support; trained on tens of millions of hours.

Community

Star History

License

• FunASR toolkit source code in this repository: MIT License.
• Pretrained model weights are licensed separately. Check the license shown on each model card; when a model card links to the FunASR Model Open Source License Agreement, those terms apply.

Citations

@inproceedings{gao2023funasr,
  author={Zhifu Gao and others},
  title={FunASR: A Fundamental End-to-End Speech Recognition Toolkit},
  booktitle={INTERSPEECH},
  year={2023}
}