Alibaba Qwen3-TTS

Alibaba's Tongyi Qianwen team has just open-sourced its new Qwen3-TTS series of text-to-speech models. This release includes five model variants, with the smallest containing only 0.6 billion parameters and the largest at 1.7B, making them compact enough to potentially run on mobile devices. The team claims these models outperform several commercial closed-source systems, including OpenAI's GPT-4o-Audio and ElevenLabs, marking the first time the Qwen team has open-sourced a full TTS model family.

The Qwen3-TTS release focuses on two core capabilities: Voice Design and Voice Cloning, aiming to address pain points in emotional expression, personalization, and multilingual adaptability found in traditional TTS systems.

Model Overview

The open-sourced Qwen3-TTS series features five models, all built around two parameter scales: 1.7B and 0.6B. The core technology relies on the Qwen3-TTS-Tokenizer-12Hz multicodebook speech encoder, enabling efficient compression and high-fidelity audio reconstruction.

The models are divided into three distinct versions:

• VoiceDesign: Generates voices from user descriptions via natural language, supporting instruction control and streaming.
• CustomVoice: Allows users to specify target voice characteristics via instructions, offering 9 high-quality preset voices covering various genders, ages, languages, and dialects, with instruction control and streaming support.
• Base: A foundational model that supports rapid 3-second voice cloning, can be fine-tuned for other tasks, and supports streaming.

Model Name	Parameters	Key Features
Qwen3-TTS-12Hz-1.7B-VoiceDesign	1.7B	Voice design from descriptions, instruction control, streaming
Qwen3-TTS-12Hz-1.7B-CustomVoice	1.7B	Instruction-based voice control, 9 preset voices, streaming
Qwen3-TTS-12Hz-1.7B-Base	1.7B	3-second voice cloning, fine-tuning base, streaming
Qwen3-TTS-12Hz-0.6B-CustomVoice	0.6B	9 preset voices, streaming
Qwen3-TTS-12Hz-0.6B-Base	0.6B	3-second voice cloning, fine-tuning base, streaming

Core Features

Despite their compact size, the Qwen3-TTS models offer several distinctive capabilities:

1. Voice Design via Natural Language Unlike traditional TTS systems limited to preset voices, Qwen3-TTS lets users describe desired voice characteristics in natural language. You can specify tone, emotion, prosody, age, gender, or even background character traits. For example, instructing the model to generate a "hoarse middle-aged male voice with an urgent sales pitch" or a "gentle young woman reading bedtime stories." This lowers the barrier for creating character voices for games, audiobooks, and similar applications.

2. 3-Second Voice Cloning The system can clone a speaker's voice from just a 3-second audio sample. With end-to-end latency as low as 97ms after the first character input, it suits real-time applications like live streaming or interactive dialogue. Key aspects include:

• Minimal data requirement (3 seconds of reference audio)
• Cross-lingual cloning: After learning a voice from one language (e.g., Chinese), it can generate speech in 10 other languages using that same voice
• High fidelity: Alibaba's tests indicate superior similarity and naturalness compared to ElevenLabs and GPT-4o-mini-tts

3. Strong Multilingual Support The models support over 10 major languages including Chinese, English, Japanese, Korean, German, French, Spanish, Italian, Portuguese, and Russian. They also handle multiple dialects such as Cantonese, Sichuanese, and Northeastern Mandarin, capturing authentic nuances and expressions. Variants like Qwen3-TTS-Flash offer up to 49 high-quality voices across various gender, age, and language combinations, with robustness to noisy inputs, special symbols, pinyin, and rare characters.

Benchmark Performance

According to Alibaba's technical report, Qwen3-TTS shows strong performance in voice cloning, voice design, and long-form audio stability. In comparisons with top closed-source models like MiniMax, ElevenLabs, and GPT-4o, the open-source models demonstrate competitive maturity for complex tasks.

Key benchmark highlights include:

• Voice Cloning: On a 10-language TTS test set, Qwen3-TTS achieved a word error rate (WER) of 1.835% (vs. ElevenLabs' 4.470%) and speaker similarity of 0.789 (vs. 0.646), indicating fewer errors and more natural output.
• Cross-lingual Cloning: On a challenging 12-language cross-lingual task, it scored 4.418 (lower is better), significantly outperforming the previous open-source SOTA model CosyVoice3 (5.548) and Alibaba's own multimodal model Qwen3-Omni-30B-A3B.
• Voice Design: On the InstructTTS-Eval benchmark, the model scored 84.1/81.8 in instruction understanding and execution, beating the closed-source MiniMax-Voice-Design (82.3/81.6) in both instruction following and expressiveness.
• Custom Voice: In single-speaker multilingual generalization tests, Qwen3-TTS achieved an average WER of just 2.34, outperforming GPT-4o (3.02) and Gemini-pro.

Additionally, the model shows breakthrough stability in long-form audio generation. Under stress tests generating 10 minutes of continuous speech, Chinese and English word error rates were held to 2.36% and 2.81% respectively, with speaker similarity reaching 0.95 thanks to a high-fidelity tokenizer, ensuring coherence for audiobooks or speeches.

Technical Architecture

At its core, Qwen3-TTS uses a general end-to-end architecture combined with a discrete multicodebook language model (LM) for holistic speech modeling, avoiding flaws of traditional cascaded systems. The proprietary Qwen3-TTS-Tokenizer-12Hz multicodebook speech encoder compresses audio signals efficiently while preserving paralinguistic details and environmental acoustics. A lightweight non-DiT (non-diffusion Transformer) architecture enables high-speed reconstruction, while a dual-track hybrid streaming system supports both streaming and non-streaming modes for flexibility.

Open Source Availability

All five Qwen3-TTS models are open-sourced under the Apache 2.0 license, allowing free commercial use. The largest model file is 4.52 GB, making it accessible for developers with GPUs to start experimenting immediately.