AuK

What does AuK do?

AuK is "an open-source foundational model for speech generation and editing" from Tencent Hunyuan, described as "a 1.5B foundation model" trained "on millions of hours of diverse audio data." Its distinguishing idea is that every task goes through the same natural-language instruction interface: you describe what you want done to or from the audio.

The README lists 16 tasks: zero-shot text to speech, instruct text to speech, speech content editing, lyric editing, pitch, speed and volume editing, emotion editing, timbre editing, de-accenting, non-verbal editing, whisper conversion, speech enhancement, speech separation, music separation and target speaker extraction. Two variants ship: AuK, the base model, and AuK-Flash, "distilled model for fast 4-step inference." The examples are in English and Chinese; a language list is not stated. It is the baseline for the single-model track of the ICASSP 2027 Audio Editing Challenge.

Key facts

• Licence: MIT on code and weights. The LICENSE file says the grant "refers solely to training code, inference code, parameters, and weights made publicly available by Tencent in accordance with MIT"; GitHub reports NOASSERTION only because of the Tencent preamble.
• 1.5 billion parameters; two variants, AuK and AuK-Flash (fixed four inference steps, no classifier-free guidance).
• 16 tasks through one instruction interface, from zero-shot text to speech to music separation and target speaker extraction.
• Peak GPU memory on an A800 80 GB in bf16: 24.78 GiB without CPU offload, 16.75 GiB with it, for a text-only prompt and 1.5 seconds of output, per the README.
• Needs Qwen/Qwen2.5-Omni-3B as its encoder, downloaded separately; Qwen3-Omni is not currently supported.
• Fine-tuning pipeline with JSONL data, a Gradio demo, ComfyUI nodes, a Python API, and an MLX branch for Apple Silicon.
• Install: clone, then uv or Conda with Python 3.10, pip install -e . with optional gradio, comfyui and train extras; weights via hf download tencent/AuK and tencent/AuK-Flash.
• No tagged GitHub release when read; 1,082 stars on 2026-09-17; free demo spaces on Hugging Face and ModelScope.

What does it replace, and where does it fall short?

AuK replaces ElevenLabs for a team that wants to generate speech and edit existing recordings under one permissive licence: change what a speaker said, shift emotion or pace, remove an accent, clean up noise, or pull one voice out of a mix. That editing range is wider than any other entry on the open-source alternatives to ElevenLabs page, and both code and weights are MIT.

Where it falls short: it is a research release for people with an NVIDIA GPU, with no packaged app, no tagged release and no streaming API in the README. Memory use is measured on an 80 GB A800. The encoder it depends on, Qwen2.5-Omni-3B, has its own licence to read. For a hosted-style endpoint on a CPU, Kokoro FastAPI is the simpler pick.

How does AuK run?

Clone the repository, create a Python 3.10 environment with uv or Conda, and pip install -e . with the extras you want: gradio for the web demo, comfyui for the nodes, train for fine-tuning. Download the weights with hf download for tencent/AuK, tencent/AuK-Flash and Qwen/Qwen2.5-Omni-3B. Inference runs on a CUDA GPU, with a --cpu_offload flag to cut VRAM; a feat/mlx-apple-silicon branch runs on Apple Silicon. The prompt enhancer needs an OpenAI-compatible language model endpoint. Nobody sells a hosted AuK; the demos on Hugging Face and ModelScope are free.

Who is AuK for?

A team building speech editing or restoration into a product and wanting MIT weights it can ship, with GPUs to run them. A team that only needs text to speech with preset voices on modest hardware should look at Kokoro FastAPI or Piper.

What limits does the README state?

From the README: "Qwen3-Omni is not currently supported by --qwen_path"; the ComfyUI integration has a 30-second source-plus-target sequence limit; and actual memory use "depends on input length, dtype, hardware, and software versions."

Questions people ask

Is AuK open source?

Yes, code and weights. The LICENSE file wraps the MIT text in a Tencent preamble and states that the MIT grant covers the training code, inference code, parameters and weights Tencent published. The Hugging Face card repeats that AuK is released under MIT. GitHub shows NOASSERTION only because of the preamble.

Can AuK be used commercially?

Yes, under MIT, for both the code and the weights. One dependency to check separately: AuK uses Qwen2.5-Omni-3B as its encoder, which is downloaded from its own repository under its own licence, and the README does not state that licence.

What hardware does AuK need?

An NVIDIA GPU. The README measures peak memory on an A800 80 GB in bf16 at 24.78 GiB without CPU offload and 16.75 GiB with it, for a short text-only generation, and notes that real usage depends on input length and settings. Apple Silicon runs through a separate MLX branch; a CPU-only path is not described.

How does AuK compare with ElevenLabs?

AuK edits speech as well as generating it, under MIT, on your own GPU. ElevenLabs is a hosted service with a large voice library and no hardware to run. AuK has no packaged app or streaming endpoint in its README. The open-source alternatives to ElevenLabs page sets it beside Chatterbox, Kokoro and the others by need.

What can AuK edit in an existing recording?

Per the README's task list: the spoken content and lyrics, pitch, speed and volume, emotion, timbre, accent, non-verbal sounds, and conversion to a whisper. It also enhances noisy speech, separates speech from music, and extracts a target speaker from a mix, all through the same instruction interface.

Sources

• AuK README and LICENSE: github.com/Tencent-Hunyuan/AuK, read 2026-09-17.
• AuK model card, MIT: huggingface.co/tencent/AuK, read 2026-09-17.