Skip to main content

Qwen3-Omni Chat Sample

Preview: The Qwen3-Omni API (OmniPipeline and related types) is a preview feature and is subject to change in future releases.

Interactive multimodal chat with Qwen3-Omni models supporting text, image, audio, and video input, and text/speech output.

Description​

Demonstrates ov::genai::OmniPipeline for end-to-end multimodal conversations. Qwen3-Omni models accept text, images, audio, and video inputs, generating text and optionally synthesized speech responses.

Key features:

  • Multi-turn conversations with image and video context
  • Optional speech synthesis (talker mode)
  • Separate generation configs for text decoding and speech output
  • Interactive CLI interface

Prerequisites​

Prepare image files (JPG, PNG), audio files (16kHz mono WAV) and a video file for testing.

This image can be used as a sample image.

Download an example 16kHz mono WAV file:

wget https://storage.openvinotoolkit.org/models_contrib/speech/2021.2/librispeech_s5/how_are_you_doing_today.wav

Download an example video file:

wget https://storage.openvinotoolkit.org/repositories/openvino_notebooks/data/data/video/Coco%20Walking%20in%20Berkeley.mp4

Usage​

omni_chat <MODEL_DIR> <IMAGE_FILE_OR_DIR> <AUDIO_FILE> <VIDEO_FILE>

Parameters:

  • <MODEL_DIR> — Path to exported Qwen3-Omni OpenVINO model directory
  • <IMAGE_FILE_OR_DIR> — Path to input image(s) for visual context
  • <AUDIO_FILE> — Path to input audio file (16kHz mono WAV)
  • <VIDEO_FILE> — Path to an input video file

This sample takes exactly one audio file. The Python sample takes several through a repeatable --audio flag.

Example:

./build/samples/cpp/omni/omni_chat /models/qwen3-omni-ov ./coco.jpg ./audio.wav "./Coco Walking in Berkeley.mp4"

Interactive usage:

  1. Images, video and audio are loaded once at startup and sent with the first question
  2. Type questions and press Enter
  3. Model responds with streaming text; when speech output is enabled, each turn's audio is saved to output_audio_<turn>.wav
  4. Continue the conversation across multiple turns
  5. Press Ctrl+D to exit

Model Support​

Compatible with Qwen3-Omni models exported to OpenVINO format.

Export using Optimum Intel CLI or Python API.

Configuration​

Text Generation​

GenerationConfig controls text decoding (the "thinker" phase):

ov::genai::GenerationConfig text_config;
text_config.max_new_tokens = 256;

Speech Synthesis​

OmniTalkerSpeechConfig controls talker and speech output:

ov::genai::OmniTalkerSpeechConfig talker_speech_config(models_path);
talker_speech_config.return_audio = true;  // Enable speech synthesis
talker_speech_config.speaker = "Cherry";   // Select voice (optional)

Available voices vary by checkpoint. MoE models typically expose: "Ethan", "Chelsie", "Aiden", "Cherry". Check talker_config.speaker_id in the model's config.json for the full list. Leaving speaker empty selects the default voice.

Speech output: 24kHz mono PCM samples returned in OmniDecodedResults.speech_result.waveforms and saved to output_audio_<turn>.wav for each turn.

GPU Inference​

Important: Qwen3-Omni speech synthesis requires FP32 precision on GPU. FP16 causes numerical drift → codec token corruption → distorted audio. The pipeline enforces INFERENCE_PRECISION_HINT=f32 automatically for GPU devices. Change the device argument in OmniPipeline constructor:

ov::genai::OmniPipeline pipe(models_path, "GPU");

MoE models (30B-A3B) may exceed available GPU memory. Use CPU for MoE inference.

Speaker API Features​

The main sample demonstrates speaker enumeration and voice blending at startup:

  1. List speakers — shows all named voices in the model
  2. Voice blending — when 2+ speakers are available, creates a 50/50 blend and uses it for generation

Custom blending: Retrieve speaker embeddings via pipe.get_talker()->get_speaker_embedding(name), mix with custom ratios (e.g., 0.75A + 0.25B), and pass via OmniTalkerSpeechConfig::speaker.

See Also​