Automatic Speech Recognition

Convert and Optimize Model

Download and convert an automatic speech recognition model (e.g. openai/whisper-base) to OpenVINO format from Hugging Face:

Default
INT8 Static Quantization

optimum-cli export openvino --model openai/whisper-base whisper_ov

optimum-cli export openvino --model openai/whisper-base --quant-mode int8 --dataset librispeech --num-samples 32 whisper_ov_int8

See all supported Speech Recognition Models.

info

Refer to the Model Preparation guide for detailed instructions on how to download, convert and optimize models for OpenVINO GenAI.

Run Model Using OpenVINO GenAI

OpenVINO GenAI introduces the ASRPipeline pipeline for automatic speech recognition models (ASR). You can construct it straight away from the folder with the converted model. It will automatically load the model, tokenizer, detokenizer and default generation configuration.

info

ASRPipeline expects normalized audio files in WAV format at sampling rate of 16 kHz as input.

Python
C++
JavaScript

import openvino_genai as ov_genai
import librosa

def read_wav(filepath):
  raw_speech, samplerate = librosa.load(filepath, sr=16000)
  return raw_speech.tolist()

raw_speech = read_wav('sample.wav')

pipe = ov_genai.ASRPipeline(model_path, "CPU")
result = pipe.generate(raw_speech, max_new_tokens=100)
print(result)

import openvino_genai as ov_genai
import librosa

def read_wav(filepath):
  raw_speech, samplerate = librosa.load(filepath, sr=16000)
  return raw_speech.tolist()

raw_speech = read_wav('sample.wav')

pipe = ov_genai.ASRPipeline(model_path, "GPU")
result = pipe.generate(raw_speech, max_new_tokens=100)
print(result)

#include "openvino/genai/automatic_speech_recognition/pipeline.hpp"
#include "audio_utils.hpp"
#include <iostream>

int main(int argc, char* argv[]) {
  std::filesystem::path models_path = argv[1];
  std::string wav_file_path = argv[2];

  std::vector<float> raw_speech = utils::audio::read_wav(wav_file_path);

  ov::genai::ASRPipeline pipe(models_path, "CPU");
  auto result = pipe.generate(raw_speech, ov::genai::max_new_tokens(100));
  std::cout << result << std::endl;
}

#include "openvino/genai/automatic_speech_recognition/pipeline.hpp"
#include "audio_utils.hpp"
#include <iostream>

int main(int argc, char* argv[]) {
  std::filesystem::path models_path = argv[1];
  std::string wav_file_path = argv[2];

  std::vector<float> raw_speech = utils::audio::read_wav(wav_file_path);

  ov::genai::ASRPipeline pipe(models_path, "GPU");
  auto result = pipe.generate(raw_speech, ov::genai::max_new_tokens(100));
  std::cout << result << std::endl;
}

import { ASRPipeline } from 'openvino-genai-node';
import { readAudio } from './wav_utils.js';

const rawSpeech = readAudio('sample.wav');

const pipeline = await ASRPipeline(modelPath, "CPU");
const generationConfig = { max_new_tokens: 100 };
const result = await pipeline.generate(rawSpeech, { generationConfig });
console.log(result.texts[0]);

import { ASRPipeline } from 'openvino-genai-node';
import { readAudio } from './wav_utils.js';

const rawSpeech = readAudio('sample.wav');

const pipeline = await ASRPipeline(modelPath, "GPU");
const generationConfig = { max_new_tokens: 100 };
const result = await pipeline.generate(rawSpeech, { generationConfig });
console.log(result.texts[0]);

tip

Use CPU or GPU as devices without any other code change.

Additional Usage Options

tip

Check out Python, C++, and JavaScript speech recognition samples.

Use Different Generation Parameters

Generation Configuration Workflow

Get the model default config with get_generation_config()
Modify parameters
Apply the updated config using one of the following methods:
- Use set_generation_config(config)
- Pass config directly to generate() (e.g. generate(prompt, config))
- Specify options as inputs in the generate() method (e.g. generate(prompt, max_new_tokens=100))

Basic Generation Configuration

Python
C++
JavaScript

import openvino_genai as ov_genai

pipe = ov_genai.ASRPipeline(model_path, "CPU")

# Get default configuration
config = pipe.get_generation_config()

# Modify parameters
config.max_new_tokens = 100
config.temperature = 0.7
config.top_k = 50
config.top_p = 0.9
config.repetition_penalty = 1.2

# Generate text with custom configuration
result = pipe.generate(raw_speech, config)

int main() {
    ov::genai::ASRPipeline pipe(model_path, "CPU");

    // Get default configuration
    auto config = pipe.get_generation_config();

    // Modify parameters
    config.max_new_tokens = 100;
    config.temperature = 0.7f;
    config.top_k = 50;
    config.top_p = 0.9f;
    config.repetition_penalty = 1.2f;

    // Generate text with custom configuration
    auto result = pipe.generate(raw_speech, config);
}

import { ASRPipeline } from 'openvino-genai-node';

const pipeline = await ASRPipeline(modelPath, "CPU");

// Get default configuration
const config = pipeline.getGenerationConfig();

// Modify parameters
const generationConfig = {
  ...config,
  max_new_tokens: 100,
  temperature: 0.7,
  top_k: 50,
  top_p: 0.9,
  repetition_penalty: 1.2
};

// Generate text with custom configuration
const result = await pipeline.generate(rawSpeech, { generationConfig });

Understanding Basic Generation Parameters

max_new_tokens: The maximum numbers of tokens to generate, excluding the number of tokens in the prompt. max_new_tokens has priority over max_length.
temperature: Controls the level of creativity in AI-generated text:
- Low temperature (e.g. 0.2) leads to more focused and deterministic output, choosing tokens with the highest probability.
- Medium temperature (e.g. 1.0) maintains a balance between creativity and focus, selecting tokens based on their probabilities without significant bias.
- High temperature (e.g. 2.0) makes output more creative and adventurous, increasing the chances of selecting less likely tokens.
top_k: Limits token selection to the k most likely next tokens. Higher values allow more diverse outputs.
top_p: Selects from the smallest set of tokens whose cumulative probability exceeds p. Helps balance diversity and quality.
repetition_penalty: Reduces the likelihood of repeating tokens. Values above 1.0 discourage repetition.

For the full list of generation parameters, refer to the Generation Config API.

Optimizing Generation with Grouped Beam Search

Beam search helps explore multiple possible text completions simultaneously, often leading to higher quality outputs.

Python
C++
JavaScript

import openvino_genai as ov_genai

pipe = ov_genai.ASRPipeline(model_path, "CPU")

# Get default generation config
config = pipe.get_generation_config()

# Modify parameters
config.max_new_tokens = 256
config.num_beams = 15
config.num_beam_groups = 3
config.diversity_penalty = 1.0

# Generate text with custom configuration
result = pipe.generate(raw_speech, config)

int main() {
    ov::genai::ASRPipeline pipe(model_path, "CPU");

    // Get default generation config
    ov::genai::ASRGenerationConfig config = pipe.get_generation_config();

    // Modify parameters
    config.max_new_tokens = 256;
    config.num_beams = 15;
    config.num_beam_groups = 3;
    config.diversity_penalty = 1.0f;

    // Generate text with custom configuration
    auto result = pipe.generate(raw_speech, config);
}

const pipeline = await ASRPipeline(modelPath, "CPU");

// Get default generation config
const config = pipeline.getGenerationConfig();

// Modify parameters
const generationConfig = {
  ...config,
  max_new_tokens: 256,
  num_beams: 15,
  num_beam_groups: 3,
  diversity_penalty: 1.0
};

// Generate text with custom configuration
const result = await pipeline.generate(rawSpeech, { generationConfig });

Understanding Beam Search Generation Parameters

max_new_tokens: The maximum numbers of tokens to generate, excluding the number of tokens in the prompt. max_new_tokens has priority over max_length.
num_beams: The number of beams for beam search. 1 disables beam search.
num_beam_groups: The number of groups to divide num_beams into in order to ensure diversity among different groups of beams.
diversity_penalty: value is subtracted from a beam's score if it generates the same token as any beam from other group at a particular time.

For the full list of generation parameters, refer to the Generation Config API.

note

Beam search support depends on the selected model architecture. Qwen3-ASR models do not support beam search decoding.

Transcription

Language handling depends on the model architecture. Models with language identification support can automatically detect the language of the input audio. You can also specify the language to improve accuracy. The detected or specified language is available in the result via the languages field.

Python
C++
JavaScript

pipe = ov_genai.ASRPipeline(model_path, "CPU")

# Automatic language detection for models with language identification support
raw_speech = read_wav("speech_sample.wav")
result = pipe.generate(raw_speech)
print(result.languages[0])  # e.g. "en", "fr", "de"

# Explicitly specify language. In the form of `en`, `<|en|>` for Whisper and `English` for Qwen3-ASR
result = pipe.generate(raw_speech, language="en")
print(result.languages[0])  # "en"

# French speech sample
raw_speech = read_wav("french_sample.wav")
result = pipe.generate(raw_speech, language="<|fr|>")
print(result.languages[0])  # "fr"

int main() {
    ov::genai::ASRPipeline pipe(model_path, "CPU");

    // Automatic language detection for models with language identification support
    auto result = pipe.generate(raw_speech);
    std::cout << result.languages[0] << "\n";  // e.g. "en", "fr", "de"

    // Explicitly specify language. In the form of `en`, `<|en|>` for Whisper and `English` for Qwen3-ASR
    result = pipe.generate(raw_speech, ov::genai::language("<|en|>"));
    std::cout << result.languages[0] << "\n";  // "en"

    // French speech sample
    raw_speech = utils::audio::read_wav("french_sample.wav");
    result = pipe.generate(raw_speech, ov::genai::language("<|fr|>"));
    std::cout << result.languages[0] << "\n";  // "fr"
}

const pipeline = await ASRPipeline(modelPath, "CPU");

// Automatic language detection for models with language identification support
let result = await pipeline.generate(rawSpeech);

// Explicitly specify language (English)
let generationConfig = { language: "<|en|>" };
result = await pipeline.generate(rawSpeech, { generationConfig });

// French speech sample
generationConfig = { language: "<|fr|>" };
result = await pipeline.generate(frenchRawSpeech, { generationConfig });

SenseVoiceSmall-specific Options

info

These features are specific to SenseVoiceSmall used with ASRPipeline. They do not apply to other ASR model architectures.

note

SenseVoiceSmall supports automatic language identification (language="auto") and explicit language selection using language codes such as en, zh, yue, ja, and ko.

Inverse Text Normalization

SenseVoiceSmall supports inverse text normalization (ITN) during transcription. By default, use_itn=false selects the woitn mode. Set use_itn=true to select the withitn mode.

Python
C++
JavaScript

pipe = ov_genai.ASRPipeline(model_path, "CPU")

# Example output for the same audio with ITN disabled and enabled:
#   use_itn=False: "...presented him with fifty pieces of gold"
#   use_itn=True:  "...presented him with 50 pieces of gold."
result = pipe.generate(raw_speech, use_itn=True)

ov::genai::ASRPipeline pipe(model_path, "CPU");

// Enable inverse text normalization (default: false)
auto result = pipe.generate(raw_speech, ov::genai::use_itn(true));

const pipeline = await ASRPipeline(modelPath, "CPU");

// Enable inverse text normalization (default: false)
const result = await pipeline.generate(rawSpeech, {
  generationConfig: { use_itn: true },
});

note

SenseVoiceSmall does not currently generate segment-level or word-level timestamps. return_timestamps and word_timestamps are accepted but ignored, and chunks and words remain unset. Normal transcription and streaming are unaffected.

For supported languages, word-level timestamps can be generated separately using ASRForcedAligner: first transcribe the audio with SenseVoiceSmall, then pass the audio and transcription to the standalone aligner. See Word-level Timestamps Prediction for details.

Whisper-specific Features

info

These features are specific to Whisper models used with ASRPipeline. They may not apply to other ASR model architectures.

Translation

By default, Whisper performs transcription, keeping the output in the same language as the input. To translate non-English speech to English, use the translate task:

Python
C++
JavaScript

pipe = ov_genai.ASRPipeline(model_path, "CPU")

# Translate French audio to English
raw_speech = read_wav("french_sample.wav")
result = pipe.generate(raw_speech, task="translate")

int main() {
    ov::genai::ASRPipeline pipe(model_path, "CPU");

    // Translate French audio to English
    raw_speech = utils::audio::read_wav("french_sample.wav");
    result = pipe.generate(raw_speech, ov::genai::task("translate"));
}

const pipeline = await ASRPipeline(modelPath, "CPU");

// Translate French audio to English
const generationConfig = { task: "translate" };
const result = await pipeline.generate(rawSpeech, { generationConfig });

Timestamps Prediction

Whisper can predict timestamps for each segment of speech, which is useful for synchronization or creating subtitles:

Python
C++
JavaScript

pipe = ov_genai.ASRPipeline(model_path, "CPU")

# Enable timestamp prediction
result = pipe.generate(raw_speech, return_timestamps=True)

# Print timestamps and text segments
for chunk in result.chunks[0]:
    print(f"timestamps: [{chunk.start_ts:.2f}, {chunk.end_ts:.2f}] text: {chunk.text}")

int main() {
    ov::genai::ASRPipeline pipe(model_path, "CPU");

    // Enable timestamp prediction
    result = pipe.generate(raw_speech, ov::genai::return_timestamps(true));

    // Print timestamps and text segments
    for (const auto& chunk : (*result.chunks)[0]) {
        std::cout << "timestamps: [" << chunk.start_ts << ", " << chunk.end_ts
                  << "] text: " << chunk.text << "\n";
    }
}

const pipeline = await ASRPipeline(modelPath, "CPU");

// Enable timestamp prediction
const generationConfig = { return_timestamps: true, language: "<|en|>", task: "transcribe" };
const result = await pipeline.generate(rawSpeech, { generationConfig });

// Print timestamps and text segments
for (const chunk of result.chunks?.[0] ?? []) {
  console.log(`timestamps: [${chunk.startTs.toFixed(2)}, ${chunk.endTs.toFixed(2)}] text: ${chunk.text}`);
}

Word-level Timestamps Prediction

Whisper can predict timestamps for each word of speech, which provides more granular timing information compared to segment-level timestamps.

Python
C++
JavaScript

# Word timestamps require decomposition of cross-attention decoder SDPA layers,
# so word_timestamps must be passed to the pipeline constructor (not just in generation config)
pipe = openvino_genai.ASRPipeline(model_path, "CPU", word_timestamps=True)

# Enable word-level timestamp prediction
result = pipe.generate(raw_speech, word_timestamps=True)

# Print word-level timestamps
for word in result.words[0]:
    print(f"[{word.start_ts:.2f}, {word.end_ts:.2f}]: {word.text}")

int main() {
    // Word timestamps require decomposition of cross-attention decoder SDPA layers,
    // so word_timestamps must be passed to the pipeline constructor (not just in generation config)
    ov::genai::ASRPipeline pipe(model_path, "CPU", ov::genai::word_timestamps(true));

    // Enable word-level timestamp prediction
    auto result = pipe.generate(raw_speech, ov::genai::word_timestamps(true));

    // Print word-level timestamps
    std::cout << std::fixed << std::setprecision(2);
    for (const auto& word : (*result.words)[0]) {
        std::cout << "[" << word.start_ts << ", " << word.end_ts << "]: " << word.text << "\n";
    }
}

// Word timestamps require decomposition of cross-attention decoder SDPA layers,
// so word_timestamps must be passed to the pipeline constructor (not just in generation config)
const pipeline = await ASRPipeline(modelPath, "CPU", { word_timestamps: true });

// Enable word-level timestamp prediction
const generationConfig = { return_timestamps: true, word_timestamps: true, language: "<|en|>", task: "transcribe" };
const result = await pipeline.generate(rawSpeech, { generationConfig });

// Print word-level timestamps
for (const w of result.words?.[0] ?? []) {
  console.log(`[${w.startTs.toFixed(2)}, ${w.endTs.toFixed(2)}]: ${w.text}`);
}

For Qwen3-ASR word timestamps, OpenVINO GenAI uses the companion Qwen3 Forced Aligner model.

OpenVINO GenAI currently supports forced-alignment timestamps for Chinese, Cantonese, English, German, Spanish, French, Italian, Portuguese, and Russian. The Qwen3 Forced Aligner model also supports Japanese and Korean, but Japanese and Korean segmentation are not currently implemented in OpenVINO GenAI.

Qwen3-ASR and its forced aligner are distributed separately and should be exported to separate directories. Install qwen-asr before export.

pip install qwen-asr

# Required for ASRPipeline transcription with Qwen3-ASR.
optimum-cli export openvino \
    --model Qwen/Qwen3-ASR-0.6B \
    --task automatic-speech-recognition-with-past \
    --trust-remote-code \
    qwen3-asr-ov

# Required for forced alignment (standalone or ASRPipeline word timestamps).
optimum-cli export openvino \
    --model Qwen/Qwen3-ForcedAligner-0.6B \
    --task automatic-speech-recognition-with-past \
    --trust-remote-code \
    qwen3-forced-aligner-ov

Construct the ASRForcedAligner and pass the instance into ASRPipeline. The ASR pipeline and forced aligner may use different devices.

Python
C++

aligner = openvino_genai.ASRForcedAligner(forced_aligner_model_path, "CPU")
pipe = openvino_genai.ASRPipeline(asr_model_path, "GPU", forced_aligner=aligner)

result = pipe.generate(raw_speech, word_timestamps=True, language="English")

for word in result.words[0]:
    print(f"[{word.start_ts:.2f}, {word.end_ts:.2f}]: {word.text}")

#include "openvino/genai/automatic_speech_recognition/forced_aligner.hpp"
#include "openvino/genai/automatic_speech_recognition/pipeline.hpp"

int main() {
    auto aligner = std::make_shared<ov::genai::ASRForcedAligner>(forced_aligner_model_path, "CPU");
    ov::genai::ASRPipeline pipe(asr_model_path, "GPU", ov::genai::forced_aligner(aligner));

    auto result = pipe.generate(raw_speech, ov::genai::word_timestamps(true), ov::genai::language("English"));

    std::cout << std::fixed << std::setprecision(2);
    for (const auto& word : (*result.words)[0]) {
        std::cout << "[" << word.start_ts << ", " << word.end_ts << "]: " << word.text << "\n";
    }
}

When word timestamps are enabled for Qwen3-ASR, audio is processed in chunks of approximately 180 seconds. This can change chunk boundaries and therefore the transcription compared with plain transcription.

The forced aligner is also available as a standalone ASRForcedAligner class, usable independently of ASRPipeline.

Python
C++

aligner = openvino_genai.ASRForcedAligner(forced_aligner_model_path, "CPU")

words = aligner.align(raw_speech, transcript, language="english")

for word in words:
    print(f"[{word.start_ts:.2f}, {word.end_ts:.2f}]: {word.text}")

#include "openvino/genai/automatic_speech_recognition/forced_aligner.hpp"

int main() {
    ov::genai::ASRForcedAligner aligner(forced_aligner_model_path, "CPU");

    auto words = aligner.align(raw_speech, transcript, ov::genai::language("english"));

    std::cout << std::fixed << std::setprecision(2);
    for (const auto& word : words) {
        std::cout << "[" << word.start_ts << ", " << word.end_ts << "]: " << word.text << "\n";
    }
}

Long-Form Audio Processing

Whisper models are designed for audio segments up to 30 seconds in length. For longer audio, the OpenVINO GenAI ASR pipeline automatically handles Whisper processing using a sequential chunking algorithm ("sliding window"):

The audio is divided into 30-second segments
Each segment is processed sequentially
Results are combined to produce the complete transcription

This happens automatically when you input longer audio files.

Using Initial Prompts and Hotwords

You can improve transcription quality and guide the model's output style by providing initial prompts or hotwords using the following parameters:

initial_prompt: initial prompt tokens passed as a previous transcription (after <|startofprev|> token) to the first processing window.
hotwords: hotwords tokens passed as a previous transcription (after <|startofprev|> token) to the all processing windows.

Whisper models can use that context to better understand the speech and maintain a consistent writing style. However, prompts do not need to be genuine transcripts from prior audio segments. Such prompts can be used to steer the model to use particular spellings or styles:

Python
C++
JavaScript

pipe = ov_genai.ASRPipeline(model_path, "CPU")

result = pipe.generate(raw_speech)
# He has gone and gone for good answered Paul Icrom who...

result = pipe.generate(raw_speech, initial_prompt="Polychrome")
# He has gone and gone for good answered Polychrome who...

int main() {
    ov::genai::ASRPipeline pipe(model_path, "CPU");

    auto result = pipe.generate(raw_speech);
    // He has gone and gone for good answered Paul Icrom who...

    result = pipe.generate(raw_speech, ov::genai::initial_prompt("Polychrome"));
    // He has gone and gone for good answered Polychrome who...
}

const pipeline = await ASRPipeline(modelPath, "CPU");

let result = await pipeline.generate(rawSpeech);
// He has gone and gone for good answered Paul Icrom who...

const generationConfig = { initial_prompt: "Polychrome" };
result = await pipeline.generate(rawSpeech, { generationConfig });
// He has gone and gone for good answered Polychrome who...

info

For the ASR API, refer to the ASR Generation Config API.

Performance Metrics

ASRPipeline returns ASRDecodedResults with a perf_metrics field of type ASRPerfMetrics, which extends PerfMetrics.

In addition to common metrics (TTFT, TPOT, throughput, etc.), ASRPerfMetrics provides:

Metric	Type	Description
`get_features_extraction_duration()`	`MeanStdPair`, ms	Time spent extracting audio features (mel spectrogram)
`get_encode_inference_duration()`	`MeanStdPair`, ms	Encoder model inference duration
`get_decode_inference_duration()`	`MeanStdPair`, ms	Decoder model inference duration (all token generation steps)
`get_word_level_timestamps_processing_duration()`	`MeanStdPair`, ms	Time spent processing word-level timestamps

Raw values are available in the asr_raw_metrics field of ASRPerfMetrics.

Streaming the Output

Refer to the Streaming guide for more information on streaming the output with OpenVINO GenAI.

Convert and Optimize Model​

Run Model Using OpenVINO GenAI​

Additional Usage Options​

Use Different Generation Parameters​

Generation Configuration Workflow​

Basic Generation Configuration​

Optimizing Generation with Grouped Beam Search​

Transcription​

SenseVoiceSmall-specific Options​

Inverse Text Normalization​

Whisper-specific Features​

Translation​

Timestamps Prediction​

Word-level Timestamps Prediction​

Long-Form Audio Processing​

Using Initial Prompts and Hotwords​

Performance Metrics​

Streaming the Output​

Convert and Optimize Model

Run Model Using OpenVINO GenAI

Additional Usage Options

Use Different Generation Parameters

Generation Configuration Workflow

Basic Generation Configuration

Optimizing Generation with Grouped Beam Search

Transcription

SenseVoiceSmall-specific Options

Inverse Text Normalization

Whisper-specific Features

Translation

Timestamps Prediction

Word-level Timestamps Prediction

Long-Form Audio Processing

Using Initial Prompts and Hotwords

Performance Metrics

Streaming the Output