Skip to main content

Debug Logging

There are six log levels, which can be called explicitly or set via the OPENVINO_LOG_LEVEL environment variable: https://github.com/openvinotoolkit/openvino/blob/f35beb7f1355cb3f7f73b0d431933afbb6a8d3c6/src/inference/include/openvino/runtime/properties.hpp#L640

  • -1 - ov::log::Level::NO
  • 0 - ov::log::Level::ERR
  • 1 - ov::log::Level::WARNING
  • 2 - ov::log::Level::INFO
  • 3 - ov::log::Level::DEBUG
  • 4 - ov::log::Level::TRACE

When setting the environment variable OPENVINO_LOG_LEVEL > ov::log::Level::WARNING, the additional information will be printed.

export OPENVINO_LOG_LEVEL=3

After pipeline initialization and reading the model, the properties of the compiled model will be printed to the console.

Output:
NETWORK_NAME: Model0
OPTIMAL_NUMBER_OF_INFER_REQUESTS: 1
NUM_STREAMS: 1
INFERENCE_NUM_THREADS: 48
PERF_COUNT: NO
INFERENCE_PRECISION_HINT: bf16
PERFORMANCE_HINT: LATENCY
EXECUTION_MODE_HINT: PERFORMANCE
PERFORMANCE_HINT_NUM_REQUESTS: 0
ENABLE_CPU_PINNING: YES
SCHEDULING_CORE_TYPE: ANY_CORE
MODEL_DISTRIBUTION_POLICY:
ENABLE_HYPER_THREADING: NO
EXECUTION_DEVICES: CPU
CPU_DENORMALS_OPTIMIZATION: NO
LOG_LEVEL: LOG_NONE
CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE: 1
DYNAMIC_QUANTIZATION_GROUP_SIZE: 32
KV_CACHE_PRECISION: f16
AFFINITY: CORE
EXECUTION_DEVICES:
CPU: Intel(R) Xeon(R) Platinum 8468

When a Speculative Decoding or Prompt Lookup pipeline is executed, performance metrics will also be printed.

Output:
===============================
Generated tokens: 100
Total duration, sec: 26.6217
Draft model duration, sec: 1.60329
Main model duration, sec: 25.0184
Draft model duration, %: 6.02248
Main model duration, %: 93.9775
Token per sec: 3.75633
AVG acceptance rate, %: 21.6809
===============================
REQUEST_ID: 0
Main model iterations: 47
AVG acceptance rate, %: 21.6809
Generated tokens by draft model: 100
Accepted token rate, %: 51
===============================
Request_id: 0 ||| 40 0 40 20 0 0 40 40 0 20 20 20 0 40 0 0 20 80 0 80 20 0 0 0 40 80 0 40 60 40 80 0 0 0 0 40 20 20 0 40 20 40 0 20 0 0 0

For user-facing speculative decoding metrics, prefer SDPerModelsPerfMetrics from extended_perf_metrics. It exposes accepted, rejected, and total draft candidate tokens, plus get_draft_acceptance_rate(). The draft model's generated-token counter reports draft-model processing work and can include non-candidate work such as cache or state alignment.

When a GGUF model is passed to the pipeline, the detailed debug info will also be printed.

Output:
[GGUF Reader]: Loading and unpacking model from: gguf_models/qwen2.5-0.5b-instruct-q4_0.gguf
[GGUF Reader]: Loading and unpacking model done. Time: 196ms
[GGUF Reader]: Start generating OpenVINO model...
[GGUF Reader]: Save generated OpenVINO model to: gguf_models/openvino_model.xml done. Time: 466 ms
[GGUF Reader]: Model generation done. Time: 757ms

When setting the environment variable OPENVINO_LOG_LEVEL >= ov::log::Level::INFO, the information about which backend is used (PA/SDPA) will be displayed.

For example, after PagedAttention transformation is applied successfully, the following message is printed to the console in case of LLMPipeline or VLMPipeline:

Output:
[INFO] PA backend is enabled.