Skip to main content

Convert Models to OpenVINO Format

This page explains how to convert various generative AI models from Hugging Face and ModelScope to OpenVINO IR format. Refer to the Supported Models for a list of available models.

For downloading pre-converted models, see Download Pre-Converted OpenVINO Models.

Converting Models from Hugging Face​

  1. Install optimum-intel package to download, convert and optimize models:

    Note

    Some models may require additional dependencies. To convert models with optimum-cli and to run the examples, install the dependencies from ./samples/export-requirements.txt:

    pip install --requirement ./samples/export-requirements.txt

  2. Download and convert a model to the OpenVINO IR format using optimum-cli tool from Hugging Face:

    optimum-cli export openvino --model <model_id_or_path> <output_dir>
    tip

    For better performance with minimal accuracy impact, convert the model to lower precision by using --weight-format argument:

    optimum-cli export openvino --model <model_id_or_path> --weight-format int4 <output_dir>
    info

    The --trust-remote-code flag is required for some models that use custom code.

    Check a full list of conversion options here.

Converting Models from ModelScope​

ModelScope models need to be downloaded first, then converted to OpenVINO IR format.

  1. Install modelscope and optimum-intel packages to download, convert and optimize models: 
    pip install modelscope --requirement ./samples/export-requirements.txt
  2. Download the required model (e.g. Qwen/Qwen2-7b) to a local directory using modelscope tool: 
    modelscope download --model 'Qwen/Qwen2-7b' --local_dir <model_path>
  3. Convert the model (and optionally compress weights) using optimum-cli tool: 
    optimum-cli export openvino --model <model_path> --weight-format int4 <output_dir>
info

Refer to the Use Cases for detailed instructions on using models with OpenVINO GenAI.