Skip to main content

Optimization techniques

Optimization techniques to OpenVINO GenAI.

📄️Speculative Decoding

Note: This page is a work in progress.

📄️KVCache Token Eviction Algorithm

Overview

📄️Sparse Attention prefill algorithms

Overview

📄️Continuous Batching

Note: This page is a work in progress.

📄️Prefix Caching

Note: This page is a work in progress.

📄️Diffusion Caching (TaylorSeer Lite)

Overview

📄️Visual Token Pruning (CDPruner)

Overview