nncf.torch.knowledge_distillation.algo#

Contains builder and controller class definitions for the knowledge distillation.

Classes#

KnowledgeDistillationController

Controller for the knowledge distillation in PT.
class nncf.torch.knowledge_distillation.algo.KnowledgeDistillationController(target_model, original_model, kd_type, scale, temperature)[source]#

Bases: nncf.torch.compression_method_api.PTCompressionAlgorithmController

Controller for the knowledge distillation in PT.

Parameters:

  • target_model (nncf.torch.nncf_network.NNCFNetwork) –

  • original_model (torch.nn.Module) –

  • kd_type (str) –

  • scale (float) –

  • temperature (float) –

property scheduler: nncf.api.compression.CompressionScheduler[source]#

The compression scheduler for this particular algorithm combination.

Return type:

nncf.api.compression.CompressionScheduler

property loss: nncf.api.compression.CompressionLoss[source]#

The compression loss for this particular algorithm combination.

Return type:

nncf.api.compression.CompressionLoss

compression_stage()[source]#

Returns level of compression. Should be used on saving best checkpoints to distinguish between uncompressed, partially compressed and fully compressed models.

Return type:

nncf.api.compression.CompressionStage

statistics(quickly_collected_only=False)[source]#

Returns a Statistics class instance that contains compression algorithm statistics.

Parameters:

quickly_collected_only (bool) – Enables collection of the statistics that don’t take too much time to compute. Can be helpful for the case when need to keep track of statistics on each training batch/step/iteration.

Return type:

nncf.common.statistics.NNCFStatistics

distributed()[source]#

Should be called when distributed training with multiple training processes is going to be used (i.e. after the model is wrapped with DistributedDataParallel). Any special preparations for the algorithm to properly support distributed training should be made inside this function.