mmaction#

OTX Algorithms - Action Classification Dataset.

class otx.algorithms.action.adapters.mmaction.data.OTXActionClsDataset(otx_dataset: DatasetEntity, labels: List[LabelEntity], pipeline: Sequence[dict], test_mode: bool = False, modality: str = 'RGB')#

Wrapper that allows using a OTX dataset to train mmaction models.

This wrapper is not based on the filesystem, but instead loads the items here directly from the OTX DatasetEntity object.

prepare_test_frames(idx: int) → Dict[str, Any]#

Get testing data after pipeline.

Args:: idx (int): Index of data

prepare_train_frames(idx: int) → Dict[str, Any]#

Get training data and annotations after pipeline.

Args:: idx (int): Index of data

class otx.algorithms.action.adapters.mmaction.data.OTXActionDetDataset(otx_dataset: DatasetEntity, labels: List[LabelEntity], pipeline: Sequence[dict], test_mode: bool = False, person_det_score_thr: float = 0.9, num_max_proposals: int = 1000, modality: str = 'RGB', fps: int = 30)#

Wrapper that allows using a OTX dataset to train action detection models.

This wrapper is not based on the filesystem, but instead loads the items here directly from the OTX DatasetEntity object. It is adapted from AVADataset of mmaction, but it supports other dataset such as UCF and JHMDB.

evaluate(results: List[List[ndarray]], metrics: Tuple[str] = ('mAP',), logger: Optional[Logger] = None, **kwargs)#: Evaluate the prediction results and report mAP.

prepare_test_frames(idx: int) → Dict[str, Any]#

Get testing data after pipeline.

Args:: idx (int): Index of data

prepare_train_frames(idx: int) → Dict[str, Any]#

Get training data and annotations after pipeline.

Args:: idx (int): Index of data

class otx.algorithms.action.adapters.mmaction.data.RawFrameDecode#

Load and decode frames with given indices.

otx_dataset: DatasetEntity#

OTX Adapters for action recognition models - mmaction2.

class otx.algorithms.action.adapters.mmaction.models.AVAFastRCNN(backbone, roi_head, train_cfg, test_cfg, neck=None, pretrained=None)#

Implementation of Fast R-CNN for Action Detection.

Add forward_infer function for inference without pre-proposals

forward_infer(self, imgs, img_metas)#: Forward function for inference without pre-proposal.

patch_for_export()#: Patch mmdetection’s FastRCNN for exporting to onnx.

training: bool#

class otx.algorithms.action.adapters.mmaction.models.AVARoIHead(bbox_roi_extractor=None, bbox_head=None, mask_roi_extractor=None, mask_head=None, shared_head=None, train_cfg=None, test_cfg=None, pretrained=None, init_cfg=None)#

AVARoIHead for OTX.

simple_test(x, proposal_list, img_metas, proposals=None, rescale=False, **kwargs)#: This is temporary soluition, since otx.mmdet is differnt with latest mmdet.

training: bool#

class otx.algorithms.action.adapters.mmaction.models.MoViNetHead(num_classes: int, in_channels: int, hidden_dim: int, loss_cls: dict, tf_like: bool = False, conv_type: str = '3d')#

Classification head for MoViNet.

Args:: num_classes (int): Number of classes to be classified. in_channels (int): Number of channels in input feature. hidden_dim (int): Number of channels in hidden layer. tf_like (bool): If True, uses TensorFlow-style padding. Default: False. conv_type (str): Type of convolutional layer. Default: ‘3d’. loss_cls (dict): Config for building loss. Default: dict(type=’CrossEntropyLoss’). spatial_type (str): Pooling type in spatial dimension. Default: ‘avg’. dropout_ratio (float): Probability of dropout layer. Default: 0.5. init_std (float): Standard deviation for initialization. Default: 0.1.

forward(x)#

Defines the computation performed at every call.

Args:: x (torch.Tensor): The input data.
Returns:: torch.Tensor: The classification scores for input samples.

init_weights()#: Initialize the parameters from scratch.

training: bool#

class otx.algorithms.action.adapters.mmaction.models.MoViNetRecognizer(**kwargs)#

MoViNet recognizer model framework for OTX compatibility.

static load_state_dict_pre_hook(module, state_dict, prefix, *args, **kwargs)#: Redirect input state_dict to model for OTX model compatibility.

static state_dict_hook(module, state_dict, *args, **kwargs)#: Redirect model as output state_dict for OTX MoviNet compatibility.

training: bool#

class otx.algorithms.action.adapters.mmaction.models.OTXMoViNet(**kwargs)#

MoViNet wrapper class for OTX.

static fill_conv(conf, input_channels, out_channels, kernel_size, stride, padding)#

Set the values of a given Config object to conv layer.

Args:: conf (Config): The Config object to be updated. input_channels (int): The number of input channels. out_channels (int): The number of output channels. kernel_size (tuple[int]): The size of the kernel. stride (tuple[int]): The stride of the kernel. padding (tuple[int]): The padding of the kernel.
Returns:: None.

static fill_se_config(conf, input_channels, out_channels, expanded_channels, kernel_size, stride, padding, padding_avg)#

Set the values of a given Config object to SE module.

Args:: conf (Config): The Config object to be updated. input_channels (int): The number of input channels. out_channels (int): The number of output channels. expanded_channels (int): The number of channels after expansion in the basic block. kernel_size (tuple[int]): The size of the kernel. stride (tuple[int]): The stride of the kernel. padding (tuple[int]): The padding of the kernel. padding_avg (tuple[int]): The padding for the average pooling operation.
Returns:: None.

training: bool#

otx.algorithms.action.adapters.mmaction.models.register_action_backbones()#: Register action backbone to mmdetection backbones.

OTX Adapters - mmaction.utils.

class otx.algorithms.action.adapters.mmaction.utils.Exporter(recipe_cfg: Config, weights: OrderedDict, deploy_cfg: Config, work_dir: str, half_precision: bool)#

Export class for action recognition model using mmdeploy framework.

export()#: Export action model using mmdeploy apis.

otx.algorithms.action.adapters.mmaction.utils.det_eval(predictions, result_type, labels, video_infos, exclude_file, verbose=True, custom_classes=None)#: Evaluation method for AVA Dataset.

otx.algorithms.action.adapters.mmaction.utils.patch_config(config: Config, data_pipeline_path: str, work_dir: str, task_type: TaskType)#: Patch recipe config suitable to mmaction.

otx.algorithms.action.adapters.mmaction.utils.prepare_for_training(config: Union[Config, ConfigDict], train_dataset: DatasetEntity, val_dataset: DatasetEntity, time_monitor: TimeMonitorCallback, learning_curves: defaultdict) → Config#: Prepare configs for training phase.

otx.algorithms.action.adapters.mmaction.utils.set_data_classes(config: Config, labels: List[LabelEntity], task_type: TaskType)#: Setter data classes into config.