Export to OPENVINO

[glucasol]

I would like to know if the models exported to Openvino IR are yet optimized. Or if we can improve the performance/speed of the inference using the exported model.

The inference using torch (with .ckpt model) should be slower than the inference using openvino (with .bin model) ?

Thanks!

[djdameln]

Models are converted to OpenVINO IR using the Model Optimizer, which applies several default optimizations out of the box. These optimizations are meant to speed up computations on all intel hardware.

In addition to these default optimizations, the Model Optimizer also supports several optional optimizations for more advanced use cases or specific hardware configurations, such as FP16 compression. Please note that Anomalib's export functionality currently only supports the default optimizations (supporting the full range of MO options is a WIP).

Whether the OpenVINO model runs faster than the Torch model depends to a large extent on the device on which the models are deployed. When running both models on the same device (e.g. CPU), the applied optimizations will generally allow the OpenVINO model to run faster than the Torch model. However, an OpenVINO model running on CPU may not be able to beat a Torch model deployed on GPU, due to the computational advantages of GPU operations.

[glucasol]

Thanks a lot!

[glucasol]

Do you know if I can generate an annotation file for an anomaly segmentation model using openvino convert_annotation command? if so witch converter should I use?