OpenVINO Toolkit operator 1.2.0

Version 1.2.0 is a minor upgrade of operator 1.1.0.

The ModelServer CRD includes now new fields to pass to the deployed containers extra environment parameters which can be helpful for tunning the behavior of the MediaPipe calculators from the served graphs:

deployment_parameters.extra_envs_secret : Secret name including extra environment variables to be applied in the deployed pods
deployment_parameters.extra_envs_configmap : Configmap name including extra environment variables to be applied in the deployed pods

There is also simplified GUI form for enabling the metrics in a consistent manner for single model and multi-model deployments.

We also update the version of included component dependencies with security fixes.

The default image of the model server and the version of notebook git repository are refreshed to v2024.1

OpenVINO Toolkit operator 1.1.0

Version 1.1.0 is a minor upgrade of operator 1.0.0.

It includes minor changes to example and default values of some CRD fields as well as some bug fixes.

It also enables users to configure metrics for single model mode from the CRD level.

ModelServer changes:

• Added support for metrics configuration for single model mode. Previously metrics collection could be turned on only in the model server configuration file. New monitoring fields have been added to ModelServer CRD to enable setting metrics for deployments without the configuration file.

• Updated default OVMS image to 2022.3 - the current latest available.

• Fixed handling value 0 for file_system_poll_wait_seconds and sequence_cleaner_poll_wait_minutes parameters. Previously they were ineffective. Now providing 0 correctly disabling periodic model repository updates and stateful models sequence cleanup routines.

• Changed the model server sample and the default values of model_path, plugin_config and file_system_poll_wait_seconds.

  • model_path is now generic GS path: gs://<bucket_name>/<model_dir>. Users need to specify their own path for the model server to successfully load the model.

  • plugin_config has been changed to {"PERFORMANCE_HINT":"LATENCY"} to match latest OpenVINO guides for latency-oriented setup.

  • file_system_poll_wait_seconds has been changed from 0 to 1.

• Fixed typo in service type drop-down list option. Previously, in OCP form we had "Loadbalancer" available as a service type, which is wrong. Changed to "LoadBalancer".

• Fixed handling update_strategy deployment parameter.

• Fixed model server sample manifests. Changed model_config to config_path fields for multi model mode. model_config was wrong, and it caused deployment failure.

• Changed liveness and readiness. Server is now alive when gRPC service is running and ready when all desired models are loaded. It is in line with the OpenVINO Model Server 2022.3 API changes.

• Added metrics-based scaling with HPA example along with Prometheus setup.

Notebook changes:

• Fix BuildConfig labels for RHODS integration. Added opendatahub.io/build_type: notebook_image label, so that RHODS dashboard gets notified about the build status.

• Fix ImageStream annotations for RHODS integration. Removed opendatahub.io/notebook-python-dependencies annotation as we cannot guarantee dependencies versions and modified opendatahub.io/notebook-software annotation to drop dot version for Python (v3.8.6 -> v3.8).

OpenVINO Toolkit operator 1.0.0

Version 1.0.0 is a major upgrade of the operator 0.2 hosted previously in a github repository. It has non-compatible changes in the CRD records of ModelServer and Notebook. It is recommended to remove all those custom resources before upgrading the operator from v0.2 to v1.0.

Key features:

• Managing OpenVINO Model Server instance in OpenShift and Kubernetes
• GUI interface for creating new model servers in OpenShift console
• Auto scalability
• integration of the OpenVINO notebook image with the JupyterHub Spawner in Red Hat OpenShift Data Science
• updated helm chart which can be used directly also without the operator