Skip to content

Commit

Permalink
[MO] Remove Apache MxNet models support in openvino-dev (#26800)
Browse files Browse the repository at this point in the history
**Details:** Due to drop of Python 3.8, MxNet support is removed. MxNet
is archived project and distributed only up to Python 3.8

**Ticket:** 153365

---------

Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
  • Loading branch information
rkazants authored Sep 30, 2024
1 parent f1431da commit c928fde
Show file tree
Hide file tree
Showing 176 changed files with 40 additions and 7,393 deletions.
1 change: 0 additions & 1 deletion .github/ISSUE_TEMPLATE/bug.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,6 @@ body:
- Caffe
- ONNX
- PyTorch
- mxnet
- PaddlePaddle
validations:
required: false
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ jobs:
# For running TensorFlow frontend unit tests
python3 -m pip install -r ${{ github.workspace }}/src/frontends/tensorflow/tests/requirements.txt
# For MO unit tests
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_mxnet.txt
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_caffe.txt
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_kaldi.txt
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_onnx.txt
Expand Down
7 changes: 1 addition & 6 deletions .github/workflows/job_python_unit_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -99,10 +99,6 @@ jobs:
extras_to_install="caffe,kaldi,onnx,tensorflow2,pytorch"
if [[ "${{ runner.arch }}" != "ARM64" ]]; then
extras_to_install="mxnet,$extras_to_install"
fi
# Find and install OV dev wheel
pushd ${INSTALL_DIR}/tools
ov_dev_wheel_name=$(find . -name 'openvino_dev*.whl')
Expand Down Expand Up @@ -142,8 +138,7 @@ jobs:
# Skips under tickets: 133405, 122666
python3 -m pytest -s ${INSTALL_TEST_DIR}/mo/unit_tests \
--junitxml=${INSTALL_TEST_DIR}/TEST-ModelOptimizer.xml \
--ignore-glob="**/mo/unit_tests/mo/front/mxnet/**"
--junitxml=${INSTALL_TEST_DIR}/TEST-ModelOptimizer.xml
- name: Python ONNX operators tests
if: (fromJSON(inputs.affected-components).Python_API.test ||
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/mo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ jobs:
# For UT
pip install unittest-xml-reporting==3.0.2
# MO requirements
pip install -r requirements_mxnet.txt
pip install -r requirements_caffe.txt
pip install -r requirements_kaldi.txt
pip install -r requirements_onnx.txt
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/windows_vs2019_release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@ jobs:
# Find and install the dev OV wheel
$ovDevWheelPath=Get-ChildItem -Path "${{ env.INSTALL_DIR }}\tools" -Filter openvino_dev*.whl | % { $_.FullName }
python3 -m pip install "$ovDevWheelPath[mxnet,caffe,kaldi,onnx,tensorflow2,pytorch]"
python3 -m pip install "$ovDevWheelPath[caffe,kaldi,onnx,tensorflow2,pytorch]"
- name: Install Python API tests dependencies
run: |
Expand Down Expand Up @@ -316,7 +316,7 @@ jobs:
if: fromJSON(needs.smart_ci.outputs.affected_components).MO.test
shell: cmd
run: |
python3 -m pytest -s ${{ env.INSTALL_TEST_DIR }}/mo/unit_tests --ignore=${{ env.INSTALL_TEST_DIR }}/mo/unit_tests/mo/front/mxnet --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-ModelOptimizer.xml
python3 -m pytest -s ${{ env.INSTALL_TEST_DIR }}/mo/unit_tests --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-ModelOptimizer.xml
- name: Install Python Layer tests dependencies
run: |
Expand Down
2 changes: 1 addition & 1 deletion docs/articles_en/documentation/legacy-features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ offering.
Discontinued:
#############

.. dropdown:: Apache MXNet, Caffe, and Kaldi model formats
.. dropdown:: Caffe, and Kaldi model formats

| *New solution:* conversion to ONNX via external tools
| *Old solution:* model support discontinued with OpenVINO 2024.0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ Q14. What does the message "Cannot infer shape for node {} because there is no C
Q15. What does the message "Framework name can not be deduced from the given options. Use --framework to choose one of Caffe, TensorFlow, MXNet" mean?
######################################################################################################################################################
**A:** You have run Model Optimizer without a flag ``--framework caffe|tf|mxnet``. Model Optimizer tries to deduce the framework by the extension of input model file (``.pb`` for TensorFlow, ``.caffemodel`` for Caffe, ``.params`` for Apache MXNet). Your input model might have a different extension and you need to explicitly set the source framework. For example, use ``--framework caffe``.
**A:** You have run Model Optimizer without a flag ``--framework caffe|tf``. Model Optimizer tries to deduce the framework by the extension of input model file (``.pb`` for TensorFlow, ``.caffemodel`` for Caffe, ``.params`` for Apache MXNet). Your input model might have a different extension and you need to explicitly set the source framework. For example, use ``--framework caffe``.
.. _question-16:
Expand Down
1 change: 0 additions & 1 deletion docs/dev/ov_dependencies.txt
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,6 @@ mccabe
mistune
mongo-python-driver
more-itertools
mxnet
mypy
mypy-extensions
networkx
Expand Down
7 changes: 3 additions & 4 deletions docs/dev/pypi_publish/pypi-openvino-dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,6 @@ pip install openvino-dev[extras]
| :-------------------------------| :------------------------------------------------------------------------------- |
| caffe | [Caffe*](https://caffe.berkeleyvision.org/) |
| kaldi | [Kaldi*](https://github.com/kaldi-asr/kaldi) |
| mxnet | [Apache MXNet*](https://mxnet.apache.org/) |
| onnx | [ONNX*](https://github.com/microsoft/onnxruntime/) |
| pytorch | [PyTorch*](https://pytorch.org/) |
| tensorflow | [TensorFlow* 1.x](https://www.tensorflow.org/versions#tensorflow_1) |
Expand Down Expand Up @@ -149,14 +148,14 @@ Users in China might encounter errors while downloading sources via PIP during O
If you use zsh (Z shell) interpreter, that is the default shell for macOS starting with version 10.15 (Catalina), you may encounter the following error while installing `openvino-dev` package with extras:

```sh
pip install openvino-dev[tensorflow2,mxnet,caffe]
zsh: no matches found: openvino-dev[tensorflow2,mxnet,caffe]
pip install openvino-dev[tensorflow2,caffe]
zsh: no matches found: openvino-dev[tensorflow2,caffe]
```

By default zsh interprets square brackets as an expression for pattern matching. To resolve this issue, you need to escape the command with quotes:

```sh
pip install 'openvino-dev[tensorflow2,mxnet,caffe]'
pip install 'openvino-dev[tensorflow2,caffe]'
```

To avoid such issues you can also disable globbing for PIP commands by defining an alias in `~/.zshrc` file:
Expand Down
9 changes: 0 additions & 9 deletions docs/sphinx_setup/_static/download/supported_models.csv
Original file line number Diff line number Diff line change
Expand Up @@ -211,11 +211,6 @@ facenet,Object Detection,onnx,FP16,,,+
facenet,Object Detection,onnx,FP16-INT8,,,+
facenet-20180408-102900,Object Detection,tf,FP16,+,+,+
facenet-20180408-102900,Object Detection,tf,FP32,+,+,
face-recognition-mobilefacenet-arcface,Object Detection,mxnet,FP16,,,+
face-recognition-mobilefacenet-arcface,Object Detection,mxnet,FP16-INT8,,,+
face-recognition-resnet50-arcface,Object Detection,mxnet,FP16,+,+,+
face-recognition-resnet50-aws,Object Detection,mxnet,FP16,,,+
face-recognition-resnet50-aws,Object Detection,mxnet,FP32,,,+
face-reidentification-retail-0095,Object Detection,onnx,FP16,,,+
face-reidentification-retail-0095,Object Detection,onnx,FP16-INT8,,,+
facial-landmarks-35-adas-0002,Object Detection,caffe,FP16,,,+
Expand Down Expand Up @@ -428,8 +423,6 @@ ocrnet-hrnet-w18,Text Detection,paddle,FP16-INT8,+,+,
ocrnet-hrnet-w18,Text Detection,paddle,FP32,+,,
ocrnet-hrnet-w48,Text Detection,paddle,FP16,+,,
ocrnet-hrnet-w48,Text Detection,paddle,FP32,+,,
octave-resnext-101-0.25,Image Classification,mxnet,FP16,+,+,+
octave-resnext-101-0.25,Image Classification,mxnet,FP32,+,,
openchat-3.6-8b-20240522,Large Language Model,pytorch,intel-optimum default,,+,
open-closed-eye-0001,Image Classification,onnx,FP16,,,+
open-closed-eye-0001,Image Classification,onnx,FP16-INT8,,,+
Expand Down Expand Up @@ -629,8 +622,6 @@ squeezenet1.0-12,Image Classification,onnx,FP16,,,+
squeezenet1.0-12,Image Classification,onnx,FP32,+,+,+
squeezenet1.1-caffe,Image Classification,caffe,FP16,+,+,+
squeezenet1.1-caffe,Image Classification,caffe,FP32,+,+,+
squeezenet1.1-mxnet,Image Classification,mxnet,FP16,+,+,+
squeezenet1.1-mxnet,Image Classification,mxnet,FP32,+,+,+
squeezenet1.1-onnx,Image Classification,onnx,FP32,+,,
srgan-onnx,"Image Processing, Enhancement",onnx,FP16,+,,+
srgan-tf,"Image Processing, Enhancement",tf,FP16,+,+,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def get_precision(self):
return self.__model_prc

def path_to_model(model_path: os.path, prefix: str):
frameworks = {'tf', 'tf2', 'caffe', 'onnx', 'mxnet', 'paddle', 'kaldi'}
frameworks = {'tf', 'tf2', 'caffe', 'onnx', 'paddle', 'kaldi'}
precisions = {'FP16', 'FP32', 'INT8', 'INT1'}
# remove share path + model.xml
model_path = model_path.replace('\n', '')
Expand Down
2 changes: 0 additions & 2 deletions tools/constraints.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@
# files because the version differs between them:
# tensorflow, numpy
h5py>=3.1.0,<3.11.0
mxnet~=1.2.0; sys_platform == 'win32'
mxnet>=1.7.0.post2,<=1.9.1; sys_platform != 'win32'
onnx>=1.8.1,<=1.16.0
networkx<=3.1.0
pytest>=5.0,<8.4
Expand Down
4 changes: 0 additions & 4 deletions tools/mo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,6 @@ E.g. the command below will install dependencies to support ONNX\* and TensorFlo
```
pip install openvino-dev[onnx,tensorflow2]
```
To enable support of MxNet\* models run:
```
pip install openvino-dev[mxnet]
```
To enable support of all frameworks:
```
pip install openvino-dev[all]
Expand Down
112 changes: 0 additions & 112 deletions tools/mo/automation/package_BOM.txt
Original file line number Diff line number Diff line change
Expand Up @@ -252,111 +252,6 @@ openvino/tools/mo/front/LayerNorm.py
openvino/tools/mo/front/Log1p.py
openvino/tools/mo/front/MatMul_normalizer.py
openvino/tools/mo/front/MoveEmbeddedInputsToInputs.py
openvino/tools/mo/front/mxnet/__init__.py
openvino/tools/mo/front/mxnet/activation.py
openvino/tools/mo/front/mxnet/adaptive_avg_pooling_ext.py
openvino/tools/mo/front/mxnet/add_input_data_to_prior_boxes.py
openvino/tools/mo/front/mxnet/arange_ext.py
openvino/tools/mo/front/mxnet/arange_like_ext.py
openvino/tools/mo/front/mxnet/arange_like_replacer.py
openvino/tools/mo/front/mxnet/arange_replacer.py
openvino/tools/mo/front/mxnet/batch_dot_ext.py
openvino/tools/mo/front/mxnet/block_grad_ext.py
openvino/tools/mo/front/mxnet/box_nms_ext.py
openvino/tools/mo/front/mxnet/cast_ext.py
openvino/tools/mo/front/mxnet/check_softmax_node_inputs.py
openvino/tools/mo/front/mxnet/clip_ext.py
openvino/tools/mo/front/mxnet/conv_ext.py
openvino/tools/mo/front/mxnet/copy_ext.py
openvino/tools/mo/front/mxnet/crop_ext.py
openvino/tools/mo/front/mxnet/cumsum.py
openvino/tools/mo/front/mxnet/cumsum_ext.py
openvino/tools/mo/front/mxnet/custom.py
openvino/tools/mo/front/mxnet/custom_rpn_proposal.py
openvino/tools/mo/front/mxnet/deformable_conv_ext.py
openvino/tools/mo/front/mxnet/deformable_psroi_pooling_ext.py
openvino/tools/mo/front/mxnet/div_sqrt_dim.py
openvino/tools/mo/front/mxnet/dropout_ext.py
openvino/tools/mo/front/mxnet/einsum_ext.py
openvino/tools/mo/front/mxnet/elementwise_ext.py
openvino/tools/mo/front/mxnet/eltwise_scalar_replacers.py
openvino/tools/mo/front/mxnet/exp_ext.py
openvino/tools/mo/front/mxnet/expand_dims_ext.py
openvino/tools/mo/front/mxnet/extractor.py
openvino/tools/mo/front/mxnet/extractors/__init__.py
openvino/tools/mo/front/mxnet/extractors/add_n.py
openvino/tools/mo/front/mxnet/extractors/batchnorm.py
openvino/tools/mo/front/mxnet/extractors/concat.py
openvino/tools/mo/front/mxnet/extractors/l2_normalization.py
openvino/tools/mo/front/mxnet/extractors/multibox_prior.py
openvino/tools/mo/front/mxnet/extractors/relu.py
openvino/tools/mo/front/mxnet/extractors/scaleshift.py
openvino/tools/mo/front/mxnet/extractors/slice_axis.py
openvino/tools/mo/front/mxnet/extractors/utils.py
openvino/tools/mo/front/mxnet/eye_ext.py
openvino/tools/mo/front/mxnet/eye_mx_to_eye.py
openvino/tools/mo/front/mxnet/fft_ext.py
openvino/tools/mo/front/mxnet/flatten_ext.py
openvino/tools/mo/front/mxnet/fully_connected_ext.py
openvino/tools/mo/front/mxnet/gather.py
openvino/tools/mo/front/mxnet/gather_ext.py
openvino/tools/mo/front/mxnet/gluoncv_ssd_anchors.py
openvino/tools/mo/front/mxnet/instance_norm_ext.py
openvino/tools/mo/front/mxnet/layer_norm_ext.py
openvino/tools/mo/front/mxnet/leaky_relu.py
openvino/tools/mo/front/mxnet/loader.py
openvino/tools/mo/front/mxnet/lrn_ext.py
openvino/tools/mo/front/mxnet/max_ext.py
openvino/tools/mo/front/mxnet/modulated_deformable_conv_ext.py
openvino/tools/mo/front/mxnet/modulated_deformable_conv_replacer.py
openvino/tools/mo/front/mxnet/multibox_detection_ext.py
openvino/tools/mo/front/mxnet/mx_reshape_reverse.py
openvino/tools/mo/front/mxnet/mx_reshape_to_reshape.py
openvino/tools/mo/front/mxnet/MXFFTToDFT.py
openvino/tools/mo/front/mxnet/MXRepeatReplacer.py
openvino/tools/mo/front/mxnet/nd_to_params.py
openvino/tools/mo/front/mxnet/null_ext.py
openvino/tools/mo/front/mxnet/pad_ext.py
openvino/tools/mo/front/mxnet/pooling_ext.py
openvino/tools/mo/front/mxnet/proposal_ext.py
openvino/tools/mo/front/mxnet/psroi_pooling_ext.py
openvino/tools/mo/front/mxnet/random_uniform_ext.py
openvino/tools/mo/front/mxnet/register_custom_ops.py
openvino/tools/mo/front/mxnet/repeat_ext.py
openvino/tools/mo/front/mxnet/reshape_ext.py
openvino/tools/mo/front/mxnet/RNN_ext.py
openvino/tools/mo/front/mxnet/rnn_param_concat.py
openvino/tools/mo/front/mxnet/roi_pooling_ext.py
openvino/tools/mo/front/mxnet/roll_ext.py
openvino/tools/mo/front/mxnet/shape_array_ext.py
openvino/tools/mo/front/mxnet/sigmoid.py
openvino/tools/mo/front/mxnet/slice_channel_ext.py
openvino/tools/mo/front/mxnet/slice_ext.py
openvino/tools/mo/front/mxnet/slice_like_ext.py
openvino/tools/mo/front/mxnet/slice_replacers.py
openvino/tools/mo/front/mxnet/softmax.py
openvino/tools/mo/front/mxnet/softmax_activation_ext.py
openvino/tools/mo/front/mxnet/softmax_ext.py
openvino/tools/mo/front/mxnet/softmax_output_ext.py
openvino/tools/mo/front/mxnet/softsign_ext.py
openvino/tools/mo/front/mxnet/squeeze_ext.py
openvino/tools/mo/front/mxnet/ssd_anchor_reshape.py
openvino/tools/mo/front/mxnet/ssd_detection_output_replacer.py
openvino/tools/mo/front/mxnet/ssd_pattern_flatten_softmax_activation.py
openvino/tools/mo/front/mxnet/ssd_pattern_remove_flatten.py
openvino/tools/mo/front/mxnet/ssd_pattern_remove_reshape.py
openvino/tools/mo/front/mxnet/ssd_pattern_remove_transpose.py
openvino/tools/mo/front/mxnet/ssd_reorder_detection_out_inputs.py
openvino/tools/mo/front/mxnet/stack_ext.py
openvino/tools/mo/front/mxnet/swapaxis_ext.py
openvino/tools/mo/front/mxnet/take_ext.py
openvino/tools/mo/front/mxnet/tile_ext.py
openvino/tools/mo/front/mxnet/tile_replacer.py
openvino/tools/mo/front/mxnet/transpose_ext.py
openvino/tools/mo/front/mxnet/up_sampling_ext.py
openvino/tools/mo/front/mxnet/where_ext.py
openvino/tools/mo/front/mxnet/yolo_v3_mobilenet1_voc.json
openvino/tools/mo/front/mxnet/zeros_ext.py
openvino/tools/mo/front/no_op_eraser.py
openvino/tools/mo/front/non_max_suppression_normalize.py
openvino/tools/mo/front/OneHotDepthNormalizer.py
Expand Down Expand Up @@ -692,16 +587,13 @@ openvino/tools/mo/load/caffe/loader.py
openvino/tools/mo/load/kaldi/__init__.py
openvino/tools/mo/load/kaldi/loader.py
openvino/tools/mo/load/loader.py
openvino/tools/mo/load/mxnet/__init__.py
openvino/tools/mo/load/mxnet/loader.py
openvino/tools/mo/load/onnx/__init__.py
openvino/tools/mo/load/onnx/loader.py
openvino/tools/mo/load/tf/__init__.py
openvino/tools/mo/load/tf/loader.py
openvino/tools/mo/main.py
openvino/tools/mo/main_caffe.py
openvino/tools/mo/main_kaldi.py
openvino/tools/mo/main_mxnet.py
openvino/tools/mo/main_onnx.py
openvino/tools/mo/main_paddle.py
openvino/tools/mo/main_tf.py
Expand Down Expand Up @@ -827,7 +719,6 @@ openvino/tools/mo/middle/UselessSplitEraser.py
openvino/tools/mo/mo.py
openvino/tools/mo/mo_caffe.py
openvino/tools/mo/mo_kaldi.py
openvino/tools/mo/mo_mxnet.py
openvino/tools/mo/mo_onnx.py
openvino/tools/mo/mo_paddle.py
openvino/tools/mo/mo_tf.py
Expand All @@ -845,7 +736,6 @@ openvino/tools/mo/ops/__init__.py
openvino/tools/mo/ops/activation.py
openvino/tools/mo/ops/activation_ops.py
openvino/tools/mo/ops/adaptive_avg_pooling.py
openvino/tools/mo/ops/arange_like.py
openvino/tools/mo/ops/argmax.py
openvino/tools/mo/ops/argmin.py
openvino/tools/mo/ops/assert_op.py
Expand Down Expand Up @@ -882,7 +772,6 @@ openvino/tools/mo/ops/dequantize_linear.py
openvino/tools/mo/ops/detection_output_onnx.py
openvino/tools/mo/ops/DetectionOutput.py
openvino/tools/mo/ops/dft.py
openvino/tools/mo/ops/div_sqrt_dim.py
openvino/tools/mo/ops/dropoutmask.py
openvino/tools/mo/ops/einsum.py
openvino/tools/mo/ops/elementwise.py
Expand Down Expand Up @@ -932,7 +821,6 @@ openvino/tools/mo/ops/multinomial.py
openvino/tools/mo/ops/mvn.py
openvino/tools/mo/ops/mxfft.py
openvino/tools/mo/ops/mxrepeat.py
openvino/tools/mo/ops/mxreshape.py
openvino/tools/mo/ops/NextIteration.py
openvino/tools/mo/ops/nms_rotated.py
openvino/tools/mo/ops/non_max_suppression.py
Expand Down
18 changes: 0 additions & 18 deletions tools/mo/openvino/tools/mo/analysis/inputs.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,21 +37,6 @@ def fifo_queue_analysis(cls, graph: Graph, inputs_desc: dict):
'data_type': fifo_queue.types[port_ind]}
return inputs_to_ignore

@classmethod
def ignore_mxnet_softmax_inputs(cls, graph: Graph):
"""
MxNet Softmax layers may have additional inputs which should be ignored. Refer to the
openvino/tools/mo/front/mxnet/check_softmax_node_inputs.py.
"""
inputs_to_ignore = set()
softmax_nodes = []
[softmax_nodes.extend(graph.get_op_nodes(op=op)) for op in ('SoftMax', 'SoftmaxActivation', 'SoftmaxOutput')]
for softmax_node in softmax_nodes:
for i in range(1, len(softmax_node.in_nodes())):
if softmax_node.in_node(i).has_valid('op') and softmax_node.in_node(i).op == 'Parameter':
inputs_to_ignore.add(softmax_node.in_node(i).id)
return inputs_to_ignore

@classmethod
def iterator_get_next_analysis(cls, graph: Graph, inputs_desc: dict):
message = None
Expand Down Expand Up @@ -80,9 +65,6 @@ def analyze(self, graph: Graph):
inputs_desc = dict()
message = InputsAnalysis.iterator_get_next_analysis(graph, inputs_desc)
inputs_to_ignore = InputsAnalysis.fifo_queue_analysis(graph, inputs_desc)
if graph.graph['fw'] == 'mxnet':
inputs_to_ignore.update(InputsAnalysis.ignore_mxnet_softmax_inputs(graph))

inputs = graph.get_op_nodes(op='Parameter')
for input in inputs:
inputs_desc[input.name] = {'shape': input.soft_get('shape', None),
Expand Down
Loading

0 comments on commit c928fde

Please sign in to comment.