Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MO] Remove Apache MxNet models support in openvino-dev #26800

Merged
merged 5 commits into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/ISSUE_TEMPLATE/bug.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,6 @@ body:
- Caffe
- ONNX
- PyTorch
- mxnet
- PaddlePaddle
validations:
required: false
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ jobs:
# For running TensorFlow frontend unit tests
python3 -m pip install -r ${{ github.workspace }}/src/frontends/tensorflow/tests/requirements.txt
# For MO unit tests
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_mxnet.txt
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_caffe.txt
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_kaldi.txt
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_onnx.txt
Expand Down
7 changes: 1 addition & 6 deletions .github/workflows/job_python_unit_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -99,10 +99,6 @@ jobs:
extras_to_install="caffe,kaldi,onnx,tensorflow2,pytorch"
if [[ "${{ runner.arch }}" != "ARM64" ]]; then
extras_to_install="mxnet,$extras_to_install"
fi
# Find and install OV dev wheel
pushd ${INSTALL_DIR}/tools
ov_dev_wheel_name=$(find . -name 'openvino_dev*.whl')
Expand Down Expand Up @@ -142,8 +138,7 @@ jobs:
# Skips under tickets: 133405, 122666
python3 -m pytest -s ${INSTALL_TEST_DIR}/mo/unit_tests \
--junitxml=${INSTALL_TEST_DIR}/TEST-ModelOptimizer.xml \
--ignore-glob="**/mo/unit_tests/mo/front/mxnet/**"
--junitxml=${INSTALL_TEST_DIR}/TEST-ModelOptimizer.xml
- name: Python ONNX operators tests
if: (fromJSON(inputs.affected-components).Python_API.test ||
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/mo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ jobs:
# For UT
pip install unittest-xml-reporting==3.0.2
# MO requirements
pip install -r requirements_mxnet.txt
pip install -r requirements_caffe.txt
pip install -r requirements_kaldi.txt
pip install -r requirements_onnx.txt
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/windows_vs2019_release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@ jobs:

# Find and install the dev OV wheel
$ovDevWheelPath=Get-ChildItem -Path "${{ env.INSTALL_DIR }}\tools" -Filter openvino_dev*.whl | % { $_.FullName }
python3 -m pip install "$ovDevWheelPath[mxnet,caffe,kaldi,onnx,tensorflow2,pytorch]"
python3 -m pip install "$ovDevWheelPath[caffe,kaldi,onnx,tensorflow2,pytorch]"

- name: Install Python API tests dependencies
run: |
Expand Down Expand Up @@ -308,7 +308,7 @@ jobs:
if: fromJSON(needs.smart_ci.outputs.affected_components).MO.test
shell: cmd
run: |
python3 -m pytest -s ${{ env.INSTALL_TEST_DIR }}/mo/unit_tests --ignore=${{ env.INSTALL_TEST_DIR }}/mo/unit_tests/mo/front/mxnet --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-ModelOptimizer.xml
python3 -m pytest -s ${{ env.INSTALL_TEST_DIR }}/mo/unit_tests --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-ModelOptimizer.xml

- name: Install Python Layer tests dependencies
run: |
Expand Down
2 changes: 1 addition & 1 deletion docs/articles_en/documentation/legacy-features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ offering.
Discontinued:
#############

.. dropdown:: Apache MXNet, Caffe, and Kaldi model formats
.. dropdown:: Caffe, and Kaldi model formats

| *New solution:* conversion to ONNX via external tools
| *Old solution:* model support discontinued with OpenVINO 2024.0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ Q14. What does the message "Cannot infer shape for node {} because there is no C
Q15. What does the message "Framework name can not be deduced from the given options. Use --framework to choose one of Caffe, TensorFlow, MXNet" mean?
######################################################################################################################################################

**A:** You have run Model Optimizer without a flag ``--framework caffe|tf|mxnet``. Model Optimizer tries to deduce the framework by the extension of input model file (``.pb`` for TensorFlow, ``.caffemodel`` for Caffe, ``.params`` for Apache MXNet). Your input model might have a different extension and you need to explicitly set the source framework. For example, use ``--framework caffe``.
**A:** You have run Model Optimizer without a flag ``--framework caffe|tf``. Model Optimizer tries to deduce the framework by the extension of input model file (``.pb`` for TensorFlow, ``.caffemodel`` for Caffe, ``.params`` for Apache MXNet). Your input model might have a different extension and you need to explicitly set the source framework. For example, use ``--framework caffe``.

.. _question-16:

Expand Down
1 change: 0 additions & 1 deletion docs/dev/ov_dependencies.txt
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,6 @@ mccabe
mistune
mongo-python-driver
more-itertools
mxnet
mypy
mypy-extensions
networkx
Expand Down
7 changes: 3 additions & 4 deletions docs/dev/pypi_publish/pypi-openvino-dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,6 @@ pip install openvino-dev[extras]
| :-------------------------------| :------------------------------------------------------------------------------- |
| caffe | [Caffe*](https://caffe.berkeleyvision.org/) |
| kaldi | [Kaldi*](https://github.com/kaldi-asr/kaldi) |
| mxnet | [Apache MXNet*](https://mxnet.apache.org/) |
| onnx | [ONNX*](https://github.com/microsoft/onnxruntime/) |
| pytorch | [PyTorch*](https://pytorch.org/) |
| tensorflow | [TensorFlow* 1.x](https://www.tensorflow.org/versions#tensorflow_1) |
Expand Down Expand Up @@ -149,14 +148,14 @@ Users in China might encounter errors while downloading sources via PIP during O
If you use zsh (Z shell) interpreter, that is the default shell for macOS starting with version 10.15 (Catalina), you may encounter the following error while installing `openvino-dev` package with extras:

```sh
pip install openvino-dev[tensorflow2,mxnet,caffe]
zsh: no matches found: openvino-dev[tensorflow2,mxnet,caffe]
pip install openvino-dev[tensorflow2,caffe]
zsh: no matches found: openvino-dev[tensorflow2,caffe]
```

By default zsh interprets square brackets as an expression for pattern matching. To resolve this issue, you need to escape the command with quotes:

```sh
pip install 'openvino-dev[tensorflow2,mxnet,caffe]'
pip install 'openvino-dev[tensorflow2,caffe]'
```

To avoid such issues you can also disable globbing for PIP commands by defining an alias in `~/.zshrc` file:
Expand Down
9 changes: 0 additions & 9 deletions docs/sphinx_setup/_static/download/supported_models.csv
Original file line number Diff line number Diff line change
Expand Up @@ -211,11 +211,6 @@ facenet,Object Detection,onnx,FP16,,,+
facenet,Object Detection,onnx,FP16-INT8,,,+
facenet-20180408-102900,Object Detection,tf,FP16,+,+,+
facenet-20180408-102900,Object Detection,tf,FP32,+,+,
face-recognition-mobilefacenet-arcface,Object Detection,mxnet,FP16,,,+
face-recognition-mobilefacenet-arcface,Object Detection,mxnet,FP16-INT8,,,+
face-recognition-resnet50-arcface,Object Detection,mxnet,FP16,+,+,+
face-recognition-resnet50-aws,Object Detection,mxnet,FP16,,,+
face-recognition-resnet50-aws,Object Detection,mxnet,FP32,,,+
face-reidentification-retail-0095,Object Detection,onnx,FP16,,,+
face-reidentification-retail-0095,Object Detection,onnx,FP16-INT8,,,+
facial-landmarks-35-adas-0002,Object Detection,caffe,FP16,,,+
Expand Down Expand Up @@ -428,8 +423,6 @@ ocrnet-hrnet-w18,Text Detection,paddle,FP16-INT8,+,+,
ocrnet-hrnet-w18,Text Detection,paddle,FP32,+,,
ocrnet-hrnet-w48,Text Detection,paddle,FP16,+,,
ocrnet-hrnet-w48,Text Detection,paddle,FP32,+,,
octave-resnext-101-0.25,Image Classification,mxnet,FP16,+,+,+
octave-resnext-101-0.25,Image Classification,mxnet,FP32,+,,
openchat-3.6-8b-20240522,Large Language Model,pytorch,intel-optimum default,,+,
open-closed-eye-0001,Image Classification,onnx,FP16,,,+
open-closed-eye-0001,Image Classification,onnx,FP16-INT8,,,+
Expand Down Expand Up @@ -629,8 +622,6 @@ squeezenet1.0-12,Image Classification,onnx,FP16,,,+
squeezenet1.0-12,Image Classification,onnx,FP32,+,+,+
squeezenet1.1-caffe,Image Classification,caffe,FP16,+,+,+
squeezenet1.1-caffe,Image Classification,caffe,FP32,+,+,+
squeezenet1.1-mxnet,Image Classification,mxnet,FP16,+,+,+
squeezenet1.1-mxnet,Image Classification,mxnet,FP32,+,+,+
squeezenet1.1-onnx,Image Classification,onnx,FP32,+,,
srgan-onnx,"Image Processing, Enhancement",onnx,FP16,+,,+
srgan-tf,"Image Processing, Enhancement",tf,FP16,+,+,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def get_precision(self):
return self.__model_prc

def path_to_model(model_path: os.path, prefix: str):
frameworks = {'tf', 'tf2', 'caffe', 'onnx', 'mxnet', 'paddle', 'kaldi'}
frameworks = {'tf', 'tf2', 'caffe', 'onnx', 'paddle', 'kaldi'}
precisions = {'FP16', 'FP32', 'INT8', 'INT1'}
# remove share path + model.xml
model_path = model_path.replace('\n', '')
Expand Down
2 changes: 0 additions & 2 deletions tools/constraints.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@
# files because the version differs between them:
# tensorflow, numpy
h5py>=3.1.0,<3.11.0
mxnet~=1.2.0; sys_platform == 'win32'
mxnet>=1.7.0.post2,<=1.9.1; sys_platform != 'win32'
onnx>=1.8.1,<=1.16.0
networkx<=3.1.0
pytest>=5.0,<8.4
Expand Down
4 changes: 0 additions & 4 deletions tools/mo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,6 @@ E.g. the command below will install dependencies to support ONNX\* and TensorFlo
```
pip install openvino-dev[onnx,tensorflow2]
```
To enable support of MxNet\* models run:
```
pip install openvino-dev[mxnet]
```
To enable support of all frameworks:
```
pip install openvino-dev[all]
Expand Down
112 changes: 0 additions & 112 deletions tools/mo/automation/package_BOM.txt
Original file line number Diff line number Diff line change
Expand Up @@ -252,111 +252,6 @@ openvino/tools/mo/front/LayerNorm.py
openvino/tools/mo/front/Log1p.py
openvino/tools/mo/front/MatMul_normalizer.py
openvino/tools/mo/front/MoveEmbeddedInputsToInputs.py
openvino/tools/mo/front/mxnet/__init__.py
openvino/tools/mo/front/mxnet/activation.py
openvino/tools/mo/front/mxnet/adaptive_avg_pooling_ext.py
openvino/tools/mo/front/mxnet/add_input_data_to_prior_boxes.py
openvino/tools/mo/front/mxnet/arange_ext.py
openvino/tools/mo/front/mxnet/arange_like_ext.py
openvino/tools/mo/front/mxnet/arange_like_replacer.py
openvino/tools/mo/front/mxnet/arange_replacer.py
openvino/tools/mo/front/mxnet/batch_dot_ext.py
openvino/tools/mo/front/mxnet/block_grad_ext.py
openvino/tools/mo/front/mxnet/box_nms_ext.py
openvino/tools/mo/front/mxnet/cast_ext.py
openvino/tools/mo/front/mxnet/check_softmax_node_inputs.py
openvino/tools/mo/front/mxnet/clip_ext.py
openvino/tools/mo/front/mxnet/conv_ext.py
openvino/tools/mo/front/mxnet/copy_ext.py
openvino/tools/mo/front/mxnet/crop_ext.py
openvino/tools/mo/front/mxnet/cumsum.py
openvino/tools/mo/front/mxnet/cumsum_ext.py
openvino/tools/mo/front/mxnet/custom.py
openvino/tools/mo/front/mxnet/custom_rpn_proposal.py
openvino/tools/mo/front/mxnet/deformable_conv_ext.py
openvino/tools/mo/front/mxnet/deformable_psroi_pooling_ext.py
openvino/tools/mo/front/mxnet/div_sqrt_dim.py
openvino/tools/mo/front/mxnet/dropout_ext.py
openvino/tools/mo/front/mxnet/einsum_ext.py
openvino/tools/mo/front/mxnet/elementwise_ext.py
openvino/tools/mo/front/mxnet/eltwise_scalar_replacers.py
openvino/tools/mo/front/mxnet/exp_ext.py
openvino/tools/mo/front/mxnet/expand_dims_ext.py
openvino/tools/mo/front/mxnet/extractor.py
openvino/tools/mo/front/mxnet/extractors/__init__.py
openvino/tools/mo/front/mxnet/extractors/add_n.py
openvino/tools/mo/front/mxnet/extractors/batchnorm.py
openvino/tools/mo/front/mxnet/extractors/concat.py
openvino/tools/mo/front/mxnet/extractors/l2_normalization.py
openvino/tools/mo/front/mxnet/extractors/multibox_prior.py
openvino/tools/mo/front/mxnet/extractors/relu.py
openvino/tools/mo/front/mxnet/extractors/scaleshift.py
openvino/tools/mo/front/mxnet/extractors/slice_axis.py
openvino/tools/mo/front/mxnet/extractors/utils.py
openvino/tools/mo/front/mxnet/eye_ext.py
openvino/tools/mo/front/mxnet/eye_mx_to_eye.py
openvino/tools/mo/front/mxnet/fft_ext.py
openvino/tools/mo/front/mxnet/flatten_ext.py
openvino/tools/mo/front/mxnet/fully_connected_ext.py
openvino/tools/mo/front/mxnet/gather.py
openvino/tools/mo/front/mxnet/gather_ext.py
openvino/tools/mo/front/mxnet/gluoncv_ssd_anchors.py
openvino/tools/mo/front/mxnet/instance_norm_ext.py
openvino/tools/mo/front/mxnet/layer_norm_ext.py
openvino/tools/mo/front/mxnet/leaky_relu.py
openvino/tools/mo/front/mxnet/loader.py
openvino/tools/mo/front/mxnet/lrn_ext.py
openvino/tools/mo/front/mxnet/max_ext.py
openvino/tools/mo/front/mxnet/modulated_deformable_conv_ext.py
openvino/tools/mo/front/mxnet/modulated_deformable_conv_replacer.py
openvino/tools/mo/front/mxnet/multibox_detection_ext.py
openvino/tools/mo/front/mxnet/mx_reshape_reverse.py
openvino/tools/mo/front/mxnet/mx_reshape_to_reshape.py
openvino/tools/mo/front/mxnet/MXFFTToDFT.py
openvino/tools/mo/front/mxnet/MXRepeatReplacer.py
openvino/tools/mo/front/mxnet/nd_to_params.py
openvino/tools/mo/front/mxnet/null_ext.py
openvino/tools/mo/front/mxnet/pad_ext.py
openvino/tools/mo/front/mxnet/pooling_ext.py
openvino/tools/mo/front/mxnet/proposal_ext.py
openvino/tools/mo/front/mxnet/psroi_pooling_ext.py
openvino/tools/mo/front/mxnet/random_uniform_ext.py
openvino/tools/mo/front/mxnet/register_custom_ops.py
openvino/tools/mo/front/mxnet/repeat_ext.py
openvino/tools/mo/front/mxnet/reshape_ext.py
openvino/tools/mo/front/mxnet/RNN_ext.py
openvino/tools/mo/front/mxnet/rnn_param_concat.py
openvino/tools/mo/front/mxnet/roi_pooling_ext.py
openvino/tools/mo/front/mxnet/roll_ext.py
openvino/tools/mo/front/mxnet/shape_array_ext.py
openvino/tools/mo/front/mxnet/sigmoid.py
openvino/tools/mo/front/mxnet/slice_channel_ext.py
openvino/tools/mo/front/mxnet/slice_ext.py
openvino/tools/mo/front/mxnet/slice_like_ext.py
openvino/tools/mo/front/mxnet/slice_replacers.py
openvino/tools/mo/front/mxnet/softmax.py
openvino/tools/mo/front/mxnet/softmax_activation_ext.py
openvino/tools/mo/front/mxnet/softmax_ext.py
openvino/tools/mo/front/mxnet/softmax_output_ext.py
openvino/tools/mo/front/mxnet/softsign_ext.py
openvino/tools/mo/front/mxnet/squeeze_ext.py
openvino/tools/mo/front/mxnet/ssd_anchor_reshape.py
openvino/tools/mo/front/mxnet/ssd_detection_output_replacer.py
openvino/tools/mo/front/mxnet/ssd_pattern_flatten_softmax_activation.py
openvino/tools/mo/front/mxnet/ssd_pattern_remove_flatten.py
openvino/tools/mo/front/mxnet/ssd_pattern_remove_reshape.py
openvino/tools/mo/front/mxnet/ssd_pattern_remove_transpose.py
openvino/tools/mo/front/mxnet/ssd_reorder_detection_out_inputs.py
openvino/tools/mo/front/mxnet/stack_ext.py
openvino/tools/mo/front/mxnet/swapaxis_ext.py
openvino/tools/mo/front/mxnet/take_ext.py
openvino/tools/mo/front/mxnet/tile_ext.py
openvino/tools/mo/front/mxnet/tile_replacer.py
openvino/tools/mo/front/mxnet/transpose_ext.py
openvino/tools/mo/front/mxnet/up_sampling_ext.py
openvino/tools/mo/front/mxnet/where_ext.py
openvino/tools/mo/front/mxnet/yolo_v3_mobilenet1_voc.json
openvino/tools/mo/front/mxnet/zeros_ext.py
openvino/tools/mo/front/no_op_eraser.py
openvino/tools/mo/front/non_max_suppression_normalize.py
openvino/tools/mo/front/OneHotDepthNormalizer.py
Expand Down Expand Up @@ -692,16 +587,13 @@ openvino/tools/mo/load/caffe/loader.py
openvino/tools/mo/load/kaldi/__init__.py
openvino/tools/mo/load/kaldi/loader.py
openvino/tools/mo/load/loader.py
openvino/tools/mo/load/mxnet/__init__.py
openvino/tools/mo/load/mxnet/loader.py
openvino/tools/mo/load/onnx/__init__.py
openvino/tools/mo/load/onnx/loader.py
openvino/tools/mo/load/tf/__init__.py
openvino/tools/mo/load/tf/loader.py
openvino/tools/mo/main.py
openvino/tools/mo/main_caffe.py
openvino/tools/mo/main_kaldi.py
openvino/tools/mo/main_mxnet.py
openvino/tools/mo/main_onnx.py
openvino/tools/mo/main_paddle.py
openvino/tools/mo/main_tf.py
Expand Down Expand Up @@ -827,7 +719,6 @@ openvino/tools/mo/middle/UselessSplitEraser.py
openvino/tools/mo/mo.py
openvino/tools/mo/mo_caffe.py
openvino/tools/mo/mo_kaldi.py
openvino/tools/mo/mo_mxnet.py
openvino/tools/mo/mo_onnx.py
openvino/tools/mo/mo_paddle.py
openvino/tools/mo/mo_tf.py
Expand All @@ -845,7 +736,6 @@ openvino/tools/mo/ops/__init__.py
openvino/tools/mo/ops/activation.py
openvino/tools/mo/ops/activation_ops.py
openvino/tools/mo/ops/adaptive_avg_pooling.py
openvino/tools/mo/ops/arange_like.py
openvino/tools/mo/ops/argmax.py
openvino/tools/mo/ops/argmin.py
openvino/tools/mo/ops/assert_op.py
Expand Down Expand Up @@ -882,7 +772,6 @@ openvino/tools/mo/ops/dequantize_linear.py
openvino/tools/mo/ops/detection_output_onnx.py
openvino/tools/mo/ops/DetectionOutput.py
openvino/tools/mo/ops/dft.py
openvino/tools/mo/ops/div_sqrt_dim.py
openvino/tools/mo/ops/dropoutmask.py
openvino/tools/mo/ops/einsum.py
openvino/tools/mo/ops/elementwise.py
Expand Down Expand Up @@ -932,7 +821,6 @@ openvino/tools/mo/ops/multinomial.py
openvino/tools/mo/ops/mvn.py
openvino/tools/mo/ops/mxfft.py
openvino/tools/mo/ops/mxrepeat.py
openvino/tools/mo/ops/mxreshape.py
openvino/tools/mo/ops/NextIteration.py
openvino/tools/mo/ops/nms_rotated.py
openvino/tools/mo/ops/non_max_suppression.py
Expand Down
18 changes: 0 additions & 18 deletions tools/mo/openvino/tools/mo/analysis/inputs.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,21 +37,6 @@ def fifo_queue_analysis(cls, graph: Graph, inputs_desc: dict):
'data_type': fifo_queue.types[port_ind]}
return inputs_to_ignore

@classmethod
def ignore_mxnet_softmax_inputs(cls, graph: Graph):
"""
MxNet Softmax layers may have additional inputs which should be ignored. Refer to the
openvino/tools/mo/front/mxnet/check_softmax_node_inputs.py.
"""
inputs_to_ignore = set()
softmax_nodes = []
[softmax_nodes.extend(graph.get_op_nodes(op=op)) for op in ('SoftMax', 'SoftmaxActivation', 'SoftmaxOutput')]
for softmax_node in softmax_nodes:
for i in range(1, len(softmax_node.in_nodes())):
if softmax_node.in_node(i).has_valid('op') and softmax_node.in_node(i).op == 'Parameter':
inputs_to_ignore.add(softmax_node.in_node(i).id)
return inputs_to_ignore

@classmethod
def iterator_get_next_analysis(cls, graph: Graph, inputs_desc: dict):
message = None
Expand Down Expand Up @@ -80,9 +65,6 @@ def analyze(self, graph: Graph):
inputs_desc = dict()
message = InputsAnalysis.iterator_get_next_analysis(graph, inputs_desc)
inputs_to_ignore = InputsAnalysis.fifo_queue_analysis(graph, inputs_desc)
if graph.graph['fw'] == 'mxnet':
inputs_to_ignore.update(InputsAnalysis.ignore_mxnet_softmax_inputs(graph))

inputs = graph.get_op_nodes(op='Parameter')
for input in inputs:
inputs_desc[input.name] = {'shape': input.soft_get('shape', None),
Expand Down
Loading
Loading