adding pytorch quantized sparsified aware training

.ci/spellcheck/.pyspelling.wordlist.txt

@@ -323,6 +323,7 @@ JIT
 Joao
 JS
 JSON
+json
 JT
 JuggernautXL
 Jupyter
@@ -691,6 +692,8 @@ softvc
 SoftVC
 SOTA
 Sovits
+sparsity
+Sparisty
 sparsified
 sparsify
 spectrogram
@@ -858,4 +861,4 @@ Zongyuan
 ZeroScope
 zeroscope
 zh
-xformers
+xformers

notebooks/README.md

@@ -256,6 +256,7 @@
 - [From Training to Deployment with TensorFlow and OpenVINO™](./tensorflow-training-openvino/tensorflow-training-openvino.ipynb)
 - [Quantization Aware Training with NNCF, using TensorFlow Framework](./tensorflow-quantization-aware-training/tensorflow-quantization-aware-training.ipynb)
 - [Quantization Aware Training with NNCF, using PyTorch framework](./pytorch-quantization-aware-training/pytorch-quantization-aware-training.ipynb)
+- [Quantization Sparsity Aware Training with NNCF, using PyTorch framework](./pytorch-quantization-sparsity-aware-training/pytorch-quantization-sparsity-aware-training.ipynb)
 
 ## Optimize
 

notebooks/pytorch-quantization-sparsity-aware-training/README.md

@@ -0,0 +1,24 @@
+# Optimizing PyTorch models with Neural Network Compression Framework of OpenVINO™ by 8-bit quantization.
+
+This tutorial demonstrates how to use [NNCF](https://github.com/openvinotoolkit/nncf) 8-bit sparse quantization to optimize the 
+[PyTorch](https://pytorch.org/) model for inference with [OpenVINO Toolkit](https://docs.openvino.ai/). 
+For more advanced usage, refer to these [examples](https://github.com/openvinotoolkit/nncf/tree/develop/examples).
+
+This notebook is based on 'ImageNet training in PyTorch' [example](https://github.com/pytorch/examples/blob/master/imagenet/main.py).
+This notebook uses a [ResNet-50](https://arxiv.org/abs/1512.03385) model with the 
+ImageNet dataset.
+
+## Notebook Contents
+
+This tutorial consists of the following steps:
+* Transforming the original dense `FP32` model to sparse `INT8`
+* Using fine-tuning to restore the accuracy.
+* Exporting optimized and original models to OpenVINO
+* Measuring and comparing the performance of the models.
+
+## Installation Instructions
+
+This is a self-contained example that relies solely on its own code and accompanying config.json file.</br>
+We recommend running the notebook in a virtual environment. You only need a Jupyter server to start.
+For details, please refer to [Installation Guide](../../README.md).
+

notebooks/pytorch-quantization-sparsity-aware-training/config.json

@@ -0,0 +1,36 @@
+{
+    "target_device": "CPU",
+    "input_info": {
+    "sample_size": [
+          1,
+          3,
+          224,
+          224
+        ]
+    },
+    "compression": [
+    {
+      "algorithm": "magnitude_sparsity",
+      "sparsity_init": 0,
+      "params": {
+            "schedule": "multistep",
+            "sparsity_freeze_epoch": 5,
+            "multistep_steps": [2],
+            "multistep_sparsity_levels": [0.3, 0.5]
+       },
+      "ignored_scopes": [
+        "ResNet/NNCFConv2d[conv1]/conv2d_0"
+      ]
+    },
+    {
+      "algorithm": "quantization",
+      "initializer": {
+        "range": {
+          "num_init_samples": 10000
+        }
+      },
+      "ignored_scopes": [
+      ]
+    }
+  ]
+}