Skip to content

Latest commit

 

History

History
132 lines (111 loc) · 4.82 KB

README.md

File metadata and controls

132 lines (111 loc) · 4.82 KB
Raw

CHIP: CHannel Independence-based Pruning for Compact Neural Networks

Code for our NeurIPS 2021 paper: (CHIP: CHannel Independence-based Pruning for Compact Neural Networks)visitors.

Usage

Generate Feature Maps.

1. CIFAR-10
python calculate_feature_maps.py \
--arch resnet_56 \
--dataset cifar10 \
--data_dir ./data \
--pretrain_dir ./pretrained_models/resnet_56.pt \
--gpu 0
2. ImageNet
python calculate_feature_maps.py \
--arch resnet_50 \
--dataset imagenet \
--data_dir /raid/data/imagenet \
--pretrain_dir ./pretrained_models/resnet50.pth \
--gpu 0

Generate Channel Independence (CI).

This procedure is time-consuming, please be patient.

1. CIFAR-10
python calculate_ci.py \
--arch resnet_56 \
--repeat 5 \
--num_layers 55
2. ImageNet
python calculate_ci.py \
--arch resnet_50 \
--repeat 5 \
--num_layers 53

Prune and Fine-tune Models.

1. CIFAR-10
python prune_finetune_cifar.py \
--data_dir ./data \
--result_dir ./result/resnet_56/1 \
--arch resnet_56 \
--ci_dir ./CI_resnet_56 \
--batch_size 256 \
--epochs 400 \
--lr_type cos \
--learning_rate 0.01 \
--momentum 0.9 \
--weight_decay 0.005 \
--pretrain_dir ./pretrained_models/resnet_56.pt \
--sparsity [0.]+[0.4]*2+[0.5]*9+[0.6]*9+[0.7]*9 \
--gpu 0
2. ImageNet
python prune_finetune_imagenet.py \
--data_dir /raid/data/imagenet \
--result_dir ./result/resnet_50/1 \
--arch resnet_50 \
--ci_dir ./CI_resnet_50 \
--batch_size 256 \
--epochs 180 \
--lr_type cos \
--learning_rate 0.01 \
--momentum 0.99 \
--label_smooth 0.1 \
--weight_decay 0.0001 \
--pretrain_dir ./pretrained_models/resnet50.pth \
--sparsity [0.]+[0.1]*3+[0.35]*16 \
--gpu 0

Pre-trained Models

Results

We release our training logs of ResNet-56/110 model on CIFAR-10 for more epochs which can achieve better results than paper. We release our training logs of ResNet-50 model on ImageNet. Training logs can be found at link. Some results are better than papers.

CIFAR-10
Model # of Params (Reduction) Flops (Reduction) Top-1 Accuracy Sparsity Setting
ResNet-56 0.85M(0.0%) 125.49M(0.0%) 93.26% N/A
ResNet-56 0.48M(42.8%) 65.94M(47.4%) 94.16% [0.]+[0.15]*2+[0.4]*27
ResNet-56 0.24M(70.0%) 34.78M(74.1%) 92.43% [0.]+[0.4]*2+[0.5]*9+[0.6]*9+[0.7]*9
ResNet-110 1.72M(0.0%) 252.89M(0.0%) 93.50% N/A
ResNet-110 1.04M(39.1%) 140.54M(44.4%) 94.50% [0.]+[0.2]*2+[0.3]*18+[0.35]*36
ResNet-110 0.89M(48.3%) 121.09M(52.1%) 94.44% [0.]+[0.22]*2+[0.35]*18+[0.45]*36
ResNet-110 0.54M(68.3%) 71.69M(71.6%) 93.23% [0.]+[0.4]*2+[0.5]*18+[0.65]*36
VGG-16-BN 14.98M(0.0%) 313.73M(0.0%) 93.96% N/A
VGG-16-BN 2.76M(81.6%) 131.17M(58.1%) 93.86% [0.21]*7+[0.75]*5
VGG-16-BN 2.50M(83.3%) 104.78M(66.6%) 93.72% [0.3]*7+[0.75]*5
VGG-16-BN 1.90M(87.3%) 66.95M(78.6%) 93.18% [0.45]*7+[0.78]*5
ImageNet
Model # of Params (Reduction) Flops (Reduction) Top-1 Accuracy Top-5 Accuracy Sparsity Setting
ResNet-50 25.55M(0.0%) 4.11B(0.0%) 76.15% 92.87% N/A
ResNet-50 15.09M(40.8%) 2.26B(44.8%) 76.41% 93.06% [0.]+[0.1]*3+[0.35]*16
ResNet-50 14.28M(44.2%) 2.19B(48.7%) 76.35% 93.05% [0.]+[0.12]*3+[0.38]*16
ResNet-50 11.05M(56.7%) 1.52B(62.8%) 75.26% 92.53% [0.]+[0.25]*3+[0.5]*16
ResNet-50 8.02M(68.6%) 0.95B(76.7%) 73.30% 91.48% [0.]+[0.5]*3+[0.6]*16

Others

Codes are based on link.

Since I rearranged my original codes for simplicity, please feel free to open an issue if something wrong happens when you run the codes. (Please forgive me for the late response and wait for me to respond to your problems in several days.)