在线时间:8:00-16:00
迪恩网络APP
随时随地掌握行业动态
扫描二维码
关注迪恩网络微信公众号
开源软件名称:nncf_pytorch开源软件地址:https://gitee.com/openvinotoolkit-prc/nncf_pytorch开源软件介绍:Neural Network Compression Framework (NNCF)NNCF provides a suite of advanced algorithms for Neural Networks inference optimization in OpenVINO™ with minimal accuracy drop. NNCF is designed to work with models from PyTorch and TensorFlow. NNCF provides samples that demonstrate the usage of compression algorithms for three different use cases on public PyTorch andTensorFlow models and datasets: Image Classification, Object Detection and Semantic Segmentation.Compression results achievable with the NNCF-powered samples can be found in a table atthe end of this document. The framework is organized as a Python* package that can be built and used in a standalone mode. The frameworkarchitecture is unified to make it easy to add different compression algorithms for both PyTorch and TensorFlow deeplearning frameworks. Key Features
UsageThe NNCF is organized as a regular Python package that can be imported in your target training pipeline script.The basic workflow is loading a JSON configuration script containing NNCF-specific parameters determining the compression to be applied to your model, and then passing your model along with the configuration script to the Usage example with PyTorchimport torchimport nncf # Important - should be imported directly after torchfrom nncf import NNCFConfigfrom nncf.torch import create_compressed_model, register_default_init_args# Instantiate your uncompressed modelfrom torchvision.models.resnet import resnet50model = resnet50()# Load a configuration file to specify compressionnncf_config = NNCFConfig.from_json("resnet50_int8.json")# Provide data loaders for compression algorithm initialization, if necessaryimport torchvision.datasets as datasetsrepresentative_dataset = datasets.ImageFolder("/path")init_loader = torch.utils.data.DataLoader(representative_dataset)nncf_config = register_default_init_args(nncf_config, init_loader)# Apply the specified compression algorithms to the modelcompression_ctrl, compressed_model = create_compressed_model(model, nncf_config)# Now use compressed_model as a usual torch.nn.Module # to fine-tune compression parameters along with the model weights# ... the rest of the usual PyTorch-powered training pipeline# Export to ONNX or .pth when done fine-tuningcompression_ctrl.export_model("compressed_model.onnx")torch.save(compressed_model.state_dict(), "compressed_model.pth") Usage example with TensorFlowimport tensorflow as tffrom nncf import NNCFConfigfrom nncf.tensorflow import create_compressed_model, register_default_init_args# Instantiate your uncompressed modelfrom tensorflow.keras.applications import ResNet50model = ResNet50()# Load a configuration file to specify compressionnncf_config = NNCFConfig.from_json("resnet50_int8.json")# Provide dataset for compression algorithm initializationrepresentative_dataset = tf.data.Dataset.list_files("/path/*.jpeg")nncf_config = register_default_init_args(nncf_config, representative_dataset, batch_size=1)# Apply the specified compression algorithms to the modelcompression_ctrl, compressed_model = create_compressed_model(model, nncf_config)# Now use compressed_model as a usual Keras model# to fine-tune compression parameters along with the model weights# ... the rest of the usual TensorFlow-powered training pipeline# Export to Frozen Graph, TensorFlow SavedModel or .h5 when done fine-tuning compression_ctrl.export_model("compressed_model.pb", save_format='frozen_graph') For a more detailed description of NNCF usage in your training code, see this tutorial.For in-depth examples of NNCF integration, browse the sample scripts code, or the example patches to third-party repositories. Model Compression SamplesFor a quicker start with NNCF-powered compression, you can also try the sample scripts, each of which provides a basic training pipeline for classification, semantic segmentation and object detection neural network training correspondingly. To run the samples please refer to the corresponding tutorials:
Model Compression NotebooksA collection of ready-to-run Jupyter* notebooks are also available to demonstrate how to use NNCF compression algorithmsto optimize models for inference with the OpenVINO Toolkit.
Third-party repository integrationNNCF may be straightforwardly integrated into training/evaluation pipelines of third-party repositories. Used by
Git patches for third-party repositorySee third_party_integration for examples of code modifications (Git patches and base commit IDs are provided) that are necessary to integrate NNCF into the following repositories: System requirements
This repository is tested on Python* 3.6.2+, PyTorch* 1.9.1 (NVidia CUDA* Toolkit 10.2) and TensorFlow* 2.5.3 (NVidia CUDA* Toolkit 11.2). InstallationWe suggest to install or use the package in the Python virtual environment. If you want to optimize a model from PyTorch, install PyTorch by following PyTorch installation guide.If you want to optimize a model from TensorFlow, install TensorFlow by following TensorFlow installation guide. As a package built from a checked-out repository:Install the package and its dependencies by running the following in the repository root directory: python setup.py install Alternatively, If you don't install any backend you can install NNCF and PyTorch in one line with: python setup.py install --torch Install NNCF and TensorFlow in one line: python setup.py install --tf NB: For launching example scripts in this repository, we recommend replacing the As a PyPI package:NNCF can be installed as a regular PyPI package via pip: pip install nncf Alternatively, If you don't install any backend you can install NNCF and PyTorch in one line with: pip install nncf[torch] Install NNCF and TensorFlow in one line: pip install nncf[tf] NNCF is also available via conda: conda install -c conda-forge nncf From a specific commit hash using pip:pip install git+https://github.com/openvinotoolkit/nncf@bd189e2#egg=nncf Note that in order for this to work for pip versions >= 21.3, your Git version must be at least 2.22. As a Docker imageUse one of the Dockerfiles in the docker directory to build an image with an environment already set up and ready for running NNCF sample scripts. ContributingRefer to the CONTRIBUTING.md file for guidelines on contributions to the NNCF repository. NNCF Compressed Model ZooResults achieved using sample scripts, example patches to third-party repositories and NNCF configuration files providedwith this repository. See README.md files for sample scripts and example patchesto find instruction and links to exact configuration files and final checkpoints. PyTorch modelsClassification
Object detection
Semantic segmentation
NLP (HuggingFace Transformers-powered models)
TensorFlow modelsClassification
Object detection
Instance segmentation
Citing@article{kozlov2020neural, title = {Neural network compression framework for fast model inference}, author = {Kozlov, Alexander and Lazarevich, Ivan and Shamporov, Vasily and Lyalyushkin, Nikolay and Gorbachev, Yury}, journal = {arXiv preprint arXiv:2002.08679}, year = {2020}} Legal Information[*] Other names and brands may be claimed as the property of others. |
请发表评论