30dd0bd6c2
* Create doc tree index for Nano How-to Guides * Add How to guide for PyTorch Inference using ONNXRuntime * Add How to guide for PyTorch Inference using OpenVINO * Update How to guide for PyTorch Inference using OpenVINO/ONNXRuntime * Change current notebook to md and revise contents to be more concentrated * Add How-to Guide: Install BigDL-Nano in Google Colab (need further update) * Revise words in How-to Guide for PyTorch Inference using OpenVINO/ONNXRuntime * Add How-To Guide: Quantize PyTorch Model for Inference using Intel Neural Compressor * Add How-To Guide: Quantize PyTorch Model for Inference using Post-training Quantization Tools * Add API doc links and small revision * Test: syncronization through marks in py files * Test: syncronization through notebook with cells hidden from rendering in doc * Remove test commits for runnable example <-> guides synchronization * Enable rendering notebook from location out of sphinx source root * Update guide "How to accelerate a PyTorch inference pipeline through OpenVINO" to notebook under python folder * Update guide "How to quantize your PyTorch model for inference using Intel Neural Compressor" to notebook under python folder * Fix bug that markdown will be ignored inside html tags for nbconvert, and notebook revise * Update guide 'How to quantize your PyTorch model for inference using Post-training Optimization Tools' to notebook under python folder * Small updates to index and current guides * Revision based on Junwei's comments * Update how-to guides: How to install BigDL-Nano in Google Colab, and update index page * Small typo fix
2.3 KiB
2.3 KiB
Install BigDL-Nano in Google Colab
.. note::
This page is still a work in progress.
In this guide, we will show you how to install BigDL-Nano in Google Colab, and the solutions to possible version conflicts caused by pre-installed packages in Colab hosted runtime.
Please select the corresponding section to follow for your specific usage.
PyTorch
For PyTorch users, you need to install BigDL-Nano for PyTorch first:
.. tabs::
.. tab:: Latest
.. code-block:: python
!pip install bigdl-nano[pytorch]
.. tab:: Nightly-Built
.. code-block:: python
!pip install --pre --upgrade bigdl-nano[pytorch]
.. warning::
For Google Colab hosted runtime, ``source bigdl-nano-init`` is hardly to take effect as environment variables need to be set before jupyter kernel is started.
To avoid version conflicts caused by torchtext, you should uninstall it:
!pip uninstall -y torchtext
ONNXRuntime
To enable ONNXRuntime acceleration, you need to install corresponding onnx packages:
!pip install onnx onnxruntime
OpenVINO / Post-training Optimization Tools (POT)
To enable OpenVINO acceleration, or use POT for quantization, you need to install the OpenVINO toolkit:
!pip install openvino-dev
# Please remember to restart runtime to use packages with newly-installed version
.. note::
If you meet ``ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject`` when using ``Trainer.trace`` or ``Trainer.quantize`` function, you could try to solve it by upgrading ``numpy`` through:
.. code-block:: python
!pip install --upgrade numpy
# Please remember to restart runtime to use numpy with newly-installed version
Intel Neural Compressor (INC)
To use INC as your quantization backend, you need to install it:
.. tabs::
.. tab:: With no Extra Runtime Acceleration
.. code-block:: python
!pip install neural-compressor==1.11.0
.. tab:: With Extra ONNXRuntime Acceleration
.. code-block:: python
!pip install neural-compressor==1.11.0 onnx onnxruntime onnxruntime_extensions