ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Nano: update tensorflow examples (#6935)

Browse source
This commit is contained in:
Yishuo Wang 2022-12-12 13:56:05 +08:00 committed by GitHub
parent 56a2d8f1cd
commit 7e180d028e
1 changed files with 8 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
docs/readthedocs/source/doc/Nano/QuickStart/tensorflow_quantization_quickstart.md
View file

@ -19,10 +19,10 @@ By default, [Intel Neural Compressor](https://github.com/intel/neural-compressor
pip install neural-compressor==1.11.0
```
BigDL-Nano provides several APIs which can help users easily apply optimizations on inference pipelines to improve latency and throughput. The Keras Model(`bigdl.nano.tf.keras.Model`) and Sequential(`bigdl.nano.tf.keras.Sequential`) provides the APIs for all optimizations you need for inference.
BigDL-Nano provides several APIs which can help users easily apply optimizations on inference pipelines to improve latency and throughput. The Keras Model(`bigdl.nano.tf.keras.Model`) and InferenceOptimizer(`bigdl.nano.tf.keras.InferenceOptimizer`) provides the APIs for all optimizations you need for inference.
```python
from bigdl.nano.tf.keras import Model, Sequential
from bigdl.nano.tf.keras import Model, InferenceOptimizer
```
### Step 1: Loading Data
@ -71,14 +71,15 @@ model.fit(train_ds, epochs=1)
```
### Step 3: Quantization with Intel Neural Compressor
[`Model.quantize()`](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/tensorflow.html#bigdl.nano.tf.keras.Model) return a Keras module with desired precision and accuracy. Taking Resnet50 as an example, you can add quantization as below.
[`InferenceOptimizer.quantize()`](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Nano/tensorflow.html#bigdl.nano.tf.keras.InferenceOptimizer.quantize) return a Keras module with desired precision and accuracy. Taking Resnet50 as an example, you can add quantization as below.
```python
from tensorflow.keras.metrics import CategoricalAccuracy
q_model = model.quantize(calib_dataset=dataset,
metric=CategoricalAccuracy(),
tuning_strategy='basic'
)
q_model = InferenceOptimizer.quantize(model,
calib_dataset=dataset,
metric=CategoricalAccuracy(),
tuning_strategy='basic'
)
```
The quantized model can be called to do inference as normal keras model.
```python