ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

[Nano] Add how-to guide for TensorFlow Keras inference using bf16 mixed precision (#7337)

Browse source 
* Add basic structure for tf bf16 infer how-to guide

* Add how-to use BFloat16 mixed precision for TensorFlow Keras inference

* Small fixes

* Add instruction requires box

* Add outputs to show inference time diff

* Small fixes regarding hardware requirements and others

* Update based on comments

* Small fixes

* Small fixes
This commit is contained in:
Yuwen Hu 2023-02-01 13:33:54 +08:00 committed by GitHub
parent 08a556239c
commit c7674f52b2
3 changed files with 6 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
docs/readthedocs/source/_toc.yml
View file

@ -133,6 +133,7 @@ subtrees:
- file: doc/Nano/Howto/Inference/PyTorch/inference_optimizer_optimize - file: doc/Nano/Howto/Inference/PyTorch/inference_optimizer_optimize
- file: doc/Nano/Howto/Inference/TensorFlow/accelerate_tensorflow_inference_onnx - file: doc/Nano/Howto/Inference/TensorFlow/accelerate_tensorflow_inference_onnx
- file: doc/Nano/Howto/Inference/TensorFlow/accelerate_tensorflow_inference_openvino - file: doc/Nano/Howto/Inference/TensorFlow/accelerate_tensorflow_inference_openvino
- file: doc/Nano/Howto/Inference/TensorFlow/tensorflow_inference_bf16
- file: doc/Nano/Howto/Inference/TensorFlow/tensorflow_save_and_load_onnx - file: doc/Nano/Howto/Inference/TensorFlow/tensorflow_save_and_load_onnx
- file: doc/Nano/Howto/Inference/TensorFlow/tensorflow_save_and_load_openvino - file: doc/Nano/Howto/Inference/TensorFlow/tensorflow_save_and_load_openvino
- file: doc/Nano/Howto/install_in_colab - file: doc/Nano/Howto/install_in_colab

3
docs/readthedocs/source/doc/Nano/Howto/Inference/TensorFlow/tensorflow_inference_bf16.nblink Normal file
View file

@ -0,0 +1,3 @@
{
"path": "../../../../../../../../python/nano/tutorial/notebook/inference/tensorflow/tensorflow_inference_bf16.ipynb"
}

3
docs/readthedocs/source/doc/Nano/Howto/index.rst
View file

@ -41,7 +41,7 @@ TensorFlow
~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~
* `How to accelerate a TensorFlow Keras application on training workloads through multiple instances <Training/TensorFlow/accelerate_tensorflow_training_multi_instance.html>`_ * `How to accelerate a TensorFlow Keras application on training workloads through multiple instances <Training/TensorFlow/accelerate_tensorflow_training_multi_instance.html>`_
* |tensorflow_training_embedding_sparseadam_link|_ * |tensorflow_training_embedding_sparseadam_link|_
* `How to conduct BFloat16 Mixed Precision training in your TensorFlow application <Training/TensorFlow/tensorflow_training_bf16.html>`_ * `How to conduct BFloat16 Mixed Precision training in your TensorFlow Keras application <Training/TensorFlow/tensorflow_training_bf16.html>`_
.. |tensorflow_training_embedding_sparseadam_link| replace:: How to optimize your model with a sparse ``Embedding`` layer and ``SparseAdam`` optimizer .. |tensorflow_training_embedding_sparseadam_link| replace:: How to optimize your model with a sparse ``Embedding`` layer and ``SparseAdam`` optimizer
.. _tensorflow_training_embedding_sparseadam_link: Training/TensorFlow/tensorflow_training_embedding_sparseadam.html .. _tensorflow_training_embedding_sparseadam_link: Training/TensorFlow/tensorflow_training_embedding_sparseadam.html
@ -83,6 +83,7 @@ TensorFlow
~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~
* `How to accelerate a TensorFlow inference pipeline through ONNXRuntime <Inference/TensorFlow/accelerate_tensorflow_inference_onnx.html>`_ * `How to accelerate a TensorFlow inference pipeline through ONNXRuntime <Inference/TensorFlow/accelerate_tensorflow_inference_onnx.html>`_
* `How to accelerate a TensorFlow inference pipeline through OpenVINO <Inference/TensorFlow/accelerate_tensorflow_inference_openvino.html>`_ * `How to accelerate a TensorFlow inference pipeline through OpenVINO <Inference/TensorFlow/accelerate_tensorflow_inference_openvino.html>`_
* `How to conduct BFloat16 Mixed Precision inference in a TensorFlow Keras application <Inference/TensorFlow/tensorflow_inference_bf16.html>`_
* `How to save and load optimized ONNXRuntime model in TensorFlow <Inference/TensorFlow/tensorflow_save_and_load_onnx.html>`_ * `How to save and load optimized ONNXRuntime model in TensorFlow <Inference/TensorFlow/tensorflow_save_and_load_onnx.html>`_
* `How to save and load optimized OpenVINO model in TensorFlow <Inference/TensorFlow/tensorflow_save_and_load_openvino.html>`_ * `How to save and load optimized OpenVINO model in TensorFlow <Inference/TensorFlow/tensorflow_save_and_load_openvino.html>`_