ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

[Nano] Add how-to guide for TensorFlow Keras training using bf16 (#7169)

Browse source 
* Add basic doc structure for bf16 tf training how-to guide, and change the incorrect order of tf inference guides in toc

* Add how-to guide for tf bf16 training

* Add warning box for tf bf16 hardware limitations

* Add a print message to show the default policy of model after unpatch

* Small fixes

* Small github action fixes for tf bf16 training how-to guide

* diable action test for tf bf16 train for now, due to the core dumped problem on platforms without AVX512

* Updated based on comments
This commit is contained in:
Yuwen Hu 2023-01-05 17:59:43 +08:00 committed by GitHub
parent 3fd27f51a8
commit bfca337d09
3 changed files with 7 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
docs/readthedocs/source/_toc.yml
View file

@ -110,8 +110,7 @@ subtrees:
- file: doc/Nano/Howto/Training/PyTorch/use_nano_decorator_pytorch_training - file: doc/Nano/Howto/Training/PyTorch/use_nano_decorator_pytorch_training
- file: doc/Nano/Howto/Training/TensorFlow/accelerate_tensorflow_training_multi_instance - file: doc/Nano/Howto/Training/TensorFlow/accelerate_tensorflow_training_multi_instance
- file: doc/Nano/Howto/Training/TensorFlow/tensorflow_training_embedding_sparseadam - file: doc/Nano/Howto/Training/TensorFlow/tensorflow_training_embedding_sparseadam
- file: doc/Nano/Howto/Inference/TensorFlow/accelerate_tensorflow_inference_onnx - file: doc/Nano/Howto/Training/TensorFlow/tensorflow_training_bf16
- file: doc/Nano/Howto/Inference/TensorFlow/accelerate_tensorflow_inference_openvino
- file: doc/Nano/Howto/Training/General/choose_num_processes_training - file: doc/Nano/Howto/Training/General/choose_num_processes_training
- file: doc/Nano/Howto/Inference/OpenVINO/openvino_inference - file: doc/Nano/Howto/Inference/OpenVINO/openvino_inference
- file: doc/Nano/Howto/Inference/OpenVINO/openvino_inference_async - file: doc/Nano/Howto/Inference/OpenVINO/openvino_inference_async
@ -127,6 +126,8 @@ subtrees:
- file: doc/Nano/Howto/Inference/PyTorch/pytorch_save_and_load_onnx - file: doc/Nano/Howto/Inference/PyTorch/pytorch_save_and_load_onnx
- file: doc/Nano/Howto/Inference/PyTorch/pytorch_save_and_load_openvino - file: doc/Nano/Howto/Inference/PyTorch/pytorch_save_and_load_openvino
- file: doc/Nano/Howto/Inference/PyTorch/inference_optimizer_optimize - file: doc/Nano/Howto/Inference/PyTorch/inference_optimizer_optimize
- file: doc/Nano/Howto/Inference/TensorFlow/accelerate_tensorflow_inference_onnx
- file: doc/Nano/Howto/Inference/TensorFlow/accelerate_tensorflow_inference_openvino
- file: doc/Nano/Howto/install_in_colab - file: doc/Nano/Howto/install_in_colab
- file: doc/Nano/Howto/windows_guide - file: doc/Nano/Howto/windows_guide
- file: doc/Nano/Overview/known_issues - file: doc/Nano/Overview/known_issues

3
docs/readthedocs/source/doc/Nano/Howto/Training/TensorFlow/tensorflow_training_bf16.nblink Normal file
View file

@ -0,0 +1,3 @@
{
"path": "../../../../../../../../python/nano/tutorial/notebook/training/tensorflow/tensorflow_training_bf16.ipynb"
}

1
docs/readthedocs/source/doc/Nano/Howto/index.rst
View file

@ -30,6 +30,7 @@ TensorFlow
~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~
* `How to accelerate a TensorFlow Keras application on training workloads through multiple instances <Training/TensorFlow/accelerate_tensorflow_training_multi_instance.html>`_ * `How to accelerate a TensorFlow Keras application on training workloads through multiple instances <Training/TensorFlow/accelerate_tensorflow_training_multi_instance.html>`_
* |tensorflow_training_embedding_sparseadam_link|_ * |tensorflow_training_embedding_sparseadam_link|_
* `How to conduct BFloat16 Mixed Precision training in your TensorFlow application <Training/TensorFlow/tensorflow_training_bf16.html>`_
.. |tensorflow_training_embedding_sparseadam_link| replace:: How to optimize your model with a sparse ``Embedding`` layer and ``SparseAdam`` optimizer .. |tensorflow_training_embedding_sparseadam_link| replace:: How to optimize your model with a sparse ``Embedding`` layer and ``SparseAdam`` optimizer
.. _tensorflow_training_embedding_sparseadam_link: Training/TensorFlow/tensorflow_training_embedding_sparseadam.html .. _tensorflow_training_embedding_sparseadam_link: Training/TensorFlow/tensorflow_training_embedding_sparseadam.html