ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

[Nano] How-To Guides: Accelerate PyTorch Training with IPEX & Multi-instance & BF16 & Channels last (#7035)

Browse source 
* add pytorch training ipex guide

* add pytorch training multi-instance guide

* add bf16 guide

* small changes of presentation

* add channels last guide

* remove validation loader

* hide code block

* update based on comments

* add guide for reference

* update guides w.r.t. comments
This commit is contained in:
Sirui Tao 2023-01-10 13:14:47 +08:00 committed by GitHub
parent af9cdc6edd
commit d950992b91
6 changed files with 20 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
docs/readthedocs/source/_toc.yml
View file

@ -108,6 +108,10 @@ subtrees:
- file: doc/Nano/Howto/Training/PyTorchLightning/pytorch_lightning_training_bf16 - file: doc/Nano/Howto/Training/PyTorchLightning/pytorch_lightning_training_bf16
- file: doc/Nano/Howto/Training/PyTorch/convert_pytorch_training_torchnano - file: doc/Nano/Howto/Training/PyTorch/convert_pytorch_training_torchnano
- file: doc/Nano/Howto/Training/PyTorch/use_nano_decorator_pytorch_training - file: doc/Nano/Howto/Training/PyTorch/use_nano_decorator_pytorch_training
- file: doc/Nano/Howto/Training/PyTorch/accelerate_pytorch_training_ipex
- file: doc/Nano/Howto/Training/PyTorch/accelerate_pytorch_training_multi_instance
- file: doc/Nano/Howto/Training/PyTorch/pytorch_training_channels_last
- file: doc/Nano/Howto/Training/PyTorch/accelerate_pytorch_training_bf16
- file: doc/Nano/Howto/Training/TensorFlow/accelerate_tensorflow_training_multi_instance - file: doc/Nano/Howto/Training/TensorFlow/accelerate_tensorflow_training_multi_instance
- file: doc/Nano/Howto/Training/TensorFlow/tensorflow_training_embedding_sparseadam - file: doc/Nano/Howto/Training/TensorFlow/tensorflow_training_embedding_sparseadam
- file: doc/Nano/Howto/Training/TensorFlow/tensorflow_training_bf16 - file: doc/Nano/Howto/Training/TensorFlow/tensorflow_training_bf16

3
docs/readthedocs/source/doc/Nano/Howto/Training/PyTorch/accelerate_pytorch_training_bf16.nblink Normal file
View file

@ -0,0 +1,3 @@
{
"path": "../../../../../../../../python/nano/tutorial/notebook/training/pytorch/accelerate_pytorch_training_bf16.ipynb"
}

3
docs/readthedocs/source/doc/Nano/Howto/Training/PyTorch/accelerate_pytorch_training_ipex.nblink Normal file
View file

@ -0,0 +1,3 @@
{
"path": "../../../../../../../../python/nano/tutorial/notebook/training/pytorch/accelerate_pytorch_training_ipex.ipynb"
}

3
docs/readthedocs/source/doc/Nano/Howto/Training/PyTorch/accelerate_pytorch_training_multi_instance.nblink Normal file
View file

@ -0,0 +1,3 @@
{
"path": "../../../../../../../../python/nano/tutorial/notebook/training/pytorch/accelerate_pytorch_training_multi_instance.ipynb"
}

3
docs/readthedocs/source/doc/Nano/Howto/Training/PyTorch/pytorch_training_channels_last.nblink Normal file
View file

@ -0,0 +1,3 @@
{
"path": "../../../../../../../../python/nano/tutorial/notebook/training/pytorch/pytorch_training_channels_last.ipynb"
}

4
docs/readthedocs/source/doc/Nano/Howto/index.rst
View file

@ -27,6 +27,10 @@ PyTorch
~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~
* |convert_pytorch_training_torchnano|_ * |convert_pytorch_training_torchnano|_
* |use_nano_decorator_pytorch_training|_ * |use_nano_decorator_pytorch_training|_
* `How to accelerate a PyTorch application on training workloads through Intel® Extension for PyTorch* <Training/PyTorch/accelerate_pytorch_training_ipex.html>`_
* `How to accelerate a PyTorch application on training workloads through multiple instances <Training/PyTorch/accelerate_pytorch_training_multi_instance.html>`_
* `How to use the channels last memory format in your PyTorch application for training <Training/PyTorch/pytorch_training_channels_last.html>`_
* `How to conduct BFloat16 Mixed Precision training in your PyTorch application <Training/PyTorch/accelerate_pytorch_training_bf16.html>`_
.. |use_nano_decorator_pytorch_training| replace:: How to accelerate your PyTorch training loop with ``@nano`` decorator .. |use_nano_decorator_pytorch_training| replace:: How to accelerate your PyTorch training loop with ``@nano`` decorator
.. _use_nano_decorator_pytorch_training: Training/PyTorch/use_nano_decorator_pytorch_training.html .. _use_nano_decorator_pytorch_training: Training/PyTorch/use_nano_decorator_pytorch_training.html