add windows user guide (#5950)

* add windows user guide

* modifiy win user guides

* Add jupyter setup guide and known issues

* Add tips for yolov3 example

* update win guide

* update win guide

* update win guide reference

* update win guide reference

* update win guide reference

* format fix

* revise nano and chronos windows guide

* fix windows guide reference in nano install

Co-authored-by: aixideng <aixi.deng@intel.com>
This commit is contained in:
Shengsheng Huang 2022-10-11 13:14:40 +08:00 committed by GitHub
parent fb0f9a35b9
commit fbd5a57f93
8 changed files with 167 additions and 85 deletions

View file

@ -12,6 +12,7 @@ subtrees:
entries: entries:
- file: doc/UserGuide/python - file: doc/UserGuide/python
- file: doc/UserGuide/scala - file: doc/UserGuide/scala
- file: doc/UserGuide/win
- file: doc/UserGuide/colab - file: doc/UserGuide/colab
- file: doc/UserGuide/docker - file: doc/UserGuide/docker
- file: doc/UserGuide/hadoop - file: doc/UserGuide/hadoop
@ -23,7 +24,6 @@ subtrees:
- caption: Nano - caption: Nano
entries: entries:
- file: doc/Nano/Overview/nano - file: doc/Nano/Overview/nano
- file: doc/Nano/Overview/windows_guide
- file: doc/Nano/QuickStart/pytorch_train - file: doc/Nano/QuickStart/pytorch_train
- file: doc/Nano/QuickStart/pytorch_inference - file: doc/Nano/QuickStart/pytorch_inference
- file: doc/Nano/QuickStart/tensorflow_train - file: doc/Nano/QuickStart/tensorflow_train

View file

@ -2,29 +2,9 @@
## Step 1: Install WSL2 ## Step 1: Install WSL2
For Windows, we recommend using Windows Subsystem for Linux 2 (WSL2) to run BigDL-Chronos. Follow [BigDL Windows User guide](../../UserGuide/win.md) to install WSL2.
To install WSL2, you can now install everything you need by entering this command in an administrator PowerShell or Windows Command Prompt and then restarting your machine.
```powershell ## Step 2: Install Chronos
wsl --install
```
By default, this command should install the latest required components for WSL2 and install Ubuntu as default distribution for you. Follow the [Chronos Installation guide](../Overview/chronos.md#install) to install Chronos.
To run this command, you must be running Windows 10 version 2004 and higher (Build 19041 and higher) or Windows 11. If you're running an older build, or just prefer not to use the install command and would like step-by-step directions, see WSL manual installation steps for older versions.
To learn more about installation of WSL2, please Follow [this guide](https://docs.microsoft.com/en-us/windows/wsl/install-win10).
## Step 2: Install conda in WSL2
Start a new WSL2 window and setup the user information. Then download and install the conda.
```bash
wget https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh
chmod +x Miniconda3-4.5.4-Linux-x86_64.sh
./Miniconda3-4.5.4-Linux-x86_64.sh
```
## Step 3: Install Chronos
For more installation options, please refer to [Chronos User Guide](https://bigdl.readthedocs.io/en/latest/doc/Chronos/Overview/chronos.html#install).

View file

@ -75,9 +75,11 @@ PyTorch
Install Install
------------------------- -------------------------
* `How to install BigDL-Nano in Google Colab <install_in_colab.html>`_ * `How to install BigDL-Nano in Google Colab <install_in_colab.html>`_
* `How to install BigDL-Nano on Windows <windows_guide.html>`_
.. toctree:: .. toctree::
:maxdepth: 1 :maxdepth: 1
:hidden: :hidden:
install_in_colab install_in_colab
windows_guide

View file

@ -0,0 +1,37 @@
# Install BigDL-Nano on Windows
## Step 1: Install WSL2
Follow [BigDL Windows User guide](../../UserGuide/win.md) to install WSL2.
## Step 2: Install conda in WSL2
It is highly recommended to use conda to manage the python environment for BigDL-Nano. Follow [BigDL Windows User Guide/Conda Install](../../UserGuide/win.md#install-conda) to install conda.
## Step 3: Create a BigDL-Nano env
Use conda to create a new environment. For example, use `bigdl-nano` as the new environment name:
```bash
conda create -n bigdl-nano
conda activate bigdl-nano
```
## Step 4: Install BigDL-Nano from Pypi
You can install BigDL-Nano from Pypi with `pip`. Specifically, for PyTorch extensions, please run:
```
pip install bigdl-nano[pytorch]
source bigdl-nano-init
```
For Tensorflow:
```
pip install bigdl-nano[tensorflow]
source bigdl-nano-init
```

View file

@ -7,7 +7,7 @@ BigDL Nano is a Python package to transparently accelerate PyTorch and TensorFlo
--- ---
## **2. Install** ## **2. Install**
Note: For windows users, we recommend using Windows Subsystem for Linux 2 (WSL2) to run BigDL-Nano. Please refer [here](./windows_guide.md) for instructions. Note: For windows users, we recommend using Windows Subsystem for Linux 2 (WSL2) to run BigDL-Nano. Please refer to [Nano Windows install guide](../Howto/windows_guide.md) for instructions.
BigDL-Nano can be installed using pip and we recommend installing BigDL-Nano in a conda environment. BigDL-Nano can be installed using pip and we recommend installing BigDL-Nano in a conda environment.
@ -27,13 +27,13 @@ conda activate env
pip install bigdl-nano[tensorflow] pip install bigdl-nano[tensorflow]
``` ```
After installing bigdl-nano, you can run the following command to setup a few environment variables. After installing bigdl-nano, you can run the following command to setup a few environment variables.
```bash ```bash
source bigdl-nano-init source bigdl-nano-init
``` ```
The `bigdl-nano-init` scripts will export a few environment variable according to your hardware to maximize performance. The `bigdl-nano-init` scripts will export a few environment variable according to your hardware to maximize performance.
In a conda environment, `source bigdl-nano-init` will also be added to `$CONDA_PREFIX/etc/conda/activate.d/`, which will automaticly run when you activate your current environment. In a conda environment, `source bigdl-nano-init` will also be added to `$CONDA_PREFIX/etc/conda/activate.d/`, which will automaticly run when you activate your current environment.

View file

@ -1,53 +0,0 @@
# Windows User Guide
## Step 1: Install WSL2
For Windows, we recommend using Windows Subsystem for Linux 2 (WSL2) to run BigDL-Nano.
To install WSL2, you can now install everything you need by entering this command in an administrator PowerShell or Windows Command Prompt and then restarting your machine.
```powershell
wsl --install
```
By default, this command should install the latest required components for WSL2 and install Ubuntu as default distribution for you.
To run this command, you must be running Windows 10 version 2004 and higher (Build 19041 and higher) or Windows 11. If you're running an older build, or just prefer not to use the install command and would like step-by-step directions, see WSL manual installation steps for older versions.
To learn more about installation of WSL2, please Follow [this guide](https://docs.microsoft.com/en-us/windows/wsl/install-win10).
## Step 2: Install conda in WSL2
Start a new WSL2 window and set up the user information. Then download and install the conda.
```bash
wget https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh
chmod +x Miniconda3-4.5.4-Linux-x86_64.sh
./Miniconda3-4.5.4-Linux-x86_64.sh
```
## Step 3: Create a BigDL-Nano env
Use conda to create a new environment. For example, use `bigdl-nano` as the new environment name:
```bash
conda create -n bigdl-nano
conda activate bigdl-nano
```
## Step 4: Install BigDL-Nano from Pypi
You can install BigDL-Nano from Pypi with `pip`. Specifically, for PyTorch extensions, please run:
```
pip install bigdl-nano[pytorch]
source bigdl-nano-init
```
For Tensorflow:
```
pip install bigdl-nano[tensorflow]
source bigdl-nano-init
```

View file

@ -1,8 +1,8 @@
# Python User Guide # Python User Guide
--- ---
Supported Platforms: Linux and macOS. _**Note:** Windows is currently not supported._ Supported Platforms: Linux and macOS. For Windows, Refer to [Windows User Guide](./win.md).
### **1. Install** ### **1. Install**
- We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the Python environment as follows: - We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the Python environment as follows:
@ -51,7 +51,7 @@ Alternatively, you can find the list of the nightly build versions [here](https:
pip install bigdl==version pip install bigdl==version
``` ```
_**Note:** If you are using a custom URL of Python Package Index, you may need to check whether the latest packages have been sync'ed with pypi. _**Note:** If you are using a custom URL of Python Package Index, you may need to check whether the latest packages have been sync'ed with pypi.
Or you can add the option `-i https://pypi.python.org/simple` when pip install to use pypi as the index-url._ Or you can add the option `-i https://pypi.python.org/simple` when pip install to use pypi as the index-url._
You could uninstall all the packages of BigDL as follows: You could uninstall all the packages of BigDL as follows:
@ -113,7 +113,7 @@ python script.py
--- ---
### **3. Python Dependencies** ### **3. Python Dependencies**
We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to manage your Python dependencies. Libraries installed in the current conda environment will be automatically distributed to the cluster when calling `init_orca_context`. You can also add extra dependencies as `.py`, `.zip` and `.egg` files by specifying `extra_python_lib` argument in `init_orca_context`. We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to manage your Python dependencies. Libraries installed in the current conda environment will be automatically distributed to the cluster when calling `init_orca_context`. You can also add extra dependencies as `.py`, `.zip` and `.egg` files by specifying `extra_python_lib` argument in `init_orca_context`.
For more details, please refer to [Orca Context](../Orca/Overview/orca-context.md). For more details, please refer to [Orca Context](../Orca/Overview/orca-context.md).

View file

@ -0,0 +1,116 @@
# Windows User Guide
## Prerequisite
### Confirm your windows version
To use BigDL on Windows, we recommend using [Windows Subsystem for Linux 2 (WSL2)](https://learn.microsoft.com/en-us/windows/wsl/about#what-is-wsl-2). The recommended Windows versions are Windows 10 version 2004 or higher (Build 19041 and higher), or Windows 11.
### Install WSL2
To install WSL2, simply open a PowerShell or Windows Command Prompt as **administrator** and enter the below command. Restart your machine and wait until WSL2 is successfully installed.
```powershell
wsl --install
```
```eval_rst
.. note::
By default, the above command installs the latest required components for WSL2 and **Ubuntu** as default Linux distribution, and it requires Windows 10 version 2004 or higher. If you're using older versions of Windows or need customization, please refer to `WSL installation guide <https://learn.microsoft.com/en-us/windows/wsl/install>`_.
```
## Installation Guide
You can treat WSL2 shell as a normal Linux shell and run normal bash commands in it. If you're using WSL2 shell for the first time, it may require you to set up some user information. Using WSL2, you can install BigDL the same way as you do on a Linux system.
### Install Conda
Conda is the recommend way to manage the BigDL environment. Download and install conda using below commands.
```bash
wget https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh
chmod +x Miniconda3-4.5.4-Linux-x86_64.sh
./Miniconda3-4.5.4-Linux-x86_64.sh
```
```eval_rst
.. note::
On WSL2, you need to use a Linux version of Conda intead of a Windows version. For other available conda versions, refer to `conda install <https://conda.io/projects/conda/en/latest/user-guide/install/index.html>`_, or `miniconda install <https://docs.conda.io/en/main/miniconda.html>`_.
```
### Install BigDL
After installing conda, use conda to create and activate an environment for bigdl.
```bash
conda create -n bigdl-env
conda activate bigdl-env
```
Then install BigDL as a whole, or specific bigdl library the same way as you do on a Linux system. For example,
```bash
pip install bigdl
```
```eval_rst
.. card::
**Related Readings**
^^^
* `BigDL Installation Guide <../UserGuide/python>`_
* `Nano Installation Guide <../Nano/Overview/nano.html#install>`_
* `Chronos Installation Guide <../Chronos/Overview/chronos.html#install>`_
```
### Setup Jupyter Notebook Environment
Fist, install JupyterLab using pip:
```bash
pip install jupyterlab
```
Then start JupyterLab using:
```bash
jupyter lab
```
```eval_rst
.. note::
Once you started Juypterlab, it will open automatically in your browser. If it does not open automatically, you can manually enter the notebook servers URL into the browser (The URL is shown on the terminal where you run the command). The default workspace of jupyter is located at the directory where you start the jupyterlab. For more information about JupyterLab installation and usage, refer to `JupyterLab User Guide <https://jupyterlab.readthedocs.io/en/stable/getting_started/overview.html#>`_.
```
## Tips and Known Issues
### 1.ImportError: libgomp.so.1: cannot open shared object file: No such file or directory
This error may appear when you try to import torch. This is caused by Ubuntu 14.04 or later not installing libgomp1 by default. Just install libgomp1 to resolve it:
```bash
sudo apt-get install libgomp1
```
### 2.Slow PyTorch training with BF16
Using BFloat16 mixed precision in PyTorch or PyTorch-Lightning training may be much slower than FP32.
### 3.ERROR: Could not build wheels for pycocotools, which is required to install pyproject.toml-based projects
pycocotools is a dependency of Intel neural-compressor which is used for inference quantization in BigDL-Nano. This error is usually caused by GCC library not installed in system. Just install gcc to resolve it:
```bash
sudo apt-get install gcc
```
### 4.ValueError: After taking into account object store and redis memory usage, the amount of memory on this node available for tasks and actors is less than -75% of total.
When running ray applications, you need to set the `memory` and `object_store_memory` properly according to your system memory capacity. This error indicates you have used too large memory configurations and you need to decrease them. For example on a laptop with 8G memory, you may set the memory configurations as below:
```bash
python yoloV3.py --memory 2g --object_store_memory 1g
```