add windows user guide (#5950)
* add windows user guide * modifiy win user guides * Add jupyter setup guide and known issues * Add tips for yolov3 example * update win guide * update win guide * update win guide reference * update win guide reference * update win guide reference * format fix * revise nano and chronos windows guide * fix windows guide reference in nano install Co-authored-by: aixideng <aixi.deng@intel.com>
This commit is contained in:
Shengsheng Huang
GitHub
parent
fb0f9a35b9
commit
fbd5a57f93
8 changed files with 167 additions and 85 deletions
|
|
@ -12,6 +12,7 @@ subtrees:
|
|||
entries:
|
||||
- file: doc/UserGuide/python
|
||||
- file: doc/UserGuide/scala
|
||||
- file: doc/UserGuide/win
|
||||
- file: doc/UserGuide/colab
|
||||
- file: doc/UserGuide/docker
|
||||
- file: doc/UserGuide/hadoop
|
||||
|
|
@ -23,7 +24,6 @@ subtrees:
|
|||
- caption: Nano
|
||||
entries:
|
||||
- file: doc/Nano/Overview/nano
|
||||
- file: doc/Nano/Overview/windows_guide
|
||||
- file: doc/Nano/QuickStart/pytorch_train
|
||||
- file: doc/Nano/QuickStart/pytorch_inference
|
||||
- file: doc/Nano/QuickStart/tensorflow_train
|
||||
|
|
|
|||
|
|
@ -2,29 +2,9 @@
|
|||
|
||||
## Step 1: Install WSL2
|
||||
|
||||
For Windows, we recommend using Windows Subsystem for Linux 2 (WSL2) to run BigDL-Chronos.
|
||||
Follow [BigDL Windows User guide](../../UserGuide/win.md) to install WSL2.
|
||||
|
||||
To install WSL2, you can now install everything you need by entering this command in an administrator PowerShell or Windows Command Prompt and then restarting your machine.
|
||||
|
||||
```powershell
|
||||
wsl --install
|
||||
```
|
||||
## Step 2: Install Chronos
|
||||
|
||||
By default, this command should install the latest required components for WSL2 and install Ubuntu as default distribution for you.
|
||||
|
||||
To run this command, you must be running Windows 10 version 2004 and higher (Build 19041 and higher) or Windows 11. If you're running an older build, or just prefer not to use the install command and would like step-by-step directions, see WSL manual installation steps for older versions.
|
||||
|
||||
To learn more about installation of WSL2, please Follow [this guide](https://docs.microsoft.com/en-us/windows/wsl/install-win10).
|
||||
|
||||
## Step 2: Install conda in WSL2
|
||||
|
||||
Start a new WSL2 window and setup the user information. Then download and install the conda.
|
||||
|
||||
```bash
|
||||
wget https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh
|
||||
chmod +x Miniconda3-4.5.4-Linux-x86_64.sh
|
||||
./Miniconda3-4.5.4-Linux-x86_64.sh
|
||||
```
|
||||
|
||||
## Step 3: Install Chronos
|
||||
For more installation options, please refer to [Chronos User Guide](https://bigdl.readthedocs.io/en/latest/doc/Chronos/Overview/chronos.html#install).
|
||||
Follow the [Chronos Installation guide](../Overview/chronos.md#install) to install Chronos.
|
||||
|
|
|
|||
|
|
@ -75,9 +75,11 @@ PyTorch
|
|||
Install
|
||||
-------------------------
|
||||
* `How to install BigDL-Nano in Google Colab <install_in_colab.html>`_
|
||||
* `How to install BigDL-Nano on Windows <windows_guide.html>`_
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
install_in_colab
|
||||
install_in_colab
|
||||
windows_guide
|
||||
37
docs/readthedocs/source/doc/Nano/Howto/windows_guide.md
Normal file
37
docs/readthedocs/source/doc/Nano/Howto/windows_guide.md
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
# Install BigDL-Nano on Windows
|
||||
|
||||
## Step 1: Install WSL2
|
||||
|
||||
|
||||
Follow [BigDL Windows User guide](../../UserGuide/win.md) to install WSL2.
|
||||
|
||||
|
||||
## Step 2: Install conda in WSL2
|
||||
|
||||
It is highly recommended to use conda to manage the python environment for BigDL-Nano. Follow [BigDL Windows User Guide/Conda Install](../../UserGuide/win.md#install-conda) to install conda.
|
||||
|
||||
## Step 3: Create a BigDL-Nano env
|
||||
|
||||
Use conda to create a new environment. For example, use `bigdl-nano` as the new environment name:
|
||||
|
||||
```bash
|
||||
conda create -n bigdl-nano
|
||||
conda activate bigdl-nano
|
||||
```
|
||||
|
||||
|
||||
## Step 4: Install BigDL-Nano from Pypi
|
||||
|
||||
You can install BigDL-Nano from Pypi with `pip`. Specifically, for PyTorch extensions, please run:
|
||||
|
||||
```
|
||||
pip install bigdl-nano[pytorch]
|
||||
source bigdl-nano-init
|
||||
```
|
||||
|
||||
For Tensorflow:
|
||||
|
||||
```
|
||||
pip install bigdl-nano[tensorflow]
|
||||
source bigdl-nano-init
|
||||
```
|
||||
|
|
@ -7,7 +7,7 @@ BigDL Nano is a Python package to transparently accelerate PyTorch and TensorFlo
|
|||
---
|
||||
## **2. Install**
|
||||
|
||||
Note: For windows users, we recommend using Windows Subsystem for Linux 2 (WSL2) to run BigDL-Nano. Please refer [here](./windows_guide.md) for instructions.
|
||||
Note: For windows users, we recommend using Windows Subsystem for Linux 2 (WSL2) to run BigDL-Nano. Please refer to [Nano Windows install guide](../Howto/windows_guide.md) for instructions.
|
||||
|
||||
BigDL-Nano can be installed using pip and we recommend installing BigDL-Nano in a conda environment.
|
||||
|
||||
|
|
@ -27,13 +27,13 @@ conda activate env
|
|||
pip install bigdl-nano[tensorflow]
|
||||
```
|
||||
|
||||
After installing bigdl-nano, you can run the following command to setup a few environment variables.
|
||||
After installing bigdl-nano, you can run the following command to setup a few environment variables.
|
||||
|
||||
```bash
|
||||
source bigdl-nano-init
|
||||
```
|
||||
|
||||
The `bigdl-nano-init` scripts will export a few environment variable according to your hardware to maximize performance.
|
||||
The `bigdl-nano-init` scripts will export a few environment variable according to your hardware to maximize performance.
|
||||
|
||||
In a conda environment, `source bigdl-nano-init` will also be added to `$CONDA_PREFIX/etc/conda/activate.d/`, which will automaticly run when you activate your current environment.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,53 +0,0 @@
|
|||
# Windows User Guide
|
||||
|
||||
## Step 1: Install WSL2
|
||||
|
||||
For Windows, we recommend using Windows Subsystem for Linux 2 (WSL2) to run BigDL-Nano.
|
||||
|
||||
To install WSL2, you can now install everything you need by entering this command in an administrator PowerShell or Windows Command Prompt and then restarting your machine.
|
||||
|
||||
```powershell
|
||||
wsl --install
|
||||
```
|
||||
|
||||
By default, this command should install the latest required components for WSL2 and install Ubuntu as default distribution for you.
|
||||
|
||||
To run this command, you must be running Windows 10 version 2004 and higher (Build 19041 and higher) or Windows 11. If you're running an older build, or just prefer not to use the install command and would like step-by-step directions, see WSL manual installation steps for older versions.
|
||||
|
||||
To learn more about installation of WSL2, please Follow [this guide](https://docs.microsoft.com/en-us/windows/wsl/install-win10).
|
||||
|
||||
## Step 2: Install conda in WSL2
|
||||
|
||||
Start a new WSL2 window and set up the user information. Then download and install the conda.
|
||||
|
||||
```bash
|
||||
wget https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh
|
||||
chmod +x Miniconda3-4.5.4-Linux-x86_64.sh
|
||||
./Miniconda3-4.5.4-Linux-x86_64.sh
|
||||
```
|
||||
|
||||
## Step 3: Create a BigDL-Nano env
|
||||
|
||||
Use conda to create a new environment. For example, use `bigdl-nano` as the new environment name:
|
||||
|
||||
```bash
|
||||
conda create -n bigdl-nano
|
||||
conda activate bigdl-nano
|
||||
```
|
||||
|
||||
|
||||
## Step 4: Install BigDL-Nano from Pypi
|
||||
|
||||
You can install BigDL-Nano from Pypi with `pip`. Specifically, for PyTorch extensions, please run:
|
||||
|
||||
```
|
||||
pip install bigdl-nano[pytorch]
|
||||
source bigdl-nano-init
|
||||
```
|
||||
|
||||
For Tensorflow:
|
||||
|
||||
```
|
||||
pip install bigdl-nano[tensorflow]
|
||||
source bigdl-nano-init
|
||||
```
|
||||
|
|
@ -1,8 +1,8 @@
|
|||
# Python User Guide
|
||||
|
||||
---
|
||||
Supported Platforms: Linux and macOS. _**Note:** Windows is currently not supported._
|
||||
|
||||
Supported Platforms: Linux and macOS. For Windows, Refer to [Windows User Guide](./win.md).
|
||||
|
||||
### **1. Install**
|
||||
- We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the Python environment as follows:
|
||||
|
||||
|
|
@ -51,7 +51,7 @@ Alternatively, you can find the list of the nightly build versions [here](https:
|
|||
pip install bigdl==version
|
||||
```
|
||||
|
||||
_**Note:** If you are using a custom URL of Python Package Index, you may need to check whether the latest packages have been sync'ed with pypi.
|
||||
_**Note:** If you are using a custom URL of Python Package Index, you may need to check whether the latest packages have been sync'ed with pypi.
|
||||
Or you can add the option `-i https://pypi.python.org/simple` when pip install to use pypi as the index-url._
|
||||
|
||||
You could uninstall all the packages of BigDL as follows:
|
||||
|
|
@ -113,7 +113,7 @@ python script.py
|
|||
---
|
||||
### **3. Python Dependencies**
|
||||
|
||||
We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to manage your Python dependencies. Libraries installed in the current conda environment will be automatically distributed to the cluster when calling `init_orca_context`. You can also add extra dependencies as `.py`, `.zip` and `.egg` files by specifying `extra_python_lib` argument in `init_orca_context`.
|
||||
We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to manage your Python dependencies. Libraries installed in the current conda environment will be automatically distributed to the cluster when calling `init_orca_context`. You can also add extra dependencies as `.py`, `.zip` and `.egg` files by specifying `extra_python_lib` argument in `init_orca_context`.
|
||||
|
||||
For more details, please refer to [Orca Context](../Orca/Overview/orca-context.md).
|
||||
|
||||
|
|
|
|||
116
docs/readthedocs/source/doc/UserGuide/win.md
Normal file
116
docs/readthedocs/source/doc/UserGuide/win.md
Normal file
|
|
@ -0,0 +1,116 @@
|
|||
# Windows User Guide
|
||||
## Prerequisite
|
||||
|
||||
|
||||
### Confirm your windows version
|
||||
|
||||
To use BigDL on Windows, we recommend using [Windows Subsystem for Linux 2 (WSL2)](https://learn.microsoft.com/en-us/windows/wsl/about#what-is-wsl-2). The recommended Windows versions are Windows 10 version 2004 or higher (Build 19041 and higher), or Windows 11.
|
||||
|
||||
|
||||
### Install WSL2
|
||||
|
||||
To install WSL2, simply open a PowerShell or Windows Command Prompt as **administrator** and enter the below command. Restart your machine and wait until WSL2 is successfully installed.
|
||||
|
||||
```powershell
|
||||
wsl --install
|
||||
```
|
||||
|
||||
```eval_rst
|
||||
.. note::
|
||||
By default, the above command installs the latest required components for WSL2 and **Ubuntu** as default Linux distribution, and it requires Windows 10 version 2004 or higher. If you're using older versions of Windows or need customization, please refer to `WSL installation guide <https://learn.microsoft.com/en-us/windows/wsl/install>`_.
|
||||
```
|
||||
|
||||
## Installation Guide
|
||||
|
||||
You can treat WSL2 shell as a normal Linux shell and run normal bash commands in it. If you're using WSL2 shell for the first time, it may require you to set up some user information. Using WSL2, you can install BigDL the same way as you do on a Linux system.
|
||||
|
||||
|
||||
### Install Conda
|
||||
|
||||
Conda is the recommend way to manage the BigDL environment. Download and install conda using below commands.
|
||||
|
||||
```bash
|
||||
wget https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh
|
||||
chmod +x Miniconda3-4.5.4-Linux-x86_64.sh
|
||||
./Miniconda3-4.5.4-Linux-x86_64.sh
|
||||
```
|
||||
|
||||
```eval_rst
|
||||
.. note::
|
||||
On WSL2, you need to use a Linux version of Conda intead of a Windows version. For other available conda versions, refer to `conda install <https://conda.io/projects/conda/en/latest/user-guide/install/index.html>`_, or `miniconda install <https://docs.conda.io/en/main/miniconda.html>`_.
|
||||
```
|
||||
|
||||
### Install BigDL
|
||||
|
||||
After installing conda, use conda to create and activate an environment for bigdl.
|
||||
|
||||
```bash
|
||||
conda create -n bigdl-env
|
||||
conda activate bigdl-env
|
||||
```
|
||||
|
||||
Then install BigDL as a whole, or specific bigdl library the same way as you do on a Linux system. For example,
|
||||
|
||||
```bash
|
||||
pip install bigdl
|
||||
```
|
||||
|
||||
```eval_rst
|
||||
.. card::
|
||||
|
||||
**Related Readings**
|
||||
^^^
|
||||
* `BigDL Installation Guide <../UserGuide/python>`_
|
||||
* `Nano Installation Guide <../Nano/Overview/nano.html#install>`_
|
||||
* `Chronos Installation Guide <../Chronos/Overview/chronos.html#install>`_
|
||||
```
|
||||
|
||||
### Setup Jupyter Notebook Environment
|
||||
|
||||
Fist, install JupyterLab using pip:
|
||||
|
||||
```bash
|
||||
pip install jupyterlab
|
||||
```
|
||||
|
||||
Then start JupyterLab using:
|
||||
|
||||
```bash
|
||||
jupyter lab
|
||||
```
|
||||
|
||||
```eval_rst
|
||||
.. note::
|
||||
Once you started Juypterlab, it will open automatically in your browser. If it does not open automatically, you can manually enter the notebook server’s URL into the browser (The URL is shown on the terminal where you run the command). The default workspace of jupyter is located at the directory where you start the jupyterlab. For more information about JupyterLab installation and usage, refer to `JupyterLab User Guide <https://jupyterlab.readthedocs.io/en/stable/getting_started/overview.html#>`_.
|
||||
```
|
||||
|
||||
## Tips and Known Issues
|
||||
|
||||
### 1.ImportError: libgomp.so.1: cannot open shared object file: No such file or directory
|
||||
|
||||
This error may appear when you try to import torch. This is caused by Ubuntu 14.04 or later not installing libgomp1 by default. Just install libgomp1 to resolve it:
|
||||
|
||||
```bash
|
||||
sudo apt-get install libgomp1
|
||||
```
|
||||
|
||||
### 2.Slow PyTorch training with BF16
|
||||
|
||||
Using BFloat16 mixed precision in PyTorch or PyTorch-Lightning training may be much slower than FP32.
|
||||
|
||||
|
||||
### 3.ERROR: Could not build wheels for pycocotools, which is required to install pyproject.toml-based projects
|
||||
|
||||
pycocotools is a dependency of Intel neural-compressor which is used for inference quantization in BigDL-Nano. This error is usually caused by GCC library not installed in system. Just install gcc to resolve it:
|
||||
|
||||
```bash
|
||||
sudo apt-get install gcc
|
||||
```
|
||||
|
||||
### 4.ValueError: After taking into account object store and redis memory usage, the amount of memory on this node available for tasks and actors is less than -75% of total.
|
||||
|
||||
When running ray applications, you need to set the `memory` and `object_store_memory` properly according to your system memory capacity. This error indicates you have used too large memory configurations and you need to decrease them. For example on a laptop with 8G memory, you may set the memory configurations as below:
|
||||
|
||||
```bash
|
||||
python yoloV3.py --memory 2g --object_store_memory 1g
|
||||
```
|
||||
Loading…
Reference in a new issue