diff --git a/docs/readthedocs/source/_toc.yml b/docs/readthedocs/source/_toc.yml index a1708b61..2e6376b8 100644 --- a/docs/readthedocs/source/_toc.yml +++ b/docs/readthedocs/source/_toc.yml @@ -26,7 +26,7 @@ subtrees: subtrees: - entries: - file: doc/Orca/Overview/orca - title: "Orca in 5 miniutes" + title: "Orca in 5 minutes" - file: doc/Orca/Overview/install title: "Installation" - file: doc/Orca/Overview/index @@ -39,7 +39,7 @@ subtrees: - file: doc/Orca/Overview/distributed-tuning - file: doc/Orca/Overview/ray - file: doc/Orca/QuickStart/index - title: "Tutorials" + title: "Quick Examples" subtrees: - entries: - file: doc/UseCase/spark-dataframe @@ -48,6 +48,11 @@ subtrees: - file: doc/Orca/QuickStart/orca-pytorch-distributed-quickstart - file: doc/Orca/QuickStart/orca-autoestimator-pytorch-quickstart - file: doc/Orca/QuickStart/orca-autoxgboost-quickstart + - file: doc/Orca/Tutorial/index + title: "Tutorials" + subtrees: + - entries: + - file: doc/Orca/Tutorial/yarn - file: doc/Orca/Overview/known_issues title: "Tips and Known Issues" - file: doc/PythonAPI/Orca/index diff --git a/docs/readthedocs/source/doc/Orca/Overview/getstarted.rst b/docs/readthedocs/source/doc/Orca/Overview/getstarted.rst deleted file mode 100644 index cf74e658..00000000 --- a/docs/readthedocs/source/doc/Orca/Overview/getstarted.rst +++ /dev/null @@ -1,2 +0,0 @@ -Orca Key Features -================================= \ No newline at end of file diff --git a/docs/readthedocs/source/doc/Orca/Overview/install.md b/docs/readthedocs/source/doc/Orca/Overview/install.md index 9f6a7bea..5fce4620 100644 --- a/docs/readthedocs/source/doc/Orca/Overview/install.md +++ b/docs/readthedocs/source/doc/Orca/Overview/install.md @@ -1,7 +1,7 @@ # Installation -## To use Distributed Data processing, training, and/or inference +## To use basic Orca features We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the Python environment. ```bash conda create -n py37 python=3.7 # "py37" is conda environment name, you can use any name you like. @@ -14,32 +14,38 @@ You can install bigdl-orca nightly build version using pip install --pre --upgrade bigdl-orca ``` -## To use RayOnSpark +## To additionally use RayOnSpark -There're some additional dependencies required for running [RayOnSpark](ray.md). Use extra key `[ray]` to install. +If you wish to run [RayOnSpark](ray.md) or [sklearn-style Estimator APIs in Orca](distributed-training-inference.md) with "ray" backend, use the extra key `[ray]` during the installation above: ```bash pip install bigdl-orca[ray] ``` -or to install nightly build, use +or for the nightly build version, use ```bash pip install --pre --upgrade bigdl-orca[ray] ``` -## To use Orca AutoML +Note that with the extra key of [ray], `pip` will automatically install the additional dependencies for RayOnSpark, +including `ray[default]==1.9.2`, `aiohttp==3.8.1`, `async-timeout==4.0.1`, `aioredis==1.3.1`, `hiredis==2.0.0`, `prometheus-client==0.11.0`, `psutil`, `setproctitle`. -There're some additional dependencies required for Orca AutoML support. Use extra key `[automl]` to install. +## To additionally use AutoML + +If you wish to run AutoML, use the extra key `[automl]` during the installation above: ```bash pip install bigdl-orca[automl] ```` +or for the nightly build version, use +```bash +pip install --pre --upgrade bigdl-orca[automl] +``` -_Note that with extra key of [automl], `pip` will automatically install the additional dependencies for distributed hyper-parameter tuning, -including `ray[tune]==1.9.2`, `scikit-learn`, `tensorboard`, `xgboost`._ +Note that with the extra key of [automl], `pip` will automatically install the additional dependencies for distributed hyper-parameter tuning, +including `ray[tune]==1.9.2`, `scikit-learn`, `tensorboard`, `xgboost` together with the dependencies given by the extra key [ray]. -To use [Pytorch Estimator](#pytorch-autoestimator), you need to install Pytorch with `pip install torch==1.8.1`. - -To use [TensorFlow/Keras AutoEstimator](#tensorflow-keras-autoestimator), you need to install Tensorflow with `pip install tensorflow==1.15.0`. +- To use [Pytorch Estimator](#pytorch-autoestimator), you need to install Pytorch with `pip install torch==1.8.1`. +- To use [TensorFlow/Keras AutoEstimator](#tensorflow-keras-autoestimator), you need to install TensorFlow with `pip install tensorflow==1.15.0`. diff --git a/docs/readthedocs/source/doc/Orca/Overview/orca.md b/docs/readthedocs/source/doc/Orca/Overview/orca.md index 3ea31fb1..226fd26e 100644 --- a/docs/readthedocs/source/doc/Orca/Overview/orca.md +++ b/docs/readthedocs/source/doc/Orca/Overview/orca.md @@ -6,9 +6,11 @@ Most AI projects start with a Python notebook running on a single laptop; howeve --- -### Tensorflow Bite-sized Example +### TensorFlow Bite-sized Example -This section uses TensorFlow 1.15, and you should install TensorFlow before running this example: +First of all, follow the steps [here](install.md#to-use-basic-orca-features) to install Orca in your environment. + +This section uses TensorFlow 1.15, and you should also install TensorFlow before running this example: ```bash pip install tensorflow==1.15 ``` diff --git a/docs/readthedocs/source/doc/Orca/Overview/ray.md b/docs/readthedocs/source/doc/Orca/Overview/ray.md index 6175e4b5..5d3ab749 100644 --- a/docs/readthedocs/source/doc/Orca/Overview/ray.md +++ b/docs/readthedocs/source/doc/Orca/Overview/ray.md @@ -14,7 +14,7 @@ _**Note:** BigDL has been tested on Ray 1.9.2 and you are highly recommended to We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the Python environment. When installing bigdl-orca with pip, you can specify the extras key `[ray]` to install the additional dependencies -for running Ray (i.e. `ray==1.9.2`, `psutil`, `aiohttp==3.7.0`, `aioredis==1.1.0`, `setproctitle`, `hiredis==1.1.0`, `async-timeout==3.0.1`): +for running Ray (i.e. `ray[default]==1.9.2`, `aiohttp==3.8.1`, `async-timeout==4.0.1`, `aioredis==1.3.1`, `hiredis==2.0.0`, `prometheus-client==0.11.0`, `psutil`, `setproctitle`): ```bash conda create -n py37 python=3.7 # "py37" is conda environment name, you can use any name you like. diff --git a/docs/readthedocs/source/doc/Orca/QuickStart/index.md b/docs/readthedocs/source/doc/Orca/QuickStart/index.md index 89751ddc..5dd79202 100644 --- a/docs/readthedocs/source/doc/Orca/QuickStart/index.md +++ b/docs/readthedocs/source/doc/Orca/QuickStart/index.md @@ -1,4 +1,4 @@ -# Orca Tutorial +# Orca Quickstarts - [**Orca TensorFlow 1.15 Quickstart**](./orca-tf-quickstart.html) diff --git a/docs/readthedocs/source/doc/Orca/Tutorial/index.rst b/docs/readthedocs/source/doc/Orca/Tutorial/index.rst new file mode 100644 index 00000000..774e46ef --- /dev/null +++ b/docs/readthedocs/source/doc/Orca/Tutorial/index.rst @@ -0,0 +1,6 @@ +Orca Tutorials +================================= + +* `Run on Hadoop/YARN clusters `_ +* `Run on Azure Databricks <../../UserGuide/databricks.html>`_ +* `Run on Google Colab <../../UserGuide/colab.html>`_ \ No newline at end of file diff --git a/docs/readthedocs/source/doc/Orca/Tutorial/yarn.md b/docs/readthedocs/source/doc/Orca/Tutorial/yarn.md index 9914f6ce..8c5afee3 100644 --- a/docs/readthedocs/source/doc/Orca/Tutorial/yarn.md +++ b/docs/readthedocs/source/doc/Orca/Tutorial/yarn.md @@ -1,4 +1,4 @@ -# Running BigDL-Orca on Hadoop/YARN Clusters +# Run on Hadoop/YARN Clusters This tutorial provides a step-by-step guide on how to run BigDL-Orca programs on Apache Hadoop/YARN clusters, using a [PyTorch Fashin-MNIST program](https://github.com/intel-analytics/BigDL/blob/main/python/orca/tutorial/pytorch/FashionMNIST/) as a working example. diff --git a/docs/readthedocs/source/doc/UseCase/spark-dataframe.md b/docs/readthedocs/source/doc/UseCase/spark-dataframe.md index a2759c8e..8c5b1fba 100644 --- a/docs/readthedocs/source/doc/UseCase/spark-dataframe.md +++ b/docs/readthedocs/source/doc/UseCase/spark-dataframe.md @@ -1,4 +1,4 @@ -# Use Spark Dataframe for Deep Learning +# Use Spark DataFrames for Deep Learning ---