ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Chronos: update Chronos quickstart from az to bigdl (#3324)

Browse source 
* fix anomaly detector qs

* fix autoest notebook

* add updates

* add qs name

* update logo

* fix installation guide

* fix typos
This commit is contained in:
Junwei Deng 2021-10-29 12:59:21 +08:00 committed by GitHub
parent 26c087d493
commit ac96d9a35a
3 changed files with 35 additions and 35 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
docs/readthedocs/source/doc/Chronos/QuickStart/chronos-anomaly-detector.md
View file

@ -1,8 +1,8 @@
# Anomaly Detector # Anomaly Detector Quickstart
--- ---
![](../../../../image/colab_logo_32px.png)[Run in Google Colab](https://colab.research.google.com/github/intel-analytics/analytics-zoo/blob/master/docs/docs/colab-notebook/chronos/chronos_minn_traffic_anomaly_detector.ipynb)  ![](../../../../image/GitHub-Mark-32px.png)[View source on GitHub](https://github.com/intel-analytics/analytics-zoo/blob/master/docs/docs/colab-notebook/chronos/chronos_minn_traffic_anomaly_detector.ipynb) ![](../../../../image/colab_logo_32px.png)[Run in Google Colab](https://colab.research.google.com/github/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_minn_traffic_anomaly_detector.ipynb)  ![](../../../../image/GitHub-Mark-32px.png)[View source on GitHub](https://github.com/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_minn_traffic_anomaly_detector.ipynb)
--- ---
@ -10,12 +10,12 @@
### Step 0: Prepare Environment ### Step 0: Prepare Environment
We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the environment. Please refer to the [install guide](../../UserGuide/python.md) for more details. We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the environment. Please refer to the [install guide](../Overview/chronos.html#install) for more details.
```bash ```bash
conda create -n zoo python=3.7 # "zoo" is conda environment name, you can use any name you like. conda create -n my_env python=3.7 # "my_env" is conda environment name, you can use any name you like.
conda activate zoo conda activate my_env
pip install analytics-zoo[automl] # install either version 0.10 or latest nightly build pip install bigdl-chronos
``` ```
## Step 1: Prepare dataset ## Step 1: Prepare dataset
@ -27,7 +27,7 @@ For the machine_usage data, the pre-processing contains 2 parts: <br>
2. Check missing values and handle missing data. 2. Check missing values and handle missing data.
```python ```python
from zoo.chronos.data import TSDataset from bigdl.chronos.data import TSDataset
tsdata = TSDataset.from_pandas(df, dt_col="timestamp", target_col="value") tsdata = TSDataset.from_pandas(df, dt_col="timestamp", target_col="value")
df = tsdata.resample("5min")\ df = tsdata.resample("5min")\
@ -36,10 +36,10 @@ df = tsdata.resample("5min")\
``` ```
## Step 2: Use Chronos Anomaly Detector ## Step 2: Use Chronos Anomaly Detector
Chronos provides many anomaly detector for anomaly detection, here we use DBScan as an example. More anomaly detector can be found [here](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/anomaly_detectors.html). Chronos provides many anomaly detector for anomaly detection, here we use DBScan as an example. More anomaly detector can be found [here](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Chronos/anomaly_detectors.html).
```python ```python
from zoo.chronos.detector.anomaly import DBScanDetector from bigdl.chronos.detector.anomaly import DBScanDetector
ad = DBScanDetector(eps=0.3, min_samples=6) ad = DBScanDetector(eps=0.3, min_samples=6)
ad.fit(df['value'].to_numpy()) ad.fit(df['value'].to_numpy())

32
docs/readthedocs/source/doc/Chronos/QuickStart/chronos-autotsest-quickstart.md
View file

@ -1,8 +1,8 @@
# AutoTSEstimator # AutoTSEstimator Quickstart
--- ---
![](../../../../image/colab_logo_32px.png)[Run in Google Colab](https://colab.research.google.com/github/intel-analytics/analytics-zoo/blob/master/docs/docs/colab-notebook/chronos/chronos_experimental_autots_nyc_taxi.ipynb) &nbsp;![](../../../../image/GitHub-Mark-32px.png)[View source on GitHub](https://github.com/intel-analytics/analytics-zoo/blob/master/docs/docs/colab-notebook/chronos/chronos_experimental_autots_nyc_taxi.ipynb) ![](../../../../image/colab_logo_32px.png)[Run in Google Colab](https://colab.research.google.com/github/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_autots_nyc_taxi.ipynb) &nbsp;![](../../../../image/GitHub-Mark-32px.png)[View source on GitHub](https://github.com/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_autots_nyc_taxi.ipynb)
--- ---
@ -12,16 +12,16 @@
Chronos provides `AutoTSEstimator` as a highly integrated solution for time series forecasting task with hyperparameter autotuning, auto feature selection and auto preprocessing. Users can prepare a `TSDataset`(recommended, used in this notebook) or their own data creator as input data. By constructing a `AutoTSEstimator` and calling `fit` on the data, a `TSPipeline` contains the best model and pre/post data processing will be returned for further development of deployment. Chronos provides `AutoTSEstimator` as a highly integrated solution for time series forecasting task with hyperparameter autotuning, auto feature selection and auto preprocessing. Users can prepare a `TSDataset`(recommended, used in this notebook) or their own data creator as input data. By constructing a `AutoTSEstimator` and calling `fit` on the data, a `TSPipeline` contains the best model and pre/post data processing will be returned for further development of deployment.
`AutoTSEstimator` is experimental and only support LSTM, TCN, and Seq2seq model for now. `AutoTSEstimator` only support LSTM, TCN, and Seq2seq built-in models and 3rd party models for now.
### **Step 0: Prepare Environment** ### **Step 0: Prepare Environment**
We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the environment. Please refer to the [install guide](../../UserGuide/python.md) for more details. We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the environment. Please refer to the [install guide](../Overview/chronos.html#install) for more details.
```bash ```bash
conda create -n zoo python=3.7 conda create -n my_env python=3.7
conda activate zoo conda activate my_env
pip install --pre --upgrade analytics-zoo[automl] pip install --pre --upgrade bigdl-chronos[all]
``` ```
### **Step 1: Init Orca Context** ### **Step 1: Init Orca Context**
@ -40,7 +40,7 @@ This is the only place where you need to specify local or distributed mode. View
### **Step 2: Prepare a TSDataset** ### **Step 2: Prepare a TSDataset**
Prepare a `TSDataset` and call necessary operations on it. Prepare a `TSDataset` and call necessary operations on it.
```python ```python
from zoo.chronos.data import TSDataset from bigdl.chronos.data import TSDataset
from sklearn.preprocessing import StandardScaler from sklearn.preprocessing import StandardScaler
tsdata_train, tsdata_val, tsdata_test\ tsdata_train, tsdata_val, tsdata_test\
@ -56,13 +56,13 @@ There is no need to call `.roll()` or `.to_torch_data_loader()` in this step, wh
Please call `.gen_dt_feature()`(recommended), `.gen_rolling_feature()`, and `gen_global_feature()` to generate all candidate features to be selected by `AutoTSEstimator` as well as your input extra feature. Please call `.gen_dt_feature()`(recommended), `.gen_rolling_feature()`, and `gen_global_feature()` to generate all candidate features to be selected by `AutoTSEstimator` as well as your input extra feature.
Detailed information please refer to [TSDataset API doc](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/tsdataset.html#tsdataset) and [Time series data basic concepts](https://analytics-zoo.readthedocs.io/en/latest/doc/Chronos/Overview/chronos.html#data-processing-and-feature-engineering). Detailed information please refer to [TSDataset API doc](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Chronos/tsdataset.html) and [Time series data basic concepts](https://bigdl.readthedocs.io/en/latest/doc/Chronos/Overview/data_processing_feature_engineering.html).
### **Step 3: Create an AutoTSEstimator** ### **Step 3: Create an AutoTSEstimator**
```python ```python
import zoo.orca.automl.hp as hp import bigdl.orca.automl.hp as hp
from zoo.chronos.autots import AutoTSEstimator from bigdl.chronos.autots import AutoTSEstimator
auto_estimator = AutoTSEstimator(model='lstm', # the model name used for training auto_estimator = AutoTSEstimator(model='lstm', # the model name used for training
search_space='normal', # a default hyper parameter search space search_space='normal', # a default hyper parameter search space
past_seq_len=hp.randint(1, 10), # hp sampling function of past_seq_len for auto-tuning past_seq_len=hp.randint(1, 10), # hp sampling function of past_seq_len for auto-tuning
@ -72,7 +72,7 @@ We prebuild three defualt search space for each build-in model, which you can us
`past_seq_len` can be set as a hp sample function, the proper range is highly related to your data. A range between 0.5 cycle and 3 cycle is reasonable. `past_seq_len` can be set as a hp sample function, the proper range is highly related to your data. A range between 0.5 cycle and 3 cycle is reasonable.
Detailed information please refer to [AutoTSEstimator API doc](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/autotsestimator.html#id1) and some basic concepts [here](https://analytics-zoo.readthedocs.io/en/latest/doc/Orca/Overview/distributed-tuning.html#search-space-and-search-algorithms). Detailed information please refer to [AutoTSEstimator API doc](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Chronos/autotsestimator.html#autotsestimator) and basic concepts [here](https://bigdl.readthedocs.io/en/latest/doc/Chronos/Overview/forecasting.html#use-autots-pipeline).
### **Step 4: Fit with AutoTSEstimator** ### **Step 4: Fit with AutoTSEstimator**
```python ```python
@ -81,7 +81,7 @@ ts_pipeline = auto_estimator.fit(data=tsdata_train, # train dataset
validation_data=tsdata_val, # validation dataset validation_data=tsdata_val, # validation dataset
epochs=5) # number of epochs to train in each trial epochs=5) # number of epochs to train in each trial
``` ```
Detailed information please refer to [AutoTSEstimator API doc](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/autotsestimator.html#id1). Detailed information please refer to [AutoTSEstimator API doc](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Chronos/autotsestimator.html#autotsestimator).
### **Step 5: Further deployment with TSPipeline** ### **Step 5: Further deployment with TSPipeline**
The `TSPipeline` will reply the same preprcessing and corresponding postprocessing operations on the test data. You may carry out predict, evaluate or save/load for further development. The `TSPipeline` will reply the same preprcessing and corresponding postprocessing operations on the test data. You may carry out predict, evaluate or save/load for further development.
```python ```python
@ -101,10 +101,10 @@ print("Evaluate: the smape value is", smape)
my_ppl_file_path = "/tmp/saved_pipeline" my_ppl_file_path = "/tmp/saved_pipeline"
ts_pipeline.save(my_ppl_file_path) ts_pipeline.save(my_ppl_file_path)
# restore the pipeline for further deployment # restore the pipeline for further deployment
from zoo.chronos.autots import TSPipeline from bigdl.chronos.autots import TSPipeline
loaded_ppl = TSPipeline.load(my_ppl_file_path) loaded_ppl = TSPipeline.load(my_ppl_file_path)
``` ```
Detailed information please refer to [TSPipeline API doc](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/autotsestimator.html#tspipeline-experimental). Detailed information please refer to [TSPipeline API doc](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Chronos/tsdataset.html).
### **Optional: Examine the leaderboard visualization** ### **Optional: Examine the leaderboard visualization**
To view the evaluation result of "not chosen" trails and find some insight or even possibly improve you search space for a new autotuning task. We provide a leaderboard through tensorboard. To view the evaluation result of "not chosen" trails and find some insight or even possibly improve you search space for a new autotuning task. We provide a leaderboard through tensorboard.
@ -113,4 +113,4 @@ To view the evaluation result of "not chosen" trails and find some insight or ev
%load_ext tensorboard %load_ext tensorboard
%tensorboard --logdir /tmp/autots_estimator/autots_estimator_leaderboard/ %tensorboard --logdir /tmp/autots_estimator/autots_estimator_leaderboard/
``` ```
Detailed information please refer to [Visualization](https://analytics-zoo.readthedocs.io/en/latest/doc/Chronos/Overview/chronos.html#Visualization). Detailed information please refer to [Visualization](https://bigdl.readthedocs.io/en/latest/doc/Chronos/Overview/useful_functionalities.html#automl-visualization).

20
docs/readthedocs/source/doc/Chronos/QuickStart/chronos-tsdataset-forecaster-quickstart.md
View file

@ -1,8 +1,8 @@
# TSDataset and Forecaster # TSDataset and Forecaster Quickstarts
--- ---
![](../../../../image/colab_logo_32px.png)[Run in Google Colab](https://colab.research.google.com/github/intel-analytics/analytics-zoo/blob/master/docs/docs/colab-notebook/chronos/chronos_nyc_taxi_tsdataset_forecaster.ipynb) &nbsp;![](../../../../image/GitHub-Mark-32px.png)[View source on GitHub](https://github.com/intel-analytics/analytics-zoo/blob/master/docs/docs/colab-notebook/chronos/chronos_nyc_taxi_tsdataset_forecaster.ipynb) ![](../../../../image/colab_logo_32px.png)[Run in Google Colab](https://colab.research.google.com/github/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_nyc_taxi_tsdataset_forecaster.ipynb) &nbsp;![](../../../../image/GitHub-Mark-32px.png)[View source on GitHub](https://github.com/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_nyc_taxi_tsdataset_forecaster.ipynb)
--- ---
@ -10,22 +10,22 @@
### **Step 0: Prepare Environment** ### **Step 0: Prepare Environment**
We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the environment. Please refer to the [install guide](../../UserGuide/python.md) for more details. We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the environment. Please refer to the [install guide](../Overview/chronos.html#install) for more details.
```bash ```bash
conda create -n zoo python=3.7 # "zoo" is conda environment name, you can use any name you like. conda create -n my_env python=3.7 # "my_env" is conda environment name, you can use any name you like.
conda activate zoo conda activate my_env
pip install analytics-zoo[automl] # install latest nightly build pip install bigdl-chronos[all]
``` ```
### Step 1: Data transformation and feature engineering using Chronos TSDataset ### Step 1: Data transformation and feature engineering using Chronos TSDataset
[TSDataset](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/tsdataset.html) is our abstract of time series dataset for data transformation and feature engineering. Here we use it to preprocess the data. [TSDataset](https://bigdl.readthedocs.io/en/latest/doc/Chronos/Overview/data_processing_feature_engineering.html) is our abstract of time series dataset for data transformation and feature engineering. Here we use it to preprocess the data.
Initialize train, valid and test tsdataset from raw pandas dataframe. Initialize train, valid and test tsdataset from raw pandas dataframe.
```python ```python
from zoo.chronos.data import TSDataset from bigdl.chronos.data import TSDataset
from sklearn.preprocessing import StandardScaler from sklearn.preprocessing import StandardScaler
tsdata_train, tsdata_valid, tsdata_test = TSDataset.from_pandas(df, dt_col="timestamp", target_col="value", tsdata_train, tsdata_valid, tsdata_test = TSDataset.from_pandas(df, dt_col="timestamp", target_col="value",
@ -54,7 +54,7 @@ for tsdata in [tsdata_train, tsdata_valid, tsdata_test]:
### Step 2: Time series forecasting using Chronos Forecaster ### Step 2: Time series forecasting using Chronos Forecaster
After preprocessing the datasets. We can use [Chronos Forecaster](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/forecasters.html) to handle the forecasting tasks. After preprocessing the datasets. We can use [Chronos Forecaster](https://bigdl.readthedocs.io/en/latest/doc/Chronos/Overview/forecasting.html#use-standalone-forecaster-pipeline) to handle the forecasting tasks.
Transform TSDataset to sampled numpy ndarray and feed them to forecaster. Transform TSDataset to sampled numpy ndarray and feed them to forecaster.
@ -68,7 +68,7 @@ forecaster = TCNForecaster(past_seq_len=lookback, # number of steps to look bac
future_seq_len=horizon, # number of steps to predict future_seq_len=horizon, # number of steps to predict
input_feature_num=x.shape[-1], # number of feature to use input_feature_num=x.shape[-1], # number of feature to use
output_feature_num=y.shape[-1]) # number of feature to predict output_feature_num=y.shape[-1]) # number of feature to predict
res = forecaster.fit((x, y), validation_data=(x_val, y_val), epochs=3) res = forecaster.fit(data=(x, y), epochs=3)
``` ```
### Step 3: Further deployment with fitted forecaster ### Step 3: Further deployment with fitted forecaster