Chronos: update Chronos quickstart from az to bigdl (#3324)
* fix anomaly detector qs * fix autoest notebook * add updates * add qs name * update logo * fix installation guide * fix typos
This commit is contained in:
Junwei Deng
GitHub
parent
26c087d493
commit
ac96d9a35a
3 changed files with 35 additions and 35 deletions
|
|
@ -1,8 +1,8 @@
|
|||
# Anomaly Detector
|
||||
# Anomaly Detector Quickstart
|
||||
|
||||
---
|
||||
|
||||
[Run in Google Colab](https://colab.research.google.com/github/intel-analytics/analytics-zoo/blob/master/docs/docs/colab-notebook/chronos/chronos_minn_traffic_anomaly_detector.ipynb) [View source on GitHub](https://github.com/intel-analytics/analytics-zoo/blob/master/docs/docs/colab-notebook/chronos/chronos_minn_traffic_anomaly_detector.ipynb)
|
||||
[Run in Google Colab](https://colab.research.google.com/github/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_minn_traffic_anomaly_detector.ipynb) [View source on GitHub](https://github.com/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_minn_traffic_anomaly_detector.ipynb)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -10,12 +10,12 @@
|
|||
|
||||
### Step 0: Prepare Environment
|
||||
|
||||
We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the environment. Please refer to the [install guide](../../UserGuide/python.md) for more details.
|
||||
We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the environment. Please refer to the [install guide](../Overview/chronos.html#install) for more details.
|
||||
|
||||
```bash
|
||||
conda create -n zoo python=3.7 # "zoo" is conda environment name, you can use any name you like.
|
||||
conda activate zoo
|
||||
pip install analytics-zoo[automl] # install either version 0.10 or latest nightly build
|
||||
conda create -n my_env python=3.7 # "my_env" is conda environment name, you can use any name you like.
|
||||
conda activate my_env
|
||||
pip install bigdl-chronos
|
||||
```
|
||||
|
||||
## Step 1: Prepare dataset
|
||||
|
|
@ -27,7 +27,7 @@ For the machine_usage data, the pre-processing contains 2 parts: <br>
|
|||
2. Check missing values and handle missing data.
|
||||
|
||||
```python
|
||||
from zoo.chronos.data import TSDataset
|
||||
from bigdl.chronos.data import TSDataset
|
||||
|
||||
tsdata = TSDataset.from_pandas(df, dt_col="timestamp", target_col="value")
|
||||
df = tsdata.resample("5min")\
|
||||
|
|
@ -36,10 +36,10 @@ df = tsdata.resample("5min")\
|
|||
```
|
||||
|
||||
## Step 2: Use Chronos Anomaly Detector
|
||||
Chronos provides many anomaly detector for anomaly detection, here we use DBScan as an example. More anomaly detector can be found [here](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/anomaly_detectors.html).
|
||||
Chronos provides many anomaly detector for anomaly detection, here we use DBScan as an example. More anomaly detector can be found [here](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Chronos/anomaly_detectors.html).
|
||||
|
||||
```python
|
||||
from zoo.chronos.detector.anomaly import DBScanDetector
|
||||
from bigdl.chronos.detector.anomaly import DBScanDetector
|
||||
|
||||
ad = DBScanDetector(eps=0.3, min_samples=6)
|
||||
ad.fit(df['value'].to_numpy())
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
# AutoTSEstimator
|
||||
# AutoTSEstimator Quickstart
|
||||
|
||||
---
|
||||
|
||||
[Run in Google Colab](https://colab.research.google.com/github/intel-analytics/analytics-zoo/blob/master/docs/docs/colab-notebook/chronos/chronos_experimental_autots_nyc_taxi.ipynb) [View source on GitHub](https://github.com/intel-analytics/analytics-zoo/blob/master/docs/docs/colab-notebook/chronos/chronos_experimental_autots_nyc_taxi.ipynb)
|
||||
[Run in Google Colab](https://colab.research.google.com/github/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_autots_nyc_taxi.ipynb) [View source on GitHub](https://github.com/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_autots_nyc_taxi.ipynb)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -12,16 +12,16 @@
|
|||
|
||||
Chronos provides `AutoTSEstimator` as a highly integrated solution for time series forecasting task with hyperparameter autotuning, auto feature selection and auto preprocessing. Users can prepare a `TSDataset`(recommended, used in this notebook) or their own data creator as input data. By constructing a `AutoTSEstimator` and calling `fit` on the data, a `TSPipeline` contains the best model and pre/post data processing will be returned for further development of deployment.
|
||||
|
||||
`AutoTSEstimator` is experimental and only support LSTM, TCN, and Seq2seq model for now.
|
||||
`AutoTSEstimator` only support LSTM, TCN, and Seq2seq built-in models and 3rd party models for now.
|
||||
|
||||
### **Step 0: Prepare Environment**
|
||||
|
||||
We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the environment. Please refer to the [install guide](../../UserGuide/python.md) for more details.
|
||||
We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the environment. Please refer to the [install guide](../Overview/chronos.html#install) for more details.
|
||||
|
||||
```bash
|
||||
conda create -n zoo python=3.7
|
||||
conda activate zoo
|
||||
pip install --pre --upgrade analytics-zoo[automl]
|
||||
conda create -n my_env python=3.7
|
||||
conda activate my_env
|
||||
pip install --pre --upgrade bigdl-chronos[all]
|
||||
```
|
||||
|
||||
### **Step 1: Init Orca Context**
|
||||
|
|
@ -40,7 +40,7 @@ This is the only place where you need to specify local or distributed mode. View
|
|||
### **Step 2: Prepare a TSDataset**
|
||||
Prepare a `TSDataset` and call necessary operations on it.
|
||||
```python
|
||||
from zoo.chronos.data import TSDataset
|
||||
from bigdl.chronos.data import TSDataset
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
|
||||
tsdata_train, tsdata_val, tsdata_test\
|
||||
|
|
@ -56,13 +56,13 @@ There is no need to call `.roll()` or `.to_torch_data_loader()` in this step, wh
|
|||
|
||||
Please call `.gen_dt_feature()`(recommended), `.gen_rolling_feature()`, and `gen_global_feature()` to generate all candidate features to be selected by `AutoTSEstimator` as well as your input extra feature.
|
||||
|
||||
Detailed information please refer to [TSDataset API doc](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/tsdataset.html#tsdataset) and [Time series data basic concepts](https://analytics-zoo.readthedocs.io/en/latest/doc/Chronos/Overview/chronos.html#data-processing-and-feature-engineering).
|
||||
Detailed information please refer to [TSDataset API doc](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Chronos/tsdataset.html) and [Time series data basic concepts](https://bigdl.readthedocs.io/en/latest/doc/Chronos/Overview/data_processing_feature_engineering.html).
|
||||
|
||||
### **Step 3: Create an AutoTSEstimator**
|
||||
|
||||
```python
|
||||
import zoo.orca.automl.hp as hp
|
||||
from zoo.chronos.autots import AutoTSEstimator
|
||||
import bigdl.orca.automl.hp as hp
|
||||
from bigdl.chronos.autots import AutoTSEstimator
|
||||
auto_estimator = AutoTSEstimator(model='lstm', # the model name used for training
|
||||
search_space='normal', # a default hyper parameter search space
|
||||
past_seq_len=hp.randint(1, 10), # hp sampling function of past_seq_len for auto-tuning
|
||||
|
|
@ -72,7 +72,7 @@ We prebuild three defualt search space for each build-in model, which you can us
|
|||
|
||||
`past_seq_len` can be set as a hp sample function, the proper range is highly related to your data. A range between 0.5 cycle and 3 cycle is reasonable.
|
||||
|
||||
Detailed information please refer to [AutoTSEstimator API doc](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/autotsestimator.html#id1) and some basic concepts [here](https://analytics-zoo.readthedocs.io/en/latest/doc/Orca/Overview/distributed-tuning.html#search-space-and-search-algorithms).
|
||||
Detailed information please refer to [AutoTSEstimator API doc](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Chronos/autotsestimator.html#autotsestimator) and basic concepts [here](https://bigdl.readthedocs.io/en/latest/doc/Chronos/Overview/forecasting.html#use-autots-pipeline).
|
||||
|
||||
### **Step 4: Fit with AutoTSEstimator**
|
||||
```python
|
||||
|
|
@ -81,7 +81,7 @@ ts_pipeline = auto_estimator.fit(data=tsdata_train, # train dataset
|
|||
validation_data=tsdata_val, # validation dataset
|
||||
epochs=5) # number of epochs to train in each trial
|
||||
```
|
||||
Detailed information please refer to [AutoTSEstimator API doc](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/autotsestimator.html#id1).
|
||||
Detailed information please refer to [AutoTSEstimator API doc](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Chronos/autotsestimator.html#autotsestimator).
|
||||
### **Step 5: Further deployment with TSPipeline**
|
||||
The `TSPipeline` will reply the same preprcessing and corresponding postprocessing operations on the test data. You may carry out predict, evaluate or save/load for further development.
|
||||
```python
|
||||
|
|
@ -101,10 +101,10 @@ print("Evaluate: the smape value is", smape)
|
|||
my_ppl_file_path = "/tmp/saved_pipeline"
|
||||
ts_pipeline.save(my_ppl_file_path)
|
||||
# restore the pipeline for further deployment
|
||||
from zoo.chronos.autots import TSPipeline
|
||||
from bigdl.chronos.autots import TSPipeline
|
||||
loaded_ppl = TSPipeline.load(my_ppl_file_path)
|
||||
```
|
||||
Detailed information please refer to [TSPipeline API doc](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/autotsestimator.html#tspipeline-experimental).
|
||||
Detailed information please refer to [TSPipeline API doc](https://bigdl.readthedocs.io/en/latest/doc/PythonAPI/Chronos/tsdataset.html).
|
||||
|
||||
### **Optional: Examine the leaderboard visualization**
|
||||
To view the evaluation result of "not chosen" trails and find some insight or even possibly improve you search space for a new autotuning task. We provide a leaderboard through tensorboard.
|
||||
|
|
@ -113,4 +113,4 @@ To view the evaluation result of "not chosen" trails and find some insight or ev
|
|||
%load_ext tensorboard
|
||||
%tensorboard --logdir /tmp/autots_estimator/autots_estimator_leaderboard/
|
||||
```
|
||||
Detailed information please refer to [Visualization](https://analytics-zoo.readthedocs.io/en/latest/doc/Chronos/Overview/chronos.html#Visualization).
|
||||
Detailed information please refer to [Visualization](https://bigdl.readthedocs.io/en/latest/doc/Chronos/Overview/useful_functionalities.html#automl-visualization).
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
# TSDataset and Forecaster
|
||||
# TSDataset and Forecaster Quickstarts
|
||||
|
||||
---
|
||||
|
||||
[Run in Google Colab](https://colab.research.google.com/github/intel-analytics/analytics-zoo/blob/master/docs/docs/colab-notebook/chronos/chronos_nyc_taxi_tsdataset_forecaster.ipynb) [View source on GitHub](https://github.com/intel-analytics/analytics-zoo/blob/master/docs/docs/colab-notebook/chronos/chronos_nyc_taxi_tsdataset_forecaster.ipynb)
|
||||
[Run in Google Colab](https://colab.research.google.com/github/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_nyc_taxi_tsdataset_forecaster.ipynb) [View source on GitHub](https://github.com/intel-analytics/BigDL/blob/branch-2.0/python/chronos/colab-notebook/chronos_nyc_taxi_tsdataset_forecaster.ipynb)
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -10,22 +10,22 @@
|
|||
|
||||
### **Step 0: Prepare Environment**
|
||||
|
||||
We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the environment. Please refer to the [install guide](../../UserGuide/python.md) for more details.
|
||||
We recommend using [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to prepare the environment. Please refer to the [install guide](../Overview/chronos.html#install) for more details.
|
||||
|
||||
```bash
|
||||
conda create -n zoo python=3.7 # "zoo" is conda environment name, you can use any name you like.
|
||||
conda activate zoo
|
||||
pip install analytics-zoo[automl] # install latest nightly build
|
||||
conda create -n my_env python=3.7 # "my_env" is conda environment name, you can use any name you like.
|
||||
conda activate my_env
|
||||
pip install bigdl-chronos[all]
|
||||
```
|
||||
|
||||
### Step 1: Data transformation and feature engineering using Chronos TSDataset
|
||||
|
||||
[TSDataset](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/tsdataset.html) is our abstract of time series dataset for data transformation and feature engineering. Here we use it to preprocess the data.
|
||||
[TSDataset](https://bigdl.readthedocs.io/en/latest/doc/Chronos/Overview/data_processing_feature_engineering.html) is our abstract of time series dataset for data transformation and feature engineering. Here we use it to preprocess the data.
|
||||
|
||||
Initialize train, valid and test tsdataset from raw pandas dataframe.
|
||||
|
||||
```python
|
||||
from zoo.chronos.data import TSDataset
|
||||
from bigdl.chronos.data import TSDataset
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
|
||||
tsdata_train, tsdata_valid, tsdata_test = TSDataset.from_pandas(df, dt_col="timestamp", target_col="value",
|
||||
|
|
@ -54,7 +54,7 @@ for tsdata in [tsdata_train, tsdata_valid, tsdata_test]:
|
|||
|
||||
### Step 2: Time series forecasting using Chronos Forecaster
|
||||
|
||||
After preprocessing the datasets. We can use [Chronos Forecaster](https://analytics-zoo.readthedocs.io/en/latest/doc/PythonAPI/Chronos/forecasters.html) to handle the forecasting tasks.
|
||||
After preprocessing the datasets. We can use [Chronos Forecaster](https://bigdl.readthedocs.io/en/latest/doc/Chronos/Overview/forecasting.html#use-standalone-forecaster-pipeline) to handle the forecasting tasks.
|
||||
|
||||
Transform TSDataset to sampled numpy ndarray and feed them to forecaster.
|
||||
|
||||
|
|
@ -68,7 +68,7 @@ forecaster = TCNForecaster(past_seq_len=lookback, # number of steps to look bac
|
|||
future_seq_len=horizon, # number of steps to predict
|
||||
input_feature_num=x.shape[-1], # number of feature to use
|
||||
output_feature_num=y.shape[-1]) # number of feature to predict
|
||||
res = forecaster.fit((x, y), validation_data=(x_val, y_val), epochs=3)
|
||||
res = forecaster.fit(data=(x, y), epochs=3)
|
||||
```
|
||||
|
||||
### Step 3: Further deployment with fitted forecaster
|
||||
|
|
|
|||
Loading…
Reference in a new issue