Chronos User Guide

1. Overview

Chronos is an application framework for building large-scale time series analysis applications.

You can use Chronos to do:

Data pre/post-processing and feature generation (using TSDataset)
Time Series Forecasting (using Standalone Forecasters, Auto Models (with HPO) or AutoTS (full AutoML enabled pipelines))
Anomaly Detection (using Anomaly Detectors)
Synthetic Data Generation (using Simulators)
Speed up or tune your customized time-series model (using TSTrainer and AutoTS)

2. Install

Install bigdl-chronos from PyPI. We recommened to install with a conda virtual environment. To install Conda, please refer to https://docs.conda.io/en/latest/miniconda.html#.

conda create -n my_env python=3.7 setuptools=58.0.4
conda activate my_env
pip install bigdl-chronos

You may also install bigdl-chronos with target [all] to install the additional dependencies for Chronos. This will enable distributed tuning with AutoTS.

# stable version
pip install bigdl-chronos[all]
# nightly built version
pip install --pre --upgrade bigdl-chronos[all]
# set env variables for your conda environment
source bigdl-nano-init

Some dependencies are optional and not included in bigdl-chronos[all]. You may install them when you want to use corresponding functionalities. This includes:

pip install tsfresh==0.17.0
pip install bigdl-nano[tensorflow]
pip install pmdarima==1.8.2
pip install prophet==1.0.1
pip install neural-compressor==1.8.1
pip install pyarrow=6.0.1

.. note:: 
    **Supported OS**:

     Chronos is thoroughly tested on Ubuntu (16.04/18.04/20.04). If you are a Windows user, the most convenient way to use Chronos on a windows laptop might be using WSL2, you may refer to https://docs.microsoft.com/en-us/windows/wsl/setup/environment or just install a ubuntu virtual machine.

.. note:: 
    **Supported Python Version**:

     Chronos is thoroughly tested on Python3.6/3.7. Still, it is highly recommended to use python3.7.

3. Run

Various python programming environments are supported to run a Chronos application.

3.1 Jupyter Notebook

You can start the Jupyter notebook as you normally do using the following command and run Chronos application directly in a Jupyter notebook:

jupyter notebook --notebook-dir=./ --ip=* --no-browser

3.2 Python Script

You can directly write Chronos application in a python file (e.g. script.py) and run in the command line as a normal Python program:

python script.py

.. note:: 
    **Optimization on Intel® Hardware**:
    
     Chronos integrated many optimized library and best known methods(BKMs), users can have best performance to add ``bigdl-nano-init`` before their scripts. 
     
     ``bigdl-nano-init python script.py``

     Currently, this function is under active development and we encourage our users to add ``bigdl-nano-init`` for forecaster's training.

4. Get Started

4.1 Initialization

Chronos uses Orca to enable distributed training and AutoML capabilities. Initialize orca as below when you want to:

Use the distributed mode of a forecaster.
Use automl to distributedly tuning your model.
Use XshardsTSDataset to process time series dataset in distribution fashion.

Otherwise, there is no need to initialize an orca context.

View Orca Context for more details. Note that argument init_ray_on_spark must be True for Chronos.

from bigdl.orca import init_orca_context, stop_orca_context

if __name__ == "__main__":
    # run in local mode
    init_orca_context(cluster_mode="local", cores=4, init_ray_on_spark=True)
    # run on K8s cluster
    init_orca_context(cluster_mode="k8s", num_nodes=2, cores=2, init_ray_on_spark=True)
    # run on Hadoop YARN cluster
    init_orca_context(cluster_mode="yarn-client", num_nodes=2, cores=2, init_ray_on_spark=True)

    # >>> Start of Chronos Application >>>
    # ...
    # <<< End of Chronos Application <<<

    stop_orca_context()

4.2 AutoTS Example

This example run a forecasting task with automl optimization with AutoTSEstimator on New York City Taxi Dataset. To run this example, install the following: pip install --pre --upgrade bigdl-chronos[all].

from bigdl.orca.automl import hp
from bigdl.chronos.data.repo_dataset import get_public_dataset
from bigdl.chronos.autots import AutoTSEstimator
from bigdl.orca import init_orca_context, stop_orca_context
from sklearn.preprocessing import StandardScaler

if __name__ == "__main__":
    # initial orca context
    init_orca_context(cluster_mode="local", cores=4, memory="8g", init_ray_on_spark=True)

    # load dataset
    tsdata_train, tsdata_val, tsdata_test = get_public_dataset(name='nyc_taxi')

    # dataset preprocessing
    stand = StandardScaler()
    for tsdata in [tsdata_train, tsdata_val, tsdata_test]:
        tsdata.gen_dt_feature().impute()\
              .scale(stand, fit=tsdata is tsdata_train)

    # AutoTSEstimator initalization
    autotsest = AutoTSEstimator(model="tcn",
                                future_seq_len=10)

    # AutoTSEstimator fitting
    tsppl = autotsest.fit(data=tsdata_train,
                          validation_data=tsdata_val)

    # Evaluation
    autotsest_mse = tsppl.evaluate(tsdata_test)

    # stop orca context
    stop_orca_context()

5. Details

Chronos provides flexible components for forecasting, detection, simulation and other userful functionalities. You may review following pages to fully learn how to use Chronos to build various time series related applications.

11 KiB Raw Blame History