parent
3f576e5ffa
commit
53ba2d8279
9 changed files with 53749 additions and 10 deletions
BIN
docs/readthedocs/image/trial_dataframe.png
Normal file
BIN
docs/readthedocs/image/trial_dataframe.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 22 KiB |
8924
docs/readthedocs/source/_static/hpovis/contour.html
Normal file
8924
docs/readthedocs/source/_static/hpovis/contour.html
Normal file
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
8924
docs/readthedocs/source/_static/hpovis/intermediate_values.html
Normal file
8924
docs/readthedocs/source/_static/hpovis/intermediate_values.html
Normal file
File diff suppressed because one or more lines are too long
8924
docs/readthedocs/source/_static/hpovis/optimization_history.html
Normal file
8924
docs/readthedocs/source/_static/hpovis/optimization_history.html
Normal file
File diff suppressed because one or more lines are too long
8924
docs/readthedocs/source/_static/hpovis/parallel_coordinate.html
Normal file
8924
docs/readthedocs/source/_static/hpovis/parallel_coordinate.html
Normal file
File diff suppressed because one or more lines are too long
8924
docs/readthedocs/source/_static/hpovis/param_importance.html
Normal file
8924
docs/readthedocs/source/_static/hpovis/param_importance.html
Normal file
File diff suppressed because one or more lines are too long
8924
docs/readthedocs/source/_static/hpovis/param_importance_latency.html
Normal file
8924
docs/readthedocs/source/_static/hpovis/param_importance_latency.html
Normal file
File diff suppressed because one or more lines are too long
|
|
@ -473,13 +473,14 @@ You can export the trial statistics as pandas dataframe, as shown below.
|
||||||
|
|
||||||
Below an example of the trials history we have exported as below.
|
Below an example of the trials history we have exported as below.
|
||||||
|
|
||||||
|
.. image:: ../../../../image/trial_dataframe.png
|
||||||
|
:width: 600
|
||||||
|
|
||||||
|
|
||||||
|
Plot Hyperparamter Optimization History
|
||||||
Plot optimization history and hyperparameters importance
|
|
||||||
--------------------------------------------------------
|
--------------------------------------------------------
|
||||||
|
|
||||||
You can also examine the tuning results by making plots about the optimization history, importance of hyperparameters, etc.
|
You can also plot the optimization history as shown below.
|
||||||
|
|
||||||
.. tabs::
|
.. tabs::
|
||||||
|
|
||||||
|
|
@ -501,13 +502,207 @@ You can also examine the tuning results by making plots about the optimization h
|
||||||
study = trainer.search_summary()
|
study = trainer.search_summary()
|
||||||
|
|
||||||
from bigdl.nano.automl.hpo.visualization import plot_optimization_history
|
from bigdl.nano.automl.hpo.visualization import plot_optimization_history
|
||||||
plot1=plot_optimization_history(study)
|
plot_optimization_history(study)
|
||||||
|
|
||||||
An example optimization history chart is shown below. It is an interactive chart which you can zoom-in and zoom-out and select data points.
|
Example plot as below. It is an interactive chart which you can zoom-in and zoom-out and select data points.
|
||||||
|
|
||||||
.. only:: html
|
.. only:: html
|
||||||
|
|
||||||
.. raw:: html
|
.. raw:: html
|
||||||
|
|
||||||
<iframe src='../../../_static/visualization.html' height="400px" width="100%" scrolling='no'></iframe>
|
<iframe src='../../../_static/hpovis/optimization_history.html' height="400px" width="100%" scrolling='no'></iframe>
|
||||||
|
|
||||||
|
|
||||||
|
Plot Intermediate Values
|
||||||
|
--------------------------------------------------------
|
||||||
|
|
||||||
|
You can also plot the intermediate values as shown below. This plot shows the metric result on each epoch/step of each trial, including pruned trials.
|
||||||
|
|
||||||
|
.. tabs::
|
||||||
|
|
||||||
|
.. tab:: Tensorflow
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
...
|
||||||
|
study = model.search_summary()
|
||||||
|
|
||||||
|
from bigdl.nano.automl.hpo.visualization import plot_intermediate_values
|
||||||
|
plot_intermediate_values(study)
|
||||||
|
|
||||||
|
.. tab:: PyTorch
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
...
|
||||||
|
study = trainer.search_summary()
|
||||||
|
|
||||||
|
from bigdl.nano.automl.hpo.visualization import plot_intermediate_values
|
||||||
|
plot_intermediate_values(study)
|
||||||
|
|
||||||
|
Example plot as below. It is an interactive chart which you can zoom-in and zoom-out and select data points.
|
||||||
|
|
||||||
|
.. only:: html
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
<iframe src='../../../_static/hpovis/intermediate_values.html' height="400px" width="100%" scrolling='no'></iframe>
|
||||||
|
|
||||||
|
|
||||||
|
Plot the Hyperparameters in Parallel Coordinates
|
||||||
|
------------------------------------------------
|
||||||
|
|
||||||
|
You can plot the hyperparamters in parallel coordinates chart.
|
||||||
|
|
||||||
|
|
||||||
|
.. tabs::
|
||||||
|
|
||||||
|
.. tab:: Tensorflow
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
...
|
||||||
|
study = model.search_summary()
|
||||||
|
|
||||||
|
from bigdl.nano.automl.hpo.visualization import plot_parallel_coordinate
|
||||||
|
plot_parallel_coordinate(study)
|
||||||
|
|
||||||
|
.. tab:: PyTorch
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
...
|
||||||
|
study = trainer.search_summary()
|
||||||
|
|
||||||
|
from bigdl.nano.automl.hpo.visualization import plot_parallel_coordinate
|
||||||
|
plot_parallel_coordinate(study)
|
||||||
|
|
||||||
|
|
||||||
|
Example plot as below. It is an interactive chart which you can zoom-in and zoom-out and select data points.
|
||||||
|
|
||||||
|
|
||||||
|
.. only:: html
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
<iframe src='../../../_static/hpovis/parallel_coordinate.html' height="400px" width="100%" scrolling='no'></iframe>
|
||||||
|
|
||||||
|
|
||||||
|
Plot the Hyperparameter Contour
|
||||||
|
------------------------------------------------
|
||||||
|
|
||||||
|
You can plot the hyperparameter contour chart.
|
||||||
|
|
||||||
|
|
||||||
|
.. tabs::
|
||||||
|
|
||||||
|
.. tab:: Tensorflow
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
...
|
||||||
|
study = model.search_summary()
|
||||||
|
|
||||||
|
from bigdl.nano.automl.hpo.visualization import plot_contour
|
||||||
|
plot_contour(study)
|
||||||
|
|
||||||
|
.. tab:: PyTorch
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
...
|
||||||
|
study = trainer.search_summary()
|
||||||
|
|
||||||
|
from bigdl.nano.automl.hpo.visualization import plot_contour
|
||||||
|
plot_contour(study)
|
||||||
|
|
||||||
|
|
||||||
|
Example plot as below. It is an interactive chart which you can zoom-in and zoom-out and select data points.
|
||||||
|
|
||||||
|
|
||||||
|
.. only:: html
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
<iframe src='../../../_static/hpovis/contour.html' height="400px" width="100%" scrolling='no'></iframe>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Inspect Hyperparameter Importance by accuracy
|
||||||
|
---------------------------------------------
|
||||||
|
|
||||||
|
You can plot the hyperparameter importance according to their relationship to accuracy.
|
||||||
|
|
||||||
|
|
||||||
|
.. tabs::
|
||||||
|
|
||||||
|
.. tab:: Tensorflow
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
...
|
||||||
|
study = model.search_summary()
|
||||||
|
|
||||||
|
from bigdl.nano.automl.hpo.visualization import plot_param_importances
|
||||||
|
plot_param_importances(study)
|
||||||
|
|
||||||
|
.. tab:: PyTorch
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
...
|
||||||
|
study = trainer.search_summary()
|
||||||
|
|
||||||
|
from bigdl.nano.automl.hpo.visualization import plot_param_importances
|
||||||
|
plot_param_importances(study)
|
||||||
|
|
||||||
|
|
||||||
|
Example plot as below. It is an interactive chart which you can zoom-in and zoom-out and select data points.
|
||||||
|
|
||||||
|
|
||||||
|
.. only:: html
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
<iframe src='../../../_static/hpovis/param_importance.html' height="400px" width="100%" scrolling='no'></iframe>
|
||||||
|
|
||||||
|
|
||||||
|
Inspect Hyperparameter Importance by latency
|
||||||
|
--------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
You can plot the hyperparameter importance according to their relationship to latency.
|
||||||
|
|
||||||
|
.. tabs::
|
||||||
|
|
||||||
|
.. tab:: Tensorflow
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
...
|
||||||
|
study = model.search_summary()
|
||||||
|
|
||||||
|
from bigdl.nano.automl.hpo.visualization import plot_param_importances
|
||||||
|
plot_param_importances(study, target=lambda t: t.duration.total_seconds(), target_name="duration")
|
||||||
|
|
||||||
|
.. tab:: PyTorch
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
...
|
||||||
|
study = trainer.search_summary()
|
||||||
|
|
||||||
|
from bigdl.nano.automl.hpo.visualization import plot_param_importances
|
||||||
|
plot_param_importances(study, target=lambda t: t.duration.total_seconds(), target_name="duration")
|
||||||
|
|
||||||
|
|
||||||
|
Example plot as below. It is an interactive chart which you can zoom-in and zoom-out and select data points.
|
||||||
|
|
||||||
|
|
||||||
|
.. only:: html
|
||||||
|
|
||||||
|
.. raw:: html
|
||||||
|
|
||||||
|
<iframe src='../../../_static/hpovis/param_importance_latency.html' height="400px" width="100%" scrolling='no'></iframe>
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue