From 95ef573fc47c87ff2a7e4bf7106583104002719a Mon Sep 17 00:00:00 2001
From: Yuwen Hu <54161268+Oscilloscope98@users.noreply.github.com>
Date: Mon, 31 Oct 2022 16:29:25 +0800
Subject: [PATCH] [Doc] Add contributor guide for writing new documentation
 (#6254)

* Revise toc tree & add draft version for document guide

* Add more contents and revise documentation guide

* Rename new file for documentation guide

* Add some new contents and try to change html tables to plain text tables for generated styles

* Change back to html table with specified css

* Add some new contents and small revision

* Revise contributor index page

* Minor typo fixes for landing page quicklinks

* Revise based on comments

* Add rst card components and small revisions
---
 .../readthedocs/source/_static/css/custom.css |   5 +
 .../source/_templates/sidebar_quicklinks.html |  20 +-
 docs/readthedocs/source/_toc.yml              |   9 +-
 .../source/doc/UserGuide/contributor.rst      |   5 +
 .../source/doc/UserGuide/documentation.md     | 592 ++++++++++++++++++
 5 files changed, 619 insertions(+), 12 deletions(-)
 create mode 100644 docs/readthedocs/source/doc/UserGuide/contributor.rst
 create mode 100644 docs/readthedocs/source/doc/UserGuide/documentation.md

diff --git a/docs/readthedocs/source/_static/css/custom.css b/docs/readthedocs/source/_static/css/custom.css
index e6ca444e..18f909b1 100644
--- a/docs/readthedocs/source/_static/css/custom.css
+++ b/docs/readthedocs/source/_static/css/custom.css
@@ -91,4 +91,9 @@ footer.bd-footer{
 /* the margin height is just the height of original footer, to avoid content overlapping */
 .bd-footer-article {
     margin-bottom: 5rem;
+}
+
+/* for tables in documentation guides, vertically center rows */
+.bigdl-documentation-guide-tables td{
+    vertical-align: middle;
 }
\ No newline at end of file
diff --git a/docs/readthedocs/source/_templates/sidebar_quicklinks.html b/docs/readthedocs/source/_templates/sidebar_quicklinks.html
index 87f723d2..e85e6501 100644
--- a/docs/readthedocs/source/_templates/sidebar_quicklinks.html
+++ b/docs/readthedocs/source/_templates/sidebar_quicklinks.html
@@ -4,8 +4,8 @@
         <ul class="nav">
             <li>
                 <strong class="bigdl-quicklinks-section-title">Orca QuickStart</strong>
-                <input id="quicklink-clsuter-orca" type="checkbox" class="toctree-checkbox" />
-                <label for="quicklink-clsuter-orca" class="toctree-toggle">
+                <input id="quicklink-cluster-orca" type="checkbox" class="toctree-checkbox" />
+                <label for="quicklink-cluster-orca" class="toctree-toggle">
                     <i class="fa-solid fa-chevron-down"></i>
                 </label>
                 <ul class="bigdl-quicklinks-section-nav">
@@ -22,8 +22,8 @@
             </li>
             <li>
                 <strong class="bigdl-quicklinks-section-title">Nano QuickStart</strong>
-                <input id="quicklink-clsuter-nano" type="checkbox" class="toctree-checkbox" />
-                <label for="quicklink-clsuter-nano" class="toctree-toggle">
+                <input id="quicklink-cluster-nano" type="checkbox" class="toctree-checkbox" />
+                <label for="quicklink-cluster-nano" class="toctree-toggle">
                     <i class="fa-solid fa-chevron-down"></i>
                 </label>
                 <ul class="nav bigdl-quicklinks-section-nav">
@@ -46,8 +46,8 @@
             </li>
             <li>
                 <strong class="bigdl-quicklinks-section-title">DLlib QuickStart</strong>
-                <input id="quicklink-clsuter-dllib" type="checkbox" class="toctree-checkbox" />
-                <label for="quicklink-clsuter-dllib" class="toctree-toggle">
+                <input id="quicklink-cluster-dllib" type="checkbox" class="toctree-checkbox" />
+                <label for="quicklink-cluster-dllib" class="toctree-toggle">
                     <i class="fa-solid fa-chevron-down"></i>
                 </label>
                 <ul class="nav bigdl-quicklinks-section-nav">
@@ -61,8 +61,8 @@
             </li>
             <li>
                 <strong class="bigdl-quicklinks-section-title">Chronos QuickStart</strong>
-                <input id="quicklink-clsuter-chronos" type="checkbox" class="toctree-checkbox" />
-                <label for="quicklink-clsuter-chronos" class="toctree-toggle">
+                <input id="quicklink-cluster-chronos" type="checkbox" class="toctree-checkbox" />
+                <label for="quicklink-cluster-chronos" class="toctree-toggle">
                     <i class="fa-solid fa-chevron-down"></i>
                 </label>
                 <ul class="nav bigdl-quicklinks-section-nav">
@@ -79,8 +79,8 @@
             </li>
             <li>
                 <strong class="bigdl-quicklinks-section-title">PPML QuickStart</strong>
-                <input id="quicklink-clsuter-ppml" type="checkbox" class="toctree-checkbox" />
-                <label for="quicklink-clsuter-ppml" class="toctree-toggle">
+                <input id="quicklink-cluster-ppml" type="checkbox" class="toctree-checkbox" />
+                <label for="quicklink-cluster-ppml" class="toctree-toggle">
                     <i class="fa-solid fa-chevron-down"></i>
                 </label>
                 <ul class="nav bigdl-quicklinks-section-nav">
diff --git a/docs/readthedocs/source/_toc.yml b/docs/readthedocs/source/_toc.yml
index e53706c0..d7a2df75 100644
--- a/docs/readthedocs/source/_toc.yml
+++ b/docs/readthedocs/source/_toc.yml
@@ -239,8 +239,13 @@ subtrees:
                   - file: doc/PPML/Overview/azure_ppml_occlum
 
   - entries:
-    - file: doc/UserGuide/develop
-      title: "Developer guide"
+    - file: doc/UserGuide/contributor
+      title: "Contributor guide"
+      subtrees:
+        - entries:
+          - file: doc/UserGuide/develop
+          - file: doc/UserGuide/documentation
+
 
 
   - entries:
diff --git a/docs/readthedocs/source/doc/UserGuide/contributor.rst b/docs/readthedocs/source/doc/UserGuide/contributor.rst
new file mode 100644
index 00000000..92ad50af
--- /dev/null
+++ b/docs/readthedocs/source/doc/UserGuide/contributor.rst
@@ -0,0 +1,5 @@
+Contributor Guide
+=========================
+
+* `Developer Guide <./develop.html>`_ provides instructions on how to build from source and contribute to BigDL source code.
+* `Documentation Guide <./documentation.html>`_ provides tips and guidelines for adding/modifying BigDL documents.
\ No newline at end of file
diff --git a/docs/readthedocs/source/doc/UserGuide/documentation.md b/docs/readthedocs/source/doc/UserGuide/documentation.md
new file mode 100644
index 00000000..7ef0a12a
--- /dev/null
+++ b/docs/readthedocs/source/doc/UserGuide/documentation.md
@@ -0,0 +1,592 @@
+# Documentation Guide
+
+Here list several writing tips and guidelines you could refer to if you want to add/modify documents for BigDL documentation. The source code of our documentation is available [here](https://github.com/intel-analytics/BigDL/tree/main/docs/readthedocs).
+
+```eval_rst
+.. tip::
+
+   You could refer `here <https://github.com/intel-analytics/BigDL/blob/main/docs/readthedocs/README.md>`_ if you would like to test your local changes to BigDL documentation.
+```
+
+## 1. How to add a new document
+### 1.1 Decide whether to add a reStructuredText (`.rst`) file or a CommonMark (`.md`) file
+In our documentation, both reStructuredText (`.rst`) and CommonMark (`.md`) files are allowed to use. In convension, we use `.rst` file in index pages, and `.md` files for other pages.
+
+Here shows an overview of our documentation structure tree:
+
+```eval_rst
+.. graphviz::
+
+   digraph DocStructure {
+      graph [tooltip=" " splines=ortho]
+      node [color="#0171c3" shape=box fontname="Arial" fontsize=12 tooltip=" "]
+      edge [tooltip=" "]
+      
+      N1 [label="BigDL (.rst)" style=filled fontcolor="#ffffff"]
+      
+      N1_1 [label="User guide (.rst)" style=filled fontcolor="#ffffff"]
+      N1_2 [label="Powered by (.md)" style=rounded]
+      N1_3 [label="Orca (.rst)" style=filled fontcolor="#ffffff"]
+      N1_4 [label="Nano (.rst)" style=filled fontcolor="#ffffff"]
+      N1_5 [label="DLlib (.rst)" style=filled fontcolor="#ffffff"]
+      N1_6 [label="Chronos (.rst)" style=filled fontcolor="#ffffff"]
+      N1_7 [label="Fresian (.rst)" style=filled fontcolor="#ffffff"]
+      N1_8 [label="PPML (.rst)" style=filled fontcolor="#ffffff"]
+      N1_9 [label="..." shape=plaintext]
+      
+      N1_1_1 [label="Python (.md)" style=rounded]
+      N1_1_2 [label="Scala (.md)" style=rounded]
+      N1_1_3 [label="..." shape=plaintext]
+      
+      N1_8_1 [label="PPML Intro. (.md)" style=rounded]
+      N1_8_2 [label="User Guide (.md)" style=rounded]
+      N1_8_3 [label="Tutorials (.rst)" style="filled" fontcolor="#ffffff"]
+      N1_8_4 [label="..." shape=plaintext]
+      
+      
+      N1_8_3_1 [label="..." shape=plaintext]
+      
+      N1_3_1 [label="..." shape=plaintext]
+      N1_4_1 [label="..." shape=plaintext]
+      N1_5_1 [label="..." shape=plaintext]
+      N1_6_1 [label="..." shape=plaintext]
+      N1_7_1 [label="..." shape=plaintext]
+      
+      N1 -> N1_1
+      N1 -> N1_2
+      N1 -> N1_3 -> N1_3_1
+      N1 -> N1_4 -> N1_4_1
+      N1 -> N1_5 -> N1_5_1
+      N1 -> N1_6 -> N1_6_1
+      N1 -> N1_7 -> N1_7_1
+      N1 -> N1_8
+      N1 -> N1_9
+      
+      N1_1 -> N1_1_1
+      N1_1 -> N1_1_2
+      N1_1 -> N1_1_3
+      
+      N1_8 -> N1_8_1
+      N1_8 -> N1_8_2
+      N1_8 -> N1_8_3 -> N1_8_3_1
+      N1_8 -> N1_8_4
+   }
+```
+
+Index pages (nodes filled with blue) are the ones supposed to lead to further pages. In the structure above, they are nodes with descendants.
+
+```eval_rst
+.. note::
+   
+   In convension, we use ``.rst`` file for index pages becuase various web components (such as cards, note boxes, tabs, etc.) are more straightforward to be inserted in our documentation through reStructuredText. And it is a common case in our documentation that index pages include various web components.
+```
+
+### 1.2 Add the new document to the table of contents (ToC)
+For clear navigation purposes, it is recommended to put the document in the ToC. To do this, you need to insert the relative path to the newly-added file into the [`_toc.yml`](https://github.com/intel-analytics/BigDL/blob/main/docs/readthedocs/source/_toc.yml) file, according to its position in the structure tree.
+
+```eval_rst
+.. tip::
+
+   When adding a new document, you should always check whether to put relative link directing to it inside its parent index page, or inside any other related pages.
+
+.. warning::
+
+   According to `sphinx-external-toc <https://sphinx-external-toc.readthedocs.io/en/latest/user_guide/sphinx.html#basic-structure>`_ document, "each document file can only occur once in the ToC".
+```
+
+For API related documents, we still use in-file `.. toctree::` directives instead of putting them inside `_toc.yml`. You could refer [here](https://github.com/intel-analytics/BigDL/tree/main/docs/readthedocs/source/doc/PythonAPI) for example usages.
+
+## 2. Differentiate the syntax of reStructuredText and CommonMark
+As mentioned above, our documentation includes both `.rst` and `.md` files. They have different syntax, please make sure you do not mix the usage of them.
+
+```eval_rst
+.. seealso::
+
+   You could refer `here <https://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html>`_ for reStructuredText syntax examples, and `here <https://spec.commonmark.org/>`_ for CommonMark specifications.
+```
+
+Here list several use cases where syntax in `.rst` and `.md` are often confused:
+
+<table class="table bigdl-documentation-guide-tables">
+<tr><th></th><th>reStructuredText</th><th>CommonMark</th></tr>
+<tr><td>Inline code</td>
+<td>
+
+```rst
+``inline code``
+```
+</td>
+<td>
+
+```md
+`inline code`
+```
+
+</td></tr>
+<tr><td>Hyperlinks</td>
+<td>
+
+```rst
+`Relative link text <relatve/path/to/the/file>`_ 
+`Absolute link text <https://www.example.com/>`_ 
+```
+
+</td>
+<td>
+
+```md
+[Relative link text](relatve/path/to/the/file)
+[Absolute link text](https://www.example.com/)
+```
+
+</td></tr>
+<tr><td>Italic</td>
+<td>
+
+```rst
+`italicized text`
+*italicized text*
+```
+</td>
+<td>
+
+```md
+*italicized text*
+
+```
+
+</td></tr>
+<tr><td>Italic & bold</td>
+<td>
+
+Not supported, needed help with css
+
+</td>
+<td>
+
+```md
+***italicized & bold text***
+```
+
+</td></tr>
+</table>
+
+```eval_rst
+.. note::
+
+   When linking to a ``.rst`` file in a ``.md`` file, replace the ``.rst`` with ``.html`` in the relative path to avoid errors.
+   That is, if you want to link to the ``example.rst`` in a ``.md`` file, use 
+
+   .. code-block:: md
+
+      [Example](relatve/path/to/example.html)
+```
+
+### 2.1 Tips when adding docstrings in source code for API documentation
+According to the [`sphinx.ext.autodoc`](https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#module-sphinx.ext.autodoc) document, docstrings should be written in reStructuredText. We need to make sure that we are using reStructuredText syntax in the source code docstrings for API documentation.
+
+There are two [field lists](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#field-lists) syntax often used in API documentation for parameter definition and return values. Let us take a snippet from [`bigdl.nano.pytorch.InferenceOptimizer.get_best_model`](../PythonAPI/Nano/pytorch.html#bigdl.nano.pytorch.InferenceOptimizer.get_best_model) as an example:
+```rst
+:param use_ipex: (optional) if not None, then will only find the
+       model with this specific ipex setting.
+:param accuracy_criterion: (optional) a float represents tolerable
+       accuracy drop percentage, defaults to None meaning no accuracy control.
+:return: best model, corresponding acceleration option
+```
+
+```eval_rst
+.. important::
+
+   The following lines of one parameter/return definition should be indented to be rendered correctly.
+
+.. tip::
+
+   Please always check whether corresponding API documentation is correctly rendered when changes made to the docstrings.
+```
+## 3. Common components in `.rst` files
+
+<table class="table bigdl-documentation-guide-tables">
+<tr><td>Headers</td>
+<td>
+
+```rst
+Header Level 1
+=========================
+
+Header Level 2
+-------------------------
+
+Header Level 3
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Header Level 4
+^^^^^^^^^^^^^^^^^^^^^^^^^
+```
+
+</td>
+<td>
+
+Note that the underline symbols should be at least as long as the header texts.
+
+Also, **we do not expect maually-added styles to headers.**
+
+You could refer [here](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#sections) for more information on reStructuredText sections.
+
+</td></tr>
+<tr><td>Lists</td>
+<td>
+
+```rst
+* A unordered list
+* The second item of the unordered list
+  with two lines
+
+#. A numbered list
+
+   1. A nested numbered list
+   2. The second nested numbered list
+
+#. The second item of 
+   the numbered list
+```
+
+</td>
+<td>
+
+Note that the number of spaces indented depends on the markup. That is, if we use '* '/'#. '/'10. ' for the list, the following contents belong to the list or the nested lists after it should be indented by 2/3/4 spaces.
+
+Also note that blanks lines are needed around the nested list.
+
+You could refer [here](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#lists-and-quote-like-blocks) for more information on reStructuredText lists.
+
+</td></tr>
+<tr><td>
+
+Note, <br>
+Warning, <br>
+Danger, <br>
+Tip, <br>
+Important, <br>
+See Also <br>
+boxes
+
+</td>
+<td>
+
+```rst
+.. note::
+
+   This is a note box.
+
+.. warning::
+
+   This is a warning box.
+
+.. danger::
+
+   This is a danger box.
+
+.. tip::
+
+   This is a tip box.
+
+.. important::
+
+   This is an important box.
+
+.. seealso::
+
+   This is a see also box.
+```
+
+</td>
+<td>
+
+```eval_rst
+.. note::
+
+   This is a note box.
+
+.. warning::
+
+   This is a warning box.
+
+.. danger::
+
+   This is a danger box.
+
+.. tip::
+
+   This is a tip box.
+
+.. important::
+
+   This is an important box.
+
+.. seealso::
+
+   This is a see also box.
+```
+
+</td></tr>
+<tr><td>Code blocks</td>
+<td>
+
+```rst
+.. code-block:: [language]
+
+   some code in this language
+
+.. code-block:: python
+
+   some python code
+```
+
+</td>
+<td>
+
+All the supported language argument for syntax highlighting can be found [here](https://pygments.org/docs/lexers/).
+
+</td></tr>
+<tr><td>Tabs</td>
+<td>
+
+```rst
+.. tabs::
+
+   .. tab:: Title 1
+
+      Contents for tab 1
+
+   .. tab:: Title 2
+
+      Contents for tab 2
+
+      .. code-block:: python
+
+         some python code
+```
+
+</td>
+<td>
+
+```eval_rst
+.. tabs::
+
+   .. tab:: Title 1
+
+      Contents for tab 1
+
+   .. tab:: Title 2
+
+      Contents for tab 2
+
+      .. code-block:: python
+
+         some python code
+```
+
+You could refer [here](https://sphinx-tabs.readthedocs.io/en/v3.4.0/) for more information on the usage of tabs.
+
+</td></tr>
+<tr><td>Cards in grids</td>
+<td>
+
+```rst
+.. grid:: 1 2 2 2
+   :gutter: 2
+
+   .. grid-item-card::
+
+      **Header**
+      ^^^
+      A normal card.
+      +++
+      :bdg-link:`Footer <relatve/path>`
+
+   .. grid-item-card::
+      :link: https://www.example.com/
+      :class-card: bigdl-link-card
+
+      **Header**
+      ^^^
+      A link card.
+      +++
+      Footer
+```
+
+</td>
+<td>
+
+```eval_rst
+.. grid:: 1 2 2 2
+   :gutter: 2
+
+   .. grid-item-card::
+
+      **Header**
+      ^^^
+      A normal card.
+      +++
+      :bdg-link:`Footer <relatve/path>`
+
+   .. grid-item-card::
+      :link: https://www.example.com/
+      :class-card: bigdl-link-card
+
+      **Header**
+      ^^^
+      A link card.
+      +++
+      Footer
+```
+
+You could refer [here](https://sphinx-design.readthedocs.io/en/furo-theme/cards.html) for more information on the usage of cards, and [here](https://sphinx-design.readthedocs.io/en/furo-theme/grids.html#placing-a-card-in-a-grid) for cards in grids.
+
+Note that `1 2 2 2` defines the number of cards per row in different screen sizes (from extra-small to large).
+
+</td></tr>
+</table>
+
+### 3.1 Use reStructuredText in `.md` files
+You could embed reStructuredText into `.md` files through putting reStructuredText code into `eval_rst` code block:
+~~~md
+```eval_rst
+any contents in reStructuredText syntax
+```
+~~~
+It could be useful if you want to use special boxes (e.g. note box) or tabs in your `.md` files.
+
+```eval_rst
+.. important::
+
+   Any contents inside ``eval_rst`` code block should follow the reStructuredText syntax.
+```
+
+## 4. Common components in `.md` files
+<table class="table bigdl-documentation-guide-tables">
+<tr><td>Headers</td>
+<td>
+
+```md
+# Header Level 1
+
+## Header Level 2
+
+### Header Level 3
+
+#### Header Level 4
+```
+
+</td>
+<td>
+
+Note that **we do not expect maually-added styles to headers.**
+
+</td></tr>
+<tr><td>Lists</td>
+<td>
+
+```md
+- A unordered list
+- The second item of the unordered list
+  with two lines
+
+1. A numbered list
+   * A nested unordered list
+   * The second nested unordered list
+2. The second item of 
+   the numbered list
+```
+
+</td>
+<td>
+
+Note that the number of spaces indented depends on the markup. That is, if we use '- '/'1. '/'10. ' for the list, the following contents belong to the list or the nested lists after it should be indented by 2/3/4 spaces.
+
+</td></tr>
+<tr><td>Code blocks</td>
+<td>
+
+~~~md
+```[language]
+some code in this language
+```
+
+```python
+some python code
+```
+~~~
+
+</td>
+<td>
+
+All the supported language argument for syntax highlighting can be found [here](https://pygments.org/docs/lexers/).
+
+</td>
+</tr>
+</table>
+
+## 5. How to include Jupyter notebooks directly inside our documentation
+If you want to include a Jupyter notebook into our documentation as an example, a tutorial, a how-to guide, etc., you could just put it anywhere inside [`BigDL/docs/readthedocs/source`](https://github.com/intel-analytics/BigDL/tree/main/docs/readthedocs/source) dictionary, and link it into `_toc.yml` file.
+
+However, if you want to render a Jupyter notebook located out of `BigDL/docs/readthedocs/source` dictionary into our documentation, the case is a little bit complicated. To do this, you need to add a file with `.nblink` extension into `BigDL/docs/readthedocs/source` , and link the `.nblink` file into `_toc.yml`.
+
+The `.nblink` file should have the following structure:
+```json
+{
+    "path": "relative/path/to/the/notebook/you/want/to/include"
+}
+```
+
+```eval_rst
+.. seealso::
+
+   You could find `here <https://github.com/intel-analytics/BigDL/blob/main/docs/readthedocs/source/doc/Nano/Howto/Training/PyTorchLightning/accelerate_pytorch_lightning_training_ipex.nblink>`_ for an example of ``.nblink`` usage inside our documentation.
+```
+
+### 5.1 How to hide a cell from rendering
+
+If you want to hide a notebook markdown/code cell from rendering into our documentation, you could simply add `"nbsphinx": "hidden"` into the cell's `metadata`.
+
+Here shows an examlpe of a markdown cell hidden from rendering:
+
+```json
+{
+"cell_type": "markdown",
+"metadata": {
+   "nbsphinx": "hidden"
+},
+"source": [
+   ...
+]
+}
+```
+
+```eval_rst
+.. tip::
+
+   You could simply open the notebook through text editor to edit the ``metadata`` of each cell.
+
+.. note::
+
+   Currently we could not hide the output/input code cell individually from rendering as they have the same ``metadata``. 
+```
+
+### 5.2 Note/Warning/Related Readings boxes
+In convension, in the markdown cell of notebooks, we create note/warning/related reading boxes with the help of quote blocks and emoji:
+
+```md
+> 📝 **Note**
+>
+> This is a note box in notebooks.
+
+> ⚠️ **Warning**
+> 
+> This is a warning box in notebooks.
+
+> 📚 **Related Readings**
+> 
+> This is a related readings box in notebooks.
+
+```
\ No newline at end of file