ColossalAI/docs/README.md

113 lines
5.4 KiB
Markdown

# 📕 Documentation
## 🔗 Table of Contents
- [📕 Documentation](#-documentation)
- [🔗 Table of Contents](#-table-of-contents)
- [📝 Overview](#-overview)
- [🗺 Module Structure](#-module-structure)
- [🧱 Our Documentation System](#-our-documentation-system)
- [🎊 Contribution](#-contribution)
- [🖊 Adding a New Documentation](#-adding-a-new-documentation)
- [🧹 Doc Testing](#-doc-testing)
- [💉 Auto Documentation](#-auto-documentation)
## 📝 Overview
We evaluated various existing solutions for documentation in the community and discussed their advantages and disadvantages in the [issue #2651](https://github.com/hpcaitech/ColossalAI/issues/2651). Therefore, we propose to build a more modern and robust documentation system by integrating the Sphinx [autodoc](https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html) function and the [Docusaurus](https://docusaurus.io/) framework.
## 🗺 Module Structure
```text
- docs
- source
- en
- zh-Hans
- sidebars.json
- versions.json
- requirements-doc-test.txt
```
The documentation module structure is shown above:
1. source: This folder contains multi-language documentation files.
2. `sidebars.json`: The `sidebars.json` defines the table of content for the tutorials. You need to update this file when a new doc is added/deleted.
3. `versions.json`: The `versions.json` in the **main branch** in the **latest commit** will be used to control the versions to be displayed on our website
## 🧱 Our Documentation System
We believe that the combination of the existing systems can yield several advantages such as simplicity, usability and maintainability:
1. Support [Markdown](https://www.markdownguide.org/). We believe is a more popular language for writing documentation compared to [RST](https://docutils.sourceforge.io/rst.html).
2. Support Autodoc. It can automatically generate documentation from the docstrings in the source code provided by [Sphinx](https://www.sphinx-doc.org/en/master/).
3. Support elegant and modern UI, which is provided by [Docusaurus](https://docusaurus.io/).
4. Support MDX for more flexible and powerful documentation, which is provided by [Docusaurus](https://docusaurus.io/).
5. Support hosting blogs/project home page/other pages besides the documentation, which is provided by [Docusaurus](https://docusaurus.io/).
Therefore, we have built the [ColossalAI-Documentation](https://github.com/hpcaitech/ColossalAI-Documentation) repository to integrate the features above.
## 🎊 Contribution
You can contribute to the documentation by directly setting up a Pull Request towards the `docs/source` folder. There are several guidelines for documentation contribution.
1. The documentation is written in Markdown. You can refer to the [Markdown Guide](https://www.markdownguide.org/) for the syntax.
2. You must ensure that the documentation exists for all languages. You can refer to the [Adding a New Documentation](#-adding-a-new-documentation) for more details.
3. You must provide a test command for your documentation, please see [Doc Testing](#-doc-testing) for more details.
4. You can embed your docstring in your markdown, please see [Auto Documentation](#-auto-documentation) for more details.
### 🖊 Adding a New Documentation
You can add a Markdown file to the `docs/source` folder`. You need to ensure that multi-language is supported in your PR.
Let's assume that you want to add a file called `your_doc.md`, your file structure will look like this.
```text
- docs
- source
- en
- your_doc.md # written in English
- zh-Hans
- your_doc.md # written in Chinese
- sidebars.json # add your documentation file name here
```
Meanwhile, you need to ensure the `sidebars.json` is updated such that it contains your documentation file. Our CI will check whether documentation exists for all languages and can be used to build the website successfully.
### 🧹 Doc Testing
Every documentation is tested to ensure it works well. You need to add the following line to the **bottom of your file** and replace `$command` with the actual command. Do note that the markdown will be converted into a Python file. Assuming you have a `demo.md` file, the test file generated will be `demo.py`. Therefore, you should use `demo.py` in your command, e.g. `python demo.py`.
```markdown
<!-- doc-test-command: $command -->
```
Meanwhile, only code labeled as a Python code block will be considered for testing.
```markdown
```python
print("hello world")
```
```
Lastly, if you want to skip some code, you just need to add the following annotations to tell `docer` to discard the wrapped code for testing.
```markdown
<!--- doc-test-ignore-start -->
```python
print("hello world")
```
<!--- doc-test-ignore-end -->
```
If you have any dependency required, please add it to `requirements-doc-test.txt` for pip and `conda-doc-test-deps.yml` for Conda.
### 💉 Auto Documentation
Lastly, you may want to include the API documentation for a class/function in your documentation for reference.
We support `autodoc` to extract the docstring and transform it into a Web element for an elegant display.
You just need to add `{{ autodoc:<mod-name> }}` in your markdown as a single line. An example is given below and you can see the outcome in [this PR](https://github.com/hpcaitech/ColossalAI-Documentation/pull/175).
```markdown
{{ autodoc:colossalai.legacy.amp.apex_amp.convert_to_apex_amp }}
```