ColossalAI/docs

📕 Documentation

🔗 Table of Contents

• 📕 Documentation
  • 🔗 Table of Contents
  • 📝 Overview
  • 🗺 Module Structure
  • 🧱 Our Documentation System
  • 🎊 Contribution
    • 🖊 Adding a New Documentation
    • 🧹 Doc Testing
    • 💉 Auto Documentation

📝 Overview

We evaluated various existing solutions for documentation in the community and discussed their advantages and disadvangtes in the issue #2651. Therefore, we propose to build a more modern and robust documentation system by integrating the Sphinx autodoc function and the Docusaurus framework.

🗺 Module Structure

- docs
    - source
        - en
        - zh-Hans
    - sidebars.json
    - versions.json
    - requirements-doc-test.txt

The documentation module structure is shown above:

1. source: This folder contains the multi-language documentation files.
2. sidebars.json: The sidebars.json defines the table of content for the tutorials. You need to update this file when a new doc is added/deleted.
3. versions.json: The versions.json in the main branch in the latest commit will be used to control the versions to be displayed on our website

🧱 Our Documentation System

We believe that there are several advantages from the existing system can be combined for simplicity, usability and maintainability:1

1. Support Markdown](https://www.markdownguide.org/), we belive is a more popular language for writing documentations comapred to RST.
2. Support Autodoc, which can automatically generate documentation from the docstrings in the source code provided by Sphinx.
3. Support elegant and modern UI, which is provided by Docusaurus.
4. Support MDX for more flexible and powerful documentation, which is provided by Docusaurus.
5. Support hosting blogs/project home page/other pages besides the documentation, which is provided by Docusaurus.

Therefore, we have built the ColossalAI-Documentation repository to integrate the features above.

🎊 Contribution

You can contribute to the documentation by directly set up a Pull Request towards the docs/source folder. There are several guidelines for documentation contribution.

1. The documentation is written in Markdown. You can refer to the Markdown Guide for the syntax.
2. You must ensure that the documentation exists for all languages. You can refer to the Adding a New Documentation for more details.
3. You must provide a test command for your documentation, please see Doc Testing for more details.
4. You can embed your docstring in your markdown, please see Auto Documentation for more details.

🖊 Adding a New Documentation

You can add a Markdown file to the docs/source folder. You need to ensure that multi-language is supported in your PR. Let's assume that you want to add a file called your_doc.md`, your file structure will look like this.

- docs
  - source
    - en
        - your_doc.md  # written in English
    - zh-Hans
        - your_doc.md  # written in Chinese
  - sidebars.json  # add your documentation file name here

Meanwhile, you need to ensure the sidebars.json is updated such that it contains your documentation file. Our CI will check whether a documentation exists for all languages and can be used to build the website successfully.

🧹 Doc Testing

Every documentation is tested to ensure it works well. You need to add the following line to the bottom of your file and replace $command with the actual command. Do note that the markdown will be converted into a Python file. Assuming you have a demo.md file, the test file generated will be demo.py. Therefore, you should use demo.py in your command, e.g. python demo.py.

<!-- doc-test-command: $command  -->

Meanwhile, only code labelled as a Python code block will be considered for testing.

    ```python
    print("hello world")
    ```

Lastly, if you want to skip some code, you just need to add the following annotations to tell docer to discard the wrapped code for testing.

<!--- doc-test-ignore-start -->

    ```python
    print("hello world")
    ```

<!--- doc-test-ignore-end -->

If you have any dependency required, please add it to requriements-doc-test.txt for pip and conda-doc-test-deps.yml for conda.

💉 Auto Documentation

Lastly, you may want to include the API documentation for a class/function in your documentation for reference. We support autodoc to extract the docstring and transform it into a Web element for elegant display. You just need to add {{ autodoc:<mod-name> }} in your markdown as a single line. An example is given below and you can see the outcome in this PR.

{{ autodoc:colossalai.amp.apex_amp.convert_to_apex_amp }}