[docs][discuss] Convert ReStructuredText docs to Markdown?

Development pre-RFC
#1

Hi all,

Currently the documentation for the TVM site is written in ReStructuredText. However, just about everything else we do is in Markdown:

  • this forum
  • github
  • discord

Needing to know two languages makes it harder for people to contribute docs. It is possible to use Markdown with Sphinx (the tool we use to generate the docs).

What do people think about trying to convert the TVM docs site to Markdown? I did a demo of using Markdown with Sphinx. Feel free to pull and build the doc (note you may need 8828 to work around a bug with our layout in both RST and MD) to try it out.

This approach requires that we use some markdown extensions to interact with Sphinx. There’s some documentation on these here.

Would love to get thoughts/feedback on this idea!

Andrew

2 Likes
#2

Thanks @areusch , we have thought about similar things and tried it out before. To provide a bit of background, our document was initially built with Sphinx markdown parser(recommonmark) that allowed a mix of rst and md. We also made several contributions to recommonmark to structify certain markdown elements.

As we start to write more structured components like cross reference and other layouts, we found out that it is easier to keep a consistent to use reStructuredText instead. The markdown parser was OK for certain components but still do not allow us to obtain the best control of the structure component. MySt likely will have a similar problems we faced before.

Eventually we found that in order to write the best structured doc seems rst is needed and more consistent across the docstring and tutorials(note that we will likely need to use rst for tutorials due to limit of the Sphinx gallery), and some requirement on learning of reStructuredText is needed anyway(although likely be more strange to use), even in the markdown setting. Most of the popular packages such as pytorch, sklearn, numpy follows the approach of using rSt for docs and use markdown for less structured discussions. As a result we turned everything to rst and removed the dep on Sphinx markdown support.

#3

I think I’d lean towards markdown for consistency with other services, but that’s only if all other features were equal. Markdown would be nicer for reviewing, since it can be viewed from github in the browser, but the I think cross-references are the more important feature.

Would either md or rST allow for cross-references between the python API documentation and the C++ documentation? I think that would be the biggest feature for me, since it would give a clear way to indicate which functions are intended to only be wrappers.

#4

@tqchen thanks for the context. I agree that it may not be worth it if you still have to learn restructuredtext to write a single document. with that said, I do think that MyST-Parser seems to be an improvement over recommonmark.

There is also some more context on why MyST-Parser was created. It does seem that they intend to support more of the functionality of restructuredtext and there is more support for extensions.

I agree that it’s not optimal to write non-tutorials in markdown and tutorials in rst. sphinx-gallery appears to be adding support for MyST-Parser, so perhaps when that lands, it could be a good time to reconsider this.

That said, I do think the bar here should be: would the community find it easier to write docs if we support markdown? We have a lot more docs that we need to write.

The markdown parser was OK for certain components but still do not allow us to obtain the best control of the structure component. MySt likely will have a similar problems we faced before.

@tqchen could you elaborate on this point, even if just for historical record? would be great to understand which features myst-parser would need to support to overcome past problems.

#5

Would either md or rST allow for cross-references between the python API documentation and the C++ documentation? I think that would be the biggest feature for me, since it would give a clear way to indicate which functions are intended to only be wrappers.

they both do via references, i believe.

#6

In many cases we need some equivalence of structure tags in rST, most of that can be supported via embedding rST(because markdown was not designed for related constructs). Then it is a bit like inlining asm in another language. The general new constructions in myST dialect might need some time to settle down.

Because Sphinx enjoys a wider community support for rST, most features are also more well tested, sometimes the markdown translation suffered some problems. In a lot of cases we found that we need to fallback to embedding rST for control as well as the API documentation generation. At that point, rST became a more appealing choice.

We initially enabled mixed markdown and rST support with the assumption that such option would have been easier. As we start to use more structured reference tags we found that the difference was not as huge compared to the amount other effort (of putting up text and figured)