Automatically generate CLI reference documentation via sphinx-click and a custom directive

[agriyakhetarpal]

I noticed in #350 that our CLI reference page is likely to get outdated often, as we may forget to update it with new CLI flags and subcommands being added. This PR introduces an inline Sphinx extension with a directive that scans entry points added by pyodide-build in pyodide-cli and adds them to the page. See the PR preview: https://pyodide-build--355.org.readthedocs.build/en/355/reference/cli.html

We can also skip such automation if we want, but based on my experience writing customisations for the Sphinx build process, this will remain fairly stable and shouldn't break in the future. It also beats adding new commands manually, so I recommend keeping it :D

The extension doesn't support Sphinx 9 yet, as I noticed from this issue that I encountered: click-contrib/sphinx-click#159, which has a PR to fix it: click-contrib/sphinx-click#160. We don't need any cutting-edge features from Sphinx anyway and none of our extensions imposes a Sphinx >=9 bound, so we should be fine with 8.x. Also, the extension doesn't support Markdown, but that's easily workaroundable with {eval-rst}: click-contrib/sphinx-click#127.

Lastly, I dropped the requirements-doc.txt file, as it's kind of redundant with dependency groups now.

[ryanking13]

Thanks! Would it be able to hide some of the subcommands from the generation? I intentionally not included build-recipes, build-recipes-no-deps, skeleton subcommands as I don't want to distract users with recipe-related commands. Most of the users who are looking at that document don't need to be aware of the recipe.

[agriyakhetarpal]

Done! I also updated the config variables table since it was a bit outdated. Also, I'll open an issue to reorganise the documentation using Diátaxis.

[ryanking13]

Thanks! I left some minor comments, otherwise looks good to me

[agriyakhetarpal]

Thanks! Done!