docs: Special-case command line reference conversion

[armandomontanez]

Special-cases the command line reference conversion so the generated MDX is formatted in a usable way. Since the HTML is rather rigid and predictable, this is sufficient for the initial migration away from DevSite.

Eventually, the command line reference should be generated externally from e.g. proto so the generation mechanism is not tied to the bazel version.

RELNOTES: None

[armandomontanez]

@fweikert @alan707 this is blunt and complex, but it produces MUCH better CLI reference docs. The script itself is Slop™, but I'm short on free time this week.

[armandomontanez]

Ping, the command line reference is pretty broken without this.

[fweikert]

Update: I've imported this change, and it's currently running through presubmit checks.

I had to fix some smaller lint issues, and I removed the changes to src/main/java/com/google/devtools/build/lib/bazel/repository/RepositoryOptions.java since it will make cherry-picking a bit tricky. Can you please send a separate PR for this file?

[ted-xie]

@armandomontanez I understand the rationale for this PR. What I don't understand is why there appears to be a custom HTML parser in the CLR converter. Could you use lxml instead, or some other off-the-shelf parser?

[bazel-docs-pr-commenter]

✅ Bazel docs preview is ready!

Preview URL: https://bazel-pr-29439.mintlify.app/

Updated for c031e7b4322345f098fe8fc380eae61664d56d52

[armandomontanez]

I had to fix some smaller lint issues, and I removed the changes to src/main/java/com/google/devtools/build/lib/bazel/repository/RepositoryOptions.java since it will make cherry-picking a bit tricky. Can you please send a separate PR for this file?

Sent out #30701