Add support for sparse index by serving registry over HTTP as static file

[fMeow]

Serving registry over HTTP as static file is a feature described in rust-lang/rfcs#2789, and the implementation rust-lang/cargo#10470 is already merged in cargo currently as unstable feature http-registry.

This PR add support for http-registry by serving index files over HTTP, and ignored hidden file to avoid leaking .git folder.

1. add sparse_index_config in config to control HTTP registry path

   [sparse_index_config]
   path = "api/v1/crates"

2. add feature gate sparse-index to enable http registry endpoint

[gagbo]

Were you able to test this?

Sorry for coming back to this only now. It seems the feature has been renamed sparse-registry and I'll try to review this.

Basically my main issue is that I don't know where to look for the actual expected format of the registry

[gagbo]

That seems fine as is, just one small question:

Does the current implementation enforces that the same path is used in the HTTP route, and in the local folder on the filesystem? It feels like it creates a lot of friction (especially if there are changes in the unstable feature on Cargo-side)

[gagbo]

Nitpick (and I don't know if it's consistently applied in this codebase), but I'd rather have the cfg attribute here than in the sparse.rs file.

This way I know directly from the main file whether the module is relevant or not

Suggested change

	mod sparse;
	#[cfg(feature = "sparse-index")]
	mod sparse;