Make load_dataset work offline (no internet) when dataset + schema is cached.

[Kostis-S-Z]

load_dataset currently requires the MDC API and the schema registry to be reachable, even when everything needed to load the dataset is already on disk.

A user who has previously run load_dataset("my-dataset") has, locally:

• the downloaded archive (~/.mozdata/datasets/.tar.gz)
• the extracted directory
• a cached schema.yaml inside it

But on a later run with no internet (train on a plane, VM with no internet access, MDC API outage), the call fails immediately at get_dataset_details (datasets.py:208) with a raw requests.ConnectionError, before any of the local-cache logic is reached.

Most of the pipeline is already offline-safe:

• _download_dataset returns early without network if the archive exists (download.py:70)
• _extract_archive skips if the extract dir exists (archive_utils.py:34)
• _resolve_schema already has a cached-schema path (cache_schema.py:43-52)

The blocker is that we can't resolve dataset_id → archive filename / checksum / paths without the API, since that mapping only ever exists in the API response (DatasetDetails.filename).

There are few different ways to approach this.

One is through a global, machine-local index of downloaded datasets, i.e. a single JSON file (e.g. ~/.mozdata/index.json, overridable via env) maintained by the SDK, mapping each dataset the user has downloaded to everything needed to load it without the network:

• dataset id
• slug (so offline lookups work whether the user passes the id or the slug)
• archive filename + checksum
• absolute archive path and absolute extracted-directory path
• last-synced timestamp

The index is written/updated on every successful download_dataset / load_dataset, and read as a fallback when the API or registry is unreachable. It is machine-dependent by design (absolute paths, user-specific download dirs).

This also decouples "which datasets do I have locally" from "which download directory was passed on this particular call", which is currently only discoverable by guessing filenames.

Notes

• No breaking changes needed. The offline fallback can be automatic on connection errors, so load_dataset's signature is unchanged.
• Offline means freshness cannot be validated (no API checksum to compare against cache_schema.py:47) should emit a clear logger.warning that a possibly-stale cached copy is being used, never fail silently.
• _get_dataset_schema should distinguish "404, not in registry" from "registry unreachable". Currently both collapse into a RuntimeError (schema.py:202-207), which makes the existing cached-schema fallback at cache_schema.py:57-59 unreachable offline.
• Entries should be validated on read (paths may have been moved or deleted since) and pruned/refreshed rather than trusted blindly.