diff --git a/README.md b/README.md index 0b0a722..6e968a4 100644 --- a/README.md +++ b/README.md @@ -52,10 +52,10 @@ export MDC_API_KEY=your-api-key-here MDC_API_KEY=your-api-key-here ``` -3. **Get your dataset ID from the last section of the dataset URL at the MDC website**. +3. **Get your dataset ID and/or slug from the dataset's page at the MDC website**. > [!TIP] -> You can find the `dataset-id` by looking at the URL of the dataset's page on MDC platform. The ID is the unique string of characters located at the very end of the URL, after the `/datasets/` path. For example, for URL `https://mozilladatacollective.com/datasets/cminc35no007no707hql26lzk` dataset id will be `cminc35no007no707hql26lzk`. +> You can find the dataset's **ID** and/or **slug** from its page on the MDC platform. Click the **Download** button and select **API / Python Access**, then switch to the **Python Library** tab on the pop-up modal. The **Dataset ID** and **Dataset Slug** appear at the bottom of the modal. Both can be used interchangeably in the Python library, but the slug is a more user-friendly identifier. 4. **Save a dataset locally**: ``` @@ -102,6 +102,10 @@ Returns a `Dataset`, or a `DatasetDict` keyed by split name for datasets with mu ## Programmatic submissions and uploads +> [!ΝΟΤΕ] +> In order to be able to upload datasets in the MDC platform you will first need to Request Access to Upload by +> navigating to your [profile](https://mozilladatacollective.com/profile/uploads) under the `Upload` tab. + You can create dataset submissions and upload files with resumable uploads into the MDC platform programmatically using our Python SDK: ```python diff --git a/docs/index.md b/docs/index.md index cddc63e..dc55147 100644 --- a/docs/index.md +++ b/docs/index.md @@ -87,7 +87,7 @@ export MDC_LOG_PATH=/path/to/save/the/log/files **IMPORTANT NOTE:** Before trying to access any dataset, make sure you have thoroughly **read and agreed** to the specific dataset's conditions & licensing terms. !!! tip - You can find the `dataset-id` by looking at the URL of the dataset's page on MDC platform. The ID is the unique string of characters located at the very end of the URL, after the `/datasets/` path. For example, for URL `https://mozilladatacollective.com/datasets/cmflnuzw6lrt9e6ui4kwcshvn` dataset id will be `cmflnuzw6lrt9e6ui4kwcshvn`. + You can find the dataset's **ID** and/or **slug** from its page on the MDC platform. Click the **Download** button and select **API / Python Access**, then switch to the **Python Library** tab on the pop-up modal. The **Dataset ID** and **Dataset Slug** appear at the bottom of the modal. Both can be used interchangeably in the Python library, but the slug is a more user-friendly identifier. ### Download a dataset @@ -112,6 +112,11 @@ The files will be stored under `MDC_DOWNLOAD_PATH` (default `~/.mozdata/datasets The SDK supports creating dataset submissions and uploading files with resumable uploads. The upload state is stored in a JSON file alongside the archive so interrupted uploads can resume automatically. +> [!ΝΟΤΕ] +> In order to be able to upload datasets in the MDC platform you will first need to Request Access to Upload by +> navigating to your [profile](https://mozilladatacollective.com/profile/uploads) under the `Upload` tab. + + ```python from datacollective import ( DatasetSubmission, diff --git a/docs/upload.md b/docs/upload.md index 226cce1..5404786 100644 --- a/docs/upload.md +++ b/docs/upload.md @@ -19,6 +19,7 @@ The SDK also supports **resumable uploads**, meaning if an upload is interrupted Before uploading, ensure you have: +- Your Request Access to Upload **approved** in your MDC [profile](https://mozilladatacollective.com/profile/uploads). - An API key from the Mozilla Data Collective [dashboard](https://mozilladatacollective.com/api-reference) - Your dataset packaged as an archive file (`.tar.gz`, uploads use `application/gzip`) - All the required metadata for the dataset submission