Skip to content

Re-make of central functions - #43

Merged
Kostis-S-Z merged 27 commits into
Mozilla-Data-Collective:mainfrom
Kostis-S-Z:main
Nov 26, 2025
Merged

Re-make of central functions#43
Kostis-S-Z merged 27 commits into
Mozilla-Data-Collective:mainfrom
Kostis-S-Z:main

Conversation

@Kostis-S-Z

@Kostis-S-Z Kostis-S-Z commented Nov 20, 2025

Copy link
Copy Markdown
Contributor

Major updates

  • Re-implement SDK functions following functional programming, instead of OOP to remove necessity of setting up a client. The idea behind this feature is that we make it as easy as possible for devs to integrate the SDK to their codebase.

Before

from datacollective import DataCollective
client = DataCollective()
dataset = client.load_dataset("<dataset-id>")

After

from datacollective import load_dataset
dataset = load_dataset("your-dataset-id")
  • Create registry to handle dataset-specific loading functions. See README

  • Single source of truth for environment variables under api_utils.py

  • Remove setting multiple environments

  • Remove duplicate code by implementing a generic api_request() function to handle GET/POST requests

  • Remove scripts dir + code @moz-johanndiedrick was there a use for it in this repo?

Minor updates

  • Add mkdocs + GH page + CI for docs
  • Add CI for tests
  • Add logo
  • Add PyCharm files (.idea dir) to .gitignore
  • Prettify and simplify README. Docs are moved to GH page.

@Kostis-S-Z Kostis-S-Z self-assigned this Nov 24, 2025
@Kostis-S-Z
Kostis-S-Z marked this pull request as ready for review November 25, 2025 14:56
# Conflicts:
#	README.md
#	scripts/dev.py
#	src/datacollective/client.py

@moz-tello-wharton moz-tello-wharton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes to .github/workflows/publish.yml... would you be able to share a little about the intended flow? It looks like it is getting quite complex.

Specifically, I am curious about how the flow is expected to work when it is main, and when it is release... and if perhaps those should be two separate files?

Comment thread .github/workflows/publish.yml

@moz-johanndiedrick moz-johanndiedrick left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment thread .github/workflows/publish.yml
@Kostis-S-Z
Kostis-S-Z merged commit 3708db8 into Mozilla-Data-Collective:main Nov 26, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants