Experiments on calibrating the correlation between data domains and model alignment outcomes.
Install uv before proceeding.
foo@bar:~$ git clone git@github.com:BrevityD/data-calibrator.git
foo@bar:~$ cd data-calibrator
foo@bar:~/data-calibrator$ uv venv .venv
foo@bar:~/data-calibrator$ uv syncActivate your virtual environment:
foo@bar:~/data-calibrator$ source .venv/bin/activateLog in to WandB (optional):
Skip this step if not using WandB.
Samples:
Configure your hf-token in data-calibrator/datasets/download_data.py.
To obtain your hf-token, visit https://huggingface.co/settings/tokens. To access restricted datasets like livecodebench, grant permission for "Read access to contents of all public gated repos you can access".
foo@bar:~/data-calibrator$ source .venv/bin/activate
(.venv)foo@bar:~/data-calibrator$ cd datasets
(.venv)foo@bar:~/data-calibrator/datasets$ python download_data.py