small language models research
Aim:
- Do open source, state of the art language models research
- Setup a go-to language model training pipeline, similar to nanochat.
- Explore and experiment with promising architectures, datasets, training paradigms
- One stop destination to experiment and share results, models, datasets, ideas with the community