Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
# Required
version: 2

# Set the version of Python and other tools you might need
# Set the version of Python and other tools that might be needed
build:
os: ubuntu-lts-latest
tools:
Expand All @@ -18,6 +18,6 @@ build:
sphinx:
configuration: docs/source/conf.py

# Optionally declare the Python requirements required to build your docs
# Optionally declare the Python requirements required to build the docs
conda:
environment: environment.yml
96 changes: 86 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,92 @@
# The CADET-Research Data Management toolbox
# CADET-RDM

Welcome to CADET-RDM, a project by the Forschungszentrum Jülich.
[![CI](https://github.com/cadet/CADET-RDM/actions/workflows/CI.yml/badge.svg)](https://github.com/cadet/CADET-RDM/actions/workflows/CI.yml)
[![Documentation](https://readthedocs.org/projects/cadet-rdm/badge/?version=latest)](https://cadet-rdm.readthedocs.io)
[![License](https://img.shields.io/github/license/cadet/cadet-rdm)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.9%2B-blue)](https://www.python.org/)

CADET-RDM is a Research Data Management toolbox developed at Forschungszentrum Jülich.
It supports computational research projects by tracking code, data, environments, and generated results in a reproducible and shareable way.

The toolbox is domain-agnostic and can be applied to any computational project with a structured workflow.


## Scope and purpose

CADET-RDM helps manage and version

This toolbox aims to help track and version control:
- input data
- code
- software versions
- output data
- source code
- configurations and metadata
- software and environment versions
- generated output data

The primary goal is to ensure reproducibility, traceability, and reuse of computational results by explicitly linking them to the project state that produced them.


## Repository structure

A CADET-RDM project consists of two independent but coupled Git repositories:

1. **Project repository**
Contains source code, configuration files, documentation, and metadata required to execute the computations.

2. **Output repository**
Contains the results generated by running the project code, including data products, models, figures, and run-specific metadata.

Both repositories have separate Git histories and remotes. CADET-RDM provides workflows that operate on both repositories to maintain a consistent link between code and results.

## Using CADET-RDM

### Result tracking and reproducibility

Each execution of project code creates a new output branch that contains only the files generated by that run.

In addition, a central run history records

- the project repository commit used for the run
- software and environment information
- metadata required to reproduce the result

This commit structure allows results to be reproduced and inspected without manual bookkeeping.

### Interfaces

CADET-RDM can be used through

* a **command line interface (CLI)**, e.g. for scripted or automated bash workflows
* a **Python interface**, e.g. for direct context tracking of code within existing Python workflows

Additionally, CADET-RDM can be used within Jupyter Lab with some limitations.

Detailed descriptions of commands and APIs are provided in the dedicated interface documentation.

* [Command line interface](command-line-interface.md)
* [Python interface](python-interface.md)
* [Jupyter interface](jupyter-interface.md)

### Typical workflow

1. Initialize or clone a CADET-RDM project
2. Develop and commit project code
3. Execute computations with CADET-RDM result tracking
4. Generate versioned output branches automatically
5. Push project and output repositories to their remotes
6. Reuse or reference results via their output branches


Results are referenced by unique output branch names that encode the timestamp, active project branch, and project commit hash. CADET-RDM provides a local cache mechanism that allows results from previous runs or from other CADET-RDM projects to be reused as input data while preserving provenance information.


## Getting started

The full documentation is available at
https://cadet-rdm.readthedocs.io

It includes installation instructions, usage guides for the different interfaces, and detailed descriptions of repository and result management workflows.

and allow for easy sharing, integration, and reproduction of generated results.

## Documentation
## Project information

The documentation contains a user guide with helpful information on how to install CADET-RDM, how to quickly start working with it and a more detailed explaination of its tools.
The documentation can be found [here](https://cadet-rdm.readthedocs.io).
- **License:** see [LICENSE](LICENSE)
- **Authors and contributors:** see [AUTHORS](AUTHORS)
2 changes: 1 addition & 1 deletion docs/Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line.
# These variables can be set from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = CADET-RDM
Expand Down
4 changes: 2 additions & 2 deletions docs/source/bibliography.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,6 @@
```

```{bibliography} ./references.bib
:all:
:style: unsrt
```

```
14 changes: 10 additions & 4 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# extensions coming with Sphinx (named 'sphinx.ext.*') or custom
# ones.

# Extensions
Expand All @@ -43,7 +43,7 @@
'.rst': 'restructuredtext',
'.ipynb': 'myst-nb',
'.myst': 'myst-nb',
'.md': 'myst-nb',
'.md': 'myst-nb'
}

## Numpydoc
Expand Down Expand Up @@ -74,12 +74,15 @@
## Viewcode
extensions.append("sphinx.ext.viewcode")

## View figures
extensions.append("sphinx_subfigure")

## Copy Button
extensions.append("sphinx_copybutton")

## BibTeX
extensions.append("sphinxcontrib.bibtex")
bibtex_bibfiles = ['references.bib']
bibtex_bibfiles = ["references.bib"]

# -- Internationalization ------------------------------------------------
# specifying the natural language populates some key tags
Expand All @@ -94,6 +97,9 @@
sitemap_locales = [None]
sitemap_url_scheme = "{link}"

### Figure
extensions.append("sphinx_subfigure")

# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]

Expand All @@ -107,7 +113,7 @@
myst_enable_extensions = [
"dollarmath",
"amsmath",
"colon_fence",
"colon_fence"
]

# -- Options for HTML output -------------------------------------------------
Expand Down
5 changes: 3 additions & 2 deletions docs/source/index.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
```{include} ../../README.md
```{include} ./user_guide/introduction.md
```

```{toctree}
:maxdepth: 2
:caption: User guide
:hidden:

user_guide/introduction
user_guide/installation
user_guide/getting-started
user_guide/CLI-interface
user_guide/command-line-interface
user_guide/python-interface
user_guide/jupyter-interface
```
Expand Down
115 changes: 0 additions & 115 deletions docs/source/user_guide/CLI-interface.md

This file was deleted.

Loading