Skip to content

Add (categorized) stats#222

Open
cen1 wants to merge 7 commits intokpcyrd:mainfrom
cen1:feature/stats
Open

Add (categorized) stats#222
cen1 wants to merge 7 commits intokpcyrd:mainfrom
cen1:feature/stats

Conversation

@cen1
Copy link
Copy Markdown
Contributor

@cen1 cen1 commented Mar 16, 2026

This essentially brings the possibility of the cool Debian graphs (reproducibility over time) and categorized stats (rebuilderd_stats.py) natively into rebuilderd for all distros.

2 new REST endpoints

  • POST /api/v1/stats : creates a stats datapoint (snapshot), e.g. daily. Collect GOOD, BAD etc + BAD categorized stats
  • GET /api/v1/stats : fetch stats (to draw graphs or whatever you want to do with the data)

A new rebuilderd-stats.conf file

Since each distro can have it's own specific error categorizations, I've split the backends per distro and you can specify different categories for each. There is also one .all backend which applies to all distros.

The categorization logic is essentially "stolen" directly from rebuilderd_stats.py as well as initial categorizations for Debian, thus the direct attribution there.

I've also done some quick analysis of my Arch rebuilder logs and added some categories but there are probably much more. The hope is that rebuilders could contribute these as they encounter them on their instances. I also have some categories from my FreeBSD work but that'd come a bit later.

A new systemd timer
Added a sample timer that triggers stats snapshot once a day via rebuildctl (similar to how -sync is done). Maybe a sccheduler inside daemon code could be done but I think this is good enough.

Simple contrib with html/js sample graphs
This was mostly added just so I could quickly verify the viability of the GET API and it's a nice direct visual representation of the feature but ultimately I can remove this from the PR. CORS was also added just so I could test on localhost, can also be removed, but doesn't hurt to have it (actually, it should probably be expanded in the future with full config options).

For my Debian/sid/amd64 it looks something like this with 3 snapshots:
deb_stats

State of the code

I'd say "it seems to be working" but not sure yet on correctness, since I am still not completely familiar with the db model I need to do a few more passes, especially for daemon/src/api/v1/stats.rs. The rest should be mostly fine though.

@cen1 cen1 changed the title Add (categorized) stats endpoints Add (categorized) stats Mar 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant