Skip to content

Commit f15744e

Browse files
mstrathmanclaude
mstrathman
and
claude
committed
docs: OSS publishing setup — README, governance, licensing, CI
Prepare the repo for the PureStorage-OpenConnect org: - License: correct copyright to Pure Storage, Inc.; SPDX headers on all first-party sources; NOTICE for the Apache half; declare MIT OR Apache-2.0 in the README. - Docs: real README (operations, receipts/replay, models, build), ARCHITECTURE, SECURITY (trust model + private reporting), CONTRIBUTING (DCO, checks), CHANGELOG. Code of Conduct adopts the org-wide OpenConnect CoC, matching ts-pure-client. All prose humanized (no em dashes). - CI (GitHub Actions): build+test on Linux and macOS, ASan/UBSan, native valgrind, and a 60s libFuzzer smoke run; a tag-driven release workflow that publishes prebuilt binaries. PR/issue templates and .editorconfig. - Makefile: add a standalone 'soak' target for CI valgrind. 96 tests green; ASan/UBSan/valgrind clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: mstrathman <matthew.strathman@gmail.com>
1 parent eca1990 commit f15744e

22 files changed

Lines changed: 526 additions & 12 deletions

‎.editorconfig‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
root = true
2+
3+
[*]
4+
charset = utf-8
5+
end_of_line = lf
6+
insert_final_newline = true
7+
trim_trailing_whitespace = true
8+
9+
[*.{c,h}]
10+
indent_style = space
11+
indent_size = 2
12+
13+
[*.py]
14+
indent_style = space
15+
indent_size = 4
16+
17+
[Makefile]
18+
indent_style = tab
19+
20+
[*.md]
21+
trim_trailing_whitespace = false

‎.github/ISSUE_TEMPLATE/config.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
blank_issues_enabled: true
2+
contact_links:
3+
- name: Security vulnerability
4+
url: https://github.com/PureStorage-OpenConnect/sqlite-predict/security/advisories/new
5+
about: Please report security issues privately, not as a public issue. See SECURITY.md.

‎.github/pull_request_template.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
<!-- Thanks for contributing. Please keep this checklist honest. -->
2+
3+
## What and why
4+
5+
<!-- What does this change do, and what problem does it solve? -->
6+
7+
## Checklist
8+
9+
- [ ] Tests added or updated (adversarial cases where they apply)
10+
- [ ] `make test` passes
11+
- [ ] `make test-asan` is clean
12+
- [ ] Commits are signed off (`git commit -s`, per the DCO)
13+
- [ ] If the result shape of an operation changed, `predict_replay`
14+
expectations were updated deliberately

‎.github/workflows/ci.yml‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
workflow_dispatch:
8+
9+
permissions:
10+
contents: read
11+
12+
concurrency:
13+
group: ci-${{ github.ref }}
14+
cancel-in-progress: true
15+
16+
jobs:
17+
test:
18+
name: build + test (${{ matrix.os }})
19+
runs-on: ${{ matrix.os }}
20+
strategy:
21+
fail-fast: false
22+
matrix:
23+
os: [ubuntu-latest, macos-latest]
24+
steps:
25+
- uses: actions/checkout@v4
26+
- name: Install uv
27+
uses: astral-sh/setup-uv@v5
28+
- name: Build loadable extension
29+
run: make loadable
30+
- name: Run test suite
31+
run: make test
32+
33+
sanitizers:
34+
name: ASan + UBSan
35+
runs-on: ubuntu-latest
36+
steps:
37+
- uses: actions/checkout@v4
38+
- name: Run sanitizer soak
39+
run: make test-asan CC=clang
40+
41+
valgrind:
42+
name: valgrind
43+
runs-on: ubuntu-latest
44+
steps:
45+
- uses: actions/checkout@v4
46+
- name: Install valgrind
47+
run: sudo apt-get update -qq && sudo apt-get install -y -qq valgrind
48+
- name: Build soak
49+
run: make soak CC=gcc
50+
- name: Run under valgrind
51+
run: |
52+
valgrind --leak-check=full --error-exitcode=9 \
53+
--errors-for-leak-kinds=definite ./dist/soak
54+
55+
fuzz-smoke:
56+
name: fuzz smoke (60s)
57+
runs-on: ubuntu-latest
58+
steps:
59+
- uses: actions/checkout@v4
60+
- name: Build + run libFuzzer
61+
run: |
62+
make vendor/sqlite3ext.h sqlite-predict.h
63+
clang -std=c99 -g -O1 -fsanitize=fuzzer,address \
64+
-DSQLITE_CORE -DSQLITE_PREDICT_STATIC -Ivendor/ -I./ \
65+
fuzz/fuzz_predict.c sqlite-predict.c predict-forecast.c \
66+
predict-tabular.c predict-receipts.c vendor/sha256.c \
67+
vendor/sqlite3.c -o fuzz_predict -lm -lpthread -ldl
68+
./fuzz_predict -max_total_time=60 -max_len=512 fuzz/seeds

‎.github/workflows/release.yml‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
name: Release
2+
3+
on:
4+
push:
5+
tags: ["v*"]
6+
7+
permissions:
8+
contents: write
9+
10+
jobs:
11+
build:
12+
name: build (${{ matrix.os }})
13+
runs-on: ${{ matrix.os }}
14+
strategy:
15+
fail-fast: false
16+
matrix:
17+
include:
18+
- os: ubuntu-latest
19+
ext: so
20+
- os: macos-latest
21+
ext: dylib
22+
steps:
23+
- uses: actions/checkout@v4
24+
- name: Build loadable extension
25+
run: make loadable
26+
- name: Stage artifact
27+
run: |
28+
mkdir -p out
29+
cp dist/predict0.${{ matrix.ext }} \
30+
out/predict0-${{ runner.os }}-${{ runner.arch }}.${{ matrix.ext }}
31+
- uses: actions/upload-artifact@v4
32+
with:
33+
name: predict0-${{ runner.os }}-${{ runner.arch }}
34+
path: out/*
35+
36+
release:
37+
needs: build
38+
runs-on: ubuntu-latest
39+
steps:
40+
- uses: actions/download-artifact@v4
41+
with:
42+
path: artifacts
43+
merge-multiple: true
44+
- name: Publish release
45+
uses: softprops/action-gh-release@v2
46+
with:
47+
files: artifacts/*
48+
generate_release_notes: true

‎ARCHITECTURE.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Architecture
2+
3+
A short map of how `sqlite-predict` is put together, for contributors.
4+
5+
## Shape
6+
7+
The extension is pure C99, built as a loadable SQLite/libSQL extension
8+
(`predict0.{dylib,so,dll}`) with a single entry point,
9+
`sqlite3_predict_init`. It has no third-party runtime dependencies; the
10+
SQLite amalgamation headers are fetched at build time and never
11+
redistributed.
12+
13+
Source layout:
14+
15+
| File | Responsibility |
16+
| --- | --- |
17+
| `sqlite-predict.c` | entry point, function registration, shared helpers (timestamp parse/format, ULID, options parsing, normal-quantile) |
18+
| `predict-forecast.c` | `forecast()` and `detect_anomalies()` vtabs, the statistical models, and the shared `collect_series()` helper |
19+
| `predict-tabular.c` | `predict()` vtab and the in-context k-NN model |
20+
| `predict-receipts.c` | model registry, receipts, canonical hashing, the logical-digest anchor, and `predict_replay()` |
21+
| `predict-internal.h` | shared types, contract constants, error codes, internal prototypes |
22+
| `vendor/sha256.c` | a self-contained FIPS 180-4 SHA-256 |
23+
24+
## Operations are table-valued functions
25+
26+
Each operation is an [eponymous virtual table module][tvf]. Arguments
27+
arrive as hidden columns (`query`, `horizon`, `options`), resolved in
28+
`xBestIndex` and consumed in `xFilter`, which does all the work and
29+
materializes result rows the cursor then walks. This is the same mechanism
30+
SQLite's own `generate_series` uses.
31+
32+
`forecast()` and `detect_anomalies()` share `collect_series()`: prepare and
33+
validate the inner query (read-only, single statement), resolve or infer
34+
the time/value/group columns, and collect rows into per-series buffers.
35+
36+
## Models
37+
38+
Models are looked up in a registry table (`_predict_models`) by id, with a
39+
content hash and a license tag. The bundled models are pure-C statistical
40+
methods. Foundation models are out-of-process teachers reached via
41+
distillation (roadmap), not per-query serving paths. The benchmarks in
42+
`benchmarks/` drove that decision.
43+
44+
## Receipts, anchoring, and replay
45+
46+
Every operation writes a row to `_predict_receipts` (unless
47+
`'{"receipt":0}'`): the model id and hash, an anchor for the data state
48+
read, the canonical call parameters, the inner SQL, and a hash of the
49+
result set.
50+
51+
- **Canonical result hash** (`predict-receipts.c`): rows are hashed in a
52+
defined order with type-tagged fields, separated by `0x1F` between fields
53+
and `0x1E` between rows. Integers hash as decimal text, reals as their
54+
big-endian IEEE-754 bit pattern, and text as UTF-8. This keeps the hash
55+
stable across runs on the same machine.
56+
- **Logical-digest anchor**: a hash of the user tables' schema and rows
57+
(excluding `_predict_%` and `sqlite_%`). A page-level file digest can
58+
never replay-match, because writing the receipt itself changes the file;
59+
the logical digest is indifferent to receipt writes and to `VACUUM`.
60+
- **Replay** re-executes the recorded call read-only against the anchored
61+
state and compares result hashes. It refuses to run if the current state
62+
no longer matches the anchor.
63+
64+
## Determinism
65+
66+
Statistical inference is deterministic on a given machine, which is what
67+
makes replay meaningful. Cross-machine and cross-backend determinism is
68+
not guaranteed (see the notes on GPU backends in `benchmarks/notes.md`);
69+
replay verification is same-machine.
70+
71+
## Deviations from the spec
72+
73+
The implementation surfaced amendments queued for the design spec: the
74+
`ts-stat`/`tabular-stat` model kinds, the `logical-digest` anchor kind, a
75+
`{"train","apply"}` JSON `input_sql` for two-query operations, and null
76+
option values meaning "key omitted." These are noted at the top of the
77+
files that introduce them.
78+
79+
[tvf]: https://www.sqlite.org/vtab.html#tabfunc2

‎CHANGELOG.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Changelog
2+
3+
All notable changes to this project are documented here. The format is
4+
based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this
5+
project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6+
7+
## [Unreleased]
8+
9+
### Added
10+
11+
- `forecast()`, `detect_anomalies()`, and `predict()` table-valued
12+
functions with a trailing JSON options argument.
13+
- Replayable receipts on every prediction, and `predict_replay()` to verify
14+
a recorded call reproduces against its anchored data state.
15+
- Bundled zero-dependency models: `theta-classic`, `stub-seasonal-naive`
16+
(time series) and `knn5-incontext` (tabular).
17+
- `predict_ulid()` and `predict_version()` utility functions.
18+
- Test suite, AddressSanitizer/UBSan and valgrind soak targets, and a
19+
libFuzzer harness.
20+
21+
_This project is pre-alpha; everything above is subject to change before a
22+
tagged release._

‎CODE_OF_CONDUCT.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Code of Conduct
2+
3+
This project adopts the Pure Storage OpenConnect Code of Conduct.
4+
5+
Please read it here: https://github.com/PureStorage-OpenConnect/Code-of-Conduct
6+
7+
By participating in this project you agree to abide by its terms.

‎CONTRIBUTING.md‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Contributing to sqlite-predict
2+
3+
Thanks for your interest. This project is pre-alpha; the API and formats
4+
are unstable, so it's a good time to influence them.
5+
6+
## Ground rules
7+
8+
- **Correctness is verified, not asserted.** Every change must keep the
9+
test suite green and the sanitizers clean. New behavior needs tests,
10+
including adversarial ones that try to break it. The suite has caught
11+
real memory bugs this way.
12+
- **Fail loudly.** No silent fallbacks. If an input is invalid, raise a
13+
`PREDICT_ERR_*` error; don't guess or degrade silently.
14+
- **The receipt is a contract.** If you change what an operation returns,
15+
the result-hash layout changes and `predict_replay` round-trips will
16+
fail. That is the intended tripwire; update it deliberately.
17+
18+
## Developer Certificate of Origin (DCO)
19+
20+
Contributions are accepted under the [DCO](https://developercertificate.org/).
21+
Sign off each commit to certify you wrote it or have the right to submit it:
22+
23+
```sh
24+
git commit -s -m "your message"
25+
```
26+
27+
This appends a `Signed-off-by: Your Name <you@example.com>` trailer.
28+
29+
## Workflow
30+
31+
1. Fork and branch from `main`.
32+
2. Make your change with tests.
33+
3. Run the checks locally:
34+
```sh
35+
make test # pytest suite
36+
make test-asan # AddressSanitizer + UBSan
37+
make test-valgrind # valgrind (needs Docker)
38+
```
39+
4. Open a pull request. CI runs the same checks on Linux and macOS.
40+
41+
## Code style
42+
43+
- C99, no compiler warnings (`-Wall -Wextra`), formatted with
44+
`clang-format` (`make format`).
45+
- Follow the existing conventions: `predict_*` for SQL-visible functions,
46+
`predict0_*` for internals; contract constants live in
47+
`predict-internal.h`.
48+
- Comments explain *why*, not *what*.
49+
50+
By contributing, you agree that your contributions are dual-licensed under
51+
MIT OR Apache-2.0, matching the project license.

‎LICENSE-MIT‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
MIT License
22

3-
Copyright (c) 2026 Matthew Strathman
3+
Copyright (c) 2026 Pure Storage, Inc.
44

55
Permission is hereby granted, free of charge, to any person obtaining a copy
66
of this software and associated documentation files (the "Software"), to deal

0 commit comments

Comments
 (0)