feat: add cachelon (name pending) crates with multi-level caching abstractions by schgoo · Pull Request #240 · microsoft/oxidizer

schgoo · 2026-01-29T21:45:08Z

Cachelon = Cache + Echelon

TODO more description

github-actions · 2026-01-29T21:47:12Z

⚠️ Breaking Changes Detected

error: failed to retrieve local crate data from git revision

Caused by:
    0: failed to retrieve manifest file from git revision source
    1: possibly due to errors: [
         failed to parse /home/runner/work/oxidizer/oxidizer/target/semver-checks/git-origin_main/2627e94555b3f837af373a6e9930fa063b1d8e87/Cargo.toml: no `package` table,
         failed when reading /home/runner/work/oxidizer/oxidizer/target/semver-checks/git-origin_main/2627e94555b3f837af373a6e9930fa063b1d8e87/scripts/crate-template/Cargo.toml: TOML parse error at line 9, column 26
         |
       9 | keywords = ["oxidizer", {{CRATE_KEYWORDS}}]
         |                          ^
       missing key for inline table element, expected key
       : TOML parse error at line 9, column 26
         |
       9 | keywords = ["oxidizer", {{CRATE_KEYWORDS}}]
         |                          ^
       missing key for inline table element, expected key
       ,
       ]
    2: package `cachelon` not found in /home/runner/work/oxidizer/oxidizer/target/semver-checks/git-origin_main/2627e94555b3f837af373a6e9930fa063b1d8e87

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
   1: cargo_semver_checks::rustdoc_gen::RustdocFromProjectRoot::get_crate_source
   2: cargo_semver_checks::rustdoc_gen::StatefulRustdocGenerator<cargo_semver_checks::rustdoc_gen::CoupledState>::prepare_generator
   3: cargo_semver_checks::Check::check_release::{{closure}}
   4: cargo_semver_checks::Check::check_release
   5: cargo_semver_checks::exit_on_error
   6: cargo_semver_checks::main
   7: std::sys::backtrace::__rust_begin_short_backtrace
   8: main
   9: <unknown>
  10: __libc_start_main
  11: _start

If the breaking changes are intentional then everything is fine - this message is merely informative.

Remember to apply a version number bump with the correct severity when publishing a version with breaking changes (1.x.x -> 2.x.x or 0.1.x -> 0.2.x).

crates/cachelon/Cargo.toml

crates/cachelon/src/runtime.rs

… stack size

…ature.

…ments.

…mentation cleanup.

crates/cachelon_tier/src/tier.rs

crates/cachelon_tier/src/entry.rs

Vaiz · 2026-02-05T13:09:20Z

crates/cachelon_tier/src/entry.rs

+#[derive(Clone, Debug, PartialEq, Eq)]
+pub struct CacheEntry<V> {
+    value: V,
+    cached_at: Option<SystemTime>,


what if we merge cached_at and ttl into cached_until?

As I said above, cached_at does get set by the cache. But I guess it might be confusing if you can set it here... The reason for being configurable here is in case they want manual control, or they want to be able to test it. In other words, cached_at is never useful without ttl, but ttl is always useful without cached_at. Also, I don't really think exposing cached_at is even useful to consumers - it's really more for internal testing of the expiration functionality. Still, I think what I'll do is have these methods

pub fn new(value: V) -> Self pub fn expires_after(value: V, ttl: Duration) -> Self pub fn expires_at(value: V, ttl: Duration, cached_at: SystemTime) -> Self pub(crate) fn ensure_cached_at(cached_at: SystemTime) // checks for pre-existing cached_at value

Vaiz · 2026-02-05T13:10:27Z

crates/cachelon_tier/src/entry.rs

+#[derive(Clone, Debug, PartialEq, Eq)]
+pub struct CacheEntry<V> {
+    value: V,
+    cached_at: Option<SystemTime>,


SystemTime is not very reliable, how about Instant?

Instant is limited to in-process manipulations, while this abstraction needs to work with timestamps that are externally stored. For example, when redis entry was stored.

in that case we should provide a choice to the user because uses cases might be different. There are plenty of cases when cache is used only for process internal data

It creates API split though. I would argue, that for local data one uses ttl anyway. Btw, would it make more sense to have expires_at field? What capabilities does redis expose for this?

In Redis you set TTL as a duration in seconds. They don't expose a cached timestamp of any kind. I don't really have any opinion on Instant vs SystemTime, other than that I don't like that SystemTime::duration_since() is fallible.

crates/cachelon/examples/refresh.rs

martintmk · 2026-02-05T14:10:03Z

crates/cachelon/src/telemetry/ext.rs

+/// Extension trait for timing async operations.
+pub trait ClockExt {
+    /// Times an async operation and returns both the result and elapsed duration.
+    fn timed_async<F, R>(&self, f: F) -> impl Future<Output = TimedResult<R>>


imho, this should stay internal to the crate. If these APIs are useful enough, we can just add them to tick.

Yeah, it should have been internal, my mistake. I could see this being useful for the new telemetry crate (@Vaiz), but I'll leave them here until someone else has a use case.

BTW this was doubling the sizes of the futures, so I added a small struct that implements Future to do the timing, it's much more efficient now, and might be more useful this way.

crates/testing_aids/src/metrics.rs

crates/cachelon/src/cache.rs

crates/cachelon/src/fallback.rs

crates/cachelon/src/wrapper.rs

martintmk · 2026-02-05T14:19:55Z

crates/cachelon/src/wrapper.rs

+    S: CacheTier<K, V> + Send + Sync,
+{
+    async fn get(&self, key: &K) -> Result<Option<CacheEntry<V>>, Error> {
+        let timed = self.clock.timed_async(self.inner.get(key)).await;


imho, what's wrong with just having let stopwatch = clock.stopwatch(); and tracking the duration like that? Is extra layer around timed_async necessary here?

What's wrong with it is I didn't think of it first. I do like timed_async because otherwise I have to write the stopwatch boilerplate in many places; this feels cleaner from the callsite. I do think timed_async might be better using stopwatch in the implementation, though

crates/cachelon_service/src/request.rs

…lace When variant with WhenBoxed variant. Use stopwatch for timed_async.

…nt fixes.

schgoo added 7 commits January 15, 2026 16:45

Initial commit

fd7e4ef

clippy, fmt, tests, copyrights, readmes, etc.

6ca9232

Merge with main

886d860

More compat with github version

80cb4c1

Merge with main

15d42ff

Use uniflight and anyspawn

d0980a4

Streamline examples and benchmarks. Move stampede protection to builder.

dad0181

martintmk reviewed Jan 30, 2026

View reviewed changes

crates/cachelon/Cargo.toml Outdated Show resolved Hide resolved

martintmk reviewed Jan 30, 2026

View reviewed changes

crates/cachelon/src/runtime.rs Outdated Show resolved Hide resolved

schgoo added 13 commits January 30, 2026 13:15

Clippy fixes

cdc9700

Box fallback get/insert/invalidate futures to put a ceiling on future…

4fbc541

… stack size

Make all API methods fallible - return Result

276ca12

Test to track cache future sizes. Other formatting etc.

b3b64c8

Fix copyrights and licenses. Add allowed external types

5e2db24

Simplify telemetry tests. Remove otel_sdk dependency. Remove tokio fe…

07affe8

…ature.

Fix benches

7dbe8ec

clippy

b650a7d

Improve test quality

cacc05e

Move to SystemTime instead of Instant

30deca4

Spelling. More test cleanup. Formatting. Clean up telemetry cfg state…

cb5ba04

…ments.

Spelling. More test cleanup. Formatting. More telemetry cleanup. Docu…

45a9ca2

…mentation cleanup.

Merge branch 'main' of https://github.com/microsoft/oxidizer into cache

d680f31

Vaiz reviewed Feb 5, 2026

View reviewed changes

crates/cachelon_tier/src/tier.rs Outdated Show resolved Hide resolved

Vaiz reviewed Feb 5, 2026

View reviewed changes

crates/cachelon_tier/src/tier.rs Show resolved Hide resolved

Vaiz reviewed Feb 5, 2026

View reviewed changes