Skip to content
Aleix Raventós
~2 min read

We built kache to speed up Kunobi's development. Kunobi is a Tauri app with a large Rust dependency tree, and we were spending too much time waiting for recompilation across worktrees and CI runners. This post explains what kache does and how it works.

A common mental model for a Rust build cache is "it remembers what you just compiled, so you don't recompile it next time." This is what cargo's incremental cache does, and it's good at that job: making the second build of a project on the same machine, with the same target/ directory, much faster than the first. It's path-keyed and tied to one target/ because that's the right design for tight inner-loop iteration.

kache is for a different question: has anyone, anywhere compiled this exact code before? When you switch git worktrees, spin up a fresh CI runner, or hand the project to a teammate, you're outside the territory cargo's incremental cache was designed for. kache is a RUSTC_WRAPPER that picks up there, with a cache keyed by the contents of your source, your dependencies, and your toolchain. The project's location on disk doesn't enter into it.

What happens when cargo asks for a build

Setting RUSTC_WRAPPER=kache tells cargo to invoke kache in place of rustc on every compilation. Cargo doesn't know it's been intercepted; it just calls what the env var says to call. kache receives the real rustc path as its first argument and decides what to do from there.

For each call:

A cache hit doesn't copy any bytes. kache restores the cached artifact by linking it into the path under target/ where cargo expects it. On filesystems that support copy-on-write (APFS, btrfs, XFS-with-reflink), it creates a reflink, which is a lightweight clone that shares the underlying disk blocks with the cached copy, but behaves like an independent file (modifications to one don't affect the other). On other filesystems, it falls back to a hardlink (a second directory entry pointing to the same inode), so both paths resolve to the exact same data on disk. The point being: one physical copy with several paths pointing at it. A 200 MB compiled library stays as one file no matter how many worktrees end up using it.

Cargo doesn't notice. It just cares about the file being there, then it moves on.

There's also a build lock that matters for workspaces (projects with multiple crates that cargo builds together). In a workspace, cargo compiles many crates in parallel, and some of those crates share dependencies. Two parallel rustc processes might both need to compile the same crate (say, serde). Without a lock, both would redundantly compile it and then race to write the same cache entry. With a per-key build lock, the first process to grab it does the compile and caches the result; any other process that needs the same crate waits for the lock, wakes up to find the cached artifact already there, and skips the compilation entirely.

kache skips binary crates, dynamic libraries, and proc-macros by default. Their outputs depend on the linker, may need code signing on macOS, and are more expensive to restore correctly. Compile time rarely hides there anyway. The expensive work is in the dependency tree: libraries like serde, tokio, and the procedural macros they pull in. Those are what kache caches. If you want to cache everything, set KACHE_CACHE_EXECUTABLES=1.

While kache is active, cargo's built-in incremental compilation is turned off (CARGO_INCREMENTAL=0). Both strategies solve the same problem (avoiding recompilation of unchanged crates) but running them simultaneously can corrupt build artifacts due to how APFS handles the overlapping file operations. kache's cache works across machines, not just within one target/ directory, so it takes over that responsibility while it's running.

The cache key

Two compilations match when their cache keys do. The key is a blake3 hash of every input that affects what rustc would produce:

  • The full rustc version, including commit hash and host triple.
  • The target triple.
  • Crate name, crate types (lib, rlib, proc-macro), and edition.
  • All -C codegen flags, sorted alphabetically.
  • Feature flags and --cfg flags, sorted.
  • Source files, picked up via a rustc --emit=dep-info pre-pass that also catches module trees, include!() targets, and build script outputs (not just the crate root).
  • Env vars referenced by env!() and option_env!().
  • Hashes of extern rlibs and rmetas (so a changed dependency invalidates everything downstream).
  • RUSTFLAGS, with machine-local path prefixes (your home directory, the cargo registry, the workspace root) replaced with stable placeholders, so the same flags produce the same key on a different machine.

What kache leaves out is as important: absolute paths, machine identity, the incremental codegen flag (which encodes machine-specific paths), and anything else that would vary between two otherwise-equivalent builds on different machines.

The key comes out the same on your laptop, on a teammate's laptop, and on a CI runner, as long as the toolchain, target, source, and flags agree. If it didn't, a remote cache would be useless.

If something keeps missing when you expected a hit, kache why-miss <crate> walks every input in the key and shows you which one changed.

Watching it work

kache monitor opens a live TUI dashboard with four tabs: Build, Projects, Store, and Transfer. The Build tab streams every rustc invocation kache intercepts, with the outcome (local hit, remote hit, skipped, or miss), the elapsed time, and the artifact size. The Store tab lists everything currently cached. The Projects tab shows which target/ directories have hardlinks into the store, which is how you find out that three of your worktrees are pointing at the same physical bytes. The Transfer tab tracks uploads and downloads to the remote backend.

The header bar shows the numbers that actually matter:

Store: 44.4 GiB / 50.0 GiB / 90.0%    11642 entries
Hit rate: 92% count | 83% weighted | 70% miss-time

The raw hit rate counts every crate equally. The weighted one accounts for how much compile time each crate represents. The gap between the two tells you whether kache is mostly catching cheap crates or expensive ones. A wide gap means you're winning on serde_derive-class things and missing on tauri-class ones, which is the moment a remote cache starts to matter.

For a quick non-interactive snapshot, kache stats prints a summary with the same metrics and exits. When something feels off (builds are slower than they should be, or the monitor shows misses you didn't expect), kache doctor is the first thing to run. It checks the binary, the wrapper config, the cargo config, the store, and the daemon, and points at whatever's wrong.

Remote caching

The same key that worked across worktrees also works across machines. kache speaks to any S3-compatible backend (AWS, Cloudflare R2, Ceph, MinIO) with a couple of lines in the config:

[cache.remote]
type = "s3"
bucket = "my-build-cache"

For GitHub Actions there's a shorter path. The official kunobi-ninja/kache-action wires kache up to GitHub's built-in cache without needing an S3 bucket at all, so CI runs share artifacts with each other out of the box. To share the cache between CI and your laptop, point both at the same S3 bucket instead.

A populated remote is what shifts the weighted hit rate. Local caching alone tends to catch the cheap, frequently-built crates; the expensive ones (tauri, kube, tokio) usually need to come from somewhere they were already built.

Wrapping up

kache and cargo's incremental cache solve different problems. Cargo's cache is keyed by the identity of one target/ directory (the right design when you're iterating on a single project on a single machine). kache's cache is keyed by the contents of your source, your dependencies, and your toolchain, because the question is different: has anyone, anywhere compiled this exact code before?

The rest is straightforward plumbing: a wrapper in front of rustc, a content-addressed store on disk, hardlinks to keep restores cheap, a daemon to push artifacts to a shared backend in the background. The cache key is where the real work is. Everything else exists to make that key fast to compute and fast to look up.

If you want to go deeper, here's a 5-minute walkthrough of the internals, covering the cache key and the content-addressed store.

kache is open source and lives at github.com/kunobi-ninja/kache. If your Rust builds feel slower than they should, install it, point RUSTC_WRAPPER at it, and run kache monitor while you work. The first build won't be faster, but the second one across a fresh worktree or CI runner usually is. If it saves you time, a star on the repo helps other people find it. Bug reports and PRs are welcome, and we read every issue. We built kache because we needed it for Kunobi, and the more people who run it against different dependency trees, the better it gets.

$ tail -f /dev/blog

Cluster updates, in your inbox.

Kubernetes deep dives, GitOps field notes, and platform-engineering essays from the team building Kunobi. Two posts a month. No fluff.

$ also availableThe Kunobi desktop app. Every cluster, one window.
Try Kunobi now
Available for:
Apple macOS logomacOSMicrosoft Windows logoWindowsLinux logoLinux
Download Kunobi