Sync

kache sync transfers artifacts between the local store and S3. It runs directly against S3 without requiring the daemon, which makes it useful for CI steps that want explicit control over cache population timing.

Basic usage

kache sync              # pull missing artifacts + push new local artifacts
kache sync --pull       # download only
kache sync --push       # upload only
kache sync --all        # pull all artifacts, ignoring Cargo.lock filtering
kache sync --workspace  # pull scope = workspace members only (one LIST per member)
kache sync --dry-run    # preview what would transfer, make no changes

Workspace filtering

The pull and push directions filter by two different mechanisms:

Pull filters downloads to every crate listed in the current directory's Cargo.lock (all direct and transitive dependencies). This avoids pulling the entire bucket on every sync — in a shared cache used by multiple projects, you only download what your project needs.
Push filters uploads to workspace member packages, derived by running cargo metadata --no-deps on the current Cargo.toml (or the --manifest-path you pass). Non-workspace local artifacts are silently skipped; a push run outside any workspace (no Cargo.toml) uploads everything.

# in your project directory (Cargo.lock must exist)
kache sync --pull

# pull everything in the bucket regardless of Cargo.lock
kache sync --pull --all

If kache can't find a Cargo.lock, it falls back to pulling everything. The same full-bucket fallback applies if Cargo.lock is present but yields no package names.

`--workspace`: scope the pull to workspace members

By default the pull issues one ListObjectsV2 per crate in the Cargo.lock dependency set — often hundreds to thousands of LIST calls. --workspace instead scopes discovery to the workspace members (the same cargo metadata --no-deps set used by the push filter), issuing one LIST per member (typically tens). It conflicts with --all.

This only narrows the up-front batch pull — dependency artifacts are still resolved on demand during the build (the rustc wrapper fetches a local miss from S3 via the daemon, and the daemon prefetches by build intent), so deps are not recompiled. It's most useful when the dependency artifacts are already present locally — e.g. a prebuilt cargo chef deps image whose compiled deps sit in target/ — where they're local hits that never need an S3 round-trip.

# discovery issues one LIST per workspace member instead of one per dependency
kache sync --pull --workspace

Unlike the default pull, --workspace does not fall back to a full-bucket scan: if the workspace set can't be resolved (cargo metadata failed, or you're not in a Cargo workspace), the command errors rather than silently listing the entire bucket.

--manifest-path controls the push-side workspace filter only (which local crates get uploaded). The pull filter always reads Cargo.lock from the current working directory and ignores --manifest-path, so to pull for a different project, run kache sync from that project's directory.

# narrows which local crates get pushed; does NOT change the pull filter
kache sync --manifest-path /path/to/project/Cargo.toml --push

Workspace filtering shells out to cargo metadata. If cargo is missing or metadata fails, kache prints a warning and — when --manifest-path was given — yields an empty filter that pushes nothing; otherwise it disables push filtering and uploads everything.

How it fits into a build workflow

In CI, the typical pattern is:

# before the build: warm the local cache from S3
kache sync --pull

# run the build (kache serves local hits, misses compile normally)
cargo build --release

# after the build: push newly compiled artifacts to S3
kache sync --push

The --pull step fills the local store so that cargo build finds local hits instead of remote hits. This is faster because local hits don't involve a daemon RPC round-trip.

When the daemon is running and remote is configured, it handles uploads automatically in the background after each compilation. Use kache sync --push only in workflows where you want an explicit, synchronous upload step — for example, when the daemon isn't running or you want to ensure the cache is populated before the runner terminates.

kache sync is safe to run alongside the daemon and alongside other concurrent syncs. Before downloading, it re-checks each entry and skips any that already landed on disk; imports use INSERT OR REPLACE; and pushes skip entries that vanished (via GC or purge) mid-run.

--push still performs a full, unfiltered LIST of all S3 keys to determine what is already uploaded — workspace filtering applies to the upload set, not to this listing. On a large shared bucket that LIST is a non-trivial cost.

S3 layout

Each cached entry is stored as two objects under a versioned v3 layout — a small manifest used for listing and existence checks, and the packed artifact itself:

{prefix}/v3/manifests/{crate_name}/{cache_key}.json      # manifest (listing / existence)
{prefix}/v3/packs/{crate_name}/{cache_key}.tar.zst       # packed artifacts

The default prefix is artifacts, so serde packs live under artifacts/v3/packs/serde/.... Organizing by crate name keeps filtered listing efficient: a workspace-filtered pull scans only the per-crate manifest prefix {prefix}/v3/manifests/{crate_name}/.

Dry run

--dry-run is useful for understanding what's in the remote cache without transferring anything:

kache sync --dry-run

It prints the plan (the count of artifacts to pull and push) followed by one line per artifact showing a truncated cache key and the crate name — for example:

Plan: pull 2 artifacts, push 1 artifact
  pull  a1b2c3d4e5f6g7h8... (serde)
  pull  9i8j7k6l5m4n3o2p... (tokio)
  push  q1r2s3t4u5v6w7x8... (my-crate)

Sizes are not shown — listing does not fetch object sizes.

Concurrency

kache sync uses up to s3_concurrency (default 16) concurrent S3 operations. You can lower this if your S3 provider throttles requests:

KACHE_S3_CONCURRENCY=4 kache sync --push