Bug 16454

Summary: sstate: refreshing mtime on cache use hides object age, consider atime-only refresh
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Michael Haener <michael.haener>
Component: coreAssignee: Unassigned <unassigned>
Status: NEW --- QA Contact:
Severity: enhancement    
Priority: Undecided CC: ccasciato, meta.mr.watcher, meta.watcher
Version: unspecified   
Target Milestone: 0.0.0   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Michael Haener 2026-10-15 20:54:59 UTC
Since cc07895f95 ("sstate: Update mtime/atime for sig/siginfo files in
sstate_checkhashes()") sstate consistently refreshes both atime and mtime
whenever an object in the sstate cache is used. As a result, the mtime no
longer tells when an object was created, and atime == mtime for every used
object.

This was discussed on the list for the patch
"[v3] sstate: fix access to a cached object rewriting its modification time":
https://patchwork.yoctoproject.org/project/oe-core/patch/20260911070121.39436-1-michael.haener@siemens.com/
Richard asked for an analysis of the use cases a change could break and for
buy-in from other users before changing this. This bug is meant to collect
that.

Why atime-only would help:

- Shared caches can tell objects that are old but still in use from objects
  that were written once and never used again (atime > mtime). On our
  shared CI cache this showed that a few large recipes cause most of the
  never-reused data, which is not visible today.
- scripts/sstate-cache-management.py --remove-duplicated keeps the entry
  with the newest st_mtime. Today, building an older configuration last
  keeps its object and removes the newer one.

Analysis of the concerns raised:

1. Limited write access (shared caches with multiple users):
   Setting only the atime (touch -a, os.utime(ns=...), utimensat with
   UTIME_OMIT) requires file ownership. Setting both to "now" (touch,
   os.utime(path, None)) only requires write access. Verified on Linux with
   a world-writable file owned by root:
     touch /dev/null            -> ok
     touch -a /dev/null         -> Operation not permitted
   So an atime-only refresh must fall back to the current full touch on
   EPERM, otherwise objects in group-writable shared caches would never get
   a refreshed atime and an atime based cleanup would delete them.

2. noatime / relatime mounts:
   These only suppress implicit atime updates on read. An explicit
   utimensat() still sets the atime, so sstate's explicit refresh keeps
   working, as it does today.

3. NFS (ext4/zfs backed):
   The refresh is an explicit SETATTR, the same as today. It does not rely
   on the server updating atime on read.

4. Read-only caches:
   EROFS is ignored, same as today.

5. Users relying on mtime as "last used":
   - The autobuilder prunes its shared cache on the storage server with
     yocto-autobuilder-helper scripts/sstate-clean.sh:
       find .../pub/sstate \( -mtime +15 -a -atime +15 \) -type f \
         \( -name '*.tar*' -o -name '*.tgz*' \)
     An object is only removed when both timestamps are older than 15 days,
     so today either one keeps a used object alive. With an atime-only
     refresh the mtime stays at creation time, and after 15 days the atime
     alone decides. This is safe as long as the explicit atime refresh from
     the workers reliably reaches the server (NFS SETATTR on ZFS). If it is
     silently dropped (EPERM for files owned by another user, dataset or
     export settings), used objects would be pruned after 15 days where
     today the mtime would have saved them. This needs to be checked on the
     autobuilder setup before changing the default.
   - Other cleanup scripts using "find -mtime" alone would have to switch to
     "-atime". This needs a migration note.
   - meta/lib/oe/sstatesig.py find_siginfo() orders siginfo files by
     st_mtime (bitbake-diffsigs, bitbake -S printdiff). With atime-only the
     order would be by creation instead of last use.

Proposal:
Refresh only the atime and fall back to the current behaviour (atime and
mtime) when that is not permitted. Optionally make this configurable
(e.g. SSTATE_TOUCH_MTIME = "1" by default) so it can be enabled per
installation first.

Questions:
- Which other tools or CI setups rely on the mtime of sstate objects?
- Do the autobuilder workers access the sstate cache as the owner of the
  files, and does the ZFS/NFS setup keep an explicitly set atime?