Repository navigation
Implement locking & co-ordination between cache cleans - #3644
peterebden wants to merge 2 commits into
Conversation
| func (cache *dirCache) Store(target *core.BuildTarget, key []byte, files []string) { | ||
| cacheDir := cache.getPath(target, key, "") | ||
| lockFile, err := core.AcquireExclusiveFileLock(cacheDir + ".lock") | ||
| lockFile, err := lockEntry(cacheDir, syscall.LOCK_EX) |
There was a problem hiding this comment.
Am I right that cacheDir sometimes isn't actually a directory but a file? I was a little confused by why you are writing a lock file under it but then delete the entire directory at line 48
There was a problem hiding this comment.
Yeah, there's a mode where it stores compressed tarballs for each entry. It mostly gets used for CI where we're more concerned about the cache's size.
I wonder sometimes if it's worth it though, it does add some complexity.
| } | ||
| cache.clean(policy, freeSpace) | ||
| now := time.Now() | ||
| if err := os.WriteFile(timestampFile, nil, 0644); err != nil { |
There was a problem hiding this comment.
Should we write the timestamp into the file regardless? It's not using the timestamp in the name, it may be unintuitive to find out how to read the timestamp. Everyone knows how to use ls and cat, not everyone knows how to use stat.
There was a problem hiding this comment.
Yeah that's not a bad idea, might as well while we're writing the file anyway.
06315c8 to
5fe0851
Compare
Third and final part of #3629 , stacked on #3642
Implements a lockfile for the cache so if a clean is in progress, another process will see that and will not try to do it as well.
Also implements cleaning of stale lock files across the cache so they don't build up forever.