From bfc57cfaa02f584d3a937707297f193ae44f3507 Mon Sep 17 00:00:00 2001
From: vickeykumar <kumarvickey45@gmail.com>
Date: Sat, 14 Nov 2026 17:17:16 +0530
Subject: [PATCH] Log every snapshot that is made, with the old day archives
 and versions the bucket cleanup removed

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---
 docs/lld/15-object-storage.md | 2 +-
 src/homesnap/snapper.go       | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/docs/lld/15-object-storage.md b/docs/lld/15-object-storage.md
index 1af03659..5a5d3944 100644
--- a/docs/lld/15-object-storage.md
+++ b/docs/lld/15-object-storage.md
@@ -36,7 +36,7 @@ A client with the standard library only (like `persist/firestore.go`): AWS Signa
 - **Who:** the node that is the authority for a home: a single server, or the gateway (which holds a copy of every home, LLD 12). Never a worker (`homesnap.Default.Enable()` is not called in worker mode; workers have no keys). Only signed-in users' homes: `guest-*` are never kept.
 - **Registered by use:** `touchHome` (`server/identity.go`) and the gateway's `PrepareHome` call `Touch(home)` when a signed-in user's request resolves its home. The first time a home is touched in a process: if its folder is missing or empty it is restored (below); from then on it is followed. A home that nobody touches is not asked about again.
 - **Snapshot (`SnapshotHome`):** a job every 5 minutes snapshots the followed homes whose fingerprint changed and in which nothing was written for 30 seconds; at a clean stop every changed home, without waiting. A snapshot holds plain files and folders (empty ones too), never a link, a socket or anything named `.wsync-*`; a file over 50 MB, or one that changes while it is read, is left out and noted in the manifest; a home over 100 MB is not snapshotted (an error in the job's result).
-- **Storage (changed 2026-11-13):** one object for each UTC day, `homes/<name>/<home>/<home>-<YYYY-MM-DD>.tar.gz` (`daily.go`), overwritten by every snapshot of that day (metadata: `sha256`, `generation`, `created`), and `latest.json` (generation, key, SHA-256, writer). The newest 3 days are kept (`Keep`). Where the bucket keeps versions (Backblaze B2 does, R2 does not), the older versions of a day's file stay in the bucket until its lifecycle rule removes them, and the profile page lists them (`objstore.ListVersions`, `GetVersion`). Objects of the earlier scheme (`<generation>-<time>-<writer>.tar.gz`) are still read and are removed when older than 3 days and a day's file exists. **Versions (2026-11-14, `homesnap/versions.go`):** a versioned bucket (B2) keeps every version an overwrite replaces, which with a snapshot every few minutes is dozens of copies of one day's archive, all billed. After each snapshot of a home, `pruneVersions` lists the versions of the home's files (the days' archives and `latest.json`) and removes for good (`objstore.Client.DeleteVersion`) all but the newest version of each file and, of the older ones, the last version of each clock hour (UTC) of the last 6 hours (`VersionHours`); delete markers are never removed. The thinning is not only of today: it takes in the versions that piled up before it, at the next snapshot of the home. A store that keeps no versions (R2), a failing listing, or an application key without the right to delete files (B2: "Read and Write" has it) leaves everything as it is, and the log says so. The profile page's list of earlier versions of a day therefore shows at most the newest and one a clock hour for six hours.
+- **Storage (changed 2026-11-13):** one object for each UTC day, `homes/<name>/<home>/<home>-<YYYY-MM-DD>.tar.gz` (`daily.go`), overwritten by every snapshot of that day (metadata: `sha256`, `generation`, `created`), and `latest.json` (generation, key, SHA-256, writer). The newest 3 days are kept (`Keep`). Where the bucket keeps versions (Backblaze B2 does, R2 does not), the older versions of a day's file stay in the bucket until its lifecycle rule removes them, and the profile page lists them (`objstore.ListVersions`, `GetVersion`). Objects of the earlier scheme (`<generation>-<time>-<writer>.tar.gz`) are still read and are removed when older than 3 days and a day's file exists. **Versions (2026-11-14, `homesnap/versions.go`):** a versioned bucket (B2) keeps every version an overwrite replaces, which with a snapshot every few minutes is dozens of copies of one day's archive, all billed. After each snapshot of a home, `pruneVersions` lists the versions of the home's files (the days' archives and `latest.json`) and removes for good (`objstore.Client.DeleteVersion`) all but the newest version of each file and, of the older ones, the last version of each clock hour (UTC) of the last 6 hours (`VersionHours`); delete markers are never removed. The thinning is not only of today: it takes in the versions that piled up before it, at the next snapshot of the home. A store that keeps no versions (R2), a failing listing, or an application key without the right to delete files (B2: "Read and Write" has it) leaves everything as it is, and the log says so. Every snapshot that is made logs one line, `homesnap: <home> snapshot made (generation N, X MB, F files); D old day archives and V old versions removed` (search `homesnap:`; a snapshot or a cleanup that works writes no other line, a failure has its own). The profile page's list of earlier versions of a day therefore shows at most the newest and one a clock hour for six hours.
 - **Two writers** (an overlapping deploy): the writer first looks (HEAD) at the day's file and gives up (`ErrConflict`) when it holds a generation not older than its own; it reads latest before and again after its upload and gives up too if the generation moved, so the pointer is only ever moved by a writer that was the newest. Without conditional writes this is a check, not a lock: a race inside the few milliseconds between the checks and the write can still happen, and then costs one version of the day, never a damaged file (an object is whole or not there).
 - **Restore (`Ensure`):** `latest.json`, the archive, its hash, `Extract` (a path that leaves the folder, a link, a file the manifest does not list, a wrong size or hash: refused and nothing is written), into `.restore-<home>-*` and a rename onto the home. A damaged latest falls back to the newest good one (three tried). The sync engine is untouched: after a restore the gateway attaches the home to the worker as always and its three-way compare keeps the worker's newer files. The sync record is not part of a snapshot, so the first round after a restore treats every path as new and decides by modification time (LLD 12, "Three-way comparison").
 - **A snapshot that does not match its manifest (2026-11-13, found in a log after a deploy: `.bash_history does not match its manifest`, four tries, no restore):** `newestValid` reads each candidate as a restore would (`verify`) and passes over one that fails for the next older. When every readable candidate fails, the newest of them is restored as it is (`extractForced`: the manifest is not held against the files; the path, link and size checks stay) and the log says so, rather than leave the home empty.
diff --git a/src/homesnap/snapper.go b/src/homesnap/snapper.go
index d280f439..5ecd5063 100644
--- a/src/homesnap/snapper.go
+++ b/src/homesnap/snapper.go
@@ -385,6 +385,9 @@ func (s *Snapper) snapshot(ctx context.Context, home string, force, settle bool)
 	res := Result{Made: true, Generation: gen, Size: int64(len(archive)), Files: len(m.Entries), Skipped: len(m.Skipped)}
 	res.Deleted = s.prune(ctx, c, home, key)
 	res.VersionsRemoved = s.pruneVersions(ctx, c, home)
+	// said in the log so that what the bucket did can be searched for ("homesnap: ... snapshot"); a failure has its own lines
+	log.Printf("homesnap: %s snapshot made (generation %d, %.1f MB, %d files); %d old day archives and %d old versions removed",
+		home, res.Generation, float64(res.Size)/(1024*1024), res.Files, res.Deleted, res.VersionsRemoved)
 	return res, nil
 }