You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi all 👋. While working on the sendwal extraction (#148), I ran into something worth splitting into its own proposal: SegmentFromName from github.com/cloudnative-pg/cloudnative-pg/pkg/postgres is used in three places in Klio: core/pkg/sendwal/receiver.go, core/internal/server/walserver/upload.go, and core/internal/consumer/wal.go, purely to parse a WAL segment file name back into an LSN/timeline. That's the only reason those packages pull in the full operator module as a dependency.
The reverse direction already lives in machinery (types.Int64ToLSN(...).WALFileName(tli, segmentSize), used in wal_handler.go), so there's precedent for this kind of helper being small and dependency-free — it's just that today the two directions (name → LSN and LSN → name) live in two different modules, one of which is much heavier than it needs to be for this.
I'd like to propose a small, self-contained package (no operator dependency, no I/O) covering both directions plus timeline handling, that Klio, the sendwal extraction, the CNPG operator and others could all import instead of each carrying their own copy or a heavy transitive dependency for one function.
This one is independent of the sendwal extraction itself (which is currently paused, see the note below) and looks like the simplest of the pending extraction candidates, so I'd like to start here.
Related: sendwal's extraction (#148) is currently paused, since PR #124 (dev/98) is reworking the exact write path we'd be decoupling, with an ack-based/durability redesign. Once that settles we'll pick sendwal back up; in the meantime I'd like to move on this and the other two extraction candidates (compression, encryption, opened as separate discussions) that don't have any PR in flight to conflict with.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi all 👋. While working on the
sendwalextraction (#148), I ran into something worth splitting into its own proposal:SegmentFromNamefromgithub.com/cloudnative-pg/cloudnative-pg/pkg/postgresis used in three places in Klio:core/pkg/sendwal/receiver.go,core/internal/server/walserver/upload.go, andcore/internal/consumer/wal.go, purely to parse a WAL segment file name back into an LSN/timeline. That's the only reason those packages pull in the full operator module as a dependency.The reverse direction already lives in
machinery(types.Int64ToLSN(...).WALFileName(tli, segmentSize), used inwal_handler.go), so there's precedent for this kind of helper being small and dependency-free — it's just that today the two directions (name → LSN and LSN → name) live in two different modules, one of which is much heavier than it needs to be for this.I'd like to propose a small, self-contained package (no operator dependency, no I/O) covering both directions plus timeline handling, that Klio, the
sendwalextraction, the CNPG operator and others could all import instead of each carrying their own copy or a heavy transitive dependency for one function.This one is independent of the
sendwalextraction itself (which is currently paused, see the note below) and looks like the simplest of the pending extraction candidates, so I'd like to start here.Related:
sendwal's extraction (#148) is currently paused, since PR #124 (dev/98) is reworking the exact write path we'd be decoupling, with an ack-based/durability redesign. Once that settles we'll picksendwalback up; in the meantime I'd like to move on this and the other two extraction candidates (compression, encryption, opened as separate discussions) that don't have any PR in flight to conflict with.All reactions