Skip to content

New scheme for managing writeback (and completion) - #684

Open
rtjohnso wants to merge 53 commits into
mainfrom
robj-writeback
Open

New scheme for managing writeback (and completion)#684
rtjohnso wants to merge 53 commits into
mainfrom
robj-writeback

Conversation

@rtjohnso

@rtjohnso rtjohnso commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

@rtjohnso
cache layer of new writeback scheme
94888d3 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@netlify

netlify Bot commented Jul 31, 2026 •
edited
Loading

Copy link
Copy Markdown

❌ Deploy Preview for splinterdb failed.

Name Link
🔨 Latest commit 3bb4cbe
🔍 Latest deploy log https://app.netlify.com/projects/splinterdb/deploys/6a83ae09ea1e230008433fdf

@rtjohnso
Fix writeback races
bc9c1c1 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
rtjohnso
rtjohnso commented Jul 31, 2026
View reviewed changes

@rtjohnso rtjohnso left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

rtjohnso added 2 commits July 31, 2026 13:12
@rtjohnso
fix some error handling in writeback
d997a56 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
definitely die on io error in writeback
1d4dd95 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
rtjohnso
rtjohnso commented Jul 31, 2026
View reviewed changes

@rtjohnso rtjohnso left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

rtjohnso added 2 commits July 31, 2026 13:53
@rtjohnso
start to handle writeback errors
2e6fffa 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
0c54d7c 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
rtjohnso
rtjohnso commented Jul 31, 2026
View reviewed changes

@rtjohnso rtjohnso left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

rtjohnso added 6 commits July 31, 2026 14:18
@rtjohnso
more writeback error handling
1ddcac3 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
19face6 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
still working on writeback
6807f75 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
add async_io_state pool
a8b7a0b 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
00e15e7 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
03bafcf 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
rtjohnso
rtjohnso commented Aug 2, 2026
View reviewed changes

@rtjohnso rtjohnso left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

rtjohnso added 2 commits August 1, 2026 19:52
@rtjohnso
add buffering to the log
7552e2e 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
1d9500b 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
rtjohnso
rtjohnso commented Aug 2, 2026
View reviewed changes

@rtjohnso rtjohnso left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

rtjohnso added 5 commits August 1, 2026 20:13
@rtjohnso
log groups
b4904dd 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
57c97e0 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
fix annoying clang options warning/error on github
89533e7 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
log cleanups
5e326b9 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
log bugfixes and playback end detection
75d8a77 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
rtjohnso
rtjohnso commented Aug 2, 2026
View reviewed changes

@rtjohnso rtjohnso left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to refactor log replay eventually, but not needed yet.

rtjohnso added 4 commits August 2, 2026 01:37
@rtjohnso
handle log sealing errors
d7db242 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
822477d 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
clean up unmount
46ce935 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
fc77ea9 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
rtjohnso added 30 commits August 2, 2026 23:35
@rtjohnso
recovery space allocation work
a978e0c 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
complete recovery
6769a7f 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
e54edb9 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
minor cleanups
20b9f69 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
cleanup blob and log recovery
a417120 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
cleanup rc_allocator and replay bug in core
542e44f 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
make unmount return error only when there might be data loss
fbd2591 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
switch to io_range_is_readable
a3aa492 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
remove live_head
64b163b 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
tons of bugfixes
8768092 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
continuing checkpoint refactoring
c18f782 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
blob cleanup
ea86fd1 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
core durable
1e061ff 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
group page combining
68a2ed5 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
cleaning up log.h
fa29090 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
make log more lock-free
0211da7 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
shard log cleanup
e2e8d69 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
shard log cleanup
9cf5480 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
shard log cleanup
86aadd6 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
shard log cleanup
10787a2 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
shard log cleanups
cb785b2 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
shard log simplification
f846351 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
shard log simplification
0e13b35 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
get rid of durability status checks
ddc7c1d 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
minor cleanups
cff043c 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
add log grace period to avoid forced rotations
89f2bbc 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
core cleanup
780c12e 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
dfa03cf 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
formatting
922bf4f 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
@rtjohnso
add missing files
3bb4cbe 
Signed-off-by: Rob Johnson <rob@robjohnson.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@rtjohnso