fix(backend): close the ReadCloser returned by PullBlob in push - #509
Conversation
There was a problem hiding this comment.
Code Review
This pull request addresses a resource leak in pkg/backend/push.go by ensuring that blob content is properly closed using a defer statement. Feedback was provided to simplify the accompanying comment for better readability and to move historical context to the commit message.
|
Simplified the close comment per review feedback. |
65d271c to
d81ffec
Compare
pushIfNotExist pulled each blob's content from the source storage and then handed it to Blobs().Push wrapped in io.NopCloser. The NopCloser is there on purpose (Close on the distribution reader returns an error, see modelpack#50), but it also means the original ReadCloser from PullBlob was never closed on any path, leaking a file descriptor or HTTP body per blob. Add a `defer content.Close()` immediately after the nil-error check so the original reader is released on both success and error paths. The existing NopCloser wrapper still prevents Push from calling Close itself, so the workaround for modelpack#50 is preserved. Fixes modelpack#491 Signed-off-by: SAY-5 <SAY-5@users.noreply.github.com> Signed-off-by: Sai Asish Y <say.apm35@gmail.com>
Signed-off-by: say <say.apm35@gmail.com> Signed-off-by: Sai Asish Y <say.apm35@gmail.com>
d81ffec to
e8ce87c
Compare
aftersnow
left a comment
There was a problem hiding this comment.
Verified against current main: pushIfNotExist never closes the ReadCloser from PullBlob, so every non-manifest blob leaks a file handle on push.
Checked that closing is actually the right call here. fileReader.Close() in distribution v3.1.0 goes through closeWithErr, which does fr.rc.Close() to release the reader chain and only then returns the fileReader: closed sentinel. So the handle is genuinely released, and ignoring the returned error is correct rather than sloppy. The guard on fr.err also makes a second Close a no-op, so there is no double-close risk alongside the existing io.NopCloser wrapper, which stays necessary to keep that sentinel out of Blobs().Push.
Test-merged onto main (9f887eb), built and ran ./pkg/backend/... ./pkg/storage/.... Failure set is identical to the baseline on clean main (all from a local sandbox blocking ~/.docker/config.json), so no regressions.
Problem
pushIfNotExistpulls each blob's content from the source storage andthen hands it to
Blobs().Pushwrapped inio.NopCloser:The
io.NopCloseris intentional:Closeon the distribution readerreturns an error (see #50), so we don't let
Pushpropagate that error.But the wrapper also prevents the original
contentfrom ever beingclosed. On both success and error paths, the underlying file descriptor
(or HTTP body) leaks for every blob we push, and
Pushis called onceper object in the image. #491.
Fix
Add
defer content.Close()immediately after the nil-error check. Theexisting
NopCloserstill preventsPushfrom callingCloseon itsown, so the workaround for #50 is preserved.
deferhandles both thesuccess path and every early-return error path.
content, err := src.PullBlob(ctx, repo, desc.Digest.String()) if err != nil { return err } + defer content.Close() reader := pb.Add(prompt, desc.Digest.String(), desc.Size, content)Fixes #491