fix: anchor tar member validation to extract_path in sagemaker-core - #6195
Conversation
custom_extractall_tarfile falls back to filtering members with
_get_safe_members when tarfile.data_filter is unavailable (Python < 3.12,
before the 3.9.17 / 3.10.12 / 3.11.4 backports). That fallback had two
containment defects that combined to let an archive member be written
outside the extraction directory:
- _get_safe_members anchored its check to the process working directory
(_get_resolved_path("")) rather than the directory the archive is
extracted into. It now takes the base as an argument, and
custom_extractall_tarfile passes the resolved extract_path.
- _is_bad_path compared paths with str.startswith, so a sibling directory
sharing a textual prefix with the base (e.g. base "/tmp/extract" and
"/tmp/extract-evil/f") was treated as contained. Containment is now
checked with os.path.commonpath, via a shared _is_within_base helper
that _validate_extracted_paths uses as well.
Absolute member paths are now rejected outright, since joinpath would
otherwise silently discard the base for them.
Both defects were required for the escape: relative member resolution is
base-independent under normpath, so a plain "../../x" member does not
escape on its own. The working-directory anchoring is what made a
prefix-matching sibling ("<cwd>" vs "<cwd>evil") pass validation while
extraction still wrote outside extract_path. _validate_extracted_paths
only walks extract_path, so it did not catch the escape either.
This is the shared utility every v3 extraction path routes through (model
unpack, local/image.py, serve TGI and DJL prepare, pipeline repack). An
already-correct implementation of the same logic exists in
sagemaker-mlops/src/sagemaker/mlops/workflow/_repack_model.py; this brings
common_utils.py in line with it.
Adds regression tests covering the end-to-end escape, the sibling-prefix
bypass, absolute members, and that validation is anchored to extract_path.
|
Claude finished @nargokul's task in 1m 3s —— View job Review: anchor tar member validation to
|
Issue
custom_extractall_tarfile()insagemaker-core/src/sagemaker/core/common_utils.pyfalls back to_get_safe_members()whentarfile.data_filteris unavailable (Python < 3.12, before the 3.9.17 / 3.10.12 / 3.11.4 backports). Two defects in that fallback let an archive member be written outside the extraction directory:_get_safe_members()anchored its check to_get_resolved_path("")— the process working directory — not theextract_pathmembers are actually extracted into._is_bad_path()usedstr.startswith(base), so a sibling sharing a textual prefix with the base (/tmp/extractvs/tmp/extract-evil/f) was treated as contained.Both are required for the escape: relative resolution is base-independent under
normpath, so../../xalone does not escape. The working-directory anchoring let a prefix-matching sibling (<cwd>vs<cwd>evil) pass validation while extraction wrote outsideextract_path._validate_extracted_paths()only walksextract_path, so it did not catch it either.This utility backs every v3 extraction path: model unpack,
local/image.py,serveTGI/DJL prepare, pipeline repack.Fix
_get_safe_members(members, base)takes the base as an argument;custom_extractall_tarfile()passes the resolvedextract_path.os.path.commonpathvia a new_is_within_base()helper, also used by_validate_extracted_paths().sagemaker-mlops/src/sagemaker/mlops/workflow/_repack_model.pyalready implements this correctly; this bringscommon_utils.pyin line with it.custom_extractall_tarfile()'s signature is unchanged and the two modified functions are private with no callers outside this module.Testing
New
TestTarExtractionPathTraversalinsagemaker-core/tests/unit/test_common_utils.py, patching outdata_filterso the fallback runs on any interpreter. Covers the end-to-end escape, absolute members, the sibling-prefix bypass, base anchoring, and a benign archive still extracting.On Python 3.9.18: 6 of the new tests fail against
masterand all pass with this change.test_common_utils.py253 passed; fullsagemaker-core/tests/unit3540 passed, 22 skipped.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.