fix(serve): make torch an optional dependency - #6166
Open
goelakash wants to merge 1 commit into
Open
Conversation
goelakash
had a problem deploying
to
manual-approval
August 7, 2026 03:29 — with
GitHub Actions
Error
goelakash
had a problem deploying
to
manual-approval
August 7, 2026 03:29 — with
GitHub Actions
Error
goelakash
had a problem deploying
to
manual-approval
August 7, 2026 03:29 — with
GitHub Actions
Error
DEFAULT_SERIALIZERS_BY_FRAMEWORK instantiated TorchTensorSerializer at module scope, so importing sagemaker.serve required torch. Store the classes instead and instantiate on lookup, and move torch to an extra. Fixes aws#5531
goelakash
force-pushed
the
fix-optional-torch-serve
branch
from
August 7, 2026 03:32
1725779 to
38b75f5
Compare
goelakash
temporarily deployed
to
manual-approval
August 7, 2026 03:32 — with
GitHub Actions
Inactive
goelakash
had a problem deploying
to
manual-approval
August 7, 2026 03:32 — with
GitHub Actions
Error
goelakash
temporarily deployed
to
manual-approval
August 7, 2026 03:32 — with
GitHub Actions
Inactive
goelakash
marked this pull request as ready for review
August 7, 2026 03:33
goelakash
temporarily deployed
to
manual-approval
August 7, 2026 03:33 — with
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5531
sagemaker-serveliststorch>=2.0.0as a required dependency, so any install ofsagemaker,sagemaker-serve, orsagemaker-mlopspulls torch (and on Linux, the CUDA/cuDNN/NCCL stack) even for API-only use.sagemaker-corealready treats torch as an extra.The blocker was
DEFAULT_SERIALIZERS_BY_FRAMEWORKinserve/constants.py, which instantiatedTorchTensorSerializer()at module scope. That runsfrom torch import Tensoron anyimport sagemaker.serve. Every other torch reference in the package is already lazy.Changes
DEFAULT_SERIALIZERS_BY_FRAMEWORK; instantiate at lookup in_fetch_serializer_and_deserializer_for_framework.torch>=2.0.0to atorchextra, matchingsagemaker-core.tests/unit/test_optional_torch_dependency.py, mirroring the existingsagemaker-coresubprocess pattern.Installing with the
torchextra is unchanged.TorchTensorSerializer()still raises the sameImportErrorif torch is missing when actually used.Testing
tests/unitin a venv with and without torch: no new failures vs.master(53 pre-existing failures in both).masterand pass with this change.from sagemaker.serve import ModelBuilderandimport sagemaker.mlopsboth succeed.