[justinwlin]

Flash → SDK 2.0 fix stack

Status as of 2026-10-22. 1–5 are merged; 10 is stacked on 9. Hover a link for its status.

#	PR	Fixes	Needs
1	#627	✅ merged: CI e2e fixed on main	—
2	#623	✅ merged: torch pulled in by peft/accelerate no longer bundled	—
3	👉 #624 (this PR)	✅ merged: requested GPU devices are pinned, not whole pools	—
4	#625	✅ merged: rp flash dev deletes its endpoints when stopped by a signal	—
5	#626	✅ merged: vol.path works on deployed workers	—
6	#628	❌ closed as a duplicate of #641, which was also closed unmerged. The API key is still copied into every endpoint	—
7	—	Release runpod 2.0.1 with 2–5 (release PR #594). 2.0.0rc2 (2026-10-20) predates them	2–5 merged ✅
8	runpod-workers/sdk-runtime#1	Rebuild runtime images (dev sessions run an old SDK)	independent
9	runpod/runpod-plugins-official#89	Plugin docs for the fixed SDK (flash lane → SDK 2.0)	2.0.1 on PyPI
10	runpod/runpod-plugins-official#90	Plugin VolumeCache guidance: global volumes, measured numbers, golden paths 08/09 (stacked on #89)	runpod/runpod-plugins-official#89

gpu=[...] with device names deploys onto whole pools, so workers can run on GPUs you didn't ask for. In a live test, ["NVIDIA GeForce RTX 4090", "NVIDIA L4", "NVIDIA RTX A5000"] ran on an RTX PRO 6000 Blackwell Server Edition MIG 1g.24gb slice.

Cause

gpu_ids_value maps each device to its pool and sends only the pool ids. The platform's AMPERE_24 pool also holds the RTX 3090 and the PRO 6000 MIG 24 GB slice. runpod-flash excluded the pool's other devices with -<device id> tokens, but only the ones in its static pool table. That table lacks the MIG slices, so Flash would land there too.

Change

• For pools selected through a device name, exclude every other member with -<device id>. Pools named directly ("AMPERE_24") still allow all their devices.
• Read pool membership from the live catalog (GET https://api.runpod.io/v2/catalog/gpus, the pool field), cached per process. Fall back to the static table, with a warning, when the catalog is unreachable. A failed lookup is retried after a minute.

Testing

Same 3-device list (4090, L4, A5000)	main	runpod-flash 1.21	This PR
Stored gpuIds	ADA_24,AMPERE_24	pools plus -<device> for its static table only	ADA_24,AMPERE_24,-NVIDIA GeForce RTX 3090,-NVIDIA RTX PRO 6000 Blackwell Server Edition MIG 1g.24gb
Where jobs ran	a job landed on RTX PRO 6000 … MIG 1g.24gb	would land on the MIG slice too (missing from its table)	4 of 4 parallel jobs on NVIDIA GeForce RTX 4090

How we tested and compared

• Live, before and after. The same queue app returned torch.cuda.get_device_name() and was deployed with rp flash deploy from main and then from this branch. We sent parallel jobs to force several workers and read the device name from each result.
• Flash comparison by code, not live. runpod-flash builds its negations from the static pool table. The live catalog (GET /v2/catalog/gpus, pool field) puts the MIG slice in AMPERE_24, and neither static table lists it. That is why this PR reads pool membership from the live catalog.
• Unit tests cover live-catalog exclusions, a pool named directly (no exclusions), the static fallback, catalog parsing, a failed lookup that is not retried on every call, and the retry after the window. tests/test_apps/conftest.py keeps the other tests offline. pytest tests: 1483 passed, coverage 91.67%.

Review (Codex, two rounds)

Finding	Resolution
A failed catalog lookup was logged at debug and cached for the whole process, so pins quietly lost the MIG exclusion	Fixed in 911ea98. A failure now logs a warning, and the catalog is retried after 60 s.
Exact pins still fall back to the static table while the catalog is down	Kept on purpose. Deploys need the same API, and failing every deploy on a catalog blip is worse than a warned, broader pin.
The catalog request uses the global API key, not an explicit AppsApiClient(api_key=...)	Not changed. The CLI always sets the global key. Programmatic explicit-key-only use falls back to the static table, with the warning above.

[promptless]

Promptless documentation updates

• docs(sdk): Document that device GPU selections exclude other GPUs in the pool corrects the note on the SDK GPU types page: asking for a specific device now keeps workers on that device, while naming a pool still allows any GPU in it.