Summary
model_free_ptq currently maintains two code paths. One for standard schemes and one for microscale (NVFP4, MXFP4).
There exists a TODO at microscale.py:104-108 that describes ideal cleanup as follows:
- Promote
MicroscaleConverter from an inline class to a top-level class fulfilling the full Converter Protocol
- Remove
build_microscale_inverse_weight_maps entirely, replacing it with the standard build_inverse_weight_maps
- Merge
process_file and process_file_microscale_scheme into a single function
This was left as a follow-up after #2498 introduced the inverse_weight_map approach, and #2737 completed the deprecation of reindex_fused_weights.
Proposed Approach
1. Promote MicroscaleConverter to a top-level Converter
Go with moving MicroscaleConverter out of the build_microscale_inverse_weight_maps function body and implement the full Protocol:
2. Remove build_microscale_inverse_weight_maps
Once MicroscaleConverter is a proper Converter, _build_jobs can instantiate it and pass it in the converters list to the standard build_inverse_weight_maps which is the same code path as non-microscale.
3. Merge process_file and process_file_microscale_scheme
A single process_file function that conditionally enables fused-group handling when the scheme uses TENSOR_GROUP compression strategy
4. Simplify _build_jobs
Remove the if is_microscale_scheme branch. Single build_inverse_weight_maps call, single process_file function, microscale converter passed via the converters list.
Happy to take a stab at it, but would love feedback on gameplan if needed/necessary
Related: #2919 (Q3 Roadmap — model_free_ptq is called out under "Custom Pytorch Definition Support")
Predecessor PRs: #2498, #2737, #2545
Summary
model_free_ptqcurrently maintains two code paths. One for standard schemes and one for microscale (NVFP4, MXFP4).There exists a TODO at
microscale.py:104-108that describes ideal cleanup as follows:MicroscaleConverterfrom an inline class to a top-level class fulfilling the full Converter Protocolbuild_microscale_inverse_weight_mapsentirely, replacing it with the standardbuild_inverse_weight_mapsprocess_fileandprocess_file_microscale_schemeinto a single functionThis was left as a follow-up after #2498 introduced the inverse_weight_map approach, and #2737 completed the deprecation of
reindex_fused_weights.Proposed Approach
1. Promote
MicroscaleConverterto a top-levelConverterGo with moving
MicroscaleConverterout of thebuild_microscale_inverse_weight_mapsfunction body and implement the full Protocol:2. Remove
build_microscale_inverse_weight_mapsOnce
MicroscaleConverteris a properConverter,_build_jobscan instantiate it and pass it in the converters list to the standardbuild_inverse_weight_mapswhich is the same code path as non-microscale.3. Merge
process_fileandprocess_file_microscale_schemeA single
process_filefunction that conditionally enables fused-group handling when the scheme usesTENSOR_GROUPcompression strategy4. Simplify
_build_jobsRemove the
if is_microscale_schemebranch. Singlebuild_inverse_weight_mapscall, singleprocess_filefunction, microscale converter passed via the converters list.Happy to take a stab at it, but would love feedback on gameplan if needed/necessary
Related: #2919 (Q3 Roadmap — model_free_ptq is called out under "Custom Pytorch Definition Support")
Predecessor PRs: #2498, #2737, #2545