compressed-tensors packed weights and ModelOpt NVFP4 names
compressed-tensors keeps weights in safetensors. The pack-quantized compressor stores an int32 weight_packed, a weight_scale, and a weight_shape, and packs the zero point only for asymmetric group or channel quantization. ModelOpt NVFP4 checkpoints use different names (uint8 weight, weight_scale, weight_scale_2, input_scale). ModelOptNvfp4Converter in the compressed-tensors project renames those to the compressed-tensors NVFP4 names and inverts the two global scales. It does not reorder the packed nibbles. Container layout is the safetensors spec. The scheme (bits, group size, strategy, symmetric) lives in config.json, not in the tensor header.
.safetensors
What it is
weight pack
Byte order
little
Dimension order
row major
Compression
none
Metadata
json
Loading
Loading it into a library does not execute code.
How to read it
Where the values live
name
module prefix
dtype
int32 or uint8 carrier. Logical type comes from config (int bits, or FP4 for NVFP4).
shape
weight_shape for pack-quantized; the uint8 shape for ModelOpt, which is not the logical FP4 shape
data_offset
safetensors offset of weight_packed or weight
nbytes
packed tensor nbytes
alignment
1. bit stream is dense; the last int32 is not required to be full
PackQuantizedModule
weight_packed
int32. shape [..., ceil(cols * num_bits / 32)] when packed_dim is 1
weight_scale
floating tensor
weight_shape
int tensor. the shape before packing
weight_zero_point
int32. present for asymmetric GROUP or CHANNEL; packed on dim 0
ModelOptNvfp4Module
weight
uint8. two FP4 values per byte. Embeddings that are not uint8 are left unchanged.
weight_scale
float8 e4m3 in the converter comment
weight_scale_2
float32. inverted to weight_global_scale
input_scale
float32. inverted to input_global_scale
Reader
The procedure that finds each tensor's bytes.
read
pseudo-python
def unpack_pack_quantized(packed, num_bits, shape):
# inverse of pack_to_int32 along dim 1: element i is the num_bits field at bit i*num_bits
cols = shape[-1]
codes = empty(shape, int32)
flat_bits = packed.view(int32) # little-endian words
for i in range(cols):
bit = i * num_bits
word = bit // 32
off = bit % 32
lo_bits = min(32 - off, num_bits) # a field may cross a word; mask only the bits in this word
field = (flat_bits[..., word] >> off) & ((1 << lo_bits) - 1)
if lo_bits < num_bits:
hi_bits = num_bits - lo_bits
field |= (flat_bits[..., word + 1] & ((1 << hi_bits) - 1)) << lo_bits
codes[..., i] = field
return (codes - (1 << (num_bits - 1))).to(int8)
Reject the file when
Listed in the spec's order. Only rules marked MUST that reject the file are shown. The other IDs are warnings or implementation limits.
CT-001. pack-quantized num_bits is in 1..8 and the tensor passed to pack_to_int32 is int8
CT-002. Stored bits are the int8 code plus 1<<(num_bits-1), so the on-disk field is unsigned
CT-003. Element i starts at bit i*num_bits along the packed dimension, including when that range crosses an int32 boundary
CT-004. Use weight_shape for the logical shape. The packed width only bounds the last dimension to within 32/num_bits.
CT-005. ModelOpt uint8 weight is copied to weight_packed without a nibble swap. Invert weight_scale_2 and input_scale (new = 1/old) when converting to compressed-tensors names.
Example
Example file. compressed-tensors pack_to_int32 wrote this safetensors file. The specs say a CPU unpack matched the int8 codes. This file does not settle which nibble is first in the ModelOpt layout. Names, dtypes, shapes, and stored bytes match the recorded hashes. This page does not unpack the weights. The header below is spaced for reading; the file pads that JSON with spaces out to a multiple of 8.