bitsandbytes stores a tensor as packed codes plus a QuantState. The default 4-bit quantizer (backends/default/ops.py at 833649043474) writes uint8 bytes whose high nibble is the even element and whose low nibble is the next element, then a per-block absmax. quant_type is nf4 or fp4. The lookup tables live in functional.get_4bit_type (16 float32 values). 8-bit blockwise quantization uses the 256-value dynamic map from create_dynamic_map, with absmax per block of 4096 by default. compress_statistics reuses that same map for the nested absmax, at block size 256, and as_dict stores it as nested_quant_map. Safetensors needs the packed QuantState.as_dict form because JSON metadata cannot hold the non-tensor fields.
.safetensors .pt .bin
What it is
weight pack
Byte order
little
Dimension order
row major
Compression
none
Metadata
json
Loading
Loading it into a library can execute code, depending on the loader and the file.
How to read it
Where the values live
name
the parameter name; quant state keys hang off it
dtype
restored float dtype from quant_state.dtype
shape
quant_state.shape, C order
data_offset
container offset of the packed uint8
nbytes
ceil(prod(shape)/2) for 4-bit uint8 storage, plus absmax
alignment
1. packed uint8 is not padded to a nibble boundary beyond a final even element
QuantState
functional.QuantState. as_dict(packed=False) returns these fields. Older checkpoints may still be the list layout documented on QuantState.__getitem__.
absmax
float32 vector. one absmax per block
quant_map
float32[16]. the code attribute. Index order is the 4-bit code.
blocksize
int
quant_type
string. nf4 or fp4
dtype
string. float16, bfloat16, or float32
shape
tuple of int
nested_absmax
float32 vector. present only when nested. absmax of the outer absmax.
nested_blocksize
int. present only when nested. 256 when compress_statistics quantizes the absmax. The standalone 8-bit path defaults to 4096.
nested_quant_map
float32[256]. present only when nested. The nested state's code attribute: the 8-bit dynamic map from create_dynamic_map, not the 16-value 4-bit table.
nested_dtype
string. present only when nested
nested_offset
float. present only when nested. The offset attribute, stored as a Python float.
Reader
The procedure that finds each tensor's bytes.
read
pseudo-python
def dequant_4bit(packed_u8, state):
# default backend: out[::2] = byte >> 4; out[1::2] = byte & 0xF
codes = empty(packed_u8.shape[0] * 2, int32)
codes[0::2] = packed_u8 >> 4
codes[1::2] = packed_u8 & 0xF
n = prod(state.shape)
codes = codes[:n]
table = GET_4BIT_TYPE[state.quant_type] # 16 floats, then divided by max(abs)
values = table[codes]
# state.code is quant_map (16 floats). When nested, state.state2.code is nested_quant_map,
# the 256-value 8-bit dynamic map, and state.state2.blocksize is 256. state.offset is nested_offset.
if state.nested:
absmax = dequant_blockwise_8bit(state.absmax, state.state2) + state.offset
else:
absmax = state.absmax
block = state.blocksize
for b, scale in enumerate(absmax):
values[b * block:(b + 1) * block] *= scale
return values[:n].reshape(state.shape).astype(state.dtype)
Reject the file when
Listed in the spec's order. Only rules marked MUST that reject the file are shown. The other IDs are warnings or implementation limits.
BNB-001. 4-bit quant_type in {nf4, fp4} and blocksize in {32, 64, 128, 256, 512, 1024, 2048, 4096}
BNB-002. High nibble is element 2j and low nibble is element 2j+1
BNB-003. Dequant indexes get_4bit_type(quant_type) with the 4-bit code. NF4 is not an IEEE float.
BNB-004. If nested_absmax is present, dequant the absmax with the nested quant state (nested_quant_map, nested_blocksize, nested_dtype) and add nested_offset before scaling the codes
Example
Example file. bitsandbytes 0.50.2 quantize_4bit (nf4, block size 64) wrote this safetensors file on CPU. The 0.50.2 wheel is older than commit 833649043474 (0.50.3.dev0). Names, dtypes, shapes, and stored bytes match the recorded hashes. This page does not unpack the weights. The header below is spaced for reading; the file pads that JSON with spaces out to a multiple of 8.