GGUF (GGML Universal File)

How to read it

header (incl. all metadata KV) | tensor_info[tensor_count] | zero padding to ALIGNMENT (absolute file offset) | tensor_data. Every field is packed (no implicit alignment) except the padding before tensor_data and between tensors. ALIGNMENT is only known after parsing metadata (general.alignment).

header

gguf_header

tensor_infos

gguf_tensor_info[header.tensor_count]

padding

u8[align_up(pos, ALIGNMENT) - pos]. ggml only seeks to the aligned offset when tensor_count > 0

tensor_data

bytes. each tensor at data_start + info.offset; gap between tensors zero-padded to ALIGNMENT

0        4        8                16               24
+--------+--------+----------------+----------------+-----------------------------+
| "GGUF" | ver u32| tensor_cnt u64 | kv_count u64   | kv[0] kv[1] ... (packed)    |
+--------+--------+----------------+----------------+-----------------------------+
| tensor_info[0] ... tensor_info[n-1] (packed)                                    |
+----------------------------------------------------------------------------------+
| 0x00 padding up to multiple of ALIGNMENT (absolute file offset)                  |
+----------------------------------------------------------------------------------+  <- data_start
| tensor 0 data | pad | tensor 1 data | pad | ...                                  |
+----------------------------------------------------------------------------------+

One tensor

name

tensor_info.name (UTF-8)

dtype

tensor_info.type (ggml_type) -> dtypes[].canonical

shape

reversed(tensor_info.dimensions) # canonical row-major shape

data_offset

data_start + tensor_info.offset (absolute)

nbytes

see nbytes_formula

order

row-major (C) in canonical shape terms; no strides

Example

The specs test vector. Shapes are outermost first: the file stores that list reversed, fastest dimension first. w_q8_0 is the Q8_0 blocks for w_f32. e_bf16 is in this file. s_f32 is not: a scalar is stored as one dimension of length 1, and that stored shape must not replace the canonical shape [].

small.gguf1,792 bytes

Version

3

general.architecture

llama

test.answer

42

test.note

tensor-format-specs