Crosswalk

hf_config_to_gguf

HF checkpoints keep hyperparameters in config.json (+ tokenizer.json / tokenizer_config.json / generation_config.json). GGUF keeps hyperparameters as typed KV inside the file. "{arch}" = general.architecture (e.g. llama, qwen2, gemma3). The converter tries the HF keys IN ORDER and takes the first one present. GGUF types are those gguf-py writes (u32 for counts, f32 for floats).

gguf	type	hf	notes
general.architecture	string	architectures[0] -> converter registry name	e.g. LlamaForCausalLM -> llama
general.name	string	_name_or_path / model directory name
general.file_type	u32		llama_ftype of the dominant tensor type
general.quantization_version	u32		always written by convert_hf_to_gguf.py, GGML_QUANT_VERSION = 2
{arch}.block_count	u32	n_layers num_hidden_layers n_layer num_layers
{arch}.context_length	u32	max_position_embeddings n_ctx n_positions max_length max_sequence_length model_max_length
{arch}.embedding_length	u32	hidden_size n_embd dim
{arch}.feed_forward_length	u32	prefix_dense_intermediate_size dense_intermediate_size intermediate_size n_inner hidden_dim
{arch}.attention.head_count	u32	num_attention_heads n_head n_heads
{arch}.attention.head_count_kv	u32	num_key_value_heads n_kv_heads	absent -> equals head_count (MHA)
{arch}.attention.key_length value_length	u32	head_dim	written when head_dim is given explicitly
{arch}.rope.freq_base	f32	global_rope_theta rope_global_theta rope_theta_global rope_theta rotary_emb_base
{arch}.rope.dimension_count	u32	head_dim or hidden_size // num_attention_heads (x partial_rotary_factor)
{arch}.rope.scaling.type .factor .original_context_length .yarn_*	string f32 u32	rope_scaling{rope_type\|type, factor, original_max_position_embeddings, ...}
{arch}.attention.layer_norm_rms_epsilon	f32	rms_norm_eps norm_eps
{arch}.attention.layer_norm_epsilon	f32	layer_norm_eps layer_norm_epsilon norm_epsilon
{arch}.expert_count	u32	num_local_experts num_experts n_routed_experts
{arch}.expert_used_count	u32	num_experts_per_tok num_experts_per_token top_k_experts
{arch}.vocab_size	u32	vocab_size	written by some converters (e.g. llama)
{arch}.attention.sliding_window	u32	sliding_window
tokenizer.ggml.model	string	set by the converter class's set_vocab(): tokenizer.model (SentencePiece) -> llama; BPE tokenizer.json -> gpt2; WordPiece -> bert; T5 Unigram -> t5 (also rwkv, none, and others for specific models)
tokenizer.ggml.pre	string	converter detects the pre-tokenizer by hashing the tokenizer's output (get_vocab_base_pre)
tokenizer.ggml.tokens scores token_type merges	array[string] array[f32] array[i32] array[string]	tokenizer.json vocab, merges; added_tokens -> token_type CONTROL/USER_DEFINED
tokenizer.ggml.bos_token_id eos_token_id unknown_token_id padding_token_id seperator_token_id	u32	tokenizer_config.json / special_tokens_map.json / config.json *_token_id	'seperator' (sic) spelling is the real key
tokenizer.ggml.add_bos_token add_eos_token	bool	tokenizer_config.add_bos_token add_eos_token
tokenizer.chat_template	string	tokenizer_config.chat_template chat_template.jinja

hf_tensor_names_to_gguf

gguf-py tensor_mapping.py (TensorNameMap) maps per-architecture HF names to GGUF names; '.weight'/'.bias' suffix kept. LLaMA-family examples:

hf	gguf	notes
model.embed_tokens.weight	token_embd.weight
model.norm.weight	output_norm.weight
lm_head.weight	output.weight	absent when tie_word_embeddings: GGUF then has no output.weight and the runtime reuses token_embd
model.layers.{N}.input_layernorm.weight	blk.{N}.attn_norm.weight
model.layers.{N}.self_attn.q_proj.weight	blk.{N}.attn_q.weight	rows PERMUTED for llama-family
model.layers.{N}.self_attn.k_proj.weight	blk.{N}.attn_k.weight	permuted with n_head_kv
model.layers.{N}.self_attn.v_proj.weight	blk.{N}.attn_v.weight
model.layers.{N}.self_attn.o_proj.weight	blk.{N}.attn_output.weight
model.layers.{N}.post_attention_layernorm.weight	blk.{N}.ffn_norm.weight
model.layers.{N}.mlp.gate_proj.weight	blk.{N}.ffn_gate.weight
model.layers.{N}.mlp.up_proj.weight	blk.{N}.ffn_up.weight
model.layers.{N}.mlp.down_proj.weight	blk.{N}.ffn_down.weight
model.layers.{N}.self_attn.rotary_emb.inv_freq	(dropped)	recomputed at runtime

6. LLaMA Q/K row permutation (HF <-> GGUF)

HF LLaMA-family checkpoints store q_proj/k_proj rows in the "rotate-half" RoPE layout. llama.cpp expects interleaved pairs. convert_hf_to_gguf.py (LlamaModel.permute) applies:

def permute(w, n_head, n_head_kv):          # w: [rows, cols] numpy, canonical order
    if n_head_kv is not None and n_head != n_head_kv:
        n_head = n_head_kv                   # used for k_proj
    return (w.reshape(n_head, 2, w.shape[0] // n_head // 2, *w.shape[1:])
             .swapaxes(1, 2).reshape(w.shape))
def unpermute(w, n_head, n_head_kv):         # inverse (GGUF -> HF)
    if n_head_kv is not None and n_head != n_head_kv: n_head = n_head_kv
    return (w.reshape(n_head, w.shape[0] // n_head // 2, 2, *w.shape[1:])
             .swapaxes(1, 2).reshape(w.shape))

q uses (n_head, n_head), k uses (n_head, n_head_kv). The permutation only reorders whole rows (axis 0). ggml blocks never span rows, so it can also be applied to already-quantized tensors by moving each row's block bytes as a unit. Other architectures have their own transforms (e.g. fused QKV split, norm +1 offsets in Gemma, exp of A_log in Mamba). Check the modify_tensors method of the specific converter class.

Implement it per architecture and test against llama.cpp output. Do not try to derive it generically.

gguf_to_hf_config

The reverse (GGUF -> HF, e.g. transformers' gguf loading in integrations/ggml.py) maps the same keys back. Lossy points: GGUF does not store torch_dtype, architectures class names, generation_config, or per-architecture extras that the converter dropped. Tokenizers must be rebuilt from tokenizer.ggml.* (merges/scores), which is not always byte-identical to the original tokenizer.json.

safetensors_metadata_to_gguf

safetensors __metadata__ is string->string (typically only {"format": "pt"}). Copy it, if needed, to GGUF keys under a private prefix (e.g. "safetensors.metadata.<key>" as STRING). Reverse: GGUF typed KV -> safetensors __metadata__ by JSON-encoding each value with a type tag, e.g. "gguf.<key>": "{\"type\":\"UINT32\",\"value\":4096}". Large arrays (tokenizer vocab) usually do not belong in safetensors metadata; write them to tokenizer.json instead (header limit 100 MB).

pytorch_to_safetensors

state_dict keys map 1:1. Tied/shared tensors must be de-duplicated (safetensors refuses aliasing); HF save_pretrained drops the tied copy and records tie_word_embeddings in config.json.

mlx_lm_quantization

mlx-lm stores quantization beside the weights, not inside the safetensors header. save_config copies quantization onto quantization_config, sorts keys, and tags non-finite floats as {"__float__": "Infinity"|"-Infinity"|"NaN"}. Tensor names are the module path from tree_flatten plus a suffix. group_size, bits, and mode are Python attributes and are not tensors.

key	type	notes
quantization.group_size	int	affine default 64; mxfp4 and mxfp8 32; nvfp4 16
quantization.bits	int	affine default 4; mxfp8 8
quantization.mode	string	affine \| mxfp4 \| nvfp4 \| mxfp8. Load treats a missing mode as affine.
quantization.<module path>	false \| {group_size, bits, mode, global_scale?}	false leaves the layer dense. global_scale true records an nvfp4 per-expert scale tensor.
quantization_config	object	duplicate of quantization on save. On load, used only when quantization is absent; quant_method then selects a foreign importer (awq, gptq, mxfp4, mxfp8, bitnet, compressed-tensors, modelopt, fp8).
quantize_activations	bool	optional. Rebuilds QuantizedLinear as QQLinear.

suffix	dtype	shape	notes
.weight	uint32	[..., K * bits / 32]	packed codes. K is the logical input features.
.scales	w.dtype (affine) or uint8 (mxfp4, nvfp4, mxfp8)	[..., K / group_size]
.biases	same as scales	same as scales	affine only. fp modes omit it.
.bias	original floating dtype		optional dense bias, not quantized
.global_scale	float32		optional nvfp4 tensor scale, one scalar or one per expert

key	value	notes
format	mlx	written on every shard by save_model

mlx_gguf

What mx.load / mx.save_gguf and mlx_lm.gguf.convert_to_gguf do, on top of hf_config_to_gguf. Dimension order is reversed (ggml ne[0] innermost <-> MLX last axis).

gguf_type	mlx
F32, F16, I8, I16, I32	same dtype, shape reversed
Q4_0	affine group_size 32 bits 4, f16 scales, biases = -8 * scale, nibble-repacked uint32
Q4_1	affine group_size 32 bits 4, f16 scale and the block min as biases
Q8_0	affine group_size 32 bits 8, f16 scales, biases = -128 * scale, codes = int8 XOR 0x80
BF16, Q2_K, Q4_K, Q6_K	one float16 tensor (gguflib dequant)
other	load throws

mlx	gguf
F32, F16, I8, I16, I32	same type, shape reversed to ne[]
bf16, uint*, i64, f64, complex64, packed uint32	save_gguf throws

Refuses config.quantization. Hardcodes general.architecture = llama. Renames a llama-style subset (translate_weight_names in mlx_lm/gguf.py). Permutes q_proj and k_proj with the section-6 permute. Casts bfloat16 to float16 via float32, and any tensor whose name contains 'norm' to float32. general.file_type and general.quantization_version are both the integer 1, not GGML_QNT_VERSION.

Sources

Nick Reynolds · @goldnode · rev 2.5