PUBLISHED

Rigel

Reverse-Engineering the Apple M4 Max Metal 4.1 Tensor Path

arXiv 2606.12765

What It Is

An empirical microbenchmark study that reverse-engineers Apple's Metal 4.1 tensor compute path — the Metal Performance Primitives matmul2d operation — on the M4 Max GPU, recovering eleven hidden or contradicted hardware specifications Apple's documentation omits.

The Problem

Apple documents that Metal 4.1 exposes a tensor matmul2d, but the actual hardware behavior is opaque. Is fp8 hardware-accelerated or emulated? What is the accumulator width? What is the cooperative-tensor fragment layout? With no public answers, kernel authors optimize blind — and a naive fp8 port of a working fp16 kernel fails to even compile.

Approach

Checksum-gated, provenance-tracked microbenchmark harness — every cell carries a config hash, code commit, seed, and environment snapshot; float64 reference cancels input-quantization error
Three-signal dispatch-target triangulation: throughput ceiling vs the scalar-ALU roofline, a four-way GEMM shootout, and GPU power-rail attribution
Pre-registered cheap-baseline falsification gate — fp8 must clear 1.5× fp16 to count as acceleration, not emulation
Hostile accumulator probe (one large term + 127 tiny terms) to expose accumulation precision bit-exactly
MIT-licensed release: every figure redraws from a committed CSV via a single make reproduce target, no GPU required

Results

fp8 (E4M3) matmul2d is emulated, not accelerated — 0.94× fp16 throughput despite reading half the operand bytes; it is a memory/dynamic-range feature, not a speedup
matmul2d executes on GPU shader cores — no dedicated matrix unit exists on M4 Max (it predates the M5/A19 Neural Accelerators)
Accumulation is at least fp32: a deliberately hostile cell returns 455.9375 bit-exactly, which an fp16 accumulator could not
Recovered the undocumented unified 8×8 cooperative-tensor fragment layout — the lever for layout-aware kernels
Hand-fused GEMM+bias+GELU kernel using the recovered layout: +12.9% at 1024³, +6.5% at 2048³ in the cache-resident regime

Stack

Metal 4.1Apple M4 Max GPUMPP matmul2dMetal Shading LanguageC++PythonMicrobenchmarking

All Research

The Problem

Approach

Checksum-gated, provenance-tracked microbenchmark harness — every cell carries a config hash, code commit, seed, and environment snapshot; float64 reference cancels input-quantization error

Three-signal dispatch-target triangulation: throughput ceiling vs the scalar-ALU roofline, a four-way GEMM shootout, and GPU power-rail attribution

Pre-registered cheap-baseline falsification gate — fp8 must clear 1.5× fp16 to count as acceleration, not emulation

Hostile accumulator probe (one large term + 127 tiny terms) to expose accumulation precision bit-exactly

MIT-licensed release: every figure redraws from a committed CSV via a single make reproduce target, no GPU required

Results

fp8 (E4M3) matmul2d is emulated, not accelerated — 0.94× fp16 throughput despite reading half the operand bytes; it is a memory/dynamic-range feature, not a speedup

matmul2d executes on GPU shader cores — no dedicated matrix unit exists on M4 Max (it predates the M5/A19 Neural Accelerators)

Accumulation is at least fp32: a deliberately hostile cell returns 455.9375 bit-exactly, which an fp16 accumulator could not

Recovered the undocumented unified 8×8 cooperative-tensor fragment layout — the lever for layout-aware kernels

Hand-fused GEMM+bias+GELU kernel using the recovered layout: +12.9% at 1024³, +6.5% at 2048³ in the cache-resident regime