Profiling

The honest page: Metal has no Nsight Compute, and the gap shapes how performance work is done on this platform.

What you have, in decreasing order of usefulness:

Command-buffer timestamps. GPUStartTime / GPUEndTime on a completed command buffer: wall-clock GPU seconds, programmatic, cheap, reliable. This is what every benchmark harness in the case studies uses. Granularity is the command buffer, so isolate what you're measuring into its own buffer, warm up (first execution includes pipeline compilation), and take medians.

Xcode's GPU capture (Metal Debugger). The closest thing to a real profiler: per-encoder timings, occupancy estimates, memory-bandwidth counters, shader profiling. Two structural limits: it's a GUI (no CI, no scripted regression checks), and capture perturbs heavily on big ML workloads, with inference projects reporting 50-100× slowdowns and multi-GB traces. Usable for staring at one dispatch; unusable as a feedback loop.

Metal 4's counter API: timestamps only, through macOS 26. The hardware counters that would give you achieved occupancy, stall reasons, or DRAM traffic per kernel are not in the public programmatic API through macOS 26. The GUI capture does surface them: flash-moe's I/O exploration notes↗ pulled a full instruction-cost breakdown from a Metal GPU trace (63.9% math, 25% type conversion from bf16 scale/bias, 93.4% L1 hit rate, 2.4% GPU utilization: the model was I/O-bound, not compute-bound). And the story is changing: macOS 27 adds always-on performance metrics, a metalperftrace command-line tool, and a StateReporting API (WWDC26 session 388↗), which would give this page its first scriptable feedback loop. Until that ships and proves out, the methodology below stands.

So the working methodology, which the war stories independently converge on:

1. Timestamps + arithmetic. Compute achieved GFLOPS and GB/s from problem size and measured time; compare against roofline ceilings you measured yourself (a STREAM-style bandwidth probe↗ for DRAM; metal-benchmarks↗ tables for ALU).
2. Differential benchmarking. Can't see stall reasons? Change one thing and re-measure. The m5-gemm README's "things that did not help" section is this method producing knowledge.
3. Disassembly when arithmetic says impossible. applegpu↗'s compiler_explorer.py shows what was actually emitted: the tool of last resort for spill hunting.
4. Methodology hygiene, because the tools won't catch you: fix your clocks story (thermals: locking fans changed long-run throughput decay from 50% to 6.7% in one measured case), report end-to-end numbers next to kernel numbers (synthetic wins that vanish end-to-end are the classic failure), and state matrix sizes, since the winner flips with size.