Part 6 · Speaking performance
Reading a profiler
The only tool in this course that measures rather than models — and the place where everything you have learned finally meets reality.
What this lesson is trying to break (3)
- The profiler tells you which line is slow.
- High instruction count means the hot line is doing too much work.
- Attribution in optimised builds is exact.
Every other instrument in this course models. A profiler measures. That makes it the final authority and also the one with the subtlest failure modes.
Two kinds of measurement
Sampling interrupts the program thousands of times per second and records where
it was. Cheap, works on production, gives you a statistical picture. This is
perf record and Instruments’ Time Profiler, and it is what you want almost
always.
Instrumentation adds counting code to every function. Exact call counts, and it distorts what it measures — often changing inlining decisions and therefore the thing you were trying to observe.
Start with sampling. Reach for instrumentation when you need exact counts of something rare.
Hardware counters are the interesting part
Beyond “where was I”, the CPU counts events, and these map directly onto Part 4.
perf stat -e cycles,instructions,cache-misses,branch-misses ./prog| Counter | Question it answers | Part |
|---|---|---|
instructions / cycles = IPC |
Are we executing or stalling? | 4.1 |
cache-misses, LLC-load-misses |
Are we memory-bound? | 4.4 |
branch-misses |
Is a branch unpredictable? | 4.3 |
stalled-cycles-backend |
Waiting on data or a unit? | 4.2 |
IPC is the first number to look at. Modern cores sustain 3–4 instructions per cycle on good code. Below ~1 means you are stalling most of the time, and the other counters tell you on what.
That single ratio reframes the whole optimisation question: if IPC is low, do not remove instructions — the core is idle, not overloaded. Removing work from an already-idle machine achieves nothing. Fix what it is waiting for.
A loop shows:
cycles 12,400,000,000
instructions 3,100,000,000 → IPC 0.25
cache-misses 190,000,000
branch-misses 410,000Memory-bound. Do not touch the arithmetic.
IPC 0.25 means the core executes one instruction every four cycles — idle roughly 90% of the time. Branch misses are negligible (0.4M against 3.1B instructions). Cache misses are enormous: 190M misses against 3.1B instructions is one miss per ~16 instructions, and at a couple of hundred cycles each that accounts for essentially all the time.
So the actions are all about data, not code:
- improve locality — contiguous instead of pointer-chasing (4.4)
- shrink the working set — smaller structs, fewer padding bytes (3.5)
- restructure access order — blocking or tiling so data is reused while cached
- consider prefetching, if the pattern is predictable but the prefetcher is not catching it
And the thing not to do: hand-optimise the loop body. You could halve the instruction count and change the runtime by almost nothing, because the core is already waiting. This is the single most common way engineering effort is wasted on performance work, and this one ratio prevents it.
Attribution lies, in specific ways
Three effects to know before trusting a per-line number.
Skid. Sampling is interrupt-driven, and the interrupt arrives some cycles after
the event. The sample gets attributed to whatever was retiring then, which may be
several instructions past the load that actually stalled. So the “hot line” is
often just after the real culprit. Precise event modes (perf’s :pp suffix,
PEBS on Intel) reduce this considerably.
Inlining. A hot function may not exist (2.5). Its time is attributed to whoever inlined it. If you cannot find a function you know is hot, that is why.
Missing frames. Without frame pointers, stack walking may truncate
(2.3) — the [unknown] frames in a flame graph.
So: trust the aggregate, distrust the specific line. “This loop is 60% of runtime and memory-bound” is reliable. “Line 47 is the problem” is a hypothesis.
On your machine
You are on an M1, so perf is not available. The equivalents:
# quick and always available
xcrun xctrace record --template 'Time Profiler' --launch -- ./prog
# or use Instruments.app: Time Profiler for stacks,
# CPU Counters for the hardware events aboveApple’s counters are named differently but measure the same things. Instruments’ CPU Counters template is where you find cycles, instructions and cache events.
And the honest caveat: your M1 is not your production server. Cache sizes, memory latency and vector width all differ (5.1), and memory-bound results in particular do not transfer. Profile where it runs.
A colleague shows a flame graph where one function is 40% of runtime, and proposes rewriting it in hand-optimised assembly.
Write your response. Name the one measurement you want before agreeing, and say what two different answers would each imply for the plan.
What you should now be able to say
- Sampling for almost everything; instrumentation distorts what it measures.
- IPC first. Low IPC means stalling — do not remove instructions from an idle core.
- Counters map onto Part 4: cache-misses → memory, branch-misses → prediction, IPC → stalls.
- Attribution is unreliable per-line because of skid, inlining and missing frames. Trust the aggregate.
- Your M1 is not your server, and memory-bound results especially do not transfer.