On "When impressive performance gains do not matter"

When impressive performance gains do not matter is a very nice article covering some ways in which going after performance alone is not sufficient without considering the wider picture. It resonated a lot with how I think about performance. If there are multiple bottlenecks in the pipeline—and with these systems, this is common—the overall throughput will not improve until every last bottleneck is removed. His focus is on distributed systems bottlenecks, but I’ve hit the same “do-nothing” speedups when optimizing client side programs. Usually this comes from spending a lot of time thinking something was the bottleneck when it wasn’t. CPU profiling is where this bites me most: it tells me “function X is taking 30% of the cycles” and I think “oooo, there’s a lot of gains to be made there”. I build a microbenchmark for X, optimize it and there’s only a marginal gain at the high level. While disappointing, I’ve become used to it over time and internalized that performance is highly…