Pitfalls of Benchmarking on Modern Systems 0 ▲ Alcides Fonseca 86 days ago · Tech · hide · Fediverse post What could it be? It could be that the operating system decided to give it different physical memory, or run it on a different kind of core (many CPUs these days have 2 or more different types of cores with different performance). Perhaps, we ran out of thermal budget and the CPU ran at a lower clock speed to avoid overheating? Or, since we are on a JVM, perhaps the compiler saw slightly different information for the types/behavior seen in the first and second iterations, and thus, made slightly different optimization decisions? This is possible because compilation happens on a background thread, and thus, even when the benchmark is deterministic, the used profiling information is to some degree racy. — Pitfalls of Benchmarking on Modern Systems by Stefan Marr In my PhD I’ve spent a lot of time doing careful benchmarking of parallel programs. During that time, I’ve learn a lot of variables that matter when doing this type of work: CPU, RAM, interpreter/compiler, room temperature,… Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town