Reposted by Stefan Marr
Brief weekend reading on another benchmarking pitfall:
The Limitations of Running a Workload Generator In the Same JVM as the System-Under-Test
norlinder.nu/posts/The-Li...
Spoiler: the GC may conspire against you and your data collection!
norlinder.nu
The Limitations of Running a Workload Generator In the Same JVM as the System-Under-Test
If you care about tail latency in SPECjbb2015, Composite mode hides GC impact. We quantify the discrepancy and recommend Distributed or MultiJVM.