The Test That Only Failed in the Summer

In the summer, while I was working on a feature locally, I noticed a test started failing. I initially dismissed it: I assumed there was some flakiness, and CI was green, so everything must’ve been good, right? Until it kept failing, only locally. A test that a few days before was working as normal. It was not just a single test. Multiple tests were failing, so the immediate thought was that they must be related. The question was: why are they failing on my machine, but not on CI? The exact same code, the exact same setup — or so I thought. After cleaning caches and dependencies, and retrying multiple times, it was definitely not flaky, nor a cache issue. Some background: the failing tests were all about X.509 certificate validity. We create leaf certificates for mdoc, whose specification (ISO/IEC 18013-5) limits their validity to 457 days. The service rejects certificates that are valid for longer than that, and the tests made sure that the limit was correctly enforced: create a…