AI bots and the open internet

This is slightly different to my usual posts but it’s something I’ve been having to deal with since making the Plane Makers Database available, which is AI bots. The site was being crawled by AI bots sending multiple bursts of ~10,000 requests in under a minute along with a constant ~30 requests per minute. I … Continue reading...