A new form of human interaction

Few days ago, Cloudflare published a blog post saying Perplexity is using undeclared crawlers to evade no-crawl directives. This sparked some interesting debates and I was particularly piqued by one arguing Perplexity is no different from a regular browser and should ignore no-crawl directives. As ridiculous as it initially sounded, I found the argument surprisingly valid and worth further thought. (Perplexity has since replied in the same defense). Bots, Crawlers, robots.txt and no-crawl Bo...