You need account for that!

I have mentioned here a few times the problems that “AI crawlers” are causing to a lot of open source projects. For example in Creepy crawlies or the problem of the LLM crawlers; plus my own frustration and the changes I had to make in my humble self-hosting solution for my git repos. It is all part of how the current crop of LLMs (or what people call just “AI”) is making things worse. If the search engines were already struggling because the proliferation of websites designed to attract clicks and show ads (which are, at the end, the incentive for this behaviour), with the availability of LLMs to generate the text, finding what you are looking for is very hard (there was an interesting conversation about this in lobsters: How are you dealing with all the negative side effects of AI on the web?). Which is resulting in more AI, because more and more often people are content with the AI summary and don’t bother checking the actual search results. What for? They are likely to be garbage…

Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town