Scrapers, AI and the smallweb (reply) 0 ▲ Mufusa 26 minutes ago · Tech · hide · 0 comments Reply to unrequested scrapers and aggregators (and AI) by slime journal blog (enter the void) “its also interesting to see what bots are hitting my feedunfortunately, it seems like some aggregators have found it somehow? maybe from scraping bubbles or somethingi dont usually mind, since its usually just a smallweb thing and maybe i could get some new readers off it” I agree with that and that’s what the smallweb is about, community and not trying to go trending (but things like Bubbles, Blogosphere and Hacker News do exist and people like using them to find new things) “so i imagine its scraping a lot of writing and shoving it into the stupid ai system” I have found this project (ai.robots.txt) and its sort of handy to easily block AI crawler/scraper bots that are flooding the internet with useless traffic and of course training on your content without asking first Even Google itself is losing $500bn because of AI, which is funny to think about Anyway I have been using the project for… No comments yet. Log in to reply on the Fediverse. Comments will appear here.