BubblesBot BubblesBot fetches the RSS and Atom feeds of the blogs listed on Bubbles. It does not follow links and does not crawl sites. User agent Mozilla/5.0 (compatible; BubblesBot/1.0; +https://bubbles.town) IP addresses 178.104.58.164 2a01:4f8:1c19:7e5::1 Every request comes from one of these two addresses. A request with our user agent from anywhere else is somebody else. What it requests The feed of each listed blog. A blog that posted in the last week is polled every 15 minutes, quieter blogs less often, down to once every 6 hours. Requests carry If-None-Match and If-Modified-Since, so an unchanged feed can be answered with a 304 and no body. /robots.txt of each host. The URLs of posts already listed, with a HEAD request, to notice deleted posts. A post that answers 404 or 410 is removed from Bubbles. The homepage of each listed blog, once or twice a day, to look for the bubbles:fediverse meta tag. What Bubbles shows of a post (excerpt, reading time, the content in our own feeds) is taken from your feed. Article pages are only checked for existence, their content is not read. robots.txt BubblesBot reads robots.txt before it requests anything else from your site, and the rules apply to feeds, post URLs and the homepage alike. A group addressed to BubblesBot is used if there is one, otherwise the * group applies. Allow, Disallow and the wildcards * and $ are supported, and the longest matching rule wins. If the file is missing or cannot be loaded, BubblesBot assumes there are no restrictions. To allow BubblesBot while other bots stay blocked: User-agent: BubblesBot Allow: / To keep it out: User-agent: BubblesBot Disallow: / Put the two lines in a block of their own, outside the * group. Open yourdomain.tld/robots.txt in a browser to check that they are live. When the rules forbid your feed, the blog is deactivated at the next poll and stops appearing on Bubbles. That includes a blanket Disallow: / under User-agent: *, which Bear Blog, Blogspot and some Wordpress privacy plugins set by default. A feed that works in Feedly, Inoreader or NetNewsWire can still be blocked for us, because most feed readers never look at robots.txt. A deactivated blog does not come back by itself when the rule is lifted. Mail us and we will put it back on the list. Contact If BubblesBot causes trouble on your site, or you want your blog removed or re-added, write to hello@bubbles.town.
