Aggressive caching for a Mastodon reverse proxy: what to cache, what to never cache, and why content negotiation will eventually betray you

I have written before about putting a cache in front of snac, and more recently about the HAProxy layer in front of FediMeteo. The general idea is always the same: the reverse proxy should absorb the repetitive, public work that has no business reaching the application server. This post is the same idea applied to a much louder neighbour: a Mastodon instance. The instance is mastodon.bsd.cafe, the proxy is nginx on FreeBSD, and the configuration below is what I am currently running in production. Mastodon is heavier than snac in every direction. It has Puma and Sidekiq behind it, more endpoints, more streaming, more federation patterns, and one specific characteristic that complicates everything: it serves multiple representations on the same URLs. The same path returns HTML to a browser, ActivityPub JSON to a remote instance, and sometimes plain JSON to an API client. If the proxy treats the URL as one thing, sooner or later it will return the wrong thing to the wrong client. Most of…