So You Want To Define a Well-Known URI

As one of the authors of the Well-Known URI specification and current Designated Expert for the registry, I field a lot of questions about how they should be used, and end up coaching a lot of folks on how to best use them. Below, I’ve summarised how I think about them. Note that these aren’t all requirements for registration – just what I consider good practice. What Well-Known URIs are Good For Well-known URIs work best when the client – whether it’s a browser, bot, or other software – knows the site1 and needs to discover something about the whole site in an efficient way. robots.txt is the perfect example – it pre-dated the RFC so it doesn’t use well-known URIs, but was a major part of the reason we reserved a space for them. A crawler needs to know what the access policies for the site are, and putting it in one central place for the site avoids the need to check headers and content on every response (which would defeat many of the purposes of having such a policy). A well-known…