--- title: "The .well-known Directory: The Web's Front Desk for Machines" description: "What /.well-known/ is (RFC 8615), what already lives there (security.txt, OAuth, ACME), its agent-era role for MCP, and why robots.txt and llms.txt sit at root." canonical: https://aiovsseo.com/articles/well-known-directory.html date: 2026-06-07 --- # The `.well-known` directory: the web's front desk for machines TL;DR Every domain has a reserved location — `/.well-known/` — where machines look for your capabilities and policies *without guessing*. Defined by RFC 8615, it already quietly powers your SSL certificate and OAuth logins. In the agent era it's becoming the place to advertise machine-facing capabilities, conventionally via `/.well-known/mcp.json`. Note: `robots.txt`, `llms.txt` and `sitemap.xml` live at the **root**, not here — a common confusion. There's a folder on your website you've probably never opened, yet it's doing essential work right now — proving you own your domain, telling browsers where to find your login config, maybe listing your security contact. It's `/.well-known/`, and it's the closest thing the web has to a standardized front desk where machines come to ask, "how do I deal with this site?" ## What .well-known actually is The `.well-known` directory is a path prefix reserved by **RFC 8615** (which obsoleted the earlier RFC 5785). The idea is simple and powerful: reserve one predictable location so that protocols and tools can **probe a site for capabilities without guessing URLs or polluting the root namespace**. Instead of every standard inventing its own path and hoping it doesn't collide with your pages, they all live under one roof, and **IANA maintains a public registry** of approved well-known names. The principle is *predictable discovery*: a machine that wants to know something standard about your site knows exactly where to look. ## What's already living there (you rely on it daily) This isn't new or exotic — you almost certainly use several of these without thinking: | Well-known URI | What it does | | --- | --- | | `/.well-known/acme-challenge/` | Proves domain ownership when issuing SSL certificates (every Let's Encrypt cert — including this site's — uses it) | | `/.well-known/security.txt` | Tells security researchers how to report a vulnerability | | `/.well-known/openid-configuration` | OAuth / OpenID Connect discovery for logins | | `/.well-known/change-password` | Standard redirect so password managers can find your reset page | | `/.well-known/assetlinks.json` & `apple-app-site-association` | Mobile deep linking for Android/iOS | None of these are content. They're **machine-to-machine contracts** — and they've worked invisibly for years. The agent era is simply extending the same pattern to new kinds of machines. ## .well-known in the agent era As the [machine layer](/articles/machine-to-machine-mcp-layer.html) forms, `.well-known` becomes a natural discovery point for the capabilities agents care about. ### The MCP manifest If you expose a [Model Context Protocol](/glossary/model-context-protocol.html) server, the convention is to advertise it at `/.well-known/mcp.json` — a manifest telling compatible agents what capabilities you offer and how to connect. It's the agent-era equivalent of `openid-configuration`: a predictable place for a machine to learn how to work with you. ### A lesson from a deprecated ancestor It's worth remembering that conventions churn. `/.well-known/ai-plugin.json` was the manifest behind ChatGPT's original plugin system — and it was sunset as that ecosystem gave way to other models. The takeaway: treat emerging AI well-known files as useful-but-provisional. Adopt them when cheap, don't bet the strategy on any single one. ## The confusion worth clearing up Plenty of important files are **not** in `.well-known` — they live at the site **root**: - `/robots.txt` — crawl rules and [Content-Signal](/glossary/content-signal.html) - `/sitemap.xml` — your URL list - `/llms.txt` — the (optional) AI site map; root, not [well-known](/glossary/llms-txt.html) - your `/{indexnow-key}.txt` — the IndexNow verification file Mixing these up is a common mistake. Rule of thumb: **registered standard protocols** (SSL, OAuth, security contacts, MCP) go in `/.well-known/`; **web-crawl conventions** (robots, sitemap, llms.txt) sit at the root. ## Practical guidance 1. **Audit what's already there.** Visit `/.well-known/` on your domain. You'll likely find `acme-challenge`; check for stale or unexpected entries. 2. **Add `security.txt`.** A cheap, standard trust signal that tells researchers how to reach you responsibly. 3. **Advertise machine capabilities deliberately.** If you run an MCP server or agent API, publish `/.well-known/mcp.json` — but only expose what you mean to (start read-only; see [governing access](/articles/governing-crawler-access.html)). 4. **Keep it minimal and safe.** RFC 8615 itself flags the risks: exposure of sensitive data, denial-of-service, and DNS-rebinding attacks. `.well-known` is a *public* surface — never put secrets or internal config there. 5. **Don't confuse it with the root.** Put robots, sitemap and llms.txt at `/`; put registered protocols in `/.well-known/`. > For decades, `.well-known` quietly answered one question for machines: "how do I deal with this site?" In the agent era, more and more machines are asking — and your answer is whatever you've chosen to put there. ## Frequently asked questions **What is the .well-known directory?** A reserved path on a website (/.well-known/) defined by RFC 8615. It gives protocols and tools a predictable place to find a site's capabilities and policies — SSL validation, security contacts, OAuth config — without guessing URLs or cluttering the root. IANA maintains a public registry of approved well-known names. **Is llms.txt in the .well-known directory?** No. llms.txt is served at the site root (/llms.txt), like robots.txt, sitemap.xml and your IndexNow key file. The .well-known directory is for registered well-known URIs like security.txt, acme-challenge and openid-configuration. The MCP manifest, by contrast, conventionally sits at /.well-known/mcp.json. **Do I need to add anything to .well-known for AI agents?** Only if you expose machine capabilities. Google says you don't need special AI files for generative search, so .well-known isn't required for citation. But if you run an MCP server or API, advertising it at /.well-known/mcp.json helps agents discover it. Keep it a deliberate, minimal public surface — never a place for sensitive config.