What content negotiation is
One URL, many representations. The server picks which bytes to send based on the client's Accept header.
Content negotiation is a mechanism defined in HTTP that lets a single URL serve different representations of the same resource, and lets the server pick which one to send based on what the client asked for.
The classic example is language: one URL /about, and the server returns
English, French, or German based on Accept-Language. The same machinery
applies to media types via Accept — which is how a single URL can serve
HTML to a browser and Markdown to an agent.
The request side
Every HTTP request can carry an Accept header listing the media types
the client can handle, ranked by preference:
GET /article HTTP/1.1
Host: example.com
Accept: text/markdown, text/html;q=0.8
That says: I prefer Markdown; I’ll take HTML at 80% preference if you
can’t do Markdown. The q value is a quality factor from 0 to 1. No q
means q=1 (maximum preference).
The response side
The server picks the representation that best matches, sets
Content-Type to describe what it actually sent, and sets Vary: Accept
so downstream caches know the response depends on the request’s Accept
header:
HTTP/1.1 200 OK
Content-Type: text/markdown; charset=utf-8
Vary: Accept
# How content negotiation works
One URL, many representations…
If the server cannot satisfy the request — it doesn’t have any of the
types the client will accept — it returns
406 Not Acceptable.
Why it matters now
Browsers and LLM agents want fundamentally different things from the same URL. Browsers want HTML with navigation, CSS, and scripts. Agents want the text, clean, without the chrome. Content negotiation is the only mechanism that lets the canonical URL serve both, without fragmenting the site into parallel URLs.
What it’s not
- It’s not
.mdsibling files at parallel URLs. Sibling files are fine to ship — they’re a separate URL, not the canonical one serving two representations. On their own, they leave the canonical URL returning HTML to any agent that asks. If you ship siblings, advertise them via aLink: rel="alternate"header (RFC 8288) so agents can discover them. And if you want the canonical URL itself to serve Markdown when requested, implement theAcceptheader. You can do both. - It’s not about the content being different — it’s the same content, in a different format.
References
- MDN — Content negotiation — canonical overview, the companion to this guide.
- RFC 9110 §12.5.1 — Proactive Negotiation — the normative spec.