Traefik entrypoint header-name sanitization bypassed via request trailers

Details

Root cause. All four entrypoint handlers iterate req.Header only — the doc comments promise more than the code does (pkg/server/server_entrypoint_tcp.go):

// removeAliasingHeaders removes any request header and trailer whose name contains a character
// which is neither a letter, a digit, nor a dash, as such a name aliases another header name.
func removeAliasingHeaders(h http.Handler) http.Handler {
	return http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
		for key := range req.Header {          // ← req.Trailer is never scanned
			if isAliasingHeaderName(key) {
				delete(req.Header, key)
			}
		}
		h.ServeHTTP(rw, req)
	})
}

rejectAliasingHeaders, removeHeadersWithUnderscores and rejectHeadersWithUnderscores share the identical structure (the reject variants return 400 from the same loop). The sibling sanitization forwardedheaders.DeleteXForwardedHeaders (pkg/middlewares/forwardedheaders/forwarded_header.go) also scans req.Header only, so the trusted X-Forwarded-* names whose header form Traefik strips for untrusted clients — the managed XHeadersSet, which includes X-Forwarded-Prefix and X-Forwarded-For — survive in trailer form. Go's HTTP server populates req.Trailer from chunked/HTTP/2 trailers, and Traefik's proxy layer forwards those entries, bypassing the sanitization above.

Contract provenance. The "header and trailer" wording is in the original introducing diffs — 108a52644 (underscoreHeadersStrategy) and 0331801c (aliasHeadersStrategy) — and is unchanged in master (full diff excerpts available on request). The option began as allowHeadersWithUnderscores: false (per the CVE-2026-54763 record) before becoming underscoreHeadersStrategy and then aliasHeadersStrategy. The user-facing documentation describes only "request headers".

Mechanism (why names survive, and when values do too).

1. Name pre-fill at parse time. The client's Trailer: X_Auth_User declaration makes Go's server move the declared keys into req.Trailer with nil values before the handler runs (net/http/transfer.go, fixTrailer); HTTP/2 does the same from the trailer: field in the initial HEADERS ("Setup Trailers", net/http/internal/httpcommon/httpcommon.go). The entrypoint handlers therefore cannot see the trailer name, but the proxy forwards it. Trailer keys are canonicalized with textproto.CanonicalMIMEHeaderKey, which treats dashes — not underscores — as case separators: the aliasing spelling survives canonicalization as e.g. X_auth_user (visible in the backend dumps in PoC §1) and remains detectable by isAliasingHeaderName, so the fix does not depend on the client's original spelling.
2. Value survival depends on who reads the body first. Trailer values are appended to req.Trailer only while the body is consumed (readTrailer / copyTrailersToHandlerRequest). On the bare path the reverse proxy calls Request.Clone at handler start, before any body read, so the clone captures nil values — on HTTP/1.1 the trailer field line is then omitted entirely (net/http/header.go, Header.writeSubset writes one line per value), and h2c delivers only the empty key. When a body-buffering middleware runs first, the order reverses: the retry middleware with status codes buffers the body via mirror.NewReusableRequest → io.ReadAll(req.Body) (pkg/middlewares/retry/retry.go, pkg/server/service/loadbalancer/mirror/mirror.go), the values are populated before http.Request.Clone, and they travel to the backend. Buffering triggers for idempotent methods with status alone; POST additionally requires retryNonIdempotentMethod (both measured). Retry and buffering are the two measured paths; the mirroring and failover services use the same mirror.NewReusableRequest helper (pkg/server/service/loadbalancer/mirror/mirror.go, failover/failover.go when errors.status is configured) and share its behavior (not measured). The buffering middleware drains the body eagerly before the proxy too: pkg/middlewares/buffering/buffering.go → oxy's multibuf.New → ioutil.ReadAll (github.com/mailgun/multibuf buffer.go; unset limits fall back to 1 MB DefaultMemBytes) — measured value-preserving with default limits.
3. Undeclared trailers: transit depends on whether anything else was declared (measured). On HTTP/2 the standard library server copies only pre-declared trailers ("Only copy it over it was pre-declared", net/http/internal/http2/server.go) — undeclared fields never appear. On HTTP/1.1 readTrailer parses the entire trailer section with no declaration filter, and mergeSetHeader either rebinds the map when nil (*dst = src) or blindly merges when non-nil (point 4). The rebind is why zero-declaration requests lose undeclared fields at Traefik's observability req.WithContext shallow copy (pkg/middlewares/observability/observability.go, entrypoint.go) — measured: they never leave the entrypoint even on buffered chains. But a bait declaration (any clean name, e.g. X-Dummy) keeps the map non-nil, and the blind merge then writes the undeclared field into the shared map at body EOF — measured on the retry-buffered chain: the backend receives map[X-Dummy:[1] X_auth_user:[attacker-value]] and presence-based policies flip; the bare path is unaffected and delivers only map[X-Dummy:[]].
4. Delete-mode stickiness depends on the merge semantics (measured). readTrailer merges parsed trailer fields via mergeSetHeader, whose non-nil branch is a blind maps.Copy (net/http/transfer.go) — a key deleted by a handler is re-added with its value at body EOF. Measured on a bare Go server (delete(r.Trailer, "X_auth_user") before draining the body): HTTP/1.1 — map[X_auth_user:[]] → map[X_auth_user:[attacker-value]] (re-added); HTTP/2 — map[X_auth_user:[]] → map[] (stays deleted: copyTrailersToHandlerRequest checks the live map).

Deliberate trailer-forwarding behavior (regression tests). Traefik deliberately does not forward request trailers on the bare proxy chain, locked by the regression tests pkg/proxy/httputil/trailer_test.go and pkg/proxy/fast/trailer_test.go (added 86b5642f, 2026-06-25; extended d427dccf, 2026-06-29): "trailers arrive after the body, once routing and security decisions have already been made, so forwarding them could raise security concerns in Traefik." The measured buffered-chain value survival (mechanism point 2) defeats exactly that locked invariant — the tests exercise only the bare chain — and the name-level h2c forwarding (empty keys) passes the tests' assertion (Header.Get is empty whether the key is absent or empty-valued): neither regression test catches this finding. The value-level path thus bypasses a deliberate, test-locked security invariant.

Preconditions.

1. An entrypoint whose sanitization is relied upon: aliasHeadersStrategy / underscoreHeadersStrategy set to delete or reject, or the default forwardedHeaders stripping of X-Forwarded-* for untrusted clients.
2. A request carrying the name as a declared trailer (HTTP/1.1 chunked, or HTTP/2), or — on HTTP/1.1 buffered chains only — as an undeclared trailer field riding a bait declaration (mechanism point 3).
3. For downstream impact: a backend that merges trailers into its header namespace (pre-fix libevent CVE-2026-63379 — still what Ubuntu 24.04 ships —, pre-fix blaze CVE-2026-73495, or custom code) or consumes trailer fields in a trust decision.
4. For the value-level path: the retry middleware with status codes, the buffering middleware, or another body-buffering middleware, in the chain.

Precedent and scope. This is the next variant of Traefik's own aliasing family — CVE-2026-33433 (GHSA-qr99-7898-vr7c), CVE-2026-39858 (GHSA-5m6w-wvh7-57vm), CVE-2026-54763 (GHSA-x677-9fxg-v5c5) — and Traefik's Security Decisions state the in-scope line: "a spelling that survives the entrypoint sanitisation and still reaches the backend as the trusted name". The trailer spelling is precisely that. The downstream merge class is cross-ecosystem: libevent CVE-2026-63379 (run live in PoC §3) and blaze/http4s CVE-2026-73495 (GHSA-46q4-43ph-c6fr, fixed ef3e666).

Boundaries (measured). Declaring Content-Length, Transfer-Encoding or Trailer as trailer fields is rejected with 400 by Go's server; Host and Connection pass through name-level. The FastProxy forwarding mode (opt-in [experimental] fastProxy) does not forward trailers; the default reverse-proxy path for http:// backends does (PoC §3). HTTP/3 (quic-go) trailer semantics are untested. Undeclared trailers: HTTP/2 drops them entirely; on HTTP/1.1 they transit only via a bait declaration on buffered chains (mechanism point 3).

PoC

Verified against a source-built Traefik (master @ 237f13c6, built with Go 1.27.0; all harness backends built with Go 1.27.0 — the trailer behaviors cited in Details are version-sensitive net/http internals). Complete harness (clients, backends, configs, logs) available on request; the raw chunked requests below are HTTP/1.1 and reproducible with nc/python.

1. Core bypass (aliasHeadersStrategy = reject). Static config:

[entryPoints.web]
  address = ":8090"
  [entryPoints.web.http]
    aliasHeadersStrategy = "reject"

[providers.file]
  filename = "dynamic.toml"
  watch = true

dynamic.toml: router PathPrefix(/) → service → h2c://127.0.0.1:8081 (a Go echo backend that drains the body and prints r.Trailer). Requests (CRLF line endings; 5/0 are chunk sizes):

POST / HTTP/1.1
Host: 127.0.0.1:8090
Connection: close
Transfer-Encoding: chunked
Trailer: X_Auth_User

5
hello
0
X_Auth_User: attacker-value

Results:

header  X_Auth_User   (curl -H "X_Auth_User: x")  → HTTP 400  (rejected, as designed)
trailer X_Auth_User   (request above)             → HTTP 200  (bypass: not rejected)
trailer X.Auth.User                                → HTTP 200  (bypass)
trailer X-Forwarded-Prefix                         → HTTP 200  (trusted-name trailer passes)

Backend evidence: TRAILERS: map[X_auth_user:[]], map[X.auth.user:[]], map[X-Forwarded-Prefix:[]]. The deprecated underscoreHeadersStrategy = "reject" behaves identically.

aliasHeadersStrategy = "delete" (same setup, delete in place of reject): header X_Auth_User / X.Auth.User → 200, backend HEADERS contain neither (deleted, as designed); trailer X_Auth_User → 200, backend TRAILERS: map[X_auth_user:[]] — the trailer form survives delete.

2. Bare-path downstream semantics (name-level). Same router, backend h2c://127.0.0.1:8082 running a trailer-merging backend (trailers folded over headers, CGI-style name normalization — the CVE-2026-63379 pattern) that authorizes /presence on the merged key and /value on X-Auth-User == "admin":

/presence, no trailer (control)                  → 403 DENIED
/presence, trailer X_Auth_User                   → 200 AUTHORIZED      ← presence flip, empty value
/value,    header X-Auth-User: admin + trailer   → merged-user=""       ← legitimate value erased

3. Real CVE'd component flipped through Traefik — value-level. Backend: Ubuntu 24.04's libevent-2.1-7t64 2.1.12-stable-9ubuntu2 (pre-fix; the merge was fixed only in 2.1.13) plus a small (≈100-line) evhttp server that authorizes via evhttp_find_header(req->input_headers, ...) (/prefix grants admin when X-Forwarded-Prefix == "admin"; server source available on request; build: gcc server.c -levent). Router adds the retry middleware:

[http.routers.lib]
  entryPoints = ["web"]
  rule = "PathPrefix(`/`)"
  service = "lib"
  middlewares = ["retry-lib"]

[http.middlewares.retry-lib.retry]
  attempts = 2
  status = ["500-599"]

[http.services.lib.loadBalancer.servers]
  [http.services.lib.loadBalancer.servers.s1]
    url = "http://127.0.0.1:8083"

Measured matrix (server log shows the merged input_headers):

The value survives because the retry middleware buffers the body before the proxy clone (mechanism point 2). The same value path holds on h2c outbound (merge backend logs trailer=map[X-Auth-User:[admin]] → 200 AUTHORIZED (value=admin)) and with the buffering middleware in place of retry (/value trailer → 200 AUTHORIZED (value=admin), /xff trailer → 200 ADMIN).

Bait declaration (measured). Declaring a clean Trailer: X-Dummy while additionally sending the undeclared X_Auth_User: attacker-value in the trailer section: on the retry-buffered chain the backend receives TRAILERS: map[X-Dummy:[1] X_auth_user:[attacker-value]] → 200 AUTHORIZED (presence policies flip on X_auth_user); the zero-declaration control still delivers map[]; the bare path delivers only map[X-Dummy:[]] (clone precedes the merge). Over HTTP/2 inbound with buffering (client_h2c through a retry chain with retryNonIdempotentMethod): backend trailer=map[X_auth_user:[attacker-value]] → 200 AUTHORIZED (presence).

X-Forwarded-For IP-trust (same chain, measured). Same router and retry middleware, backend h2c://127.0.0.1:8082 running the merge backend with an added /xff route that grants access when the merged X-Forwarded-For equals 203.0.113.7 — the classic IP-allowlist pattern:

header  X-Forwarded-For: 203.0.113.7                  → 403 DENIED (xff)
        backend log: merged XFF = "127.0.0.1"  (Traefik stripped the client value and set its own)
trailer X-Forwarded-For: 203.0.113.7  (GET, retry)    → 200 ADMIN (xff)
        backend log: trailer=map[X-Forwarded-For:[203.0.113.7]]
trailer X-Forwarded-For: 203.0.113.7  (POST, retry without retryNonIdempotentMethod → not buffered) → 403 DENIED — merged XFF empty (bare-path value drop)

4. Framing names and protocols. Trailer Content-Length, Host, Connection, Transfer-Encoding → 400 (Go rejects); Host, Connection → 200, backend TRAILERS: map[Connection:[] Host:[]]. HTTP/2 prior-knowledge client with trailer X_Auth_User → Traefik → h2c backend: 200, backend TRAILERS: map[X_auth_user:[]] — same name-only outcome as PoC §2. HTTP/3 untested.

Impact

Kind of vulnerability. A bypass of Traefik's documented defenses against spoofed trusted names. reject promises a 400 and delete promises removal for aliasing names; forwardedHeaders strips client-supplied X-Forwarded-* — and all of it applies to headers only, leaving the trailer channel open, with attacker-chosen values on body-buffering chains.

Who is impacted. Operators who enabled delete/reject to close the aliasing spoofing class (the documented mitigation for the CVE-2026-33433/39858/54763 family), and deployments whose backends trust X-Forwarded-* names or proxy-set identity headers — including the classic X-Forwarded-For IP-trust pattern, where Traefik strips the client's XFF from headers while the trailer form reaches trailer-merging backends. No opt-in option is required for the X-Forwarded-* path: the stripping is the default for untrusted clients. The value-level path additionally requires a body-buffering middleware (retry with status codes, or buffering) — mainstream documented features: the buffering middleware's documentation states that attaching it buffers the request body before forwarding, and the retry middleware's documentation example configures status = ["400","500-599"] — though no deployment telemetry is available to quantify their prevalence.

Verified harm scenarios.

1. Broken protection contract. Trailer-form aliasing names are neither rejected nor removed — the documented mitigation has a side door the operator believes is closed.
2. Presence-based authorization bypass. Trailer-merging upstreams authorizing on the presence of a trusted identity key flip their decision: 403 → 200 AUTHORIZED through Traefik on an h2c merge backend (PoC §2) and on the real pre-fix libevent component (PoC §3).
3. Value-level identity spoofing. On body-buffering chains the trailer carries the attacker's value: X-Forwarded-Prefix: admin delivered through Traefik authorizes as admin on the real CVE'd merge backend, while the identical header form is stripped and denied (PoC