Pillow: NULL dereference in ImagingExpand via rank filters on zero-size images

============================================================================
NULL dereference in ImagingExpand via rank filters on zero-size images
Location: src/libImaging/Filter.c:79  in ImagingExpand()
Commit:   a707c6b21c313880d0771e54965d142dd42358b6
============================================================================

Applying Pillow's median, minimum, maximum or rank filter to an image with zero width or zero height crashes the whole Python process instead of raising an error. Empty images like this are easy to produce with ordinary calls, for example a crop box whose left and right edges (or top and bottom edges) are equal, or the second page of a crafted multi-page image file. An attacker who can supply crop coordinates or an uploaded file to an application that runs one of these filters can kill its worker process. The effect is a crash only, with no memory disclosure or corruption.

The fault is in ImagingExpand() in src/libImaging/Filter.c. This is the edge-padding step that RankFilter.filter() always runs through the core expand() method, so MedianFilter, MinFilter and MaxFilter go through it too. It fills the margin by copying the input's edge pixels, reading imIn->image[y][0] and rows 0 / ysize-1, and never checks that the input has any pixels. For an empty image, ImagingAllocateArray leaves the line pointers NULL, so the first read dereferences NULL and the process dies with SIGSEGV, in every mode.

Zero-size images reach this point through public API alone:

• Image.crop(), ImageOps.crop() and Image.new();
• a multi-page TIFF opened by path whose second IFD declares ImageLength = 0, on which seek(1) and load() succeed.

Every other ImageFilter returns an empty image for the same inputs. The zero-height case has crashed since the original PIL import, and the zero-width case since Pillow 4.3.0. This is separate from the margin == 0 fix in #9755: the path is the same, but here the degenerate value is the image size rather than the kernel size. It has the same shape as the zero-dimension ImagingOffset crash fixed under GHSA-f87v-9578-p47w.

The attached reproducer shows a one-pixel-wide control crop filtering normally, while four empty crops passed through the rank filters are each killed by SIGSEGV. A patch that returns early in ImagingExpand for empty input is included.

Details

ImagingExpand() (src/libImaging/Filter.c), the edge-padding helper that ImageFilter.RankFilter runs before the actual rank filter, reads the first pixel of each input line without checking that the input has any pixels. For an image with width 0 or height 0 the line pointers are NULL, so Image.filter() with MedianFilter, MinFilter, MaxFilter or RankFilter kills the interpreter with SIGSEGV instead of returning or raising:

from PIL import Image, ImageFilter
im = Image.new("L", (10, 10)).crop((3, 0, 3, 10))   # size (0, 10); crop() accepts left == right
im.filter(ImageFilter.MedianFilter(3))               # SIGSEGV

Under gdb on the stock build the fault is at ImagingExpand (imIn=…, margin=1) at src/libImaging/Filter.c:105 (the EXPAND(UINT8, image8) expansion), called from _expand_image at src/_imaging.c:1151, with imIn->xsize = 0 and imIn->image[0] = 0x0. The same happens for every mode the rank filters accept (L, RGB, RGBA, I, F), and the core expand(m) method with m >= 1 crashes the same way in any mode, including P. The same input through BoxBlur, GaussianBlur, Kernel, ModeFilter and UnsharpMask returns a (0, 10) image; a (1, 10) crop through MedianFilter also returns normally.

The unchecked edge reads date from the PIL import (9a640e315, 2010, so every Pillow release): the zero-height case has crashed since then, and the zero-width case since fd907fbdc ("Fix 0-width and 0-height images other way", Pillow 4.3.0), which stopped backing zero-width images with a dummy block and left their row pointers NULL. It is adjacent to the margin == 0 divide-by-zero fix in #9755 — same expand() → ImagingExpand() path, but the unhandled degenerate value here is the image size rather than the kernel size.

The path:

# src/PIL/ImageFilter.py:111-116   RankFilter.filter  (MedianFilter/MinFilter/MaxFilter subclass it)
        image = image.expand(self.size // 2)
        return image.rankfilter(self.size, self.rank)

expand(m) with m > 0 is ImagingExpand(self->image, m) (src/_imaging.c:1142-1151). It validates only margin, allocates an output of (xsize+2*margin) × (ysize+2*margin) — which is non-empty, so the allocation succeeds — and then fills the margin by replicating edge pixels of the input:

// src/libImaging/Filter.c:70-101
    imOut =
        ImagingNewDirty(imIn->mode, imIn->xsize + 2 * margin, imIn->ysize + 2 * margin);
    …
#define EXPAND_LINE(type, image, yin, yout)                       \
    {                                                             \
        for (x = 0; x < margin; x++) {                            \
            imOut->image[yout][x] = imIn->image[yin][0];          \   // line 79
        }                                                         \
        …                                                         \
        for (x = 0; x < margin; x++) {                            \
            imOut->image[yout][margin + imIn->xsize + x] =        \
                imIn->image[yin][imIn->xsize - 1];                \   // [-1] when xsize == 0
        }                                                         \
    }

#define EXPAND(type, image)                                                      \
    {                                                                            \
        for (y = 0; y < margin; y++) {                                           \
            EXPAND_LINE(type, image, 0, y);                                      \   // line 0 of a 0-line image
        }                                                                        \
        …
        for (y = 0; y < margin; y++) {                                           \
            EXPAND_LINE(type, image, imIn->ysize - 1, margin + imIn->ysize + y); \   // line -1 when ysize == 0
        }                                                                        \
    }

Why the line pointers are NULL: ImagingNewPrologueSubtype callocs the im->image pointer array (src/libImaging/Storage.c:216, at least one entry), and ImagingAllocateArray returns before assigning any of them for an empty image:

// src/libImaging/Storage.c:423-426
    /* 0-width or 0-height image. No need to do anything */
    if (!im->linesize || !im->ysize) {
        return im;
    }

So with xsize == 0 every imIn->image[y] is NULL and imIn->image[yin][0] faults; with ysize == 0 the top-margin pass reads imIn->image[0] (the single calloc'd NULL slot) and faults the same way, and the bottom-margin pass would index imIn->image[-1]. ImagingRankFilter is never reached.

Zero-size images are ordinary objects in Pillow and arrive at filter() through public API only:

• Image.crop(box) rejects only inverted boxes (src/PIL/Image.py:1385-1390); left == right or upper == lower yields a 0×N / N×0 image. ImageOps.crop(im, border) with border == width // 2, and Image.new(mode, (0, n)), do the same.
• From a file: ImageFile.__init__ refuses a zero first-frame size (src/PIL/ImageFile.py:167-173), but TiffImagePlugin._setup() takes ImageWidth/ImageLength of every later IFD as-is on seek() (src/PIL/TiffImagePlugin.py:1468-1481). For an uncompressed page of an image opened by path, ImageFile.load() takes the mmap branch (src/PIL/ImageFile.py:307, 325-347) and Image.core.map_buffer builds an 8×0 core image without complaint (src/map.c:127-141), so im.seek(1); im.load() succeeds and a following im.filter(ImageFilter.MedianFilter(3)) segfaults in ImagingExpand exactly as above (verified under gdb: imIn->ysize = 0, imIn->image[0] = 0x0). A 384-byte two-page TIFF is enough.

Impact

Denial of service: the whole embedding process (web worker, pipeline process) dies with SIGSEGV rather than the request failing with a Python exception. It is reachable by a remote user in two realistic shapes: (a) an application that crops an upload to user-supplied coordinates and then applies a median/min/max filter (denoise, dilate/erode, thumbnail clean-up) — a box with x0 == x1 or y0 == y1 passes crop() and crashes the worker; (b) an application that opens an uploaded TIFF from disk by path, iterates its pages and applies a rank filter to each — a crafted second IFD with ImageLength = 0 crashes it. The faulting access is a read at address 0 (or a fixed small negative offset from the pointer array for the [-1] cases, which is not reached because the first read already faults), so there is no memory disclosure or corruption beyond the crash. The same shape, a zero-dimension image crashing C code reached through public API, was fixed in ImagingOffset in 39d14a0db (GHSA-f87v-9578-p47w).

Reproducing

repro is a single Python script that uses the installed Pillow as-is (no rebuild). It runs each case in a fresh child interpreter, with core dumps disabled. That way a segfault shows up as the child's exit status and the script itself keeps running. Each child builds Image.new("L", (10, 10)).crop(box), prints its size, calls im.filter(...) and then prints the size of the result.

• Control: a (1, 10) crop ((3, 0, 4, 10)) through MedianFilter(3). The rank filter accepts a thin image and must return normally.
• PoC: the rank filters on empty crops.
  • A (0, 10) crop (left == right, (3, 0, 3, 10)) through MedianFilter(3) and MaxFilter(5).
  • A (10, 0) crop (upper == lower, (0, 3, 10, 3)) through MinFilter(3).
  • A (0, 0) crop ((3, 3, 3, 3)) through RankFilter(3, 4).

Expected output on the stock build:

  crop(3, 0, 4, 10)    MedianFilter(3)    (1, 10) -> (1, 10)
  crop(3, 0, 3, 10)    MedianFilter(3)    (0, 10) -> killed by SIGSEGV
  crop(3, 0, 3, 10)    MaxFilter(5)       (0, 10) -> killed by SIGSEGV
  crop(0, 3, 10, 3)    MinFilter(3)       (10, 0) -> killed by SIGSEGV
  crop(3, 3, 3, 3)     RankFilter(3, 4)   (0, 0) -> killed by SIGSEGV
BUG

The size printed before -> shows that the empty image was created without error. The crash happens inside filter(), in ImagingExpand. The last line is BUG if the control returned normally and at least one rank-filter child was killed by SIGSEGV. Otherwise it is OK. The script covers the crop route only; the TIFF route described above fails the same way (im.seek(1); im.load(); im.filter(ImageFilter.MedianFilter(3)) on a by-path two-page TIFF whose second IFD has ImageLength = 0). With the attached patch, every PoC line prints the unchanged size (for example (0, 10) -> (0, 10)) and the script prints OK.

Suggested fix

The attached patch handles the empty input in ImagingExpand (src/libImaging/Filter.c), right after the margin checks and before the ImagingNewDirty allocation and the replication loops. With xsize == 0 or ysize == 0 there are no edge pixels to replicate, so it returns a zeroed (ImagingNew, not ImagingNewDirty) image of the expanded size, with the input's palette copied. This keeps ImagingExpand's size contract, so ImagingRankFilter then produces an output of xsize - 2*margin × ysize - 2*margin, i.e. the original empty size, matching every other filter. It mirrors the existing zero-dimension early returns in src/libImaging/BoxBlur.c (127004368) and src/libImaging/Offset.c (39d14a0db):

--- a/src/libImaging/Filter.c
+++ b/src/libImaging/Filter.c
@@ -67,6 +67,17 @@ ImagingExpand(Imaging imIn, int margin) {
         return (Imaging)ImagingError_ValueError("filter size too large");
     }
 
+    if (imIn->xsize == 0 || imIn->ysize == 0) {
+        /* No edge pixels to replicate (and no lines to read from), so
+           return a zeroed image of the expanded size */
+        imOut = ImagingNew(imIn->mode, imIn->xsize + 2 * margin, imIn->ysize + 2 * margin);
+        if (!imOut) {
+            return NULL;
+        }
+        ImagingCopyPalette(imOut, imIn);
+        return imOut;
+    }
+
     imOut =
         ImagingNewDirty(imIn->mode, imIn->xsize + 2 * margin, imIn->ysize + 2 * margin);
     if (!imOut) {

The patch also adds test_rankfilter_zero_dimension next to test_rankfilter_error in Tests/test_image_filter.py, which filters Image.new("L", size) for size in ((0, 6), (6, 0), (0, 0)) with MinFilter, MedianFilter, MaxFilter and RankFilter(3, 2) and asserts the size is preserved. Applied to a clean checkout and rebuilt, the reproducer prints OK and Tests/test_image_filter.py passes. A Python-level guard in RankFilter.filter (if 0 in image.size: return image.copy()) would also stop the crash, but the C-level check protects any other caller of expand().

The patch

Return a zeroed image from ImagingExpand when the input has zero width or height instead of reading NULL line pointers

Applies to src/libImaging/Filter.c at the commit above.

diff --git a/Tests/test_image_filter.py b/Tests/test_image_filter.py
index 974d4b7a3..8dc8e1a1a 100644
--- a/Tests/test_image_filter.py
+++ b/Tests/test_image_filter.py
@@ -155,6 +155,23 @@ def test_rankfilter_error(filter: ImageFilter.RankFilter) -> None:
         im.filter(filter).getpixel((1, 1))
 
 
[email protected](
+    "filter",
+    (
+        ImageFilter.MinFilter,
+        ImageFilter.MedianFilter,
+        ImageFilter.MaxFilter,
+        ImageFilter.RankFilter(3, 2),
+    ),
+)
[email protected]("size", ((0, 6), (6, 0), (0, 0)))
+def test_rankfilter_zero_dimension(
+    filter: ImageFilter.RankFilter, size: tuple[int, int]
+) -> None:
+    im = Image.new("L", size)
+    assert im.filter(filter).size == size
+
+
 def test_rankfilter_properties() -> None:
     rankfilter = ImageFilter.RankFilter(3, 2)
 
diff --git a/src/libImaging/Filter.c b/src/libImaging/Filter.c
index 3492a871f..d702db5f9 100644
--- a/src/libImaging/Filter.c
+++ b/src/libImaging/Filter.c
@@ -67,6 +67,17 @@ ImagingExpand(Imaging imIn, int margin) {
         return (Imaging)ImagingError_ValueError("filter size too large");
     }
 
+    if (imIn->xsize == 0 || imIn->ysize == 0) {
+        /* No edge pixels to replicate (and no lines to read from), so
+           return a zeroed image of the expanded size */
+        imOut = ImagingNew(imIn->mode, imIn->xsize + 2 * margin, imIn->ysize + 2 * margin);
+        if (!imOut) {
+            return NULL;
+        }
+        ImagingCopyPalette(imOut, imIn);
+        return imOut;
+    }
+
     imOut =
         ImagingNewDirty(imIn->mode, imIn->xsize + 2 * margin, imIn->ysize + 2 * margin);
     if (!imOut) {

The reproducer

<details> <summary>Reproducer — <code>repro</code>, 1,333 bytes (click to open)</summary>

Its last line is BUG when the issue triggers and OK when it does not. Re-run on a clean build of that commit it printed BUG.

#!/usr/bin/env python3
# Pillow: rank filters (Median/Min/Max/RankFilter) on a 0-wide or 0-high image
# SIGSEGV in ImagingExpand (src/libImaging/Filter.c) - NULL line pointer read.
# Each case runs in a child interpreter; BUG if the control returns and any
# zero-size case dies of SIGSEGV.
import signal, subprocess, sys

CHILD = """
import resource; resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
from PIL import Image, ImageFilter
im = Image.new("L", (10, 10)).crop(%s)      # public API; crop accepts x0==x1 / y0==y1
print(im.size, "->", end=" ", flush=True)
print(im.filter(ImageFilter.%s).size)
"""

def run(box, flt):
    p = subprocess.run([sys.executable, "-c", CHILD % (box, flt)],
                       capture_output=True, text=True)
    out = p.stdout.strip()
    if p.returncode < 0:
        out += " killed by " + signal.Signals(-p.returncode).name
    print(f"  crop{box!s:<16} {flt:<18} {out}")
    return p.returncode

ctrl = run((3, 0, 4, 10), "MedianFilter(3)")          # (1,10): must work
rcs = [run((3, 0, 3, 10), "MedianFilter(3)"),        # (0,10)
       run((3, 0, 3, 10), "MaxFilter(5)"),
       run((0, 3, 10, 3), "MinFilter(3)"),           # (10,0)
       run((3, 3, 3, 3), "RankFilter(3, 4)")]        # (0,0)
bug = ctrl == 0 and any(rc == -signal.SIGSEGV for rc in rcs)
print("BUG" if bug else "OK")

</details>

---