ImageMagick CLAHE : Unsigned underflow and division-by-zero lead to OOB pointer arithmetic and process crash (DoS)

Root cause

Missing input validation / bounds checks after computing default tile dimensions:

If either tile_info.width or tile_info.height is 0, this triggers a division by zero. Zeros can reach this point through:

1. Exact tiles: CLI clahe 0x0! (the ! forces zero to be used verbatim).
2. Auto tiles on tiny images: When a requested tile is 0 (no !), the code derives a default from the image size (e.g., dim >> 3). For images with dim < 8, this result is 0 unless clamped.

Reproduction

Unsigned underflow

Environment

Built with AddressSanitizer and UndefinedBehaviorSanitizer enabled.

export UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1
export ASAN_OPTIONS=abort_on_error=1:allocator_may_return_null=1:detect_leaks=0

Command

./magick xc:black -clahe 0x0 null:

Output

MagickCore/enhance.c:609:6: runtime error: addition of unsigned offset overflowed
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior MagickCore/enhance.c:609:6 in CLAHEImage

./magick -size 10x10 xc:black -clahe 0x0 null:

memory region corruption.

./magick -size 2000x2000 xc:black -clahe 0x0 null:

→ Significant memory consumption and evidence of memory region corruption.

./magick -size 4000x4000 xc:black -clahe 0x0 null:

→ Much larger memory usage; process appears to be aggressively consuming cache and address space.

./magick -size 8000x8000 xc:black -clahe 0x0 null:

→ Memory usage escalates further and begins exhausting available cache. If left running, the process is likely to crash (DoS) after sustained allocation attempts.

Division-by-zero

Environment: ASan/UBSan-enabled build.

export UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1
export ASAN_OPTIONS=abort_on_error=1:allocator_may_return_null=1:detect_leaks=0

Command

./magick -size 16x2 gradient: -type TrueColor -depth 8 -clahe 0x0! null:

Output

Notes: Without sanitizers, the process may terminate with just Aborted (still DoS).

Impact

• Primary: Denial-of-Service — crash or sustained resource exhaustion (memory/cache thrash) when processing crafted parameters or small images via CLI or API. Attackers can trivially trigger via clahe 0x0! or by uploading very small images to services using ImageMagick.
• Secondary (theoretical): OOB memory accesses and memory corruption could potentially be combined with other vulnerabilities to achieve more severe outcomes; however, no reliable code execution was demonstrated from these PoCs alone.

Suggested concrete patch snippets

Apply in CLAHEImage() after tile_info is computed but before any division/modulus/pointer arithmetic:

if (exact_tiles_requested && (tile_info.width == 0 || tile_info.height == 0)) {
  ThrowMagickException(exception, GetMagickModule(), OptionError,
                       "CLAHEInvalidTile", "%lux%lu",
                       (unsigned long) tile_info.width,
                       (unsigned long) tile_info.height);
  return (Image *) NULL;
}

if (!exact_tiles_requested) {
  tile_info.width  = (tile_info.width  == 0) ? MagickMax((size_t)1, image->columns >> 3) : tile_info.width;
  tile_info.height = (tile_info.height == 0) ? MagickMax((size_t)1, image->rows    >> 3) : tile_info.height;
}

if (tile_info.width == 0 || tile_info.height == 0) {
  ThrowMagickException(exception, GetMagickModule(), OptionError,
                       "CLAHEInvalidTile", "%lux%lu",
                       (unsigned long) tile_info.width,
                       (unsigned long) tile_info.height);
  return (Image *) NULL;
}

ssize_t tile_h_minus1 = (ssize_t)tile_info.height - 1;
if (tile_h_minus1 < 0) {
  ThrowMagickException(exception, GetMagickModule(), OptionError,
                       "CLAHEInvalidTile", "%lux%lu",
                       (unsigned long) tile_info.width,
                       (unsigned long) tile_info.height);
  return (Image *) NULL;
}
p += (ptrdiff_t) clahe_info->width * tile_h_minus1;

Notes about exact_tiles_requested: if the CLI/Wand parser already exposes whether ! was present, use it. If not, add a parse-time flag so CLAHEImage can know whether 0 is literal or auto.

Credit

Team Whys

Bug Hunting Master Program, HSpace/Findthegap

Youngmin Kim [email protected]

Woojin Park

@jin-156 [email protected]

Youngin Won

@amethyst0225 [email protected]

Siyeon Han

@hanbunny [email protected]

Shinyoung Won

@yosiimich [email protected]