Issue 1: EXIF orientation not normalized → The image orientation processed by the model differs from how humans view it, introducing interpretation bias.
Issue 2: PNG tRNS not explicitly flattened before converting to RGB → After conversion, transparent/semi-transparent pixels are rendered unexpectedly, making otherwise subtle overlay elements visible and distorting the input content. (This attack is similar to AlphaDog: RGBA handling is already correct in vLLM, but since tRNS permits RGB images, the correct processing path isn’t taken.)
Issue 3 : Pillow only loads the first frame when loading APNG or GIF files.
ImageOps.exif_transpose is not called to normalize EXIF orientation.tRNS in P/L/RGB + tRNS and other non-RGBA modes take the image.convert("RGB") path, which implicitly discards/remaps transparency semantics.https://github.com/vllm-project/vllm/blob/16b37f3119918c1e5a39f303e0d0892c65c07a90/vllm/multimodal/image.py#L77-L84
https://github.com/vllm-project/vllm/blob/16b37f3119918c1e5a39f303e0d0892c65c07a90/vllm/multimodal/image.py#L37-L43
https://github.com/vllm-project/vllm/blob/16b37f3119918c1e5a39f303e0d0892c65c07a90/vllm/multimodal/image.py#L26-L34
Current state:
ImageOps.exif_transposeis not used. (Although therescale_image_sizefunction (https://github.com/vllm-project/vllm/blob/main/vllm/multimodal/image.py#L14) exists and includes atransposeparameter, I’ve found that it doesn’t seem to be called anywhere outside thetestdirectory.)
Call order:
_convert_image_moderuns first; if the conditions are met,convert_image_modeis called.Issue: Only the “RGBA → RGB” path is explicitly flattened.
P,L, orRGBwithtRNSall fall back toimage.convert("RGB"). For PNGs that includetRNS,convert("RGB")directly produces 24-bit RGB, leading to:
Pmode: The transparent index becomes an actual RGB color (often black, white, or an undefined background), so transparency is lost.L/LAandRGB + tRNS:convert("RGB")doesn’t composite against a chosen background first, so elements that relied on transparency to be hidden or softened become solid.
vllm/multimodal/image.py. The existing RGBA→RGB flattening is correct; the issues center on missing EXIF normalization and non-RGBA tRNS not being explicitly composited.EXIF: http://qiniu.funxingzuo.top/exiforient180.jpg tRNS: http://qiniu.funxingzuo.top/hello.png
A fix for this vulnerability was merged here: https://github.com/vllm-project/vllm/pull/44974