RabbitMQ: JSONReader in the default JSON-RPC mapper never terminates on truncated input, causing DoS

Summary

com.rabbitmq.tools.json.JSONReader.read() never returns when its input ends inside a quoted string or a // line comment. Both scanners walk the input with StringCharacterIterator.next() but only compare against a delimiter, so once the iterator reaches CharacterIterator.DONE (￿) they loop forever. The string scanner (string(), line 210, while (c != sep)) also appends ￿ to a StringBuilder every iteration, so it fills the heap and throws OutOfMemoryError, taking down the JVM. The comment scanner (skipWhiteSpace(), lines 89-92, while (c != '\n')) pins a thread at 100% CPU with no allocation.

This is reachable with a single message. JsonRpcServer and JsonRpcClient fall back to DefaultJsonRpcMapper whenever no mapper is passed (JsonRpcServer.java:84 and :114, JsonRpcClient.java:186), and that mapper hands the raw message body straight to JSONReader.read() (DefaultJsonRpcMapper.java:42 for the server request, :52 for the client reply). A caller that can publish to the RPC request queue hangs the server; a malicious or MITM'd JSON-RPC service does the same to a client.

Proof of concept

Against amqp-client 5.36.0 from Maven Central:

import com.rabbitmq.tools.jsonrpc.DefaultJsonRpcMapper;

public class Poc {
    public static void main(String[] args) {
        DefaultJsonRpcMapper mapper = new DefaultJsonRpcMapper();
        mapper.parse("{\"method\":\"x", String.class); // unterminated string
        // mapper.parse("//", String.class);           // unterminated // comment
        System.out.println("unreachable");
    }
}

java -Xmx64m -cp amqp-client-5.36.0.jar:. Poc throws OutOfMemoryError: Java heap space in about 0.1s and never prints. Swapping in the // line spins at 100% CPU and never returns. A well-formed body such as {"method":"x"} returns immediately.

Impact

Availability. One small, unauthenticated message stops a JSON-RPC endpoint: the unterminated string exhausts the heap, the unterminated comment pins a thread forever. Neither is recoverable per request - JsonRpcServer.doCall only catches ClassCastException, and an OutOfMemoryError affects the whole process.

Scope and fix