In the Linux kernel, the following vulnerability has been resolved:
tracing: Fix race issue between cpu buffer write and swap
Warning happened in rbendcommit() at code: if (RBWARNON(cpubuffer, !localread(&cpu_buffer->committing)))
WARNING: CPU: 0 PID: 139 at kernel/trace/ringbuffer.c:3142 rbcommit+0x402/0x4a0 Call Trace: ringbufferunlockcommit+0x42/0x250 tracebufferunlockcommitregs+0x3b/0x250 traceeventbuffercommit+0xe5/0x440 traceeventbufferreserve+0x11c/0x150 traceeventraweventschedswitch+0x23c/0x2c0 _traceiterschedswitch+0x59/0x80 _schedule+0x72b/0x1580 schedule+0x92/0x120 worker_thread+0xa0/0x6f0
It is because the race between writing event into cpu buffer and swapping cpu buffer through file per_cpu/cpu0/snapshot:
Write on CPU 0 Swap buffer by percpu/cpu0/snapshot on CPU 1 -------- -------- tracingsnapshot_write() [...]
ringbufferlockreserve() cpubuffer = buffer->buffers[cpu]; // 1. Suppose find 'cpubuffera'; [...] rbreservenext_event() [...]
ring_buffer_swap_cpu()
if (local_read(&cpu_buffer_a->committing))
goto out_dec;
if (local_read(&cpu_buffer_b->committing))
goto out_dec;
buffer_a->buffers[cpu] = cpu_buffer_b;
buffer_b->buffers[cpu] = cpu_buffer_a;
// 2. cpu_buffer has swapped here.
rb_start_commit(cpu_buffer);
if (unlikely(READ_ONCE(cpu_buffer->buffer)
!= buffer)) { // 3. This check passed due to 'cpu_buffer->buffer'
[...] // has not changed here.
return NULL;
}
cpu_buffer_b->buffer = buffer_a;
cpu_buffer_a->buffer = buffer_b;
[...]
// 4. Reserve event from 'cpu_buffer_a'.
ringbufferunlockcommit() [...] cpubuffer = buffer->buffers[cpu]; // 5. Now find 'cpubufferb' !!! rbcommit(cpubuffer) rbendcommit() // 6. WARN for the wrong 'committing' state !!!
Based on above analysis, we can easily reproduce by following testcase:
#!/bin/bash
dmesg -n 7
sysctl -w kernel.panic_on_warn=1
TR=/sys/kernel/tracing
echo 7 > ${TR}/buffer_size_kb
echo "sched:sched_switch" > ${TR}/set_event
while [ true ]; do
echo 1 > ${TR}/per_cpu/cpu0/snapshot
done &
while [ true ]; do
echo 1 > ${TR}/per_cpu/cpu0/snapshot
done &
while [ true ]; do
echo 1 > ${TR}/per_cpu/cpu0/snapshot
done &
To fix it, IIUC, we can use smpcallfunction_single() to do the swap on the target cpu where the buffer is located, so that above race would be avoided.