In the Linux kernel, the following vulnerability has been resolved:
mm: hugememory: fix misused mappinglargefoliosupport() for anon folios
When I did a large folios split test, a WARNING "[ 5059.122759][ T166] Cannot split file folio to non-0 order" was triggered. But the test cases are only for anonmous folios. while mappinglargefolio_support() is only reasonable for page cache folios.
In splithugepagetolisttoorder(), the folio passed to mappinglargefoliosupport() maybe anonmous folio. The foliotestanon() check is missing. So the split of the anonmous THP is failed. This is also the same for shmemmapping(). We'd better add a check for both. But the shmemmapping() in _splithugepage() is not involved, as for anonmous folios, the end parameter is set to -1, so (head[i].index >= end) is always false. shmem_mapping() is not called.
Also add a VMWARNONONCE() in mappinglargefoliosupport() for anon mapping, So we can detect the wrong use more easily.
THP folios maybe exist in the pagecache even the file system doesn't support large folio, it is because when CONFIGTRANSPARENTHUGEPAGE is enabled, khugepaged will try to collapse read-only file-backed pages to THP. But the mapping does not actually support multi order large folios properly.
Using /sys/kernel/debug/splithugepages to verify this, with this patch, large anon THP is successfully split and the warning is ceased.