CVE-2024-26759

When skipping swapcache for SWPSYNCHRONOUSIO, if two or more threads swapin the same entry at the same time, they get different pages (A, B). Before one thread (T0) finishes the swapin and installs page (A) to the PTE, another thread (T1) could finish swapin of page (B), swapfree the entry, then swap out the possibly modified page reusing the same entry. It breaks the ptesame check in (T0) because PTE value is unchanged, causing ABA problem. Thread (T0) will install a stalled page (A) into the PTE and cause data corruption.

One possible callstack is like this:

CPU0 CPU1 ---- ---- doswappage() doswappage() with same entry <direct swapin path> <direct swapin path> <alloc page A> <alloc page B> swapreadfolio() <- read to page A swapreadfolio() <- read to page B <slow on later locks or interrupt> <finished swapin first> ... setpteat() swapfree() <- entry is free <write to page B, now page A stalled> <swap out page B to same swap entry> ptesame() <- Check pass, PTE seems unchanged, but page A is stalled! swapfree() <- page B content lost! setpte_at() <- staled page A installed!

And besides, for ZRAM, swapfree() allows the swap device to discard the entry content, so even if page (B) is not modified, if swapreadfolio() on CPU0 happens later than swapfree() on CPU1, it may also cause data loss.

To fix this, reuse swapcache_prepare which will pin the swap entry using the cache flag, and allow only one thread to swap it in, also prevent any parallel code from putting the entry in the cache. Release the pin after PT unlocked.

Racers just loop and wait since it's a rare and very short event. A scheduletimeoutuninterruptible(1) call is added to avoid repeated page faults wasting too much CPU, causing livelock or adding too much noise to perf statistics. A similar livelock issue was described in commit 029c4628b2eb ("mm: swap: get rid of livelock in swapin readahead")

Reproducer:

This race issue can be triggered easily using a well constructed reproducer and patched brd (with a delay in read path) [1]:

With latest 6.8 mainline, race caused data loss can be observed easily: $ gcc -g -lpthread test-thread-swap-race.c && ./a.out Polulating 32MB of memory region... Keep swapping out... Starting round 0... Spawning 65536 workers... 32746 workers spawned, wait for done... Round 0: Error on 0x5aa00, expected 32746, got 32743, 3 data loss! Round 0: Error on 0x395200, expected 32746, got 32743, 3 data loss! Round 0: Error on 0x3fd000, expected 32746, got 32737, 9 data loss! Round 0 Failed, 15 data loss!

This reproducer spawns multiple threads sharing the same memory region using a small swap device. Every two threads updates mapped pages one by one in opposite direction trying to create a race, with one dedicated thread keep swapping out the data out using madvise.

The reproducer created a reproduce rate of about once every 5 minutes, so the race should be totally possible in production.

After this patch, I ran the reproducer for over a few hundred rounds and no data loss observed.

Performance overhead is minimal, microbenchmark swapin 10G from 32G zram:

Before: 10934698 us After: 11157121 us Cached: 13155355 us (Dropping SWPSYNCHRONOUSIO flag)

[kasong@tencent.com: v4]

References

Affected packages

Git / git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git

Affected ranges

Type: GIT
Repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
Events: Introduced

0bcac06f27d7528591c27ac2b093ccd71c5d0168

Fixed

2dedda77d4493f3e92e414b272bfa60f1f51ed95

Type: GIT
Repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
Events: Introduced

0bcac06f27d7528591c27ac2b093ccd71c5d0168

Fixed

305152314df82b22cf9b181f3dc5fc411002079a

Type: GIT
Repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
Events: Introduced

0bcac06f27d7528591c27ac2b093ccd71c5d0168

Fixed

d183a4631acfc7af955c02a02e739cec15f5234d

Type: GIT
Repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
Events: Introduced

0bcac06f27d7528591c27ac2b093ccd71c5d0168

Fixed

13ddaf26be324a7f951891ecd9ccd04466d27458

Affected versions

v4.*

v4.15

v4.15-rc1

v4.15-rc2

v4.15-rc3

v4.15-rc4

v4.15-rc5

v4.15-rc6

v4.15-rc7

v4.15-rc8

v4.15-rc9

v4.16

v4.16-rc1

v4.16-rc2

v4.16-rc3

v4.16-rc4

v4.16-rc5

v4.16-rc6

v4.16-rc7

v4.17

v4.17-rc1

v4.17-rc2

v4.17-rc3

v4.17-rc4

v4.17-rc5

v4.17-rc6

v4.17-rc7

v4.18

v4.18-rc1

v4.18-rc2

v4.18-rc3

v4.18-rc4

v4.18-rc5

v4.18-rc6

v4.18-rc7

v4.18-rc8

v4.19

v4.19-rc1

v4.19-rc2

v4.19-rc3

v4.19-rc4

v4.19-rc5

v4.19-rc6

v4.19-rc7

v4.19-rc8

v4.20

v4.20-rc1

v4.20-rc2

v4.20-rc3

v4.20-rc4

v4.20-rc5

v4.20-rc6

v4.20-rc7

v5.*

v5.0

v5.0-rc1

v5.0-rc2

v5.0-rc3

v5.0-rc4

v5.0-rc5

v5.0-rc6

v5.0-rc7

v5.0-rc8

v5.1

v5.1-rc1

v5.1-rc2

v5.1-rc3

v5.1-rc4

v5.1-rc5

v5.1-rc6

v5.1-rc7

v5.10

v5.10-rc1

v5.10-rc2

v5.10-rc3

v5.10-rc4

v5.10-rc5

v5.10-rc6

v5.10-rc7

v5.11

v5.11-rc1

v5.11-rc2

v5.11-rc3

v5.11-rc4

v5.11-rc5

v5.11-rc6

v5.11-rc7

v5.12

v5.12-rc1

v5.12-rc1-dontuse

v5.12-rc2

v5.12-rc3

v5.12-rc4

v5.12-rc5

v5.12-rc6

v5.12-rc7

v5.12-rc8

v5.13

v5.13-rc1

v5.13-rc2

v5.13-rc3

v5.13-rc4

v5.13-rc5

v5.13-rc6

v5.13-rc7

v5.14

v5.14-rc1

v5.14-rc2

v5.14-rc3

v5.14-rc4

v5.14-rc5

v5.14-rc6

v5.14-rc7

v5.15

v5.15-rc1

v5.15-rc2

v5.15-rc3

v5.15-rc4

v5.15-rc5

v5.15-rc6

v5.15-rc7

v5.16

v5.16-rc1

v5.16-rc2

v5.16-rc3

v5.16-rc4

v5.16-rc5

v5.16-rc6

v5.16-rc7

v5.16-rc8

v5.17

v5.17-rc1

v5.17-rc2

v5.17-rc3

v5.17-rc4

v5.17-rc5

v5.17-rc6

v5.17-rc7

v5.17-rc8

v5.18

v5.18-rc1

v5.18-rc2

v5.18-rc3

v5.18-rc4

v5.18-rc5

v5.18-rc6

v5.18-rc7

v5.19

v5.19-rc1

v5.19-rc2

v5.19-rc3

v5.19-rc4

v5.19-rc5

v5.19-rc6

v5.19-rc7

v5.19-rc8

v5.2

v5.2-rc1

v5.2-rc2

v5.2-rc3

v5.2-rc4

v5.2-rc5

v5.2-rc6

v5.2-rc7

v5.3

v5.3-rc1

v5.3-rc2

v5.3-rc3

v5.3-rc4

v5.3-rc5

v5.3-rc6

v5.3-rc7

v5.3-rc8

v5.4

v5.4-rc1

v5.4-rc2

v5.4-rc3

v5.4-rc4

v5.4-rc5

v5.4-rc6

v5.4-rc7

v5.4-rc8

v5.5

v5.5-rc1

v5.5-rc2

v5.5-rc3

v5.5-rc4

v5.5-rc5

v5.5-rc6

v5.5-rc7

v5.6

v5.6-rc1

v5.6-rc2

v5.6-rc3

v5.6-rc4

v5.6-rc5

v5.6-rc6

v5.6-rc7

v5.7

v5.7-rc1

v5.7-rc2

v5.7-rc3

v5.7-rc4

v5.7-rc5

v5.7-rc6

v5.7-rc7

v5.8

v5.8-rc1

v5.8-rc2

v5.8-rc3

v5.8-rc4

v5.8-rc5

v5.8-rc6

v5.8-rc7

v5.9

v5.9-rc1

v5.9-rc2

v5.9-rc3

v5.9-rc4

v5.9-rc5

v5.9-rc6

v5.9-rc7

v5.9-rc8

v6.*

v6.0

v6.0-rc1

v6.0-rc2

v6.0-rc3

v6.0-rc4

v6.0-rc5

v6.0-rc6

v6.0-rc7

v6.1

v6.1-rc1

v6.1-rc2

v6.1-rc3

v6.1-rc4

v6.1-rc5

v6.1-rc6

v6.1-rc7

v6.1-rc8

v6.1.1

v6.1.10

v6.1.11

v6.1.12

v6.1.13

v6.1.14

v6.1.15

v6.1.16

v6.1.17

v6.1.18

v6.1.19

v6.1.2

v6.1.20

v6.1.21

v6.1.22

v6.1.23

v6.1.24

v6.1.25

v6.1.26

v6.1.27

v6.1.28

v6.1.29

v6.1.3

v6.1.30

v6.1.31

v6.1.32

v6.1.33

v6.1.34

v6.1.35

v6.1.36

v6.1.37

v6.1.38

v6.1.39

v6.1.4

v6.1.40

v6.1.41

v6.1.42

v6.1.43

v6.1.44

v6.1.45

v6.1.46

v6.1.47

v6.1.48

v6.1.49

v6.1.5

v6.1.50

v6.1.51

v6.1.52

v6.1.53

v6.1.54

v6.1.55

v6.1.56

v6.1.57

v6.1.58

v6.1.59

v6.1.6

v6.1.60

v6.1.61

v6.1.62

v6.1.63

v6.1.64

v6.1.65

v6.1.66

v6.1.67

v6.1.68

v6.1.69

v6.1.7

v6.1.70

v6.1.71

v6.1.72

v6.1.73

v6.1.74

v6.1.75

v6.1.76

v6.1.77

v6.1.78

v6.1.79

v6.1.8

v6.1.9

v6.2

v6.2-rc1

v6.2-rc2

v6.2-rc3

v6.2-rc4

v6.2-rc5

v6.2-rc6

v6.2-rc7

v6.2-rc8

v6.3

v6.3-rc1

v6.3-rc2

v6.3-rc3

v6.3-rc4

v6.3-rc5

v6.3-rc6

v6.3-rc7

v6.4

v6.4-rc1

v6.4-rc2

v6.4-rc3

v6.4-rc4

v6.4-rc5

v6.4-rc6

v6.4-rc7

v6.5

v6.5-rc1

v6.5-rc2

v6.5-rc3

v6.5-rc4

v6.5-rc5

v6.5-rc6

v6.5-rc7

v6.6

v6.6-rc1

v6.6-rc2

v6.6-rc3

v6.6-rc4

v6.6-rc5

v6.6-rc6

v6.6-rc7

v6.6.1

v6.6.10

v6.6.11

v6.6.12

v6.6.13

v6.6.14

v6.6.15

v6.6.16

v6.6.17

v6.6.18

v6.6.2

v6.6.3

v6.6.4

v6.6.5

v6.6.6

v6.6.7

v6.6.8

v6.6.9

v6.7

v6.7-rc1

v6.7-rc2

v6.7-rc3

v6.7-rc4

v6.7-rc5

v6.7-rc6

v6.7-rc7

v6.7-rc8

v6.7.1

v6.7.2

v6.7.3

v6.7.4

v6.7.5

v6.7.6

v6.8-rc1

v6.8-rc2

v6.8-rc3

v6.8-rc4

v6.8-rc5

Database specific

{
    "vanir_signatures": [
        {
            "signature_version": "v1",
            "signature_type": "Line",
            "target": {
                "file": "mm/swap.h"
            },
            "id": "CVE-2024-26759-1c849c51",
            "digest": {
                "line_hashes": [
                    "39718516807054168217327447537394706306",
                    "296831582519195844008506970780508514169",
                    "323321161364954983341358792727555890493",
                    "159050861882905319322983113646739745405",
                    "316748635175487059832119138980120133360",
                    "269000634819832113780343531488225612662",
                    "92645575816539542258467345210880675604"
                ],
                "threshold": 0.9
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@305152314df82b22cf9b181f3dc5fc411002079a"
        },
        {
            "signature_version": "v1",
            "signature_type": "Function",
            "target": {
                "file": "mm/memory.c",
                "function": "do_swap_page"
            },
            "id": "CVE-2024-26759-1cda56f3",
            "digest": {
                "length": 5466.0,
                "function_hash": "149951387997259558135992905350701058993"
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@13ddaf26be324a7f951891ecd9ccd04466d27458"
        },
        {
            "signature_version": "v1",
            "signature_type": "Line",
            "target": {
                "file": "mm/memory.c"
            },
            "id": "CVE-2024-26759-25587f6f",
            "digest": {
                "line_hashes": [
                    "251423681482968309417709745328194797379",
                    "301386656592820503659641950428983233565",
                    "265944948475336668054177471603955017480",
                    "262943209667881174946346706921263416934",
                    "231514620117896680714880386937754416057",
                    "334832162027385315547234758425647137517",
                    "222372055529068723362756080975799504255",
                    "58810024030845569896971269973012191705",
                    "285205086499470505098570783647474705378",
                    "258565968517696491771659681673363086168",
                    "322236287737640672062376763362482355863",
                    "302884683229572689255568171246727090660",
                    "178814200172191546430943224411841367054",
                    "237404558527278776149276272085200638598",
                    "128989090871277730767030190009301122890",
                    "308390541891872440474425798311665268009"
                ],
                "threshold": 0.9
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@305152314df82b22cf9b181f3dc5fc411002079a"
        },
        {
            "signature_version": "v1",
            "signature_type": "Line",
            "target": {
                "file": "mm/swap.h"
            },
            "id": "CVE-2024-26759-2a6ee4b4",
            "digest": {
                "line_hashes": [
                    "39718516807054168217327447537394706306",
                    "296831582519195844008506970780508514169",
                    "323321161364954983341358792727555890493",
                    "159050861882905319322983113646739745405",
                    "316748635175487059832119138980120133360",
                    "269000634819832113780343531488225612662",
                    "92645575816539542258467345210880675604"
                ],
                "threshold": 0.9
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@13ddaf26be324a7f951891ecd9ccd04466d27458"
        },
        {
            "signature_version": "v1",
            "signature_type": "Line",
            "target": {
                "file": "mm/swap.h"
            },
            "id": "CVE-2024-26759-347cc0bc",
            "digest": {
                "line_hashes": [
                    "39718516807054168217327447537394706306",
                    "296831582519195844008506970780508514169",
                    "323321161364954983341358792727555890493",
                    "159050861882905319322983113646739745405",
                    "316748635175487059832119138980120133360",
                    "269000634819832113780343531488225612662",
                    "92645575816539542258467345210880675604"
                ],
                "threshold": 0.9
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@d183a4631acfc7af955c02a02e739cec15f5234d"
        },
        {
            "signature_version": "v1",
            "signature_type": "Line",
            "target": {
                "file": "mm/swapfile.c"
            },
            "id": "CVE-2024-26759-494eda87",
            "digest": {
                "line_hashes": [
                    "227379574279090568708698351394905046320",
                    "166004363114173620440705315879842031629",
                    "279166466019089235287657463140775766669"
                ],
                "threshold": 0.9
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@2dedda77d4493f3e92e414b272bfa60f1f51ed95"
        },
        {
            "signature_version": "v1",
            "signature_type": "Line",
            "target": {
                "file": "mm/swapfile.c"
            },
            "id": "CVE-2024-26759-4a51058f",
            "digest": {
                "line_hashes": [
                    "227379574279090568708698351394905046320",
                    "166004363114173620440705315879842031629",
                    "279166466019089235287657463140775766669"
                ],
                "threshold": 0.9
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@d183a4631acfc7af955c02a02e739cec15f5234d"
        },
        {
            "signature_version": "v1",
            "signature_type": "Function",
            "target": {
                "file": "mm/memory.c",
                "function": "do_swap_page"
            },
            "id": "CVE-2024-26759-58e127fd",
            "digest": {
                "length": 5414.0,
                "function_hash": "39840519873329949375711191224174144176"
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@d183a4631acfc7af955c02a02e739cec15f5234d"
        },
        {
            "signature_version": "v1",
            "signature_type": "Line",
            "target": {
                "file": "mm/memory.c"
            },
            "id": "CVE-2024-26759-5bc15755",
            "digest": {
                "line_hashes": [
                    "251423681482968309417709745328194797379",
                    "301386656592820503659641950428983233565",
                    "265944948475336668054177471603955017480",
                    "262943209667881174946346706921263416934",
                    "231514620117896680714880386937754416057",
                    "334832162027385315547234758425647137517",
                    "222372055529068723362756080975799504255",
                    "58810024030845569896971269973012191705",
                    "253360370209173942550725169905830141977",
                    "248237646100415538731225836183345399467",
                    "322236287737640672062376763362482355863",
                    "302884683229572689255568171246727090660",
                    "178814200172191546430943224411841367054",
                    "237404558527278776149276272085200638598",
                    "128989090871277730767030190009301122890",
                    "308390541891872440474425798311665268009"
                ],
                "threshold": 0.9
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@2dedda77d4493f3e92e414b272bfa60f1f51ed95"
        },
        {
            "signature_version": "v1",
            "signature_type": "Function",
            "target": {
                "file": "mm/memory.c",
                "function": "do_swap_page"
            },
            "id": "CVE-2024-26759-88ebbabc",
            "digest": {
                "length": 5259.0,
                "function_hash": "213552839243171182919656944630621673319"
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@2dedda77d4493f3e92e414b272bfa60f1f51ed95"
        },
        {
            "signature_version": "v1",
            "signature_type": "Line",
            "target": {
                "file": "mm/swap.h"
            },
            "id": "CVE-2024-26759-8d05688f",
            "digest": {
                "line_hashes": [
                    "39718516807054168217327447537394706306",
                    "296831582519195844008506970780508514169",
                    "323321161364954983341358792727555890493",
                    "182445319627246454896832626401949920427",
                    "316748635175487059832119138980120133360",
                    "269000634819832113780343531488225612662",
                    "92645575816539542258467345210880675604"
                ],
                "threshold": 0.9
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@2dedda77d4493f3e92e414b272bfa60f1f51ed95"
        },
        {
            "signature_version": "v1",
            "signature_type": "Line",
            "target": {
                "file": "mm/swapfile.c"
            },
            "id": "CVE-2024-26759-99b2d546",
            "digest": {
                "line_hashes": [
                    "227379574279090568708698351394905046320",
                    "166004363114173620440705315879842031629",
                    "279166466019089235287657463140775766669"
                ],
                "threshold": 0.9
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@305152314df82b22cf9b181f3dc5fc411002079a"
        },
        {
            "signature_version": "v1",
            "signature_type": "Line",
            "target": {
                "file": "mm/memory.c"
            },
            "id": "CVE-2024-26759-b5d37c78",
            "digest": {
                "line_hashes": [
                    "251423681482968309417709745328194797379",
                    "301386656592820503659641950428983233565",
                    "265944948475336668054177471603955017480",
                    "262943209667881174946346706921263416934",
                    "231514620117896680714880386937754416057",
                    "334832162027385315547234758425647137517",
                    "222372055529068723362756080975799504255",
                    "58810024030845569896971269973012191705",
                    "285205086499470505098570783647474705378",
                    "258565968517696491771659681673363086168",
                    "322236287737640672062376763362482355863",
                    "302884683229572689255568171246727090660",
                    "178814200172191546430943224411841367054",
                    "237404558527278776149276272085200638598",
                    "128989090871277730767030190009301122890",
                    "308390541891872440474425798311665268009"
                ],
                "threshold": 0.9
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@13ddaf26be324a7f951891ecd9ccd04466d27458"
        },
        {
            "signature_version": "v1",
            "signature_type": "Line",
            "target": {
                "file": "mm/swapfile.c"
            },
            "id": "CVE-2024-26759-ee3e2fac",
            "digest": {
                "line_hashes": [
                    "227379574279090568708698351394905046320",
                    "166004363114173620440705315879842031629",
                    "279166466019089235287657463140775766669"
                ],
                "threshold": 0.9
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@13ddaf26be324a7f951891ecd9ccd04466d27458"
        },
        {
            "signature_version": "v1",
            "signature_type": "Line",
            "target": {
                "file": "mm/memory.c"
            },
            "id": "CVE-2024-26759-f87e33f2",
            "digest": {
                "line_hashes": [
                    "251423681482968309417709745328194797379",
                    "301386656592820503659641950428983233565",
                    "265944948475336668054177471603955017480",
                    "262943209667881174946346706921263416934",
                    "231514620117896680714880386937754416057",
                    "334832162027385315547234758425647137517",
                    "222372055529068723362756080975799504255",
                    "58810024030845569896971269973012191705",
                    "285205086499470505098570783647474705378",
                    "258565968517696491771659681673363086168",
                    "322236287737640672062376763362482355863",
                    "302884683229572689255568171246727090660",
                    "178814200172191546430943224411841367054",
                    "237404558527278776149276272085200638598",
                    "128989090871277730767030190009301122890",
                    "308390541891872440474425798311665268009"
                ],
                "threshold": 0.9
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@d183a4631acfc7af955c02a02e739cec15f5234d"
        },
        {
            "signature_version": "v1",
            "signature_type": "Function",
            "target": {
                "file": "mm/memory.c",
                "function": "do_swap_page"
            },
            "id": "CVE-2024-26759-ff470c6c",
            "digest": {
                "length": 5414.0,
                "function_hash": "39840519873329949375711191224174144176"
            },
            "deprecated": false,
            "source": "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git@305152314df82b22cf9b181f3dc5fc411002079a"
        }
    ]
}

Linux / Kernel

Package

Name: Kernel

Affected ranges

Type: ECOSYSTEM
Events: Introduced

4.15.0

Fixed

6.1.80

Type: ECOSYSTEM
Events: Introduced

6.2.0

Fixed

6.6.19

Type: ECOSYSTEM
Events: Introduced

6.7.0

Fixed

6.7.7