GHSA-jm3v-qxmh-hxwv

Source

https://github.com/advisories/GHSA-jm3v-qxmh-hxwv

Import Source

https://github.com/github/advisory-database/blob/main/advisories/github-reviewed/2024/05/GHSA-jm3v-qxmh-hxwv/GHSA-jm3v-qxmh-hxwv.json

JSON Data

https://api.osv.dev/v1/vulns/GHSA-jm3v-qxmh-hxwv

Published

2024-05-14T20:14:43Z

Modified

2024-05-14T20:31:02.052543Z

Severity

4.3 (Medium) CVSS_V3 - CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:N/A:N CVSS Calculator

Summary

Scrapy's redirects ignoring scheme-specific proxy settings

Details

Impact

When using system proxy settings, which are scheme-specific (i.e. specific to http:// or https:// URLs), Scrapy was not accounting for scheme changes during redirects.

For example, an HTTP request would use the proxy configured for HTTP and, when redirected to an HTTPS URL, the new HTTPS request would still use the proxy configured for HTTP instead of switching to the proxy configured for HTTPS. Same the other way around.

If you have different proxy configurations for HTTP and HTTPS in your system for security reasons (e.g., maybe you don’t want one of your proxy providers to be aware of the URLs that you visit with the other one), this would be a security issue.

Patches

Upgrade to Scrapy 2.11.2.

Workarounds

Replace the built-in retry middlewares (RedirectMiddleware and MetaRefreshMiddleware) and the HttpProxyMiddleware middleware with custom ones that implement the fix from Scrapy 2.11.2, and verify that they work as intended.

References

This security issue was reported by @redapple at https://github.com/scrapy/scrapy/issues/767.

References

Affected packages

PyPI / scrapy

Package

Name: scrapy; View open source insights on deps.dev
Purl: pkg:pypi/scrapy

Affected ranges

Type: ECOSYSTEM
Events: Introduced

0Unknown introduced version / All previous versions are affected

Fixed

2.11.2

Affected versions

0.*

0.7

0.8

0.9

0.10.4.2364

0.12.0.2550

0.14.1

0.14.2

0.14.3

0.14.4

0.16.0

0.16.1

0.16.2

0.16.3

0.16.4

0.16.5

0.18.0

0.18.1

0.18.2

0.18.3

0.18.4

0.20.0

0.20.1

0.20.2

0.22.0

0.22.1

0.22.2

0.24.0

0.24.1

0.24.2

0.24.3

0.24.4

0.24.5

0.24.6

1.*

1.0.0rc1

1.0.0rc2

1.0.0rc3

1.0.0

1.0.1

1.0.2

1.0.3

1.0.4

1.0.5

1.0.6

1.0.7

1.1.0rc1

1.1.0rc2

1.1.0rc3

1.1.0rc4

1.1.0

1.1.1

1.1.2

1.1.3

1.1.4

1.2.0

1.2.1

1.2.2

1.2.3

1.3.0

1.3.1

1.3.2

1.3.3

1.4.0

1.5.0

1.5.1

1.5.2

1.6.0

1.7.0

1.7.1

1.7.2

1.7.3

1.7.4

1.8.0

1.8.1

1.8.2

1.8.3

1.8.4

2.*

2.0.0

2.0.1

2.1.0

2.2.0

2.2.1

2.3.0

2.4.0

2.4.1

2.5.0

2.5.1

2.6.0

2.6.1

2.6.2

2.6.3

2.7.0

2.7.1

2.8.0

2.9.0

2.10.0

2.10.1

2.11.0

2.11.1