SUSE-SU-2022:1485-1

This update for python39 fixes the following issues:

• CVE-2021-3572: Fixed an improper handling of unicode characters in pip (bsc#1186819).

• Update to 3.9.10 (jsc#SLE-23849)

• Remove shebangs from from python-base libraries in _libdir. (bsc#1193179)

• Update to 3.9.9:

  • Core and Builtins
    • bpo-30570: Fixed a crash in issubclass() from infinite recursion when searching pathological bases tuples.
    • bpo-45494: Fix parser crash when reporting errors involving invalid continuation characters. Patch by Pablo Galindo.
    • bpo-45385: Fix reference leak from descrcheck. Patch by Dong-hee Na.
    • bpo-45167: Fix deepcopying of types.GenericAlias objects.
    • bpo-44219: Release the GIL while performing isatty system calls on arbitrary file descriptors. In particular, this affects os.isatty(), os.deviceencoding() and io.TextIOWrapper. By extension, io.open() in text mode is also affected. This change solves a deadlock in os.isatty(). Patch by Vincent Michel in bpo-44219.
    • bpo-44959: Added fallback to extension modules with '.sl' suffix on HP-UX
    • bpo-44050: Extensions that indicate they use global state (by setting msize to -1) can again be used in multiple interpreters. This reverts to behavior of Python 3.8.
    • bpo-45121: Fix issue where Protocol.init raises RecursionError when it's called directly or via super(). Patch provided by Yurii Karabas.
    • bpo-45083: When the interpreter renders an exception, its name now has a complete qualname. Previously only the class name was concatenated to the module name, which sometimes resulted in an incorrect full name being displayed.
    • bpo-45738: Fix computation of error location for invalid continuation characters in the parser. Patch by Pablo Galindo.
    • Library
    • bpo-45678: Fix bug in Python 3.9 that meant functools.singledispatchmethod failed to properly wrap the attributes of the target method. Patch by Alex Waygood.
    • bpo-45679: Fix caching of multi-value typing.Literal. Literal[True, 2] is no longer equal to Literal[1, 2].
    • bpo-45438: Fix typing.Signature string representation for generic builtin types.
    • bpo-45581: sqlite3.connect() now correctly raises MemoryError if the underlying SQLite API signals memory error. Patch by Erlend E. Aasland.
    • bpo-39679: Fix bug in functools.singledispatchmethod that caused it to fail when attempting to register a classmethod() or staticmethod() using type annotations. Patch contributed by Alex Waygood.
    • bpo-45515: Add references to zoneinfo in the datetime documentation, mostly replacing outdated references to dateutil.tz. Change by Paul Ganssle.
    • bpo-45467: Fix incremental decoder and stream reader in the 'raw-unicode-escape' codec. Previously they failed if the escape sequence was split.
    • bpo-45461: Fix incremental decoder and stream reader in the 'unicode-escape' codec. Previously they failed if the escape sequence was split.
    • bpo-45239: Fixed email.utils.parsedatetz() crashing with UnboundLocalError on certain invalid input instead of returning None. Patch by Ben Hoyt.
    • bpo-44904: Fix bug in the doctest module that caused it to fail if a docstring included an example with a classmethod property. Patch by Alex Waygood.
    • bpo-45406: Make inspect.getmodule() catch FileNotFoundError raised by :'func:inspect.getabsfile, and return None to indicate that the module could not be determined.
    • bpo-45262: Prevent use-after-free in asyncio. Make sure the cached running loop holder gets cleared on dealloc to prevent use-after-free in getrunningloop
    • bpo-45386: Make xmlrpc.client more robust to C runtimes where the underlying C strftime function results in a ValueError when testing for year formatting options.
    • bpo-45371: Fix clang rpath issue in distutils. The UnixCCompiler now uses correct clang option to add a runtime library directory (rpath) to a shared library.
    • bpo-20028: Improve error message of csv.Dialect when initializing. Patch by Vajrasky Kok and Dong-hee Na.
    • bpo-45343: Update bundled pip to 21.2.4 and setuptools to 58.1.0
    • bpo-41710: On Unix, if the semclockwait() function is available in the C library (glibc 2.30 and newer), the threading.Lock.acquire() method now uses the monotonic clock (time.CLOCKMONOTONIC) for the timeout, rather than using the system clock (time.CLOCKREALTIME), to not be affected by system clock changes. Patch by Victor Stinner.
    • bpo-45328: Fixed http.client.HTTPConnection to work properly in OSs that don't support the TCPNODELAY socket option.
    • bpo-1596321: Fix the threading.shutdown() function when the threading module was imported first from a thread different than the main thread: no longer log an error at Python exit.
    • bpo-45274: Fix a race condition in the Thread.join() method of the threading module. If the function is interrupted by a signal and the signal handler raises an exception, make sure that the thread remains in a consistent state to prevent a deadlock. Patch by Victor Stinner.
    • bpo-45238: Fix unittest.IsolatedAsyncioTestCase.debug(): it runs now asynchronous methods and callbacks.
    • bpo-36674: unittest.TestCase.debug() raises now a unittest.SkipTest if the class or the test method are decorated with the skipping decorator.
    • bpo-45235: Fix an issue where argparse would not preserve values in a provided namespace when using a subparser with defaults.
    • bpo-45234: Fixed a regression in copyfile(), copy(), copy2() raising FileNotFoundError when source is a directory, which should raise IsADirectoryError
    • bpo-45228: Fix stack buffer overflow in parsing J1939 network address.
    • bpo-45192: Fix the tempfile.inferreturntype function so that the dir argument of the tempfile functions accepts an object implementing the os.PathLike protocol.
    • bpo-45160: When tracing a tkinter variable used by a ttk OptionMenu, callbacks are no longer made twice.
    • bpo-35474: Calling mimetypes.guessallextensions() with strict=False no longer affects the result of the following call with strict=True. Also, mutating the returned list no longer affects the global state.
    • bpo-45166: typing.gettypehints() now works with Final wrapped in ForwardRef.
    • bpo-45097: Remove deprecation warnings about the loop argument in asyncio incorrectly emitted in cases when the user does not pass the loop argument.
    • bpo-45081: Fix issue when dataclasses that inherit from typing.Protocol subclasses have wrong init. Patch provided by Yurii Karabas.
    • bpo-24444: Fixed an error raised in argparse help display when help for an option is set to 1+ blank spaces or when choices arg is an empty container.
    • bpo-45021: Fix a potential deadlock at shutdown of forked children when using concurrent.futures module
    • bpo-45030: Fix integer overflow in pickling and copying the range iterator.
    • bpo-39039: tarfile.open raises ReadError when a zlib error occurs during file extraction.
    • bpo-44594: Fix an edge case of ExitStack and AsyncExitStack exception chaining. They will now match with block behavior when context is explicitly set to None when the exception is in flight.
  • Documentation
    • bpo-45726: Improve documentation for functools.singledispatch() and functools.singledispatchmethod.
    • bpo-45680: Amend the docs on GenericAlias objects to clarify that non-container classes can also implement class_getitem. Patch contributed by Alex Waygood.
    • bpo-45655: Add a new 'relevant PEPs' section to the top of the documentation for the typing module. Patch by Alex Waygood.
    • bpo-45604: Add level argument to multiprocessing.logtostderr function docs.
    • bpo-45464: Mention in the documentation of Built-in Exceptions that inheriting from multiple exception types in a single subclass is not recommended due to possible memory layout incompatibility.
    • bpo-45449: Add note about PEP 585 in collections.abc.
    • bpo-45516: Add protocol description to the importlib.abc.Traversable documentation.
    • bpo-20692: Add Programming FAQ entry explaining that int literal attribute access requires either a space after or parentheses around the literal.
    • bpo-45216: Remove extra documentation listing methods in difflib. It was rendering twice in pydoc and was outdated in some places.
    • bpo-45772: socket.socket documentation is corrected to a class from a function.
    • bpo-45392: Update the docstring of the type built-in to remove a redundant line and to mention keyword arguments for the constructor.
  • Tests
    • bpo-45578: Add tests for dis.distb()
    • bpo-45577: Add subtests for all pickle protocols in testzoneinfo.
    • bpo-43592: test.libregrtest now raises the soft resource limit for the maximum number of file descriptors when the default is too low for our test suite as was often the case on macOS.
    • bpo-40173: Fix test.support.importhelper.importfreshmodule().
    • bpo-45280: Add a test case for empty typing.NamedTuple.
    • bpo-45269: Cover case when invalid markers type is supplied to cmakeencoder.
    • bpo-45209: Fix UserWarning: resourcetracker warning in testmultiprocessing.TestSharedMemory.testsharedmemorycleanedafterprocesstermination
    • bpo-45195: Fix testreadline.testnonascii(): sometimes, the newline character is not written at the end, so don't expect it in the output. Patch by Victor Stinner.
    • bpo-45156: Fixes infinite loop on unittest.mock.seal() of mocks created by create_autospec().
    • bpo-45042: Fixes that test classes decorated with @hashlibhelper.requireshashdigest were skipped all the time.
    • bpo-45235: Reverted an argparse bugfix that caused regression in the handling of default arguments for subparsers. This prevented leaf level arguments from taking precedence over root level arguments.
    • bpo-45765: In importlib.metadata, fix distribution discovery for an empty path.
    • bpo-45644: In-place JSON file formatting using python3 -m json.tool infile infile now works correctly, previously it left the file empty. Patch by Chris Wesseling.
  • Build
    • bpo-43158: setup.py now uses values from configure script to build the uuid extension module. Configure now detects util-linux's libuuid, too.
    • bpo-45571: Modules/Setup now use PYCFLAGSNODIST instead of PYCFLAGS to compile shared modules.
    • bpo-45532: Update sys.version to use main as fallback information. Patch by Jeong YunWon.
    • bpo-45405: Prevent internal configure error when running configure with recent versions of non-Apple clang. Patch by David Bohman.
    • bpo-45220: Avoid building with the Windows 11 SDK previews automatically. This may be overridden by setting the DefaultWindowsSDKVersion environment variable before building.
  • C API
    • bpo-44687: BufferedReader.peek() no longer raises ValueError when the entire file has already been buffered.
    • bpo-44751: Remove crypt.h include from the public Python.h header.

• rpm-build-python dependency is available on the current Factory, not with SLE.

• BuildRequire rpm-build-python: The provider to inject python(abi) has been moved there. rpm-build pulls rpm-build-python automatically in when building anything against python3-base, but this implies that the initial build of python3-base does not trigger the automatic installation.

• Update to 3.9.7:

  • Security
    • Replaced usage of tempfile.mktemp() with TemporaryDirectory to avoid a potential race condition.
    • Add auditing events to the marshal module, and stop raising code.init events for every unmarshalled code object. Directly instantiated code objects will continue to raise an event, and audit event handlers should inspect or collect the raw marshal data. This reduces a significant performance overhead when loading from .pyc files.
    • Made the internal putcmd function in smtplib sanitize input for presence of \r and \n characters to avoid (unlikely) command injection.
  • Core and Builtins
    • Fixed pickling of range iterators that iterated for over 2**32 times.
    • Fix a race in WeakKeyDictionary, WeakValueDictionary and WeakSet when two threads attempt to commit the last pending removal. This fixes asyncio.createtask and fixes a data loss in asyncio.run where shutdownasyncgens is not run
    • Fixed a corner case bug where the result of float.fromhex('0x.8p-1074') was rounded the wrong way.
    • Refine the syntax error for trailing commas in import statements. Patch by Pablo Galindo.
    • Restore behaviour of complex exponentiation with integer-valued exponent of type float or complex.
    • Correct the ast locations of f-strings with format specs and repeated expressions. Patch by Pablo Galindo
    • Use new trashcan macros (PyTRASHCANBEGIN/END) in frameobject.c instead of the old ones (PyTRASHCANSAFEBEGIN/END).
    • Fix segmentation fault with deep recursion when cleaning method objects. Patch by Augusto Goulart and Pablo Galindo.
    • Fix bug where PyErrSetObject hangs when the current exception has a cycle in its context chain.
    • Fix reference leaks in the error paths of updatebases() and buildclass. Patch by Pablo Galindo.
    • Fix undefined behaviour in complex object exponentiation.
    • Remove uses of PyObjectGCDel() in error path when initializing types.GenericAlias.
    • Remove the pass-through for hash() of weakref.proxy objects to prevent unintended consequences when the original referred object dies while the proxy is part of a hashable object. Patch by Pablo Galindo.
    • Fix ltrace functionality when exceptions are raised. Patch by Pablo Galindo
    • Fix a crash at Python exit when a deallocator function removes the last strong reference to a heap type. Patch by Victor Stinner.
    • Fix crash when using passing a non-exception to a generator's throw() method. Patch by Noah Oxer
  • Library
    • run() now always return a TestResult instance. Previously it returned None if the test class or method was decorated with a skipping decorator.
    • Fix bugs in cleaning up classes and modules in unittest:
      • Functions registered with addModuleCleanup() were not called unless the user defines tearDownModule() in their test module.
      • Functions registered with addClassCleanup() were not called if tearDownClass is set to None.
      • Buffering in TestResult did not work with functions registered with addClassCleanup() and addModuleCleanup().
      • Errors in functions registered with addClassCleanup() and addModuleCleanup() were not handled correctly in buffered and debug modes.
      • Errors in setUpModule() and functions registered with addModuleCleanup() were reported in wrong order.
      • And several lesser bugs.
    • Made email date parsing more robust against malformed input, namely a whitespace-only Date: header. Patch by Wouter Bolsterlee.
    • Fix a crash in the signal handler of the faulthandler module: no longer modify the reference count of frame objects. Patch by Victor Stinner.
    • Method stopTestRun() is now always called in pair with method startTestRun() for TestResult objects implicitly created in run(). Previously it was not called for test methods and classes decorated with a skipping decorator.
    • argparse.BooleanOptionalAction's default value is no longer printed twice when used with argparse.ArgumentDefaultsHelpFormatter.
    • Upgrade bundled pip to 21.2.3 and setuptools to 57.4.0
    • Fix the os.setinheritable() function on FreeBSD 14 for file descriptor opened with the OPATH flag: ignore the EBADF error on ioctl(), fallback on the fcntl() implementation. Patch by Victor Stinner.
    • The @functools.totalordering() decorator now works with metaclasses.
    • sqlite3 user-defined functions and aggregators returning strings with embedded NUL characters are no longer truncated. Patch by Erlend E. Aasland.
    • Always show loop= arg deprecations in asyncio.gather() and asyncio.sleep()
    • Non-protocol subclasses of typing.Protocol ignore now the init method inherited from protocol base classes.
    • The tokenize.tokenize() doesn't incorrectly generate a NEWLINE token if the source doesn't end with a new line character but the last line is a comment, as the function is already generating a NL token. Patch by Pablo Galindo
    • Fix http.client.HTTPSConnection fails to download >2GiB data.
    • rcompleter does not call getattr() on property objects to avoid the side-effect of evaluating the corresponding method.
    • weakref.proxy objects referencing non-iterators now raise TypeError rather than dereferencing the null tpiternext slot and crashing.
    • The implementation of collections.abc.Set.hash() now matches that of frozenset.hash().
    • Fixed issue in compileall.compilefile() when sys.stdout is redirected. Patch by Stefan H�lzl.
    • Give priority to using the current class constructor in inspect.signature(). Patch by Weipeng Hong.
    • Fix memory leak in tkinter.flatten() if it is called with a sequence or set, but not list or tuple.
    • Update shutil.copyfile() to raise FileNotFoundError instead of confusing IsADirectoryError when a path ending with a os.path.sep does not exist; shutil.copy() and shutil.copy2() are also affected.
    • handle StopIteration subclass raised from @contextlib.contextmanager generator
    • Make the implementation consistency of indexOf() between C and Python versions. Patch by Dong-hee Na.
    • Fixes TypedDict to work with typing.gettypehints() and postponed evaluation of annotations across modules.
    • Fix bug with pdb's handling of import error due to a package which does not have a main module
    • Fixed an exception thrown while parsing a malformed multipart email by email.message.EmailMessage.
    • pathlib.PureWindowsPath.isreserved() now identifies a greater range of reserved filenames, including those with trailing spaces or colons.
    • Handle exceptions from parsing the arg of pdb's run/restart command.
    • The sqlite3 context manager now performs a rollback (thus releasing the database lock) if commit failed. Patch by Luca Citi and Erlend E. Aasland.
    • Improved string handling for sqlite3 user-defined functions and aggregates:
      • It is now possible to pass strings with embedded null characters to UDFs
      • Conversion failures now correctly raise MemoryError
    • Patch by Erlend E. Aasland.
    • Handle RecursionError in TracebackException's constructor, so that long exceptions chains are truncated instead of causing traceback formatting to fail.
    • Fix email.message.EmailMessage.setcontent() when called with binary data and 7bit content transfer encoding.
    • The compresslevel and preset keyword arguments of tarfile.open() are now both documented and tested.
    • Fixed a Y2k38 bug in the compileall module where it would fail to compile files with a modification time after the year 2038.
    • Fix testall on platforms lacking a shared memory implementation.
    • Pass multiprocessing BaseProxy argument managerowned through AutoProxy.
    • email.utils.getaddresses() now accepts email.header.Header objects along with string values. Patch by Zackery Spytz.
    • lib2to3 now recognizes async generators everywhere.
    • Fix TypeError when required subparsers without dest do not receive arguments. Patch by Anthony Sottile.
  • Documentation
    • Removed the othergui.rst file, any references to it, and the list of GUI frameworks in the FAQ. In their place I've added links to the Python Wiki page on GUI frameworks.
    • Update the definition of future in the glossary by replacing the confusing word 'pseudo-module' with a more accurate description.
    • Add typical examples to os.path.splitext docs
    • Clarify that shutil.makearchive() is not thread-safe due to reliance on changing the current working directory.
    • Update of three expired hyperlinks in Doc/distributing/index.rst: 'Project structure', 'Building and packaging the project', and 'Uploading the project to the Python Packaging Index'.
    • Updated the docstring and docs of filecmp.cmp() to be more accurate and less confusing especially in respect to shallow arg.
    • Match the docstring and python implementation of countOf() to the behavior of its c implementation.
    • List all kwargs for textwrap.wrap(), textwrap.fill(), and textwrap.shorten(). Now, there are nav links to attributes of TextWrap, which makes navigation much easier while minimizing duplication in the documentation.
    • Clarify that atexit uses equality comparisons internally.
    • Documentation of csv.Dialect is more descriptive.
    • Fix documentation for the return type of sysconfig.getpath().
    • Add a 'Security Considerations' index which links to standard library modules that have explicitly documented security considerations.
    • Remove the unqualified claim that tkinter is threadsafe. It has not been true for several years and likely never was. An explanation of what is true may be added later, after more discussion, and possibly after patching _tkinter.c,
  • Tests
    • Add calls of gc.collect() in tests to support PyPy.
    • Made tests relying on the asyncio C extension module optional to allow running on alternative Python implementations. Patch by Serhiy Storchaka.
    • Fix auto history tests of testreadline: sometimes, the newline character is not written at the end, so don't expect it in the output.
    • Add ability to wholesale silence DeprecationWarnings while running the regression test suite.
    • Notify users running testdecimal regression tests on macOS of potential harmless 'malloc can't allocate region' messages spewed by testdecimal.
    • Fixed floating point precision issue in turtle tests.
    • Regression tests, when run with -w, are now re-running only the affected test methods instead of re-running the entire test file.
    • Add test for nested queues when using multiprocessing shared objects AutoProxy[Queue] inside ListProxy and DictProxy

• Add building with --with-system-libmpdec option (bsc#1189356).

• test_faulthandler is still problematic under qemu linux-user emulation, disable it there

• Reenable profileopt with qemu emulation, test_faulthandler is no longer run during profiling

  • bpo-44022 (bsc#1189241, CVE-2021-3737): http.client now avoids infinitely reading potential HTTP headers after a 100 Continue status response from the server.
  • bpo-43075 (CVE-2021-3733, bsc#1189287): Fix Regular Expression Denial of Service (ReDoS) vulnerability in urllib.request.AbstractBasicAuthHandler. The ReDoS-vulnerable regex has quadratic worst-case complexity and it allows cause a denial of service when identifying crafted invalid RFCs. This ReDoS issue is on the client side and needs remote attackers to control the HTTP server.