What’s New in Python, Issue 2

Submitted by matt_eaton on Fri, 12/21/2018 - 03:09 PM

Tags

Python

What’s new in Python is a news feed of articles, trends, and stories happening in the Python open source ecosystem. All of the topics in this article are hand picked from sources such as Github, Bugs.Python.org, Twitter, and of course, python.org. This latest issue focuses on making PyMem_Malloc thread safe, enabling TCP_NODELAY, updating OpenSSL across Windows and macOS, an asyncio fix for KeyboardInterrupt on Windows, a time.time testing bug was discovered on Build Bot, the voting results of PEP 8016, and lastly dual stack address support for IPv4 and IPv6 sockets.

Make PyMem_Malloc() Thread Safe in Debug Mode

When Python is compiled in debug mode memory is allocated with the usage of pymalloc, which the c-equivalent of malloc. The issue here is that pymalloc is not thread safe leaving memory allocated by pymalloc open to all sort of race conditions. This is not an issue when compiled for a release build. This is also not present in Python 3 as PyMem_Malloc() requires the global interpreter lock. This PR makes PyMem_Malloc thread safe in debug mode.

Enable TCP_NODELAY for proactor event loop

TCP_NODELAY is a way for a socket to send data over the network with no delay and bypassing the Nagle Algorithm. For example, a packet is constructed based upon the buffer size of the receiving TCP connection. This packet construction is often delayed to fill the packet with data that is the size of the receiving read buffer. In this case, TCP_NODELAY sends the packet immediately and does not wait for the total buffer size to be reached. This results in a lot of smaller packets traveling over the network but, as the name indicates, there is no delay in sending the packet.

A pull request was just added to CPython to add the TCP_NODELAY option in for selector based event loops, starting in Python 3.6.

macOS and Windows Installers now can use OpenSSL 1.1.0j

This last week macOS and Windows installers were updated across Python’s 2.7.16, 3.6.8, and 3.7.2 to support OpenSSL 1.1.0j. Glad to see these installers updated, but this also does points out the amount of work the core team does to do to keep up with these updates. A big thanks to Ned Deily and Seve Dower for keeping this current.

Asyncio’s Event Loop Suppresses the KeyboardInterrupt on Windows

I have actually seen this happen in Windows where the “Ctrl”+”C” commands are used to force a KeyboardInterrupt and nothing happens. However, in some cases the KeyboardInterrupt will work as intended. This PR by Vladimir Matveev and Andrew Svetlov aims to fix this issue by adding a check to see if _self_reading_future is cancelled and allows for the KeyboardInterrupt to be caught before the super.run_forever() call is made.

# Lib/asyncio/windows_events.py
 
def run_forever(self):
    try:
        # if _self_reading_future is cancelled -
        # this might indicate that event loop was interrupted before
        # and self-reading routine is not hooked up now and needs
        # to be restarted
        if (self._self_reading_future is not None and
            self._self_reading_future.cancelled()):
            self._self_reading_future = None
            self.call_soon(self._loop_self_reading)
        super().run_forever()
    except KeyboardInterrupt:
        if self._self_reading_future is not None:
            self._self_reading_future.cancel()
        raise

time.time() System Clock Sometimes Goes Backward

Interesting issue where time.time was seen to be going backwards in test cases executed by Build Bot. This looks like it was an issue because time.time() was used to determine the length or timeout of a lot of unit tests in Python Lib/test/test_*. This issue was resolved by replacing time.time with time.monotonic() to make sure the time delta is measured properly.

PEP 8016 -- The Steering Council Model has been Voted as the Winner

After a vote of 94 core developers, PEP 8016 for a Python Steering council has won. What this means is that this will be the new model of governance to replace Guido van Rossum now that he has stepped down in his role as the Python Benevolent dictator for life, back in July. Based upon the Abstract of PEP 8016 the Steering Council will be breaking up large changes to the ecosystem up into smaller ones and focusing on these small changes at a more micro level instead of a macro level. From what I gathered from reading through PEP 8016 this will allow things to move carefully through the ecosystem.

Add socket.create_server_sock() convenience function

A very interesting PR has been put into Python 3.8 to as a convenience method to create a server socket that accepts both IPv4 and IPv6 connections. The conversation on the BPO and the Github thread has been rich and I would invite anyone interested in dual stack address support to read through this thread and issue to check it out. There is even some great references in the BPO to Tulip, AKA, Python's asyncio.

Member for

3 years 9 months

Matt Eaton

Long time mobile team lead with a love for network engineering, security, IoT, oss, writing, wireless, and mobile. Avid runner and determined health nut living in the greater Chicagoland area.

Twitter

Github