Submitted by matt_eaton
on Sat, 06/16/2018 - 12:42 PM
Python is about to get a nice new update with many new feature enhancement, performance improvements, and redesigned modules with the release of Python 3.7. At the time of writing this article a release candidate is available today by visiting CPython's github page and checking out branch 3.7. In this two part series I am writing about two updates to Python 3.7 that I feel will benefit the language in regards to networking. In this article I will be writing about the additions to the Python socket module because I am networking guy! I will be discussing two new additions to the socket module, why I find these additions interesting, and also provide some sample code for context. Let’s jump in!
NOTE: These examples have been tested on macOS 10.13 and Ubuntu 16.04 with the development branch of Python 3.7 downloaded and compiled locally. These examples have not been tested on Windows and may be subject to change once the full release of Python 3.7 is deployed.
Socket Close() 🚀
As a part of BPO 32454 Christian Heimes described a very real problem when dealing with sockets. How do you know that you have closed the socket correctly, and better yet, how do you use a single API to close a socket connection independent of the platform? So in BPO 32454 the universal close() API was added. I think this is a nice addition to the socket module because it attempts to remove platform dependent hacks and takes everything down one central path as a Python developer when closing a connection. Another reason this is a nice addition is because it lowers the surface area for troubleshooting issues. For example, if you are unsure that your socket connection really closed and there are multiple ways to close a connection depending upon your platform then this makes debugging and troubleshooting even more difficult because you have to keep track of how to close a connection on each platform.
Below if a sample of a client application that opens a connection to a server. In this application I used he new close() API and then try to write to the socket after it is closed just to make sure the connection did tear down correctly.
import socket HOST = 'localhost' PORT = 53091 write_index = 0 with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: try: s.connect((HOST, PORT)) except OSError as err: print("Client socket failed to open with error: {0}".format(err)) while write_index < 10: string = "Writing data at index: {0} ".format(write_index) s.sendall(str.encode(string)) # Block and write up to 512 bytes to the write buffer. data = s.recv(512) # Using bpo-32373, check to see if this socket is blocking. if s.getblocking(): print("(Blocking socket) " + data.decode()) else: print("(Non-blocking socket) " + data.decode()) write_index += 1 # Close the socket, but wait.... is it really closed??? s.close() # Just to make sure (bpo-32454), send data to validate that the # socket is closed. try: s.sendall(str.encode('closed')) print("Client socket remains open") except OSError as err: print("Client socket is closed with error: {0}".format(err))
Socket getblocking() 🐍
The next addition to the socket module I want to discuss is the getblocking() method. When working with sockets one of the most important questions you want to ask yourself when designing a networking program is if you want to tie up a thread or not while the connection is reading and writing. A block can occur at the kernel level when data is being read and wrote to a connection, and everything will wait for this action to complete until the next action can take place. In Python, when designing a program sometimes you may not have full exposure to whether a socket is blocking unless you drop down to the C level to see if you have a thread tied up. With asyncio now in Python this can be problematic. Now with the getblocking method you can have the C level module let you know by making a simple if call in your Python code. Making life a lot easier on the Python developer.
In the example below I created a simple blocking server that just listens for a client connection, blocks the serving thread, and then reads the data coming off the connection until the buffer is completely drained. Notice in the example that I included an if statement to display if the socket is blocking while reading the 512 bytes from the connection. This is how easy it is to see if the socket is blocking.
import socket from sys import platform HOST = '' PORT = 53091 with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: # Bind to a blank host:port and listen for connections. s.bind((HOST, PORT)) s.listen(1) print("Waiting for a connection...") conn, addr = s.accept() with conn: print('Connection bound with address: {0}'.format(addr[0])) while True: # Block and read up to 512 bytes from the read buffer. data = conn.recv(512) if not data: break else: read_data = len(str(data)) if s.getblocking(): print("(Blocking socket) Reading {0} " "bytes".format(read_data)) else: print("(Non-blocking socket) Reading {0} " "bytes".format(read_data)) conn.sendall(data)
As a peek inside the socketmodule.c, here is the new addition to the CPython code that returns a Py_True or Py_False for whether the socket is blocking or not.
PyDoc_STRVAR(setblocking_doc, "setblocking(flag)\n\ \n\ Set the socket to blocking (flag is true) or non-blocking (false).\n\ setblocking(True) is equivalent to settimeout(None);\n\ setblocking(False) is equivalent to settimeout(0.0)."); /* s.getblocking() method. Returns True if socket is in blocking mode, False if it is in non-blocking mode. */ static PyObject * sock_getblocking(PySocketSockObject *s) { if (s->sock_timeout) { Py_RETURN_TRUE; } else { Py_RETURN_FALSE; } }
In Summary ⌛️
In summary I think that close() and getblocking() are two nice features being added to the Python socket module coming up in Python 3.7. I think that both of these features streamline the ability for the Python developers to create their applications faster without having to debug on different platforms or drop down into the C code to identify what threads are blocking or not. This makes for a much richer experience as a Python network application developer. Looking forward to the release of Python 3.7 soon, a lot of hard work by the community has gone into it! If you would like to take a closer look at the examples from this post, you can find them on my Github account here. I hope you enjoyed this post, please leave a comment if you have a question, feedback, or a concern.
Member for
3 years 9 monthsLong time mobile team lead with a love for network engineering, security, IoT, oss, writing, wireless, and mobile. Avid runner and determined health nut living in the greater Chicagoland area.