Bonus: simple python socket client

Original A quick introduction to Python's new SocketServer module. Now with a complete example. Bonus: simple python socket client Editable
version 3 of 3



Python 2.6 and 3.0 introduced a lot of new modules. Here I'll be covering one of them, SocketServer, which provides a dead easy way to build custom servers. We'll be implementing a port-to-pipe adapter. Anything that comes in to a network port will be passed through the standard input pipe to an application. The application's output will be collected and routed back through the network port. What's good will this example be? Well, it is a recent saw that all good webapps are clones of a single unix command, so this port-to-pipe adapter is obviously worth millions. Righto.

We'll be creating a network enabled calculator. Any text received on port 2000 will be passed through to the bc command, and the output will be sent back over the same port. Security? None.

First, accessing the command line from Python. There are a lot of old ways to do this (os.call, Popen) but these are deprecated in favor of the subprocess module. pipe_command() wraps subprocess and manages standard I/O. The first line makes sure there is input before attaching the input pipe. The second line launches the command supplied by arg_list. The remaining lines pipe the input (if any) and return the output through .communicate().

def pipe_command(arg_list, standard_input=False):
    "arg_list is [command, arg1, ...], standard_input is string"
    pipe = subprocess.PIPE if standard_input else None
    subp = subprocess.Popen(arg_list, stdin=pipe, stdout=subprocess.PIPE)
    if not standard_input:
        return subp.communicate()[0]
    return subp.communicate(standard_input)[0]

 

Second, we override the default TCP handler with our own class. The first line collects a chunk of data, limited to a kilobyte. This data is passed through pipe_command(). If everything completes successfully and there is data to report, it is send back. Finally, the connection is closed.

it is send back.

Sent back.


 

def handle(self):
    # self.request is the client connection
    data = self.request.recv(1024)  # clip input at 1Kb
    reply = pipe_command(my_unix_command, data)
    if reply is not None:
        self.request.send(reply)
    self.request.close()

Third, the server is spliced together with the desired qualities. The custom handler is probably the most important part, but this is also where you configure TCP vs UDP. Also, note the ThreadingMixIn. It allows the server to automatically spawn a new thread for each request. Without it your sever will be limited to one request at a time. There are a handful of other performance related magic variables you can set, too.

class SimpleServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
    # Ctrl-C will cleanly kill all spawned threads
    daemon_threads = True
    # much faster rebinding
    allow_reuse_address = True

    def __init__(self, server_address, RequestHandlerClass):
        SocketServer.TCPServer.__init__(self, server_address, RequestHandlerClass)

Finally, everything is set up. The server is ready to be launched. Two lines make it go.

server = SimpleServer((HOST, PORT), SingleTCPHandler)
server.serve_forever()

You can test it out with netcat:

$ nc localhost 2000
2+2
4

All through this example we've been using TCP, but you are not limited to just TCP. For example, I've been fooling around writing streaming audio servers using UDP. SocketServer makes whipping up such prototypes very easy.

Here is all the code once more. Just 40 lines. For more information, see the SocketServer docs. Enjoy!

 

#! /usr/bin/env python
import SocketServer, subprocess, sys
from threading import Thread

my_unix_command = ['bc']
HOST = 'localhost'
PORT = 2000

def pipe_command(arg_list, standard_input=False):
    "arg_list is [command, arg1, ...], standard_input is string"
    pipe = subprocess.PIPE if standard_input else None
    subp = subprocess.Popen(arg_list, stdin=pipe, stdout=subprocess.PIPE)
    if not standard_input:
        return subp.communicate()[0]
    return subp.communicate(standard_input)[0]

class SingleTCPHandler(SocketServer.BaseRequestHandler):
    "One instance per connection.  Override handle(self) to customize action."
    def handle(self):
        # self.request is the client connection
        data = self.request.recv(1024)  # clip input at 1Kb
        reply = pipe_command(my_unix_command, data)
        if reply is not None:
            self.request.send(reply)
        self.request.close()

class SimpleServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
    # Ctrl-C will cleanly kill all spawned threads
    daemon_threads = True
    # much faster rebinding
    allow_reuse_address = True

    def __init__(self, server_address, RequestHandlerClass):
        SocketServer.TCPServer.__init__(self, server_address, RequestHandlerClass)

if __name__ == "__main__":
    server = SimpleServer((HOST, PORT), SingleTCPHandler)
    # terminate with Ctrl-C
    try:
        server.serve_forever()
    except KeyboardInterrupt:
        sys.exit(0)

An how do you deploy that under Apache using flup?


 

If you would rather use Python instead of netcat, here is a basic client for connecting to SimpleServer:

import socket
def client(string):
    HOST, PORT = 'localhost', 2000
    # SOCK_STREAM == a TCP socket
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    #sock.setblocking(0)  # optional non-blocking
    sock.connect((HOST, PORT))
    sock.send(string)
    reply = sock.recv(16384)  # limit reply to 16K
    sock.close()
    return reply

assert client('2+2') == '4'