Kyle raises ThreadError

Original Kyle raises ThreadError Editable
version 1 of 1



There are many libraries which use mainloops. It is hard to avoid them. But what do you do when there are two?

 

In the past, I would have recommended a Byzantine structure - give each its own thread with a few shared threadsafe queues for communication. This is suicide in Python, on multicore systems at least. Two excuses in my defense: I've only owned single core computers and Erlang was the first multithreaded language I learned. Doing stupid things (in Python) with extra threads is fine on single core computers. There is no penalty in such a case. But on multicore computers you will die. Erlang's threading model seems like the One True Way, which means much frustration in every other language.

I'm not sure this is accurate. Mainloops are a form of blocking IO...


Networking loops, true. GUI mainloops, true. But usually I add a...


 

My second response to this puzzle was to use multiple Python interpreters, each running a SocketServer. This has proved very useful on servers. An unhandled exception only breaks that call, the SocketServer continues to handle requests, providing additional stability. Without too much trouble, SocketServers can be moved between machines. Add some SSH tunnels, you don't even have to change any code.

But all this seems like overkill for the typical GUI app. How about putting both mainloops into a single thread?

It can be done! You just have to build your own mainloop() function. This is much easier than it sounds. I formerly treated the mainloop as if it were some automagical black box which is aimed once and beyond your control once started. Turns out a mainloop is three lines of code.

while running:
    library.loop_once()
    time.sleep(1.0 / update_rate)

That is it. A whole mainloop right there. Behind the scenes magic still happens in the loop_once() function, and every library will have a different name for that function. But that is the core of it.

Some real examples. Let's say you are making a chat room with irclib and pygtk. Both have their own mainloops, and we'll call each in turn. GTK events can pile up, so that half runs until it is out of events to process. IRClib is less demanding. Though in this example I've omitted all the setup the irc object requires.

 

while running:
    while gtk.events_pending():
        gtk.main_iteration(False)
    irc.process_once()
    time.sleep(0.01)

In real life you would probably use glib.io_add_watch (or at the v...


 

Lets say you want gtk to update at most 100 times per second and irc 10 times per second.

while running:
    for i in range(10):
        while gtk.events_pending():
            gtk.main_iteration(False)
        time.sleep(0.01)
    irc.process_once()

More often I've used this multiple-rate setup in games, to run a physics engine at a slower framerate than the GUI. A physics engine adds another wrinkle: it needs to know exactly how much time has passed since the last frame. Note that I make no attempt to regulate framerate minimum framerate. Makes the code simpler, and I've not needed it in practice. Here I'm using the PyMunk for physics, running it at the same framerate as the GUI, and I'm ignoring a lot of boilerplate.

while running:
    while gtk.events_pending():
        gtk.main_iteration(False)
    now = time.time()
    delta = now - old
    pymunk.space.step(delta)
    old = now
    time.sleep(0.01)

 

In hindsight all this seems really obvious. If the mainloop is dumb, ignore it and use your own. One reason it took so long for me to figure this out was my heavy use of Tkinter. Tkinter's mainloop is not written in Python and can not be ignored or replaced. There is no loop_once() for you to hack with. But I am fond of Tk's canvas (though I wish it could do AA and transparency) and really wanted to make a simple physics game in it. Most mailing list archives will say "Can't do that, use PyGame." But PyGame is overkill and there is a hidden gem in the canvas docs: canvas.after(). Properly misused, it lets you tuck a second mainloop inside Tkinter's invariable loop. Use is simple: provide after() with a number of milliseconds and a function to call. What is interesting is that the implementation is properly tail recursive. A function can call itself forever. Once again, lots of missing boilerplate but hopefully the point is made.

If you don't want to roll your own, Twisted has custom reactors fo...


You can drive Tkinter one redraw at a time, actually. Twisted's T...


 

def my_loop():
    milliseconds = int(round(1000.0 / framerate))
       # process the world and update the canvas, 
    # letting the real mainloop() worry about details
    # ...
    tk.canvas.after(milliseconds, my_loop)

My apologies that none of this code is directly useful in the copy/paste sense. GUI libs require a stupid amount of distracting line noise. Anyway, now you know how to roll your own mainloop even when conventional wisdom says you can't.