In the spirit of the wonderful "Twisted web in 60 seconds" posts, I'm going to contribute here a small post about the Twisted reactor and Deferreds. I know this can be found on the Twisted examples page, but writing it from scratch was helpful for my understanding as well :)

A mainloop

The most usual problem when trying to understand the reactor is context. In my case, the penny dropped when I saw the reactor in the context of a plain desktop application - not a script, not a web server.

Desktop applications are built around and event loop - the user moves the mouse, presses keys, the windowserver captures those events and calls various event handlers on your application.

Twisted reactor is a similar thing:

from twisted.internet import reactor

reactor.run() # this blocks!

After you call reactor.run, your program has an event loop, and the reactor takes over. When embedding Twisted in desktop applications, you need to install a reactor compatible with your main loop (gtk, cocoa and so on).

Deferreds

Event handlers in twisted parlance are called "deferreds". When you ask twisted to do something asynchronously, be it fetching a web page, sending an email, resolving a DNS name, connecting to a server, you need a way to know when that action succeeded (or failed, of course.)

Twisted deferreds can support chaining the results. More on this on a later example, though.

An example

Think about the canonical example when doing desktop application programming. You want to fetch some information from a webserver (perhaps to check for updates). Normally you would spawn a thread, fetch the page, process it, then update your user interface (of course taking care to update the UI from the GUI thread and other similar arcane issues).

In twisted there is only a single thread, and everything is set up using deferreds and the reactor. Let's write some simple code to do that:

We'll need to import the reactor and a getPage function:

from twisted.internet import reactor
from twisted.web.client import getPage

This is a desktop application, so here's a small function to update the UI:

def updateUI(message):
    print "UI:", message

Let's write our processing function:

def processPage(pageContent):
    print 'got the page'
    # process the page here
    # ...
    # update the UI
    updateUI('SUCCESS')

Remember, this will get called when twisted has fetched the page. Its return value will be passed on to the next callback down the chain. Here, we just return the length of the page.

Let's also write an error handler:

def handleError(error):
    print 'got error'
    # update the UI
    updateUI('Whoops! %s' % error)

Having written all the pieces, let's bring them together:

pageFetchedDeferred = getPage("http://orestis.gr")
pageFetchedDeferred.addCallback(processPage)
pageFetchedDeferred.addErrback(handleError)

Finally, we need to start the reactor. In a desktop app, you would've done that at the final stage of initialisation, but this is an example, right?

reactor.run()

Here's the whole program:

from twisted.internet import reactor
from twisted.web.client import getPage

def updateUI(message):
    print "UI:", message

def processPage(pageContent):
    print 'got the page'
    # process the page here
    # ...
    # update the UI
    updateUI('Received %d bytes' % len(pageContent))

def handleError(error):
    print 'got error'
    # update the UI
    updateUI('Whoops! %s' % error)

pageFetchedDeferred = getPage("http://orestis.gr")
pageFetchedDeferred.addCallback(processPage)
pageFetchedDeferred.addErrback(handleError)

reactor.run()

Remember, you need to kill your program with Ctrl-C because the reactor.run call blocks!

Conclusion

If you understand how deferreds and the reactor works, you've made an important step on using Twisted. As I use and understand Twisted more, I'll keep writing!