So we're using Pylons and Python in general for our new company platform. We just bought a server with 4 cores to help us reach our scalability goals, but there are a few questions I'm interested in asking the Pylons community.

I (mostly) understand the nature of "threads" in Python. From my understanding, the GIL locks the interpreter to executing only one Python thread at a time, but C modules can take advantage of a Python application being multithreaded, because they can operate independant of the GIL. Presumably, this would mean that there is, in fact, a benefit to using threads in Paste, because most network I/O bound stuff happens within a C module.

Given this situation, I believe that despite paste making an effort to be multithreaded, it would still be advantageous to run a cluster of four Pylons instances and proxy to these using nginx.

Using our setup we'd have four pylons instances being proxied to by four nginx worker threads.

In nginx you can set the processor affinity for each worker thread, thus placing each worker on a different core 0..3.

Here's where things get tricky: I've found a Python package that apparently allows Python applications to set their processor affinity (I'm afraid it doesn't work on OS
X):http://pypi.python.org/pypi/affinity/0.1.0

Using this, what do you guys thing on my idea to write a custom cluster controller, perhaps using supervisord, that will start nginx and the four worker processes, and then fork()'s my Pylons app into into a cluster of four?

Is this overkill? Is Paste more mulithreaded than I'm giving it credit for? Is there a better way to go about this? Does an alternative to the 'affinity' package exist?