45 messages in com.googlegroups.pylons-discussRe: Reducing pylons app memory usage?
FromSent OnAttachments
Marcin Kasperski16 Oct 2007 07:21 
Max Ischenko16 Oct 2007 10:46 
Marcin Kasperski17 Oct 2007 02:59 
Dalius Dobravolskas17 Oct 2007 03:04 
Christoph Haas17 Oct 2007 07:12 
Max Ischenko17 Oct 2007 11:02 
Ben Bangert17 Oct 2007 13:19 
Christoph Haas17 Oct 2007 14:23 
Ben Bangert17 Oct 2007 16:32 
Graham Dumpleton17 Oct 2007 16:56 
Ian Bicking17 Oct 2007 16:58 
Marcin Kasperski18 Oct 2007 03:18 
Ben Bangert18 Oct 2007 11:34 
Philip Jenvey18 Oct 2007 12:22 
Ben Bangert18 Oct 2007 12:32 
Ian Bicking18 Oct 2007 12:37 
Marcin Kasperski19 Oct 2007 02:35 
Marcin Kasperski19 Oct 2007 02:42 
Graham Dumpleton21 Oct 2007 16:12 
Jon Rosebaugh21 Oct 2007 16:48 
Graham Dumpleton21 Oct 2007 18:20 
Marcin Kasperski22 Oct 2007 05:11 
Bob Ippolito22 Oct 2007 05:28 
Bob Ippolito22 Oct 2007 05:33 
Graham Dumpleton22 Oct 2007 16:23 
Graham Dumpleton22 Oct 2007 16:25 
Cliff Wells22 Oct 2007 17:52 
Graham Dumpleton22 Oct 2007 17:59 
Cliff Wells22 Oct 2007 18:53 
Graham Dumpleton22 Oct 2007 22:58 
Marcin Kasperski31 Oct 2007 08:39 
Ben Bangert31 Oct 2007 08:56 
Marcin Kasperski31 Oct 2007 09:38 
Marcin Kasperski31 Oct 2007 09:40 
Ben Bangert31 Oct 2007 12:08 
Graham Dumpleton31 Oct 2007 15:52 
Graham Dumpleton31 Oct 2007 16:44 
Ben Bangert31 Oct 2007 17:04 
Ben Bangert31 Oct 2007 17:08 
Marcin Kasperski05 Nov 2007 03:53 
Marcin Kasperski05 Nov 2007 04:08 
Marcin Kasperski05 Nov 2007 04:33 
Alberto Valverde05 Nov 2007 04:57 
Marcin Kasperski05 Nov 2007 06:03 
Ian Bicking05 Nov 2007 06:58 
Subject:Re: Reducing pylons app memory usage?
From:Graham Dumpleton (Grah@public.gmane.org)
Date:10/21/2007 06:20:29 PM
List:com.googlegroups.pylons-discuss

On Oct 22, 9:49 am, "Jon Rosebaugh"
<chai@public.gmane.org> wrote:

On 10/18/07, Marcin Kasperski
<Marc@public.gmane.org> wrote:

I'd suggest checking out nginx or lightty.

I am watching with interest efforts to port WSGI to nginx which show up on some lists. Nevertheless, it seems it will take some time...

What needs to be ported? ngnix already supports reverse-proxying to a HTTP server, and paste.httpserver is an excellent app server.

FWIW, I also never quite understood the basis for getting the equivalent of Apache mod_wsgi working on nginx because how nginx works somewhat limits its overall usefulness. All the same, I haven't objected to the person trying and using code from Apache mod_wsgi in the process.

The big difference between how Apache works and how nginx works (correct me if I am wrong) is that nginx uses an asynchronous event driven model, running with a fixed number of worker processes. This makes it somewhat hard to extend it with embedded modules that are able to block, as when the application supported by the module blocks, the whole process blocks. Although with Apache prefork MPM any one child process can also block completely, Apache can dynamically create additional child processes to cope with a spike in demand or when existing child processes are busy.

In nginx, my understanding is that it isn't able to create additional worker processes like Apache is able to, which would be somewhat limiting. Also understand that nginx isn't internally thread safe at this point, so trying to use Python level threads to allow a blocking mode of operating to work on top of nginx asynchronous model seems also to be problematic.

Thus, in order to support still being able to handle other requests such as those for static files, or proxying requests through to another HTTP server or FASTCGI process, my understanding is that they originally expected that the nginx with embedded WSGI support would itself be run as a backend server to which you would proxy from a front end nginx instance which would do all the other work. In other words, WSGI in nginx is merely attempting to use the nginx web server stack in the belief that it would provide a better performing WSGI server, but not really provide the services of a general purpose web server at the same time.

In this respect, it isn't really any different to proxying to paste.httpserver or using a nginx FASTCGI solution.

So Jon, we may not see eye to eye on how Apache mod_wsgi is implemented or when it would be appropriate to use it, but on nginx mod_wsgi, we would at least appear to have the same view. ;-)

Graham