[sqlalchemy] Re: sqlalchemy.exceptions.TimeoutError with psycopg2 - Jian Luo - com.googlegroups.sqlalchemy

7 messages in com.googlegroups.sqlalchemy[sqlalchemy] Re: sqlalchemy.exception...

From	Sent On	Attachments
Jian Luo	29 Aug 2007 08:57
jason kirtland	29 Aug 2007 13:05
Jian Luo	29 Aug 2007 14:57
jason kirtland	29 Aug 2007 15:10
Jian Luo	29 Aug 2007 15:37
jason kirtland	29 Aug 2007 15:49
Michael Bayer	29 Aug 2007 17:23

Subject:	[sqlalchemy] Re: sqlalchemy.exceptions.TimeoutError with psycopg2
From:	Jian Luo (jian...@googlemail.com)
Date:	08/29/2007 02:57:20 PM
List:	com.googlegroups.sqlalchemy

Hi Jason,

thanks for the reply.

My pylons basecontroller takes care of the session clean up like this:

class BaseController(WSGIController): def __call__(self, environ, start_response): try: return WSGIController.__call__(self, environ, start_response) finally: model.Session.remove()

When I use model.Session.close(), everything works. But error comes whenever i change it to model.Session.remove(). AFAIK, remove() make more sense than close() on a real site with dynamic load which is learned from SA docs.

I changed your script a little to do the test again. The result is just the same as described above:

* with Session.close(), everything works just fine even 'ab -c 500' * with Session.remove(), the script caught timeout exceptions in minutes.

Best

Jian

On Aug 29, 10:05 pm, jason kirtland <j....@discorporate.us> wrote:

Jian wrote:

Hi List:

I am just doing a test of my pylons site under medium load. (ab2 -c 20 -n 5000 ...)

and got following errors:

2007-08-29 17:32:34,468 INFO [paste.httpserver.ThreadPool] kill_hung_threads status: 10 threads (10 working, 0 idle, 0 starting) ave time 2.24sec, max time 7.22sec, killed 0 workers 2007-08-29 17:32:40,874 INFO [paste.httpserver.ThreadPool] kill_hung_threads status: 10 threads (10 working, 0 idle, 0 starting) ave time 7.28sec, max time 13.63sec, killed 0 workers Error - <class 'sqlalchemy.exceptions.TimeoutError'>: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 Error - <class 'sqlalchemy.exceptions.TimeoutError'>: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30

Some Details of test environment: [...snip...]

How are you managing your sessions and connections? Most likely connections are being held somewhere, starving the pool.

A basic setup and query like you've described, served via Paste's server doesn't skip a beat for me even when run with 'ab -c 500'.

def wsgi_app(environ, start_response): start_response('200 OK', [('Content-Type', 'text-plain')]) entities = Data.query.all() page = ','.join([e.data for e in entities]) elixir.objectstore.close() return [page] paste.httpserver.serve(wsgi_app)