24 messages in org.postgresql.pgsql-jdbcRe: Limit vs setMaxRows issue
FromSent OnAttachments
Sebastiaan van ErkJun 21, 2006 2:11 am 
Dave CramerJun 21, 2006 7:56 am 
Sebastiaan van ErkJun 21, 2006 8:48 am 
Kris JurkaJun 21, 2006 8:59 am 
A.M.Jun 21, 2006 9:09 am 
Tom LaneJun 21, 2006 9:46 am 
Oliver JowettJun 21, 2006 3:52 pm 
Sebastiaan van ErkJun 22, 2006 1:35 am 
Mark LewisJun 22, 2006 9:15 am 
David WallJun 22, 2006 9:36 am 
Sebastiaan van ErkJun 22, 2006 1:13 pm 
Marc HerbertJul 10, 2006 1:50 am 
Marc HerbertJul 10, 2006 1:59 am 
Marc HerbertJul 10, 2006 2:05 am 
Oliver JowettJul 10, 2006 11:32 pm 
Oliver JowettJul 10, 2006 11:37 pm 
Marc HerbertJul 11, 2006 2:48 am 
Marc HerbertJul 11, 2006 3:00 am 
Oliver JowettJul 11, 2006 3:45 am 
Marc HerbertJul 11, 2006 5:14 am 
Oliver JowettJul 11, 2006 10:01 pm 
Marc HerbertJul 12, 2006 3:22 am 
Markus SchaberJul 12, 2006 3:59 am 
Marc HerbertJul 20, 2006 11:52 am 
Actions with this message:
Paste this link in email or IM:
Paste this link in email or IM:
Atom feed for this thread
Paste this URL into your reader:
Subject:Re: Limit vs setMaxRows issueActions...
From:Dave Cramer (pg@fastcrypt.com)
Date:Jun 21, 2006 7:56:47 am
List:org.postgresql.pgsql-jdbc

Sebastiaan,

I believe the setMaxRows will use a cursor, because you have an order by on the cursor it will have to be fully materialized

Try it without the order by

Dave On 21-Jun-06, at 5:11 AM, Sebastiaan van Erk wrote:

Hi,

When using the ps.setMaxRows() call on PreparedStatement, the jdbc driver sets the row limit via the "raw" postgres protocol. However, in the words of Tom Lane, "the row limit in the protocol only says how many rows to deliver in the first batch. The presumption is that you'll eventually grab the rest, and so the query is planned on that basis."

What this means that when we do the following query:

select action_id from actions order by action_id

with a ps.setMaxRows(100), it takes about 1.8 seconds for the query to complete. However, if we do the following query:

select action_id from actions order by action_id limit 100

without any ps.setMaxRows() the query only takes 0.156 seconds. This is more than a factor of 10 faster.

I'm not 100% sure of what JDBC says about setMaxRows (it's kind of ambiguous in the java doc, as usual), but as far as I can tell, if you call setMaxRows on the prepared statement there is no way in to ever retrieve more than that number of rows. If this is indeed the case, it seems to me that currently there is a mismatch between the JDBC api and the postgresql api, and JDBC should somehow tell postgres that this is a hard limit and it should not plan for a second batch.

Therefore, my question is: is this a bug? It is not feasable for me to add LIMIT clauses to all the SQL queries in my code, so if this IS a bug, I hope it can be fixed. If it is NOT a bug, is there an alternative workaround that does not involve changing all of my sql statements?

Thanks in advance, Sebastiaan

---------------------------(end of broadcast)--------------------------- TIP 2: Don't 'kill -9' the postmaster