atom feed4 messages in org.postgresql.pgsql-generalRe: Parallel Query should be a top pr...
FromSent OnAttachments
Mike MascariMar 27, 2005 8:58 pm 
Qingqing ZhouMar 27, 2005 10:06 pm 
Bruno Wolff IIIMar 28, 2005 7:50 am 
Oleg BartunovMar 28, 2005 7:51 am 
Subject:Re: Parallel Query should be a top priority
From:Oleg Bartunov (ol@sai.msu.su)
Date:Mar 28, 2005 7:51:10 am
List:org.postgresql.pgsql-general

Interesting, that Stonebraker in his interview said about parallel query processing
http://searchenterpriselinux.techtarget.com/qna/0,289202,sid39_gci1025832,00.html

Putting aside Larry Ellison, would you say, anything should have been done
differently? Stonebraker: We made a couple of significant mistakes. The one I most would like to have back was Informix made a nice run in the early 1990s selling parallel query processing and they were really fast and routinely beat Oracle in performance bakeoffs. Informix horizontally partitioned databases and spread them out over different processors and used multiple processors on a single query very efficiently. That was technology that Ingres started developing in 1987 and then Ingres decided to cancel that initiative, so that's one I'd like to have back. Another initiative that failed was that Ingres put a fair amount of money into writing a distributed database system and there just wasn't much of a market for distributed databases. I would have killed that one and kept alive the parallel query processing effort. Ultimately Informix got squashed by Oracle anyway. It's not clear this would have made a whole lot of difference in the outcome.

On Mon, 28 Mar 2005, Bruno Wolff III wrote:

On Sun, Mar 27, 2005 at 23:58:35 -0500, Mike Mascari <masc@mascari.com> wrote:

Without parallel query, the *only* way to decrease the execution time of a single query whose data has been fully cached is to buy the latest-and-greatest which is increasing in speed at decreasing rates, rather than scaling up the number of processors in a single box. A speed barrier to PostgreSQL's ability to execute a single query is fast approaching.

I think that is a bit extreme. For some queries you will be able to parallelize accross mutliple back ends and realize some speedup.

I would also think that this argument could also apply to cases where the data is on several sets of disks and you wanted to be reading from both sets at once rather than serially.

I love PostgreSQL, and with tablespaces, PITR, nested transactions, and more PLs than one knows what do with, it's my favorite database from a usability standpoint. But in terms of performance, the one missing piece to the performance puzzle is parallel query.

"Consider parallel processing a single query" should be moved out from under Miscellaneous on the TODO list and re-categorized as the formerly existent URGENT feature...

I think there are other things that could be done to improve optimization that will benefit more people than parallelized queries. Those are really only useful to people where the database is being used by a handful (less than the number of processors and/or disk channels) of users concurrently, who are making long running queries and waiting for the results.

---------------------------(end of broadcast)--------------------------- TIP 7: don't forget to increase your free space map settings

Regards, Oleg