On Tuesday 09 September 2008 10:06:01 Amber wrote: > From: "Andrew Sullivan" <ajs@xxxxxxxxxxxxxxxxx> > Sent: Tuesday, September 09, 2008 8:39 PM > To: <pgsql-general@xxxxxxxxxxxxxx> > Subject: Re: [GENERAL] PostgreSQL TPC-H test result? > > > On Tue, Sep 09, 2008 at 07:59:49PM +0800, Amber wrote: > >> I read something from > >> http://monetdb.cwi.nl/projects/monetdb/SQL/Benchmark/TPCH/index.html > > > > Given that the point of that "study" is to prove something about > > performance, one should be leery of any claims based on an "out of the > > box" comparison. Particularly since the "box" their own product comes > > out of is "compiled from CVS checkout". Their argument seems to be > > that people can learn how to drive CVS and to compile software under > > active development, but can't read the manual that comes with Postgres > > (and a release of Postgres well over a year old, at that). > > > > I didn't get any further in reading the claims, because it's obviously > > nothing more than a marketing effort using the principle that deriding > > everyone else will make them look better. Whether they have a good > > product is another question entirely. > > > > >Yes, we don't care about the performance results, but we do care > > >about the > > > point that PostgreSQL can't give the correct results of TPC-H queries. Given the point of those benchmarks is to make other systems look bad, I think you have to take them with a grain of salt. Since we don't know what the errors/results were, and no information is giving, we are left to wonder if this is a problem with the software or the tester. The site would have us believe the former, but I think I would lean toward the latter... case in point, I did a quick google and turned up this link: http://www.it.iitb.ac.in/~chetanv/personal/acads/db/report_html/node10.html. It isn't terribly informative, but it doesindicate one thing, someone else was able to run query #6 correctly, while the above site claims it returns an error. Now when I look at query#6 from that site, I notice it shows the following syntax: interval '1' year. when I saw that, it jumped out at me as something that could be an issue, and it is: pagila=# select now() - interval '1' year, now() - interval '1 year'; ?column? | ?column? -------------------------------+------------------------------- 2008-09-09 11:28:46.938209-04 | 2007-09-09 11:28:46.938209-04 (1 row) Now, I'm not sure if there is an issue that monet supports the first syntax and so when they ran thier test on postgres this query produced wrong results, but that seems possible. In this case I would wonder if the first syntax is sql compliant, but it doesn't really matter, the tpc-h allows for changes to queries to support syntax variations between databases; I'm pretty sure I could make suttle changes to "break" other databases as well. Incidentally, I poked Mark Wong, who used to work at the OSDL (big linux kernel hacking shop), and he noted he has successfully run the tpc-h tests before on postgres. In the end, I can't speak to what the issues are wrt monet and postgres and thier tpc-h benchmarks, but personally I don't think they are worth worring about. -- Robert Treat http://www.omniti.com Database: Scalability: Consulting:
