| Id | date | Message |
|---|---|---|
| 2008-09-09 10:42:09.0 | ||
| 149631 | 2008-09-09 10:42:09.0 | On Tue, Sep 9, 2008 at 10:06 AM, Amber <noway@spam.com> wrote: > Yes, we don't care about the performance results, but we do care about the point that PostgreSQL can't give the correct results of TPC-H queries. PostgreSQL, at least in terms of the open source databases, is probably your best bet if you are all concerned about correctness. Do not give any credence to a vendor published benchmark unless the test is published and can be independently verifed. merlin -- Sent via pgsql-general mailing list (pgsql-noway@spam.com) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general |
| 149642 | 2008-09-09 10:57:34.0 | On Tue, Sep 09, 2008 at 10:06:01PM +0800, Amber wrote: > Yes, we don't care about the performance results, but we do care about the point that PostgreSQL can't give the correct results of TPC-H queries. > I have never heard a reputable source claim this. I have grave doubts about their claim: they don't specify what implementation of TPC-H they use. They don't actually have the right, AIUI, to claim they tested under TPC-H, since their results aren't listed anywhere on http://www.tpc.org/tpch/results/tpch_results.asp?orderby=dbms. It could well be that they made up something that kinda does something like TPC-H, tailored to how their database works, and then claimed others can't do the job. That's nice marketing material, but it's not a meaningful test result. Without access to the methodology, you should be wary of accepting any of the conclusions. There is, I understand, an implementation of something like TPC-H that you could use to test it yourself. http://osdldbt.sourceforge.net/. DBT-3 is supposed to be that workload. Please note that the license does not allow you to publish competitive tests for marketing reasons. but you could see for yourself whether the claim is true that way. A -- Andrew Sullivan noway@spam.com +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-general mailing list (pgsql-noway@spam.com) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general |
| 149667 | 2008-09-09 11:37:40.0 | On Tuesday 09 September 2008 10:06:01 Amber wrote: > From: "Andrew Sullivan" <noway@spam.com> > Sent: Tuesday, September 09, 2008 8:39 PM > To: <pgsql-noway@spam.com> > Subject: Re: [GENERAL] PostgreSQL TPC-H test result? > > > On Tue, Sep 09, 2008 at 07:59:49PM +0800, Amber wrote: > >> I read something from > >> http://monetdb.cwi.nl/projects/monetdb/SQL/Benchmark/TPCH/index.html > > > > Given that the point of that "study" is to prove something about > > performance, one should be leery of any claims based on an "out of the > > box" comparison. Particularly since the "box" their own product comes > > out of is "compiled from CVS checkout". Their argument seems to be > > that people can learn how to drive CVS and to compile software under > > active development, but can't read the manual that comes with Postgres > > (and a release of Postgres well over a year old, at that). > > > > I didn't get any further in reading the claims, because it's obviously > > nothing more than a marketing effort using the principle that deriding > > everyone else will make them look better. Whether they have a good > > product is another question entirely. > > > > >Yes, we don't care about the performance results, but we do care > > >about the > > > point that PostgreSQL can't give the correct results of TPC-H queries. Given the point of those benchmarks is to make other systems look bad, I think you have to take them with a grain of salt. Since we don't know what the errors/results were, and no information is giving, we are left to wonder if this is a problem with the software or the tester. The site would have us believe the former, but I think I would lean toward the latter... case in point, I did a quick google and turned up this link: http://www.it.iitb.ac.in/~chetanv/personal/acads/db/report_html/node10.html. It isn't terribly informative, but it doesindicate one thing, someone else was able to run query #6 correctly, while the above site claims it returns an error. Now when I look at query#6 from that site, I notice it shows the following syntax: interval '1' year. when I saw that, it jumped out at me as something that could be an issue, and it is: pagila=# select now() - interval '1' year, now() - interval '1 year'; ?column? | ?column? -------------------------------+------------------------------- 2008-09-09 11:28:46.938209-04 | 2007-09-09 11:28:46.938209-04 (1 row) Now, I'm not sure if there is an issue that monet supports the first syntax and so when they ran thier test on postgres this query produced wrong results, but that seems possible. In this case I would wonder if the first syntax is sql compliant, but it doesn't really matter, the tpc-h allows for changes to queries to support syntax variations between databases; I'm pretty sure I could make suttle changes to "break" other databases as well. Incidentally, I poked Mark Wong, who used to work at the OSDL (big linux kernel hacking shop), and he noted he has successfully run the tpc-h tests before on postgres. In the end, I can't speak to what the issues are wrt monet and postgres and thier tpc-h benchmarks, but personally I don't think they are worth worring about. -- Robert Treat http://www.omniti.com Database: Scalability: Consulting: -- Sent via pgsql-general mailing list (pgsql-noway@spam.com) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general |
| 149722 | 2008-09-09 14:24:05.0 | My 02c, Pg does itself no favours by sticking with such pessimistic defaults, and a novice user wanting to try it out will find tweaking the pg configuration files for performance quite complicated. Given the general increase in typical hardware specs these days, perhaps the default pg specs could be set for higher spec systems? Or perhaps the standard install could come with 2 or 3 versions of the config files, & the user can simply rename/invoke the one that fits their system best? I figure (somewhat simplistically) that most settings are more related to available memory than anything else, so perhaps config files for typical 1Gb, 4Gb & 8Gb systems could be provided out of the box to make initial installs simpler? Cheers, Brent Wood Brent Wood DBA/GIS consultant NIWA, Wellington New Zealand >>> Andrew Sullivan <noway@spam.com> 09/10/08 3:47 AM >>> On Tue, Sep 09, 2008 at 07:59:49PM +0800, Amber wrote: > I read something from > http://monetdb.cwi.nl/projects/monetdb/SQL/Benchmark/TPCH/index.html Given that the point of that "study" is to prove something about performance, one should be leery of any claims based on an "out of the box" comparison. Particularly since the "box" their own product comes out of is "compiled from CVS checkout". Their argument seems to be that people can learn how to drive CVS and to compile software under active development, but can't read the manual that comes with Postgres (and a release of Postgres well over a year old, at that). I didn't get any further in reading the claims, because it's obviously nothing more than a marketing effort using the principle that deriding everyone else will make them look better. Whether they have a good product is another question entirely. A -- Andrew Sullivan noway@spam.com +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-general mailing list (pgsql-noway@spam.com) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general -- Sent via pgsql-general mailing list (pgsql-noway@spam.com) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general |