Category: PostgreSQL General List


Article: PostgreSQL TPC-H test result?



25 items found, displaying 16 to 20.[First/Prev] 1, 2, 3, 4, 5 [Next/Last]
Id date Message
2008-09-09 16:49:01.0
149803 2008-09-09 16:49:01.0 On Tue, 9 Sep 2008, Amber wrote:

> I read something from
> http://monetdb.cwi.nl/projects/monetdb/SQL/Benchmark/TPCH/index.html
> saying that PostgreSQL can't give the correct result of the some TPC-H
> queries

Jignesh Shah at Sun ran into that same problem. It's mentioned briefly in
his presentation at
http://blogs.sun.com/jkshah/entry/postgresql_east_2008_talk_postgresql on
pages 26 and 27. 5 of the 22 reference TCP-H queries (4, 5, 6, 10, 14)
returned zero rows immediately for his tests. Looks like the MonetDB crew
is saying it does that on queries 4,5,6,10,12,14,15 and that 20 takes too
long to run to generate a result. Maybe 12/15/20 were fixed by changes in
8.3, or perhaps there were subtle errors there that Jignesh didn't
catch--it's not like he did a formal submission run, was just kicking the
tires. I suspect the difference on 20 was that his hardware and tuning
was much better, so it probably did execute fast enough.

While some of the MonetDB bashing in this thread was unwarranted, it is
quite inappropriate that they published performance results here. Would
be nice if someone in the community were to grab ahold of the TPC-H
problems and try to shake them out.

--
* Greg Smith noway@spam.com http://www.gregsmith.com Baltimore, MD

--
Sent via pgsql-general mailing list (pgsql-noway@spam.com)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
149824 2008-09-09 17:53:44.0

Greg Smith wrote:
> On Tue, 9 Sep 2008, Amber wrote:
>
>> I read something from
>> http://monetdb.cwi.nl/projects/monetdb/SQL/Benchmark/TPCH/index.html
>> saying that PostgreSQL can't give the correct result of the some
>> TPC-H queries
>
> Jignesh Shah at Sun ran into that same problem. It's mentioned
> briefly in his presentation at
> http://blogs.sun.com/jkshah/entry/postgresql_east_2008_talk_postgresql
> on pages 26 and 27. 5 of the 22 reference TCP-H queries (4, 5, 6, 10,
> 14) returned zero rows immediately for his tests. Looks like the
> MonetDB crew is saying it does that on queries 4,5,6,10,12,14,15 and
> that 20 takes too long to run to generate a result. Maybe 12/15/20
> were fixed by changes in 8.3, or perhaps there were subtle errors
> there that Jignesh didn't catch--it's not like he did a formal
> submission run, was just kicking the tires. I suspect the difference
> on 20 was that his hardware and tuning was much better, so it probably
> did execute fast enough.
>
> While some of the MonetDB bashing in this thread was unwarranted, it
> is quite inappropriate that they published performance results here.
> Would be nice if someone in the community were to grab ahold of the
> TPC-H problems and try to shake them out.
>

Hmm This is the second time MonetDB has published PostgreSQL numbers. I
think I will try to spend few days on TPC-H again on a much smaller
scale (to match what MonetDB used) and start discussions on solving the
problem.. Keep tuned.

Regards,
Jignesh


--
Sent via pgsql-general mailing list (pgsql-noway@spam.com)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
149860 2008-09-09 20:00:10.0 On Tue, Sep 9, 2008 at 3:37 PM, Martijn van Oosterhout
<noway@spam.com> wrote:
> On Wed, Sep 10, 2008 at 07:16:35AM +1200, Brent Wood wrote:
>> Given the general increase in typical hardware specs these days,
>> perhaps the default pg specs could be set for higher spec systems?
>
> Given the default shmem configuration on operating systems these days,
> upping the default will likely cause postgresql to not run at all.

And it wont change the results much either. Changing shared_buffers
is very nuanced and can help or hurt performance, but it isn't tuning
in the sense it's a level you can pull to make the database 'go
faster' like magic. A lot of the obsessing about shared_buffers
resolves around the fact that remarkably few people understand how
memory works in modern operating systems.

The 'big ticket' .conf items are those that affect syncing in write
heavy enviroments (fsync, etc) and planner affecting settings
(work_mem, effective_cache_size). That said, PostgreSQL requires
very little tuning outside of the obvious tradeoffs between speed and
safety. This is an ongoing process...in the old days I had to agonize
about dealing with the stats collector...in modern terms there is much
less to 'trade off' (hopefully less still with 8.4).

merlin

--
Sent via pgsql-general mailing list (pgsql-noway@spam.com)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
149996 2008-09-10 04:29:36.0
On Tue, 2008-09-09 at 16:26 -0400, Tom Lane wrote:

> That's probably not good because it *looks* like we support the syntax,
> but in fact produce non-spec-compliant results. I think it might be
> better if we threw an error.

Definitely. If we accept SQL Standard syntax like this but then not do
what we should, it is clearly an ERROR. Our reputation will be damaged
if we don't, since people will think that we are blase about standards
compliance and about query correctness. Please lets move swiftly to plug
this hole, as if it were a data loss bug (it is, if it causes wrong
answers to queries for unsuspecting users).

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support


--
Sent via pgsql-general mailing list (pgsql-noway@spam.com)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Questions or comments? Feel free to post them here. No login or id required.

Valid XHTML 1.0!
Site Map >>