On 07/03/2012 07:08 PM, Reza Taheri wrote:
TPC-V is a new benchmark under development for virtualized databases. A TPC-V configuration has: - multiple virtual machines running a mix of DSS, OLTP, and business logic apps - VMs running with throughputs ranging from 10% to 40% of the total system ..
I think this would be a lot more interesting to the traditional, dedicated hardware part of the PostgreSQL community if there was a clear way to run this with only a single active machine too. If it's possible for us to use this to compare instances of PostgreSQL on dedicated hardware, too, that is enormously more valuable to people with larger installations. It might be helpful to VMWare as well. Being able to say "this VM install gets X% of the performance of a bare-metal install" answers a question I get asked all the time--when people want to decide between dedicated and virtual setups.
The PostgreSQL community could use a benchmark like this for its own performance regression testing too. A lot of that work is going to happen on a dedicated machines.
> After waving our hands through a number
of small differences between the platforms, we have calculated a CPU cost of around 3.2ms/transaction for the published MS SQL results, versus a measurement of 8.6ms/transaction for PostgreSQL. (TPC benchmarks are typically pushed to full CPU utilization. One removes all bottlenecks in storage, networking, etc., to achieve the 100% CPU usage. So CPU cost/tran is the final decider of performance.) So we need to cut the CPU cost of transactions in half to make publications with PostgreSQL comparable to commercial databases.
I appreciate that getting close to parity here is valuable. This situation is so synthetic though--removing other bottlenecks and looking at CPU timing--that it's hard to get too excited about optimizing for it. There's a lot of things in PostgreSQL that we know are slower than commercial databases because they're optimized for flexibility (the way operators are implemented is the best example) or for long-term code maintenance. Microsoft doesn't care if they have terribly ugly code that runs faster, because no one sees that code. PostgreSQL does care.
The measure that's more fair is a system cost based ones. What I've found is that a fair number of people note PostgreSQL's low-level code isn't quite as fast as some of the less flexible alternatives--hard coding operators is surely cheaper than looking them up each time--but the license cost savings more than pays for bigger hardware to offset that. I wish I had any customer whose database was CPU bound, that would be an awesome world to live in.
Anyway, guessing at causes here is premature speculation. When there's some code for the test kit published, at that point discussing the particulars of why it's not running well will get interesting.
-- Greg Smith 2ndQuadrant US greg@xxxxxxxxxxxxxxx Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-performance mailing list (pgsql-performance@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance
