On Mon, 4 Feb 2008, Jignesh K. Shah wrote:
Doing it at low scales is not attractive. Commercial databases are publishing at scale factor of 1000(about 1TB) to 10000(10TB) with one in 30TB space. So ideally right now tuning should start at 1000 scale factor.
I think what Simon was trying to get at is some sort of debunking of Monet's benchmarks which were running in-memory while not giving PostgreSQL any real memory to work with. What you're talking about is a completely separate discussion which is well worth having in addition to that.
I'm well aware of how painful it is to generate+load+index even single TB worth of data with PostgreSQL right now because I've been doing just that for weeks now (there's two processing phases in there as well for me that take even longer, but the raw operations are still a significant portion of the total time).
I would be happy to help out if folks here want to do work related to it. Infact if you have time, I can request a project in one of the Sun Benchmarking center to see what we can learn with community members interested in understanding where PostgreSQL performs and fails.
Sounds like a good 8.4 project. Maybe pick this topic back up at the East convention next month, we could talk about it then.
-- * Greg Smith gsmith@xxxxxxxxxxxxx http://www.gregsmith.com Baltimore, MD ---------------------------(end of broadcast)--------------------------- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
