Charles Sprickman wrote:
Hello all,
It seems that I'm starting to outgrow our current Postgres setup. We've
been running a handful of machines as standalone db servers. This is
all in a colocation environment, so everything is stuffed into 1U
Supermicro boxes. Our standard build looks like this:
Supermicro 1U w/SCA backplane and 4 bays
2x2.8 GHz Xeons
Adaptec 2015S "zero channel" RAID card
2 or 4 x 73GB Seagate 10K Ultra 320 drives (mirrored+striped)
2GB RAM
FreeBSD 4.11
PGSQL data from 5-10GB per box
Recently I started studying what we were running up against in our
nightly runs that do a ton of updates/inserts to prep things for the
tasks the db does during the business day (light mix of
selects/inserts/updates). While we have plenty of disk bandwidth
(according to bonnie), we are really dying on IOPS. I'm guessing this is
a mix of a rather anemic RAID controller (ever notice how adaptec
doesn't publish any real performance specs on raid cards?) and having
only two or four spindles (effectively 1 or 2 on writes).
So that's where we are...
I'm new to the whole SAN thing, but did recently pick up a few used
NetApp shelves and a Fibre Channel RAID HBA (Mylex ExtremeRAID 3000,
also used) to toy with. I started wondering if I could put something
together to both get our storage on one set of boxes and allow me to get
data striped across more drives. Our budget is not huge and we are not
adverse to getting used gear where appropriate.
What do you folks recommend? I'm just starting to look at what's out
there for SANs and NAS, and from what I've seen, our options are:
Leaving the whole SAN issue for a moment:
It would be interesting to see if moving to FreeBSD 6.0 would help you -
the vfs layer is no longer throttled by the (SMP) GIANT lock in this
version, and that may make quite a difference (given you have SMP boxes).
Another interesting thing to try is rebuilding the database ufs
filesystem(s) with 32K blocks and 4K frags (as opposed to 8K/1K or
16K/2K - can't recall the default on 4.x). I found this to give a factor
of 2 speedup on random disk access (specifically queries doing indexed
joins).
Is it mainly your 2 disk machines that are IOPS bound? if so, a cheap
option may be to buy 2 more cheetahs for them! If it's the 4's, well how
about a 2U U320 diskpack from whomever supplies you the Supermicro boxes?
I have just noticed Luke's posting - I would second the advice to avoid
SAN - in my experience it's an expensive way to buy storage.
best wishes
Mark
