On Mon, 28 Jan 2008, isplist@xxxxxxxxxxxx wrote:
How much I/O do you actually need?
My concern is not so much what I need up front but the growth path should I
start needing to add a lot of storage. LAMP services can sometimes grow very
quickly, immediately need endless ongoing storage space for uploaded media and
playback, not to mention the web services themselves.
Yes, but we are talking 10s od Gb here before you may need to alter your
approach.
Like I mentioned, I've seen not being prepared for growth and it's not fun. I
would hate to have to keep changing out technologies once things get going
because I didn't choose a nice flexible solution up front.
creates virtual software RAID stripe over them. It then exports this
back out via iSCSI. All the client nodes then connect to the single big
iSCSI node that is the aggregator.
Do you know of any software which helps to keep track of all this? This is an
interesting idea. I think I understand it and want to give it a try. I have
various types of storage where this would be a good solution.
It's pretty simple to set up. You just need to be familliar with iSCSI
tools and software RAID tools, all of which are almost certainly in your
distro's apt/yum repositories.
Let's see if I've got this right.
I need a machine which will become the aggregator, plenty of memory,
multi-port Ethernet card and of course an FC HBA.
FC storage will be attached to this machine. Then, iSCSI storage targets will
also export to this machine.
Not quite sure I follow this - you want to use FC storage and combine it
with iSCSI storage into a bigger iSCSI storage pool? No reasib why not, I
suppose.
This machine will then use virtual RAID (which I've no idea about yet) and
aggregate the storage as a volume or how many I need. Next I export this to
the servers via iSCSI for their larger ongoing storage needs.
Pretty much. Once you have the big software RAID stripe, you can use this
to back any number of iSCSI volumes.
Note that software RAID only goes up to RAID 6 (i.e. n+2). So you cannot
lose more than 2 nodes (FC or iSCSI), otherwise you lose your data.
Now I can use say a small FC GFS target to share various files such as web
pages and other web based shared files yet have the larger more easily
expandable V-RAID for media and other such things.
Once you have a big RAID-ed storage pool, you can partition it out in
whatever way you like. You also don't have to put it all into one big RAID
stripe, but a few smaller ones.
You can dynamically add disks to a software RAID stripe.
This means that I also need to find an iSCSI target driver that doesn't take
rocket science to figure out and is open source to keep things cheap while
trying this out.
yum install iscsi-target
:-)
NFS can give considerably better performance than GFS under some
circumstances. If you don't need POSIX compliant file locking, you may
As I've put all of this together, I've come to learn that I need GFS for the
shared data but the rest, I don't need anything but standard storage. I'll
maintain a cluster of web servers which use GFS to share their pages/images
but I could use this aggregator idea for the larger scale media storage.
Sure, but you aggregator could export space via iSCSI or via NFS,
whichever you prefer.
I had somehow gotten a little too caught up in GFS and I was basically not
thinking anything beyond that. This makes things so much simpler than where I
was heading.
It happens to the best of us. :)
I suspect that possibly overkill. It's NIC I/O you'll need more than
anything. Jumbo frames, as big as your hardware can handle, will also help.
Doesn't NIC I/O take up a lot of CPU time?
Not really. Topping out your CPU with NIC I/O load isn't all that trivial.
There are also NICs that can offload the entire TCP/IP stack off the CPU
onto the NIC, but I don't know that the driver support for Linux is like.
There is no reason why the aggregator couldn't mind it's own exports,
and run as one of the client cluster nodes.
I just mean for fail over. It might be nice to have a little redundancy there.
Sure, you could set up heartbeat and preferably some fencing. I suspect
that double activating a software RAID stripe would be quite destructive
for your data (I asked about that in a different thread, but nobody
stepped up to clarify yet), so fencing is a good idea, "just to make
sure". When a node needs to take over, it fences the other node, connects
the iSCSI shares, starts up the RAID on them, assumes the floating IP and
exports the iSCSI/NFS shares.
Gordan
--
Linux-cluster mailing list
Linux-cluster@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/linux-cluster
