Hi. Thanks for the tips. But, if there is a deadlock problem with memory, what wold be a good solution to get each one of the drives on my 4 nodes look like one big shared drive using gfs? I think that I have a wrong idea of what GFS is, can you explain it to me please? is only a file system ? is a way to share storage with distributed lock? ----- Original Message ----- From: "Benjamin Marzinski" <bmarzins@xxxxxxxxxx> To: "linux clustering" <linux-cluster@xxxxxxxxxx> Sent: Monday, November 06, 2006 11:01 PM Subject: Re: gfs mounted but not working > On Mon, Nov 06, 2006 at 01:38:40AM -0300, romero.cl@xxxxxxxxx wrote: > > Hi. > > > > Now i'm trying this and it works! for now... > > > > Two nodes: node3 & node4 > > node4 export his /dev/sdb2 with gnbd_export as "node4_sdb2" > > node3 import node4's /dev/sdb2 with gnbd_import (new /dev/gnbd/node4_sdb2) > > > > on node3: gfs_mkfs -p lock_dlm -t node3:node3_gfs -j 4 /dev/gnbd/node4_sdb2 > > mount -t gfs /dev/gnbd/node4_gfs /users/home > > > > on node4: mount -t gfs /dev/sdb2 /users/home > > > > and both nodes can read an write ths same files on /users/home!!! > > > > Now i'm going for this: > > > > 4 nodes on a dedicated 3com 1Gbit ethernet switch: > > > > node2 exporting with gnbd_export /dev/sdb2 as "node2_sdb2" > > node3 exporting with gnbd_export /dev/sdb2 as "node3_sdb2" > > node4 exporting with gnbd_export /dev/sdb2 as "node4_sdb2" > > > > node1 (main) will import all "nodeX_sdb2" and create a logical volume named > > "main_lv" including: > > > > /dev/sdb2 (his own) > > /dev/gnbd/node2_sdb2 > > /dev/gnbd/node3_sdb2 > > /dev/gnbd/node4_sdb2 > > > > Next I will try to export the new big logical volume with "gnbd_export" and > > then do gnbd_import on each node. > > With that each node will see "main_lv", then mount it on /users/home as gfs > > and get a big shared filesystem to work toghether. > > > > Is this the correct way to do the work??? possibly a deadlock??? > > Sorry. This will not work. There are a couple of problems. > > 1. A node shouldn't ever gnbd import a device it has exported. This can > cause memory deadlock. When memory pressure is high nodes try to write > their buffers to disk. Once the buffer is written to disk, the node can drop > it from memory, reducing memory pressure. When you do this over gnbd, for every > buffer that you write out on the client, a new buffer request come into the gnbd > server. If you import a device you have exported (even indirectly through the > logical volume on node1 in this setup) that new request just comes back to you. > This means that you suddenly double your buffers in memory, just when memory was > running low. > > The solution is to only access the local device directly, but never through > gnbd. Oh, just a note, if you are planning on accessing the local device > directly, you must not use the "-c" option when you are exporting the device. > This will eventually lead to corruption. The "-c" option is only for dedicated > gnbd servers. > > 2. Theoretically, you could just have every node export the devices to every > other node, and then build a logical volume on top of all the devices on each > node, but you should not do this. It totally destroys the benefit of having > a cluster. Since your GFS filesystem would then depend on having access to the > block devices of every machine, if ANY machine in your cluster went down, the > whole cluster would crash, because a piece of your filesystem would just > disappear. > > > Without shared storage, your gnbd server will be a single point of failure. > The most common way that people set up gnbd is with one dedicated gnbd server > machine, that is only used to serve gnbd blocks, so that it is unlikely to > crash. > > > Sorry if my english isn't very good ;) > > > > ----- Original Message ----- > > From: "Kevin Anderson" <kanderso@xxxxxxxxxx> > > To: "linux clustering" <linux-cluster@xxxxxxxxxx> > > Sent: Sunday, November 05, 2006 10:12 PM > > Subject: Re: gfs mounted but not working > > > > > > > On 11/5/06, *romero.cl@xxxxxxxxx <mailto:romero.cl@xxxxxxxxx>* > > > <romero.cl@xxxxxxxxx <mailto:romero.cl@xxxxxxxxx>> wrote: > > > > > > > > Hi. > > > > > > > > I'm trying your method, but still have a problem: > > > > > > > > Note: /dev/db2/ is a local partition on my second SCSI hard drive > > > > (no RAID) > > > > runing on HP ProLiant. > > > > > > > GFS requires that all storage is equally accessible by all nodes in the > > > cluster. Your other nodes have no path to the storage you set up so it > > > is impossible for them to share the data. > > > > > > Kevin > > > > > > -- > > > Linux-cluster mailing list > > > Linux-cluster@xxxxxxxxxx > > > https://www.redhat.com/mailman/listinfo/linux-cluster > > > > -- > > Linux-cluster mailing list > > Linux-cluster@xxxxxxxxxx > > https://www.redhat.com/mailman/listinfo/linux-cluster > > -- > Linux-cluster mailing list > Linux-cluster@xxxxxxxxxx > https://www.redhat.com/mailman/listinfo/linux-cluster -- Linux-cluster mailing list Linux-cluster@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/linux-cluster
