On Fri, 2004-10-15 at 19:15, Daniel McNeil wrote: > I had more time to test GFS. Reminder of the setup > (note: I added more memory so the machines are up to 1GB). > 3 machines each: > 2 processor (800 MHZ Pentium 3) > 1GB of memory > 2 100Mb ethernet (1 public, 1 private) > 1 2-port Qlogic FC host adapter > 2 F/C sitches cascaded together > 1 - 10 disk - dual controller FASTT200 (36GB 10,000rpm drives) Is this the latest cvs version of cman, dlm and gfs? > > The command run was 'time tar xf /Views/linux-2.6.8.1.tar; > time sync' where /Views is an NFS mounted file system and > the current working directory is in a clean file system on > a 5-disk stripe 64k stripe width). For the 2 node case, > I ran the command in separate directories on each node. > For comparison, the ext3 file system in on a single scsi > disk in data=ordered. Can you rerun the ext3 numbers to the same storage as GFS? Would also be interesting to run from two nodes simultaneously to separate ext3 filesystems on the storage as well? Doing dd's of the logical partitions would also be interesting to see what bandwidth the storage is capable of providing, first single node then simultaneous from both nodes. > > Tar > --- real user sys > ext3 tar 0m6.535s 0m0.429s 0m4.010s > ext3 sync 0m21.953s 0m0.000s 0m0.574s > > gfs 1 node tar 1m15.286s 0m0.787s 0m17.085s > gfs 1 node sync 0m7.734s 0m0.000s 0m0.190s > > gfs 2 node tar 3m58.337s 0m0.844s 0m17.082s > gfs 2 node sync 0m3.119s 0m0.000s 0m0.117s > gfs 2 node tar 3m55.147s 0m0.911s 0m17.529s > gfs 2 node sync 0m1.862s 0m0.001s 0m0.043s > > > du -s linux-2.6.8.1 (after 1st mount) > ----- real user sys > ext3 0m5.361s 0m0.039s 0m0.318s > gfs 1 node 0m46.077s 0m0.097s 0m5.144s > gfs 2 node 0m40.835s 0m0.069s 0m3.218s > gfs 2 node 0m41.313s 0m0.089s 0m3.348s > > Doing a 2nd du -s should be cached. On ext3 is always > seems to be. On gfs the numbers vary quite a bit. > > 2nd du -s > --------- > ext3 0m0.130s 0m0.028s 0m0.101s > gfs 1 node 0m20.95s 0m0.075s 0m3.102s > gfs 1 node 0m0.453s 0m0.044s 0m0.408s > gfs 2 node 0m0.446s 0m0.046s 0m0.400s > gfs 2 node 0m0.456s 0m0.028s 0m0.428s > > rm -rf linux-2.6.8.1 > -------------------- > ext3 0m5.050s 0m0.019s 0m0.822s > gfs 1 node 0m28.583s 0m0.094s 0m8.354s > gfs 2 node 7m16.295s 0m0.073s 0m7.785s > gfs 2 node 8m30.809s 0m0.086s 0m7.759s > > > Comment/questions: > > Tar on gfs on 1 node is nearly 3x slower than ext3. > Tar on 2 gfs nodes in parallel is showing reverse scaling: > 2 nodes take 4 minutes. > > Is there some reason why sync is so fast on gfs? > ext3 shows fast tar then long sync, gfs show long > tar and fairly fast sync. > > 1 time du is around 8 times slow than ext3. This must the > time in instantiate and acquire the DLM locks for the > inodes. > > Do you know the expected time to get instantiate and acquire a > DLM lock? > > rm is 6 times slower on gfs than ext3. Reverse scaling > on removes happening on 2 nodes in parallel. These are > in separate directories, so one would not expect DLM > conflicts. > > Thoughts? > > Daniel > > -- > > Linux-cluster@xxxxxxxxxx > http://www.redhat.com/mailman/listinfo/linux-cluster
