Hi Dave, you might also want to have a look at: http://www.opensharedroot.org/Members/marc/blog/blog-on-dlm/red-hat-dlm-__find_lock_by_id/oprofile-analysis Perhaps you have something to add there as well. Thanks Marc. On Wednesday 12 September 2007 22:45:25 David Teigland wrote: > On Thu, Sep 06, 2007 at 10:58:40AM +0200, Mark Hlawatschek wrote: > > Hi, > > > > during a performance analysis and tuning session, I did some profiling > > with oprofile on GFS and dlm. > > I got some weird results ... > > > > The installed software is: > > RHEL4u5, kernel 2.6.9-55.0.2.ELsmp > > GFS: 2.6.9-72.2.0.2 > > DLM: 2.6.9-46.16.0.1 > > > > The configuration includes 2 clusternodes. > > > > I put the following load on one cluster node: > > > > 100 processes are doing in parallel: > > - create 1000 files with 100kb size each (ie altogether we have 100.000 > > files) - flock 1000 files > > - unlink 1000 files. > > > > The following oprofile output shows, that the system spends about 49% > > (75%*65%*) of the time in gfs_unlinked_get. > > Looking into the code whe can see, that this is related to unlinked.c: > > 53 9394211 58.7081 : ul = list_entry(tmp, > > struct gfs_unlinked, ul_list); > > > > It can also be observed, that dlm spends more than 50% of its time in > > searching for hashes... > > > > Is this the expected behaviour or can this be tuned somewhere ? > > Thanks for doing this, it's very interesting. For the dlm > search_hashchain, could you try changing rsbtbl_size to 1024 (the default > is 256). echo 1024 > /proc/.../rsbtbl_size after loading the dlm module, > but before the lockspace is created. > > For gfs, I haven't looked very closely, but the linked list could probably > be simply turned into a hash table. We'd want to study it more closely to > make sure that the long non-hashed list is really the right thing to fix > (i.e. we don't want to just fix a symptom of something else). > > Dave > > -- > Linux-cluster mailing list > Linux-cluster@xxxxxxxxxx > https://www.redhat.com/mailman/listinfo/linux-cluster -- Gruss / Regards, Marc Grimme Phone: +49-89 452 3538-14 http://www.atix.de/ http://www.open-sharedroot.org/ ** ATIX Informationstechnologie und Consulting AG Einsteinstr. 10 85716 Unterschleissheim Deutschland/Germany Phone: +49-89 452 3538-0 Fax: +49-89 990 1766-0 Registergericht: Amtsgericht Muenchen Registernummer: HRB 168930 USt.-Id.: DE209485962 Vorstand: Marc Grimme, Mark Hlawatschek, Thomas Merz (Vors.) Vorsitzender des Aufsichtsrats: Dr. Martin Buss -- Linux-cluster mailing list Linux-cluster@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/linux-cluster
