Hi,
On Tue, 11 Mar 2008, Ferenc Wagner wrote:
> Kadlecsik Jozsef <kadlec@xxxxxxxxxxxxxxx> writes:
>
> > Which knobs should we tune in order to achieve the fastest lock
> > processing?
>
> Try starting gfs_controld with -l0,
We run our GFS cluster with 'gfs_controld -l0' from the beginning, thanks
to your messages on the mailing list :-).
> and maybe decrease demote_secs (you don't access the same file
> repeatedly after all).
Increasing demote_secs to 600 or decreasing it to 150 did not help.
We set glock_purge to 50 and it did not increase the performance either.
The volumes are mounted with noatime, but quota is enabled.
There are max ~100 users (including POP/IMAP) using the cluster at peak
and sometimes it takes 30-50s to load in a 30MB folder. As one of our
users expressed "It is so slow as if I connected to the cluster via mobile
GPRS." The blind users who use jaws screen reader complain that the reader
simply goes out of sync at waiting to be displayed something to read.
This is cluster-2.01.00. Some stats:
# gfs_tool df /gfs/home
/gfs/home:
SB lock proto = "lock_dlm"
SB lock table = "kfki:home"
SB ondisk format = 1309
SB multihost format = 1401
Block size = 4096
Journals = 16
Resource Groups = 4088
Mounted lock proto = "lock_dlm"
Mounted lock table = "kfki:home"
Mounted host data = "jid=2:id=196609:first=0"
Journal number = 2
Lock module flags = 0
Local flocks = FALSE
Local caching = FALSE
Oopses OK = FALSE
Type Total Used Free use%
------------------------------------------------------------------------
inodes 592913 592913 0 100%
metadata 2748600 42475 2706125 2%
data 264538979 12255826 252283153 5%
And a counter-snapshot:
# gfs_tool counters /gfs/home
locks 5778
locks held 4913
freeze count 0
incore inodes 737
metadata buffers 884
unlinked inodes 0
quota IDs 0
incore log buffers 0
log space used 0.20%
meta header cache entries 0
glock dependencies 0
glocks on reclaim list 0
log wraps 332
outstanding LM calls 0
outstanding BIO calls 0
fh2dentry misses 0
glocks reclaimed 244939160
glock nq calls 382928115
glock dq calls 376758954
glock prefetch calls 101297055
lm_lock calls 155817206
lm_unlock calls 148752006
lm callbacks 304980555
address operations 50855951
dentry operations 71076079
export operations 0
file operations 56213431
inode operations 114720407
super operations 207151835
vm operations 9
block I/O reads 0
block I/O writes 0
Is there anything which indicates what's wrong or what could be tuned?
Best regards,
Jozsef
--
E-mail : kadlec@xxxxxxxxxxxx, kadlec@xxxxxxxxxxxxxxxxx
PGP key: http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address: KFKI Research Institute for Particle and Nuclear Physics
H-1525 Budapest 114, POB. 49, Hungary
--
Linux-cluster mailing list
Linux-cluster@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/linux-cluster
