Alex Attarian wrote:
I'll give that a try Gordan! Thanks for the info!
My other question is does gluster perform a read for the extended
attributes on all servers regardless read-subvolume?
This is what my tcpdump shows:
,3..h...:O....5....*.....L.........
......+....c...c..............D........z/production/public_images/15932/images/JetLease_Aircraft_Wanted_For_Lease_Oct_09.gif.JetLease_Aircraft_Wanted_For_Lease_Oct_09.gif.............glusterfs.open-fd-count.0.........trusted.afr.gfs3.12.........trusted.afr.gfs2.12.........trusted.afr.gfs1.12.
09:17:23.498739 IP 192.168.27.26.6996 > 192.168.13.14.1022: P
10821:11057(236) ack 13336 win 1525 <nop,nop,timestamp 1758080706 741598866>
......]...........
h...,3..:O.........*.....L.........
......................@....................................c...c....KFGX....J.......J..........p............glusterfs.open-fd-count.0.........trusted.afr.gfs2.....................trusted.afr.gfs1.............
09:17:23.511931 IP 192.168.13.14.1022 > 192.168.27.26.6996: P
13336:13625(289) ack 11057 win 1525 <nop,nop,timestamp 741598867 1758080706>
..J..................
,3..h...:O.........*.....L.........
......+....c...c..............D........z/production/public_images/15932/images/Jet_1Hawker_Multi_July_09.gif.Jet_1Hawker_Multi_July_09.gif.............glusterfs.open-fd-count.0.........trusted.afr.gfs3.12.........trusted.afr.gfs2.12.........trusted.afr.gfs1.12.
09:17:23.512218 IP 192.168.27.26.6996 > 192.168.13.14.1022: P
11057:11293(236) ack 13625 win 1525 <nop,nop,timestamp 1758080707 741598867>
..................
h...,3..:O.........*.....L.........
......................A[.......g...........................c...c....KBp.....Jw......Jw.........p............glusterfs.open-fd-count.0.........trusted.afr.gfs2.....................trusted.afr.gfs1.............
09:17:23.525473 IP 192.168.13.14.1022 > 192.168.27.26.6996: P
13625:13944(319) ack 11293 win 1525 <nop,nop,timestamp 741598869 1758080707>
..J........-..
Does it have to read the trusted.afr.* attributes from every server
every time it reads a file?
In short - yes, AFAIK. Healing is done when files are accessed, thus the
metadata checks have to be done across all the servers when a file is
accessed to verify that they are all in sync.
The end result is that this ends up being the dominant performance
factor for small I/Os and negligible for large I/O. So if your files are
small and many (e.g. maildir), performance will be poor. But then again
all cluster file systems suffer from that.
Gordan
