On Tue, 2019-10-29 at 10:59 +0100, Riccardo Murri wrote: > Hello, > > I recently upgraded[2] our servers from GlusterFS 3.8 (old GlusterFS > repo for Ubuntu 16.04) to 6.0 (gotten from the GlusterFS PPA for > Ubuntu 16.04 "xenial"). > > The sustained write performance nearly dropped to half it was before. > We copy a large (a few 10'000s) number of image files (each 2 to 10 > MB > size) from the microscope where they were acquired to a SAMBA server > which mounts[1] the GlusterFS volume; before the upgrade, we could > write at about 40MB/s, after the upgrade, this dropped to 20MB/s. > > This is the version of server and client software installed: > ``` > $ dpkg -l '*gluster*' > Desired=Unknown/Install/Remove/Purge/Hold > > Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig- > > aWait/Trig-pend > > / Err?=(none)/Reinst-required (Status,Err: uppercase=bad) > > > / > > > Name Version Architecture > Description > +++-==============================-====================- > ====================- > ================================================================== > ii glusterfs-client 6.5-ubuntu1~xenial1 amd64 > clustered file-system (client package) > ii glusterfs-common 6.5-ubuntu1~xenial1 amd64 > GlusterFS common libraries and translator modules > ii glusterfs-server 6.5-ubuntu1~xenial1 amd64 > clustered file-system (server package) > ``` > Op version has been upped to 60000: > ``` > $ sudo gluster volume get all cluster.op-version > Option Value > ------ ----- > cluster.op-version 60000 > > $ sudo gluster volume get all cluster.max-op-version > Option Value > ------ ----- > cluster.max-op-version 60000 > ``` > > Running `sudo gluster volume status all clients` reports that all > clients are on op-version 60000, too. > > Any suggestions on what to look for or changes to try out? > > Thanks, > Riccardo > > [1]: I have tried both the config where SAMBA 4.8 is using the > vfs_glusterfs.so backend, and the one where `smbd` is just writing to > a locally-mounted directory. Doesn't seem to make a difference. Samba v4.8 is an EOL ed version. Please consider updating Samba to at least v4.9(rather v4.10) or higher. Can you paste the output of `testparm -s` along with the output of `gluster volume info <VOLNAME>` ? > [2]: Actually, since the servers are VMs on an OpenStack cloud, I > created new virtual machines, installed GlusterFS 6 fresh, mounted > the > old bricks in the same brick locations, How did you mount old bricks in the new location? > and restarted the cluster. I > had to fiddle a bit with the files in `/var/lib/glusterfs` because > the > hostnames and IPs changed but did not do anything else than `sed -e > s/old_hostname/new_hostname/` or similarly renaming files. In > particular, I did not touch the extended attributes in the brick > directory. > ________ > > Community Meeting Calendar: > > APAC Schedule - > Every 2nd and 4th Tuesday at 11:30 AM IST > Bridge: https://bluejeans.com/118564314 > > NA/EMEA Schedule - > Every 1st and 3rd Tuesday at 01:00 PM EDT > Bridge: https://bluejeans.com/118564314 > > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx > https://lists.gluster.org/mailman/listinfo/gluster-users ________ Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/118564314 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/118564314 Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
