Thanks for that, Ira.
So far the combined effect of:
kernel oplocks = no
oplocks = yes
level2 oplocks = yes
on a FUSE-mounted GlusterFS share appears to keep everything happy.
So far no data corruption, and a mostly-happy user base. That doesn't
work with vfs_glusterfs, and other combinations result in locking
problems (again, Photoshop, Word and Excel are the three most
troublesome).
A real bummer too, because vfs_glusterfs is noticeably faster,
particularly at directory traversal.
-Dan
----------------
Dan Mons
Skunk Works
Cutting Edge
http://cuttingedge.com.au
On 17 February 2014 13:12, Ira Cooper <ira@xxxxxxxxxx> wrote:
> Fake oplocks opens up the possibility of file corruption. You'd be better off disabling oplocks if you care about your data.
>
> Thanks,
>
> -Ira
>
> ----- Original Message -----
>> More testing of vfs_glusterfs reveals problems with Microsoft and
>> Adobe software, and their silly locking. (For example, Microsoft
>> Excel opening a .xlsx, or Adobe Photoshop opening a .jpg or .psd).
>> Dozens of other production applications we use are all fine, for what
>> it's worth (the bigger the vendor, the more likely it seems their
>> software is broken. How amusing).
>>
>> The following config (without vfx_glusterfs) is the only way I can
>> make these particular applications play ball. Still playing with
>> "fake oplocks" as well to see what broader effect that has.
>>
>> [prodbackup]
>> vfs object = streams_xattr
>> path = /prodbackup
>> Comment = prodbackup
>> browseable = yes
>> writable = yes
>> guest ok = no
>> valid users = +prod
>> create mask = 0660
>> force create mode = 0660
>> directory mask = 0770
>> force directory mode = 0770
>> hide dot files = no
>> security mask = 0660
>> force security mode = 0660
>> directory security mask = 0770
>> force directory security mode = 0770
>> kernel share modes = no
>> kernel oplocks = no
>> ea support = yes
>> oplocks = yes
>> level2 oplocks = yes
>>
>> -Dan
>>
>> ----------------
>> Dan Mons
>> Skunk Works
>> Cutting Edge
>> http://cuttingedge.com.au
>>
>>
>> On 17 February 2014 07:58, Dan Mons <dmons@xxxxxxxxxxxxxxxxxx> wrote:
>> > Thank you Daniel and Lala,
>> >
>> > "kernel share modes = no" was the magic incantation. This is working
>> > nicely for me now. Much appreciated.
>> >
>> > -Dan
>> >
>> > ----------------
>> > Dan Mons
>> > Skunk Works
>> > Cutting Edge
>> > http://cuttingedge.com.au
>> >
>> >
>> > On 14 February 2014 16:52, Daniel Müller <mueller@xxxxxxxxxxxxxxx> wrote:
>> >> HI again,
>> >>
>> >> I had the same issue. I got it working by:
>> >> Adding -- kernel share modes = No to the sahres
>> >> EX.:
>> >> [home]
>> >> comment=gluster test
>> >> vfs objects=glusterfs
>> >> glusterfs:volume= sambacluster
>> >> glusterfs:volfile_server = 172.17.1.1
>> >> path=/ads/home
>> >> read only=no
>> >> posix locking =NO
>> >> kernel share modes = No
>> >>
>> >> By the way running Centos6.4 Samba 4.1.4, gluster 3.4.1qa2.
>> >>
>> >>
>> >> Good Luck
>> >> Daniel
>> >>
>> >> EDV Daniel Müller
>> >>
>> >> Leitung EDV
>> >> Tropenklinik Paul-Lechler-Krankenhaus
>> >> Paul-Lechler-Str. 24
>> >> 72076 Tübingen
>> >> Tel.: 07071/206-463, Fax: 07071/206-499
>> >> eMail: mueller@xxxxxxxxxxxxxxx
>> >> Internet: www.tropenklinik.de
>> >> "Der Mensch ist die Medizin des Menschen"
>> >>
>> >>
>> >>
>> >>
>> >> Von: gluster-users-bounces@xxxxxxxxxxx
>> >> [mailto:gluster-users-bounces@xxxxxxxxxxx] Im Auftrag von Lalatendu
>> >> Mohanty
>> >> Gesendet: Freitag, 14. Februar 2014 07:25
>> >> An: Dan Mons; Kaleb KEITHLEY
>> >> Cc: gluster-users@xxxxxxxxxxx; Jose Rivera; Gluster Devel
>> >> Betreff: Re: [Gluster-devel] RPMs for Samba 4.1.3 w/
>> >> Gluster
>> >> VFS plug-in for RHEL, CentOS, etc., now available
>> >>
>> >> On 02/14/2014 06:37 AM, Dan Mons wrote:
>> >> Hi,
>> >>
>> >> This is failing for me. I've had the same problems after trying to build
>> >> my
>> >> own vfs_glusterfs from source. I'm certain I'm doing something stupid.
>> >>
>> >> Client is a Windows Server 2008R2 64bit machine with AD authentication.
>> >>
>> >> Server is CentOS 6.4, Gluster 3.4.1GA, Samba 4.1.4 with matching
>> >> samba_vfs_glusterfs as per this thread, AD authentication (sssd for the
>> >> Linux/PAM/nsswitch side, and Samba is configured as a member server). The
>> >> Gluster volume is working fine (this is our production test/backup
>> >> cluster,
>> >> and has been in operation for over a year).
>> >>
>> >> Samba works fine when pointing to a local FUSE mount (this is how we run
>> >> in
>> >> production today for Windows and Mac clients). When I change to
>> >> vfs_glusterfs, it all goes wrong.
>> >>
>> >> Client errors include:
>> >>
>> >> Action: On Windows regular windows explorer directory browsing
>> >> Result: All good. Much faster than regular Samba to FUSE.
>> >>
>> >> Action: On Windows: Right click -> New -> Text Document
>> >> Result: "Unable to create the file "New Text Document.txt". The system
>> >> cannot find the file specified.
>> >>
>> >> Action:
>> >> On a Linux box: dmesg > test.txt
>> >> On Windows: double-click test.txt
>> >> Result: The process cannot access the file because it is in use by another
>> >> process
>> >>
>> >> Action: On Windows: drag and drop a text file to a share
>> >> Result: nothing (file not copied, no error dialog).
>> >>
>> >> Samba logs:
>> >> [2014/02/14 10:44:52.972999, 0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >> prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:46:31.020793, 0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >> glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:47:03.326100, 0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >> prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:47:08.449040, 0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >> glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:48:21.007241, 0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >> prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:48:21.068066, 0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >> glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:51:36.683883, 0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >> prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:51:36.743577, 0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >> glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:53:14.160588, 0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >> glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:53:57.229060, 0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >> prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:53:57.288750, 0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >> glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:54:47.062171, 0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >> prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:54:47.121809, 0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >> glfs_stat(./..) failed: No data available
>> >> [2014/02/14 10:55:16.602058, 0]
>> >> ../source3/modules/vfs_glusterfs.c:292(vfs_gluster_connect)
>> >> prodbackup: Initialized volume from server localhost
>> >> [2014/02/14 10:55:16.670562, 0]
>> >> ../source3/modules/vfs_glusterfs.c:627(vfs_gluster_stat)
>> >> glfs_stat(./..) failed: No data available
>> >>
>> >>
>> >>
>> >> The log seems similar to bug
>> >> https://bugzilla.redhat.com/show_bug.cgi?id=1062674
>> >>
>> >> Please put "kernel share modes = No" for the shares and let us know if it
>> >> works for you.
>> >>
>> >> -Lala
>> >>
>> >> /etc/samba/smb.conf:
>> >>
>> >> [global]
>> >> workgroup = BLAH
>> >> server string = Samba Server Version %v
>> >> log file = /var/log/samba/log.%m
>> >> max log size = 50
>> >> security = ads
>> >> passdb backend = tdbsam
>> >> realm = BLAH
>> >> domain master = no
>> >> domain logons = no
>> >> wins support = yes
>> >> wins proxy = yes
>> >> dns proxy = yes
>> >> load printers = no
>> >> ## Shares
>> >> [prodbackup]
>> >> vfs object = glusterfs
>> >> glusterfs:volume = prodbackup
>> >> glusterfs:volfile_server = localhost
>> >> path = /
>> >> Comment = prodbackup
>> >> browseable = yes
>> >> writable = yes
>> >> guest ok = no
>> >> valid users = +prod
>> >> create mask = 0660
>> >> force create mode = 0660
>> >> directory mask = 0770
>> >> force directory mode = 0770
>> >> hide dot files = no
>> >>
>> >> Any pointers would be appreciated.
>> >>
>> >> -Dan
>> >>
>> >>
>> >>
>> >> ----------------
>> >> Dan Mons
>> >> Skunk Works
>> >> Cutting Edge
>> >> http://cuttingedge.com.au
>> >>
>> >> On 7 February 2014 07:12, Dan Mons <dmons@xxxxxxxxxxxxxxxxxx> wrote:
>> >> Fantastic work, thank you.
>> >>
>> >> -Dan
>> >> ----------------
>> >> Dan Mons
>> >> Skunk Works
>> >> Cutting Edge
>> >> http://cuttingedge.com.au
>> >>
>> >>
>> >> On 6 February 2014 22:50, Kaleb KEITHLEY <kkeithle@xxxxxxxxxx> wrote:
>> >>>
>> >>> YUM repo at http://download.gluster.org/pub/gluster/glusterfs/samba/ now
>> >> has
>> >>> el6 RPMs for RHEL6, CentOS6; along with RPMs for Fedora 18 and Fedora 19.
>> >>>
>> >>> Fedora 20 and Fedora 21 (rawhide) have Samba 4.1.3 by default.
>> >>>
>> >>> --
>> >>>
>> >>> Kaleb
>> >>> _______________________________________________
>> >>> Gluster-users mailing list
>> >>> Gluster-users@xxxxxxxxxxx
>> >>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>> >>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Gluster-devel mailing list
>> >> Gluster-devel@xxxxxxxxxx
>> >> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>> >>
>> >>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel@xxxxxxxxxx
>> https://lists.nongnu.org/mailman/listinfo/gluster-devel
>>
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users
