Following up on this: * GlusterFS 3.4.1 solves the problem of renaming over CIFS from a Windows client (via Samba3 and vfs_glusterfs/libgfapi). Happy days! * I still see 4 lines of this sort of thing for each file read in /var/log/glusterfs/bricks/*.log: "[2013-10-11 04:01:37.554826] E [posix.c:2668:posix_getxattr] 0-prodbackup-posix: getxattr failed on /gback_brick0/vfx: system.posix_acl_default (No data available)" I have the following Gluster options set: diagnostics.client-log-level: ERROR diagnostics.brick-log-level: ERROR When Samba3 is configured to read from a FUSE mount instead (which has been configured at mount time with "-o log-level=CRITICAL"), the errors do no appear in the brick log. Next up is trying to find a way to tell Samba not to pass on getxattr when vfs_glusterfs is used. Stacking VFS module "xattr_tdb" has no effect with respect to this error (xattr_tdb works, but the errors still fill the brick log). -Dan On 4 October 2013 09:21, Dan Mons <dmons at cuttingedge.com.au> wrote: > Thanks for the pointers Dave. That got me to compiling the module. > > I'm still using the RPM-installed GlusterFS and Samba, but have > compiled/installed matching source versions into a --prefix=/localdev > folder and used that to build the samba-glusterfs-vfs module with > relevant paths during the "./configure" step. > > The module was copied to /usr/lib64/samba/vfs, vfs_glusterfs.so was > symlinked to glusterfs.so, and appears to be working. > > Relevant smb.conf > > [sharename] > vfs object = glusterfs > glusterfs:volume = volumename > path = / > > Testing so far has been much faster for read/browse operations, so > that much is good. > > However, bugs that already make it unusable for me at the moment: > > * renaming files and folders from a Windows host gives a "does not > exist" error in the Explorer GUI, and a "duplicate filename exists" > error from cmd.exe. > > * Brick logs fill up with 4 lines of "getxattr failed" messages for > each file or folder read, and this is with all GlusterFS error log > levels already set to "critical". With a network full of users, I'm > going to fill disks with logs in minutes. I had seen this reported in > 3.3.1 and marked as fixed in 3.4.0, but it appears to still be around. > I have XFS formatted disk backing each brick with 512 byte inodes and > default mount options. I don't wish to turn on user_xattr, as I want > to just discard any of that sort of data coming from Mac and Windows > hosts, as it's unnecessary for our production data. But at the same > time, I don't want GlusterFS telling me it's an error when it's not > there (INFO level would suffice, so I can ignore that in my logs). > > -Dan > > > On 3 October 2013 10:59, David Gibbons <david.c.gibbons at gmail.com> wrote: >> Dan, >> >> I had the same trouble yesterday. As it happens, I created a doc to help >> script installs of future nodes. I did not snip out the portions that apply >> to what you're looking for, but the script below works for me. >> >> The biggest issue was that some modules were apparently installed in lib and >> the build process was looking for them in lib64. In any event, the vfs >> module builds, installs and runs cleanly after this. >> >> ** The big win for me was finding this command, that allowed me to figure >> out where it was looking for modules in the wrong lib directory. >> ldd /usr/local/samba/lib/vfs/glusterfs.so >> >> I'm sure there is any easier way to to do this :)... >> >> Cheers, >> Dave >> >> >> -- >> >> #!/bin/bash >> >> yum groupinstall "Development Tools" >> >> yum install git openssl-devel wget >> >> yum install libtalloc libtdb >> >> >> # set up gluster >> >> cd /usr/src && git clone https://github.com/gluster/glusterfs.git >> >> cd /usr/src/glusterfs && ./autogen.sh && ./configure && make >> >> make install >> >> >> # set up samba 3.6.9 >> >> cd /usr/src && wget http://ftp.samba.org/pub/samba/stable/samba-3.6.9.tar.gz >> && tar -zxvf samba-3.6.9.tar.gz >> >> cd /usr/src/samba-3.6.9/source3 && ./configure && make >> >> make install >> >> >> ln -s /usr/local/samba/lib/libwbclient.so.0 /usr/lib64/libwbclient.so.0 >> >> >> # then install the RPM samba version >> >> yum install samba >> >> >> # set up vfs_glusterfs >> >> cd /usr/src && git clone >> git://forge.gluster.org/samba-glusterfs/samba-glusterfs-vfs.git >> >> >> ln -s /usr/local/include/glusterfs /usr/include/glusterfs >> >> >> cd /usr/src/samba-vfs/glusterfs >> >> ./configure --with-samba-source=/usr/src/samba-3.6.9/source3 >> >> >> ln -s /usr/local/samba/lib/vfs/glusterfs.so >> /usr/lib64/samba/vfs/glusterfs.so >> >> >> # link the other modules >> >> ln -s /usr/local/lib/libgfapi.so /usr/lib64/ >> >> ln -s /usr/local/lib/libgfapi.so.0 /usr/lib64/ >> >> ln -s /usr/local/lib/libgfapi.la /usr/lib64/ >> >> ln -s /usr/local/lib/libglusterfs.la /usr/lib64/ >> >> ln -s /usr/local/lib/libglusterfs.so /usr/lib64/ >> >> ln -s /usr/local/lib/libglusterfs.so.0 /usr/lib64/ >> >> ln -s /usr/local/lib/libglusterfs.so.0.0.0 /usr/lib64/ >> >> EOF >> >> >> On Tue, Oct 1, 2013 at 10:38 PM, Dan Mons <dmons at cuttingedge.com.au> wrote: >>> >>> Hi folks, >>> >>> I've got CentOS6.4 with Samba 3.6.9 installed from the standard CentOS >>> repos via yum. I also have GlusterFS 3.4.0 GA installed from RPMs >>> direct from gluster.org. >>> >>> I'm trying to build the glusterfs VFS module for Samba to take >>> advantage of libgfapi for our Windows users, and migrate them off the >>> current Samba-on-FUSE setup we have currently. >>> >>> I've downloaded the appropriate source trees for all projects >>> (GlusterFS from gluster.org, Samba from the matching CentOS6 SRPM, and >>> samba-glusterfs-vfs from the git repo), but am facing troubles early >>> on just finding appropriate headers. >>> >>> [root at bne-gback000 samba-glusterfs-vfs]# find >>> /usr/local/src/glusterfs-3.4.0 -type f -name glfs.h >>> /usr/local/src/glusterfs-3.4.0/api/src/glfs.h >>> >>> [root at bne-gback000 samba-glusterfs-vfs]# ./configure >>> --with-glusterfs=/usr/local/src/glusterfs-3.4.0 >>> *snip* >>> checking api/glfs.h usability... no >>> checking api/glfs.h presence... no >>> checking for api/glfs.h... no >>> Cannot find api/glfs.h. Please specify --with-glusterfs=dir if necessary >>> >>> If I install glusterfs-api-devel-3.4.0-8.el6.x86_64.rpm, I need to >>> copy /usr/include/glusterfs/api/glfs.h to /usr/include for it to be >>> found (even using --with-glusterfs= doesn't work), and then I get >>> further errors about not being able to link to glfs_init: >>> >>> [root at bne-gback000 samba-glusterfs-vfs]# rpm -ivh >>> /tmp/glusterfs-api-devel-3.4.0-8.el6.x86_64.rpm >>> [root at bne-gback000 samba-glusterfs-vfs]# cp >>> /usr/include/glusterfs/api/glfs.h /usr/include/ >>> [root at bne-gback000 samba-glusterfs-vfs]# ./configure >>> *snip* >>> checking api/glfs.h usability... yes >>> checking api/glfs.h presence... yes >>> checking for api/glfs.h... yes >>> checking for glfs_init... no >>> Cannot link to gfapi (glfs_init). Please specify --with-glusterfs=dir >>> if necessary >>> >>> If anyone can point me in the right direction, that would be greatly >>> appreciated. >>> >>> Cheers, >>> >>> -Dan >>> _______________________________________________ >>> Gluster-users mailing list >>> Gluster-users at gluster.org >>> http://supercolony.gluster.org/mailman/listinfo/gluster-users >> >> > > > > -- > Dan Mons > R&D SysAdmin > Cutting Edge > http://cuttingedge.com.au -- Dan Mons R&D SysAdmin Cutting Edge http://cuttingedge.com.au
