Re: vfs_gluster broken

Terry McGuire <tmcguire@xxxxxxxxxxx> · Thu, 20 Sep 2018 14:58:38 -0600

On Sep 19, 2018, at 06:37, Anoop C S <anoopcs@xxxxxxxxxxxxx> wrote:

On Wed, 2018-09-12 at 10:37 -0600, Terry McGuire wrote:
Can you please attach the output of `testparm -s` so as to look through how Samba is setup?

I have a setup where I could browse and work with a GlusterFS volume share made available to Windows
via vfs_glusterfs module on CentOS 7.5.1804 with glusterfs-3.10.12-1.el7 and samba-4.7.1-9.el7_5.

What am I missing? Are there any specific operation that leads to abnormal behaviours?

It seems that, when using smbclient to keep things simple, the first command in the share’s root works, even if it’s a write.  The following commands, even if it’s just an ls, fails.

One difference that might be making the difference is that my gluster volume is distributed-dispersed.  This seems also to break the vfs_fruit module.

From our test server (“nomodule-nofruit” is currently the only well-behaved share):

root@mfsuat-01 ~]#testparm -s
Load smb config files from /etc/samba/smb.conf
rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
Processing section "[share1]"
Processing section "[share2]"
Processing section "[nomodule]"
Processing section "[nomodule-nofruit]"
Processing section "[module]"
Processing section "[IPC$]"
WARNING: No path in service IPC$ - making it unavailable!
NOTE: Service IPC$ is flagged unavailable.

On an unrelated note:

I don't think your intention to make [IPC$] unavailable using the 'available' parameter would work
at all.

I assume this IPC$ stuff is just an artifact of how I built my smb.conf file.  I put "vfs objects = glusterfs” into the global stanza, so it didn’t need to be added to each share stanza, but I discovered that this broke IPC$ in such a way as to make no share accessible.  To fix this, I added a share stanza for IPC$ that includes only “vfs objects =".  Maybe that’s dumb, but it worked, and, I guess, results in what we see here.

Loaded services file OK.
idmap range not specified for domain '*'
ERROR: Invalid idmap range for domain *!

On an unrelated note:

Why haven't you specified range for default configuration? I think it is a must to set range for the
default configuration.

I guess because I’m clueless enough not to know that I should.  We’re authenticating samba users from Windows Active Directory, and I’m really clueless about that.  We got things to the point that they were working, and called it a day.

WARNING: You have some share names that are longer than 12 characters.
These may not be accessible to some older clients.
(Eg. Windows9x, WindowsMe, and smbclient prior to Samba 3.0.)
WARNING: some services use vfs_fruit, others don't. Mounting them in conjunction on OS X clients
results in undefined behaviour.

Server role: ROLE_DOMAIN_MEMBER

# Global parameters
[global]
	log file = /var/log/samba/log.%m
	map to guest = Bad User
	max log size = 50
	realm = XXXX.AD.UALBERTA.CA
	security = ADS
	workgroup = STS
	glusterfs:volume = mfs1
	idmap config * : backend = tdb
	access based share enum = Yes
	force create mode = 0777
	force directory mode = 0777
	include = /mfsmount/admin/etc/mfs/smb_shares.conf
	kernel share modes = No
	read _only_ = No
	smb encrypt = desired
	vfs objects = glusterfs
[share1]
	path = /share1
	valid users = @mfs-sa1@xxxxxxxxxxxxxxxxxxx
[share2]
	path = /share2
	valid users = @mfs-test-group@xxxxxxxxxxxxxxxxxxx

Oh.. you are sharing sub-directories which is also fine.

We *only* share subdirs of the gluster volume, never the volume itself.

[nomodule]
	kernel share modes = Yes
	path = /mfsmount/share1
	valid users = @mfs-sa1@xxxxxxxxxxxxxxxxxxx
	vfs objects = fruit streams_xattr

Interesting..

Even this FUSE mounted GlusterFS share is not behaving normal? What errors do you see in glusterfs
fuse mount log(/var/log/glusterfs/mfsmount-.log) while accessing this share?

This one isn’t affected by the current samba issue, but it’s affected by another issue.  (I actually posted about it a while ago - subject “vfs_fruit and extended attributes”- and I think you responded, but it didn’t go anywhere.)  We discovered that the vfs_fruit module seems to interact poorly with our distributed-dispersed gluster volume.  To learn this, we made a test replicated volume and it worked fine.

Ironically, we decided we could live without vfs_fruit when we discovered vfs_glusterfs.  Now we have neither.  :-(

[nomodule-nofruit]
	kernel share modes = Yes
	path = /mfsmount/share1
	valid users = @mfs-sa1@xxxxxxxxxxxxxxxxxxx
	vfs objects = 

[module]
	path = /share1
	valid users = @mfs-sa1@xxxxxxxxxxxxxxxxxxx
[IPC$]
	available = No
	vfs objects = 

You may remove the whole [IPC$] section.

IPC$ stuff explained above.

Terry

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users