On Thu, Sep 27, 2018 at 7:37 AM Chad W Seys <cwseys@xxxxxxxxxxxxxxxx> wrote:
Hi all,
I am exporting cephfs using samba. It is much slower over samba than
direct. Anyone know how to speed it up?
Benchmarked using bonnie++ 5 times either directly to cephfs mounted
by kernel (v4.18.6) module:
bonnie++ -> kcephfs
or through a cifs kernel-module-mounted (protocol version 3.02) Samba
(v4.8.5) share on the same machine.
bonnie++ -> Samba -> kcephfs
Abbreviated results for 5 runs:
kcephfs: min max
file created 555 619 files/sec
sequential block input: 106.44 108.13 MB/sec
sequential block output: 102.82 110.61 MB/sec
(There is a gigabit network between the client and the ceph cluster, so
the block in/out is pleasing.)
samba -> kcephfs: min max
file created 4 5 files/sec
sequential block input: 22.85 29.5 MB/sec
sequential block output: 27.95 30.01 MB/sec
The block input/output is okay fast, but the files created per second is
low. Anyone know how to tweak Samba to speed it up?
That seems to be a samba tuning issue, which this isn't the right list for. :/
Would Samba vfs_ceph speed up access? At the moment vfs_ceph in
Debian depends on libceph1 10.2.5, so not too modern.
Well, good news and bad news here. Good news is that since vfs_ceph uses the native cephfs client library, it ought to be faster (though I don't have any data on how much going through samba itself costs against our client).
Bad news is that samba's connection model doesn't play very nicely with Ceph's — if you use vfs_ceph, every samba connection will turn into a brand new Ceph connection — smb literally runs a fork on every incoming connection — so you will see RAM/network connections/etc scale on a per-smb-client basis. This works fine for a small number of clients but not so great if you're planning to attach a bunch of them. :(
-Greg
Current Samba settings:
[global]
dns proxy = No
hostname lookups = Yes
kerberos method = secrets and keytab
logging = syslog@1 /var/log/samba/log.%m
max log size = 100000
panic action = "" %d
realm = PHYSICS.WISC.EDU
security = USER
server signing = required
server string = %h server
workgroup = PHYSICS
fruit:nfs_aces = no
idmap config * : backend = tdb
[smb]
ea support = Yes
inherit acls = Yes
inherit permissions = Yes
msdfs root = Yes
path = /srv/smb
read _only_ = No
smb encrypt = desired
vfs objects = catia fruit streams_xattr
fruit:encoding = native
Thanks!
Chad.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
