Hi all, Just wondering if the original issue has been resolved. I have the same problems with inconsistent nfs and samba directory listings. I'm running Hammer. Is it a confirmed seekdir bug in de kernel client? On 01/14/2016 04:05 AM, Yan, Zheng wrote: > On Thu, Jan 14, 2016 at 3:37 AM, Mike Carlson <mike@xxxxxxxxxxxx> wrote: >> Hey Greg, >> >> The inconsistent view is only over nfs/smb on top of our /ceph mount. >> >> When I look directly on the /ceph mount (which is using the cephfs kernel >> module), everything looks fine >> >> It is possible that this issue just went unnoticed, and it only being a >> infernalis problem is just a red herring. With that, it is oddly >> coincidental that we just started seeing issues. > > This seems like seekdir bugs in kernel client, could you try 4.0+ kernel. > > Besides, do you enable "mds bal frag" for ceph-mds > > > Regards > Yan, Zheng > > > >> >> On Wed, Jan 13, 2016 at 11:30 AM, Gregory Farnum <gfarnum@xxxxxxxxxx> wrote: >>> >>> On Wed, Jan 13, 2016 at 11:24 AM, Mike Carlson <mike@xxxxxxxxxxxx> wrote: >>>> Hello. >>>> >>>> Since we upgraded to Infernalis last, we have noticed a severe problem >>>> with >>>> cephfs when we have it shared over Samba and NFS >>>> >>>> Directory listings are showing an inconsistent view of the files: >>>> >>>> >>>> $ ls /lts-mon/BD/xmlExport/ | wc -l >>>> 100 >>>> $ sudo umount /lts-mon >>>> $ sudo mount /lts-mon >>>> $ ls /lts-mon/BD/xmlExport/ | wc -l >>>> 3507 >>>> >>>> >>>> The only work around I have found is un-mounting and re-mounting the nfs >>>> share, that seems to clear it up >>>> Same with samba, I'd post it here but its thousands of lines. I can add >>>> additional details on request. >>>> >>>> This happened after our upgrade to infernalis. Is it possible the MDS is >>>> in >>>> an inconsistent state? >>> >>> So this didn't happen to you until after you upgraded? Are you seeing >>> missing files when looking at cephfs directly, or only over the >>> NFS/Samba re-exports? Are you also sharing Samba by re-exporting the >>> kernel cephfs mount? >>> >>> Zheng, any ideas about kernel issues which might cause this or be more >>> visible under infernalis? >>> -Greg >>> >>>> >>>> We have cephfs mounted on a server using the built in cephfs kernel >>>> module: >>>> >>>> lts-mon:6789:/ /ceph ceph >>>> name=admin,secretfile=/etc/ceph/admin.secret,noauto,_netdev >>>> >>>> >>>> We are running all of our ceph nodes on ubuntu 14.04 LTS. Samba is up to >>>> date, 4.1.6, and we export nfsv3 to linux and freebsd systems. All seem >>>> to >>>> exhibit the same behavior. >>>> >>>> system info: >>>> >>>> # uname -a >>>> Linux lts-osd1 3.13.0-63-generic #103-Ubuntu SMP Fri Aug 14 21:42:59 UTC >>>> 2015 x86_64 x86_64 x86_64 GNU/Linux >>>> root@lts-osd1:~# lsb >>>> lsblk lsb_release >>>> root@lts-osd1:~# lsb_release -a >>>> No LSB modules are available. >>>> Distributor ID: Ubuntu >>>> Description: Ubuntu 14.04.3 LTS >>>> Release: 14.04 >>>> Codename: trusty >>>> >>>> >>>> package info: >>>> >>>> # dpkg -l|grep ceph >>>> ii ceph 9.2.0-1trusty >>>> amd64 distributed storage and file system >>>> ii ceph-common 9.2.0-1trusty >>>> amd64 common utilities to mount and interact with a ceph storage >>>> cluster >>>> ii ceph-fs-common 9.2.0-1trusty >>>> amd64 common utilities to mount and interact with a ceph file >>>> system >>>> ii ceph-mds 9.2.0-1trusty >>>> amd64 metadata server for the ceph distributed file system >>>> ii libcephfs1 9.2.0-1trusty >>>> amd64 Ceph distributed file system client library >>>> ii python-ceph 9.2.0-1trusty >>>> amd64 Meta-package for python libraries for the Ceph libraries >>>> ii python-cephfs 9.2.0-1trusty >>>> amd64 Python libraries for the Ceph libcephfs library >>>> >>>> >>>> What is interesting, is a directory or file will not show up in a >>>> listing, >>>> however, if we directly access the file, it shows up in that instance: >>>> >>>> >>>> # ls -al |grep SCHOOL >>>> # ls -alnd SCHOOL667055 >>>> drwxrwsr-x 1 21695 21183 2962751438 Jan 13 09:33 SCHOOL667055 >>>> >>>> >>>> Any tips are appreciated! >>>> >>>> Thanks, >>>> Mike C >>>> >>>> >>>> _______________________________________________ >>>> ceph-users mailing list >>>> ceph-users@xxxxxxxxxxxxxx >>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>> >> >> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Kramer M.D. Infrastructure Engineer ........................................................................ Nederlands Forensisch Instituut Digitale Technologie & Biometrie Laan van Ypenburg 6 | 2497 GB | Den Haag Postbus 24044 | 2490 AA | Den Haag ........................................................................ T 070 888 64 30 M 06 29 62 12 02 d.kramer@xxxxxxxxxxxxxx / dennis@xxxxxxxxx PGP publickey: http://www.holmes.nl/dennis.asc www.forensischinstituut.nl ........................................................................ Nederlands Forensisch Instituut. In feiten het beste. ........................................................................ _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |