Hi, Yeah the negative pid is interesting. AFAICT we use a negative pid to indicate that the lock was taken on another host: https://github.com/torvalds/linux/blob/master/fs/ceph/locks.c#L119 https://github.com/torvalds/linux/commit/9d5b86ac13c573795525ecac6ed2db39ab23e2a8 "Finally, we convert remote filesystems to present remote pids using negative numbers. Have lustre, 9p, ceph, cifs, and dlm negate the remote pid returned for F_GETLK lock requests." The good news is that my colleagues managed to clear this filelock by restarting dovecot on a couple nodes. But I'm still curious if others have a nice way to debug such things. Cheers, Dan On Mon, Nov 9, 2020 at 8:11 PM Anthony D'Atri <anthony.datri@xxxxxxxxx> wrote: > > Looks like a - in front of the 9605 — signed/unsigned int flern? > > > On Nov 9, 2020, at 4:59 AM, Dan van der Ster <dan@xxxxxxxxxxxxxx> wrote: > > > > Hi all, > > > > MDS version v14.2.11 > > Client kernel 3.10.0-1127.19.1.el7.x86_64 > > > > We are seeing a strange issue with a dovecot use-case on cephfs. > > Occasionally we have dovecot reporting a file locked, such as: > > > > Nov 09 13:55:00 dovecot-backend-00.cern.ch dovecot[27710]: > > imap(reguero)<23945><fRA6B6yznq68uE28>: Error: Mailbox Deleted Items: > > Timeout (180s) while waiting for lock for transaction log file > > /mail/users/r/reguero//mdbox/mailboxes/Deleted > > Items/dbox-Mails/dovecot.index.log (WRITE lock held by pid -9605) > > > > We checked all hosts that have mounted the cephfs -- there is no pid 9605. > > > > Is there any way to see who exactly created the lock? ceph_filelock > > has a client id, but I didn't find a way to inspect the > > cephfs_metadata to see the ceph_filelock directly. > > > > Otherwise, are other Dovecot/CephFS users seeing this? Did you switch > > to flock or lockfile instead of fnctlk locks? > > > > Thanks! > > > > Dan > > > > P.S. here is the output from print locks tool from the kernel client: > > > > Read lock: > > Type: 1 (0: Read, 1: Write, 2: Unlocked) > > Whence: 0 (0: start, 1: current, 2: end) > > Offset: 0 > > Len: 1 > > Pid: -9605 > > Write lock: > > Type: 1 (0: Read, 1: Write, 2: Unlocked) > > Whence: 0 (0: start, 1: current, 2: end) > > Offset: 0 > > Len: 1 > > Pid: -9605 > > > > and same file from a 15.2.5 fuse client : > > > > Read lock: > > Type: 1 (0: Read, 1: Write, 2: Unlocked) > > Whence: 0 (0: start, 1: current, 2: end) > > Offset: 0 > > Len: 0 > > Pid: 0 > > Write lock: > > Type: 1 (0: Read, 1: Write, 2: Unlocked) > > Whence: 0 (0: start, 1: current, 2: end) > > Offset: 0 > > Len: 0 > > Pid: 0 > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |