On Mon, 2019-04-29 at 07:42 -0400, Amir Goldstein wrote: > On Sun, Apr 28, 2019 at 8:57 PM Trond Myklebust < > trondmy@xxxxxxxxxxxxxxx> wrote: > > On Sun, 2019-04-28 at 18:33 -0400, Amir Goldstein wrote: > > > On Sun, Apr 28, 2019 at 6:08 PM Trond Myklebust < > > > trondmy@xxxxxxxxxxxxxxx> wrote: > > > > On Sun, 2019-04-28 at 18:00 -0400, Amir Goldstein wrote: > > > > > On Sun, Apr 28, 2019 at 11:06 AM Trond Myklebust > > > > > <trondmy@xxxxxxxxxxxxxxx> wrote: > > > > > > On Sun, 2019-04-28 at 09:45 -0400, Amir Goldstein wrote: > > > > > > > On Sun, Apr 28, 2019 at 8:09 AM Jeff Layton < > > > > > > > jlayton@xxxxxxxxxx> > > > > > > > wrote: > > > > > > > > On Sat, 2019-04-27 at 16:16 -0400, Amir Goldstein > > > > > > > > wrote: > > > > > > > > > [adding back samba/nfs and fsdevel] > > > > > > > > > > > > > > > > > > > > > > > > > cc'ing Pavel too -- he did a bunch of work in this area > > > > > > > > a > > > > > > > > few > > > > > > > > years > > > > > > > > ago. > > > > > > > > > > > > > > > > > On Fri, Apr 26, 2019 at 6:22 PM Jeff Layton < > > > > > > > > > jlayton@xxxxxxxxxx> > > > > > > > > > wrote: > > > > > > > > > > On Fri, 2019-04-26 at 10:50 -0400, J. Bruce Fields > > > > > > > > > > wrote: > > > > > > > > > > > On Fri, Apr 26, 2019 at 04:11:00PM +0200, Amir > > > > > > > > > > > Goldstein > > > > > > > > > > > wrote: > > > > > > > > > > > > On Fri, Apr 26, 2019, 4:00 PM J. Bruce Fields < > > > > > > > > > > > > bfields@xxxxxxxxxxxx> wrote: > > > > > > > > > > > > > > > > > > > > > > That said, we could also look at a vfs-level mount > > > > > > > > > > option > > > > > > > > > > that > > > > > > > > > > would > > > > > > > > > > make the kernel enforce these for any opener. That > > > > > > > > > > could > > > > > > > > > > also > > > > > > > > > > be useful, > > > > > > > > > > and shouldn't be too hard to implement. Maybe even > > > > > > > > > > make > > > > > > > > > > it > > > > > > > > > > a > > > > > > > > > > vfsmount- > > > > > > > > > > level option (like -o ro is). > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yeh, I am humbly going to leave this struggle to > > > > > > > > > someone > > > > > > > > > else. > > > > > > > > > Not important enough IMO and completely independent > > > > > > > > > effort to > > > > > > > > > the > > > > > > > > > advisory atomic open&lock API. > > > > > > > > > > > > > > > > Having the kernel allow setting deny modes on any open > > > > > > > > call > > > > > > > > is > > > > > > > > a > > > > > > > > non- > > > > > > > > starter, for the reasons Bruce outlined earlier. This > > > > > > > > _must_ be > > > > > > > > restricted in some fashion or we'll be opening up a > > > > > > > > ginormous > > > > > > > > DoS > > > > > > > > mechanism. > > > > > > > > > > > > > > > > My proposal was to make this only be enforced by > > > > > > > > applications > > > > > > > > that > > > > > > > > explicitly opt-in by setting O_SH*/O_EX* flags. It > > > > > > > > wouldn't > > > > > > > > be > > > > > > > > too > > > > > > > > difficult to also allow them to be enforced on a per-fs > > > > > > > > basis > > > > > > > > via > > > > > > > > mount > > > > > > > > option or something. Maybe we could expand the meaning > > > > > > > > of > > > > > > > > '-o > > > > > > > > mand' > > > > > > > > ? > > > > > > > > > > > > > > > > How would you propose that we restrict this? > > > > > > > > > > > > > > > > > > > > > > Our communication channel is broken. > > > > > > > I did not intend to propose any implicit locking. > > > > > > > If samba and nfsd can opt-in with O_SHARE flags, I do not > > > > > > > understand why a mount option is helpful for the cause of > > > > > > > samba/nfsd interop. > > > > > > > > > > > > > > If someone else is interested in samba/local interop than > > > > > > > yes, a mount option like suggested by Pavel could be a > > > > > > > good > > > > > > > option, > > > > > > > but it is an orthogonal effort IMO. > > > > > > > > > > > > If an NFS client 'opts in' to set share deny, then that > > > > > > still > > > > > > makes > > > > > > it > > > > > > a non-optional lock for the other NFS clients, because all > > > > > > ordinary > > > > > > open() calls will be gated by the server whether or not > > > > > > their > > > > > > application specifies the O_SHARE flag. There is no flag in > > > > > > the > > > > > > NFS > > > > > > protocol that could tell the server to ignore deny modes. > > > > > > > > > > > > IOW: it would suffice for 1 client to use O_SHARE|O_DENY* > > > > > > to > > > > > > opt > > > > > > all > > > > > > the other clients in. > > > > > > > > > > > > > > > > Sorry for being thick, I don't understand if we are in > > > > > agreement > > > > > or > > > > > not. > > > > > > > > > > My understanding is that the network file server > > > > > implementations > > > > > (i.e. samba, knfds, Ganesha) will always use share/deny > > > > > modes. > > > > > So for example nfs v3 opens will always use O_DENY_NONE > > > > > in order to have correct interop with samba and nfs v4. > > > > > > > > > > If I am misunderstanding something, please enlighten me. > > > > > If there is a reason why mount option is needed for the sole > > > > > purpose > > > > > of interop between network filesystem servers, please > > > > > enlighten > > > > > me. > > > > > > > > > > > > > > > > > > Same difference. As long as nfsd and/or Ganesha are translating > > > > OPEN4_SHARE_ACCESS_READ and OPEN4_SHARE_ACCESS_WRITE into share > > > > access > > > > locks, then those will conflict with any deny locks set by > > > > whatever > > > > application that uses them. > > > > > > > > IOW: any open(O_RDONLY) and open(O_RDWR) will conflict with an > > > > O_DENY_READ that is set on the server, and any open(O_WRONLY) > > > > and > > > > open(O_RDWR) will conflict with an O_DENY_WRITE that is set on > > > > the > > > > server. There is no opt-out for NFS clients on this issue, > > > > because > > > > stateful NFSv4 opens MUST set one or more of > > > > OPEN4_SHARE_ACCESS_READ > > > > and OPEN4_SHARE_ACCESS_WRITE. > > > > > > > > > > Urgh! I *think* I understand the confusion. > > > > > > I believe Jeff was talking about implementing a mount option > > > similar to -o mand for local fs on the server. > > > With that mount option, *any* open() by any app of file from > > > that mount will use O_DENY_NONE to interop correctly with > > > network servers that explicitly opt-in for interop on share > > > modes. > > > I agree its a nice feature that is easy to implement - not > > > important > > > for first version IMO. > > > > > > I *think* you are talking on nfs client mount option for > > > opt-in/out of share modes? there was no such intention. > > > > > > > No. I'm saying that whether you intended to or not, you _are_ > > implementing a mandatory lock over NFS. No talk about O_SHARE flags > > and > > it being an opt-in process for local applications changes the fact > > that > > non-local applications (i.e. the ones that count ) are being > > subjected > > to a mandatory lock with all the potential for denial of service > > that > > implies. > > So we need a mechanism beyond O_SHARE in order to ensure this > > system > > cannot be used on sensitive files that need to be accessible to > > all. It > > could be an export option, or a mount option, or it could be a more > > specific mechanism (e.g. the setgid with no execute mode bit as > > using > > in POSIX mandatory locks). > > > > I see. Thanks for making that concern clear. > > If server owner wishes to have samba/nfs interop obviously > server owner should configure both samba and nfs for interop. > nfs should thus have it configurable via export options IMO > and not via mount option (server's responsibility). > > Preventing O_DENY_X on a certain file... hmm > We can do that but, if nfs protocol has O_DENY what's the > logic that we would want to override it? It was added in order to support Windows clients. There is also optional support for mandatory byte range locks. However the fact that the protocol supports it doesn't automatically make it a good idea. Design by committee... > What we need is a way to track, blame the resource holder and > release the resource administratively. > > For that matter, assuming the nfsd and smbd (etc) can contain > their own fds without leaking them to other modules (minus bugs) > then provided with sufficient sysfs/procfs info (i.e. Bruce's new > open > files tracking), admin should be able to kill the offending nfs/smb > client > to release the hogged file. > > I believe that is the Windows server solution to the DoS that is > implied > from O_DENY. > Relying on being able to access the clients is not good enough. In general, server admins tend not to have access to the clients. However it should indeed be possible to create a tool on the server to revoke locks and open state. Most commercial servers have that kind of functionality, and I would agree that makes sense for dealing with rogue processes. -- Trond Myklebust Linux NFS client maintainer, Hammerspace trond.myklebust@xxxxxxxxxxxxxxx
