On Wed, Aug 21, 2019 at 04:48:10PM -0300, Jason Gunthorpe wrote: > On Wed, Aug 21, 2019 at 11:57:03AM -0700, Ira Weiny wrote: > > > > Oh, I didn't think we were talking about that. Hanging the close of > > > the datafile fd contingent on some other FD's closure is a recipe for > > > deadlock.. > > > > The discussion between Jan and Dave was concerning what happens when a user > > calls > > > > fd = open() > > fnctl(...getlease...) > > addr = mmap(fd...) > > ib_reg_mr() <pin> > > munmap(addr...) > > close(fd) > > I don't see how blocking close(fd) could work. Well Dave was saying this _could_ work. FWIW I'm not 100% sure it will but I can't prove it won't.. Maybe we are all just touching a different part of this elephant[1] but the above scenario or one without munmap is very reasonably something a user would do. So we can either allow the close to complete (my current patches) or try to make it block like Dave is suggesting. I don't disagree with Dave with the semantics being nice and clean for the filesystem. But the fact that RDMA, and potentially others, can "pass the pins" to other processes is something I spent a lot of time trying to work out. > > Write it like this: > > fd = open() > uverbs = open(/dev/uverbs) > fnctl(...getlease...) > addr = mmap(fd...) > ib_reg_mr() <pin> > munmap(addr...) > <sigkill> > > The order FD's are closed during sigkill is not deterministic, so when > all the fputs happen during a kill'd exit we could end up blocking in > close(fd) as close(uverbs) will come after in the close > list. close(uverbs) is the thing that does the dereg_mr and releases > the pin. Of course, that is a different scenario which needs to be fixed in my patch set. Now that my servers are back up I can hopefully make progress. (Power was down for them yesterday). > > We don't need complexity with dup to create problems. No but that complexity _will_ come unless we "zombie" layout leases. Ira [1] https://en.wikipedia.org/wiki/Blind_men_and_an_elephant > > Jason >
