Re: [Cluster-devel] [PATCH 0/2] scop GFP_NOFS api

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

On 29/04/16 06:35, NeilBrown wrote:
On Tue, Apr 26 2016, Michal Hocko wrote:

Hi,
we have discussed this topic at LSF/MM this year. There was a general
interest in the scope GFP_NOFS allocation context among some FS
developers. For those who are not aware of the discussion or the issue
I am trying to sort out (or at least start in that direction) please
have a look at patch 1 which adds memalloc_nofs_{save,restore} api
which basically copies what we have for the scope GFP_NOIO allocation
context. I haven't converted any of the FS myself because that is way
beyond my area of expertise but I would be happy to help with further
changes on the MM front as well as in some more generic code paths.

Dave had an idea on how to further improve the reclaim context to be
less all-or-nothing wrt. GFP_NOFS. In short he was suggesting an opaque
and FS specific cookie set in the FS allocation context and consumed
by the FS reclaim context to allow doing some provably save actions
that would be skipped due to GFP_NOFS normally.  I like this idea and
I believe we can go that direction regardless of the approach taken here.
Many filesystems simply need to cleanup their NOFS usage first before
diving into a more complex changes.>
This strikes me as over-engineering to work around an unnecessarily
burdensome interface.... but without details it is hard to be certain.

Exactly what things happen in "FS reclaim context" which may, or may
not, be safe depending on the specific FS allocation context?  Do they
need to happen at all?

My research suggests that for most filesystems the only thing that
happens in reclaim context that is at all troublesome is the final
'evict()' on an inode.  This needs to flush out dirty pages and sync the
inode to storage.  Some time ago we moved most dirty-page writeout out
of the reclaim context and into kswapd.  I think this was an excellent
advance in simplicity.
If we could similarly move evict() into kswapd (and I believe we can)
then most file systems would do nothing in reclaim context that
interferes with allocation context.
evict() is an issue, but moving it into kswapd would be a potential problem for GFS2. We already have a memory allocation issue when recovery is taking place and memory is short. The code path is as follows:

 1. Inode is scheduled for eviction (which requires deallocation)
2. The glock is required in order to perform the deallocation, which implies getting a DLM lock
 3. Another node in the cluster fails, so needs recovery
4. When the DLM lock is requested, it gets blocked until recovery is complete (for the failed node)
 5. Recovery is performed using a userland fencing utility
 6. Fencing requires memory and then blocks on the eviction
7. Deadlock (Fencing waiting on memory alloc, memory alloc waiting on DLM lock, DLM lock waiting on fencing)

It doesn't happen often, but we've been looking at the best place to break that cycle, and one of the things we've been wondering is whether we could avoid deallocation evictions from memory related contexts, or at least make it async somehow.

The issue is that it is not possible to know in advance whether an eviction will result in mearly writing things back to disk (because the inode is being dropped from cache, but still resides on disk) which is easy to do, or whether it requires a full deallocation (n_link==0) which may require significant resources and time,

Steve.

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [Linux Filesystem Development]     [Linux USB Development]     [Linux Media Development]     [Video for Linux]     [Linux NILFS]     [Linux Audio Users]     [Yosemite Info]     [Linux SCSI]

  Powered by Linux