> On Dec 4, 2016, at 11:46 PM, Dave Chinner <david@xxxxxxxxxxxxx> wrote: > > On Sun, Dec 04, 2016 at 05:47:04PM -0800, Cyril Peponnet wrote: >>> On Dec 4, 2016, at 5:22 PM, Dave Chinner <david@xxxxxxxxxxxxx> >>> wrote: What deadlock is that? XFS is reporting memory allocation >>> issues, not that there is a filesystem deadlock. Your comments >>> that dropping caches make the problem go away indicate that >>> there isn't any deadlock, just blocking on memory allocation >>> that is taking a long time to resolve… >> >> You are right by deadlock I meant that mount point hang for ls >> commands for instance (from the server it self) and basically the >> glusterfs mount on the hypervisors is also hanging for all vms (it >> makes the vms to remount RO). > > So everything is /blocked/, not deadlocked. If the memory allocation > then makes progress, we're all ok. > >> On this server I have 1200 living snapshots, usually not too much >> write in it except for some of them… >> >> Is there a way to optimize the memory allocation? Or we likely >> need to add more ram… > > More RAM won't help - it's likely a memory fragmentation problem > made worse by the fact that older kernels won't do memory compaction > (i.e. memory defrag) when high order memory allocation fails. If the > problem is file fragmentation, then the best you can probably do > right now is limit fragmentation. > > Stopping qcow2 from fragmenting the crap out of the files will > prevent the problem from re-occurring. This is the primary use case > for extent size hints in XFS these days, though I know you can't > actually control the gluster back end to use this. perhaps there are > other tweaks to qcow formats (cluster size?) or gluster config to > limit the amount of fragmentation…. I had the issue again but I don’t have more output in dmesg or journalctl even with the echo 11 > /proc/sys/fs/xfs/error_level set. Is there another location where I should look at ? Thanks > > Cheers, > > Dave. > -- > Dave Chinner > david@xxxxxxxxxxxxx -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
