On 12/15/2012 04:25 PM, Dave Chinner wrote: > On Fri, Dec 14, 2012 at 11:13:00AM -0500, Brian Foster wrote: >> On 12/12/2012 09:25 PM, Dave Chinner wrote: >>> On Wed, Dec 05, 2012 at 11:47:57AM -0500, Brian Foster wrote: >> ... ... >> >> I'm not following you here. xfs_prealloc_dquot_max() should return the >> maximum allowed preallocation for a quota by calculating some percentage >> of the free space in the quota (regardless of the existence of a soft >> limit; that serves as a throttling watermark and percentage modifier). > > > Say you have a quota and 50GB of free space and we are trying to > preallocate 8GB of that, xfs_prealloc_dquot_max() will always > return a max of 2.5GB. Hence preallocation will be limited even > though there's plenty of space available for a preallocation that is > larger than this.... > >> xfs_prealloc_quota_max() just extends that across all quota types and >> returns the most limiting value. For an inode with plenty of space in >> its quota set, we should get a large value back for max_quota_prealloc, >> which is then ignored because alloc_blocks doesn't violate that limit. > > But if we are trying to do a large file allocation (as per above), > it does limit it atificially. IOWs, files without quota soft limits set by > the user are treated as _always over the soft limit_. > Ok, I interpreted your comment originally as a bug in the implementation, but you're describing a more general flaw in the linear throttling approach. So the logarithmic approach should allow us to run harder and faster into the real EDQUOT condition, likely with the same effective results via the more aggressive throttling (considering since we do this already for ENOSPC). Point taken, thanks for the explanation. ... > >> Are you suggesting we >> set a default soft limit value on all quotas with a hard limit? > > Yes. Your code already does this, effectively, by treating files > that have no softlimit set as always being over the soft limit. > True, but only with respect to preallocation throttling. IOW, presumably there are other reasons to set (or not set) a soft limit from an administrative standpoint. Though I guess there's no reason to have a timer with no soft limit, so perhaps it doesn't matter. I'll play with it... To further clarify, is there any special behavior intended for a soft limit set in this manner? For example, the user creates quota, sets a hard limit and we set the new associated default soft limit. Should the new soft limit behave precisely as if the soft limit was set by the user from this point forward, or could it for example, remain volatile to the hard limit unless a soft limit value is explicitly set by the user (i.e., it is not written to disk and it is recalculated in memory if the filesystem is remounted; it is re-adjusted automatically if the hard limit is adjusted iff soft-limit hasn't been set explicitly, etc.). >> I'm a little unsure about applying a shift selected for one limit >> against the preallocation size of another, but I suppose it can't hurt >> to be aggressive. > > Which is what this sort of throttling needs to be. > > Essentially, the idea is that if we are close to a quota limit and > ENOSPC at the same time, whichever limit we are closer to exceeding > should determine the amount of throttling that will take place (i.e. > the shift value). > > The size of the preallocation is a separate calculation to this - > the quota thresholds only ever reduce the size, but that doesn't > change how close we are to throttle thresholds. IOWs, even if the > base prealloc is smaller than the max quota prealloc size, we still > should throttle it according to the quota throttle if we are closer > to exceeding the quota thresholds than we are the ENOSPC > thresholds... > Ok, sounds logical. Thanks again. Brian >> This design should bubble up all the relevant >> parameters to a single point anyways, so that should be easier to reason >> about and measure when I have some code. > > Exactly. You've just put words to the concept I was struggling to > describe. :) > > Cheers, > > Dave. > _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
