On Thu, Nov 24, 2016 at 09:13:28AM +1100, Dave Chinner wrote: > On Wed, Nov 23, 2016 at 08:55:59AM -0500, Zygo Blaxell wrote: > > On Wed, Nov 23, 2016 at 03:26:32PM +1100, Dave Chinner wrote: > > > On Tue, Nov 22, 2016 at 09:02:10PM -0500, Zygo Blaxell wrote: > > > > On Thu, Nov 17, 2016 at 04:07:48PM -0800, Omar Sandoval wrote: > > > > > 3. Both XFS and Btrfs cap each dedupe operation to 16MB, but the > > > > > implicit EOF gets around this in the existing XFS implementation. I > > > > > copied this for the Btrfs implementation. > > > > > > > > Somewhat tangential to this patch, but on the dedup topic: Can we raise > > > > or drop that 16MB limit? > > > > > > > > The maximum btrfs extent length is 128MB. Currently the btrfs dedup > > > > behavior for a 128MB extent is to generate 8x16MB shared extent references > > > > with different extent offsets to a single 128MB physical extent. > > > > These references no longer look like the original 128MB extent to a > > > > userspace dedup tool. That raises the difficulty level substantially > > > > for a userspace dedup tool when it tries to figure out which extents to > > > > keep and which to discard or rewrite. > > > > > > That, IMO, is a btrfs design/implementation problem, not a problem > > > with the API. Applications are always going to end up doing things > > > that aren't perfectly aligned to extent boundaries or sizes > > > regardless of the size limit that is placed on the dedupe ranges. > > > > Given that XFS doesn't have all the problems btrfs does, why does XFS > > have the same aribitrary size limit? Especially since XFS demonstrably > > doesn't need it? > > Creating a new-but-slightly-incompatible jsut for XFS makes no > sense - we have multiple filesystems that support this functionality > and so they all should use the same APIs and present (as far as is > possible) the same behaviour to userspace. OK. Let's just remove the limit on all the filesystems then. XFS doesn't need it, and btrfs can be fixed. > IOWs it's more important to use existing APIs than to invent a new > one that does almost the same thing. This way userspace applications > don't need to be changed to support new XFS functionality and we > make life easier for everyone. Except removing the limit doesn't work that way. An application that didn't impose an undocumented limit on itself wouldn't break when moved to a filesystem that imposed no such limit, i.e. if XFS had no limit, an application that moved from btrfs to XFS would just work. > A shiny new API without warts would > be nice, but we've already got to support the existing one forever, > it does the job we need and so it's less burden on everyone if we > just use it as is. > > Cheers, > > Dave. > -- > Dave Chinner > david@xxxxxxxxxxxxx >
Attachment:
signature.asc
Description: Digital signature
