Ashish Sangwan wrote:
On Wed, Aug 20, 2014 at 2:03 PM, Howard Chu <hyc@xxxxxxxxx> wrote:
Lukáš Czerner wrote:
On Wed, 20 Aug 2014, Howard Chu wrote:
Date: Wed, 20 Aug 2014 00:00:56 -0700
From: Howard Chu <hyc@xxxxxxxxx>
To: Lukáš Czerner <lczerner@xxxxxxxxxx>
Cc: linux-fsdevel <linux-fsdevel@xxxxxxxxxxxxxxx>
Subject: Re: truncate head of file?
Lukáš Czerner wrote:
On Tue, 19 Aug 2014, Howard Chu wrote:
Date: Tue, 19 Aug 2014 22:45:16 -0700
From: Howard Chu <hyc@xxxxxxxxx>
To: linux-fsdevel <linux-fsdevel@xxxxxxxxxxxxxxx>
Subject: truncate head of file?
Was thinking it would be very handy to have a truncate() variant that
deletes
pages from the head of a file. This could be leveraged to make logfiles
easier
to maintain, as an example. Anyone else interested, think this would be
nice
to have?
(Note - not the same as just punching holes in the beginning of the
file -
we
want the beginning of the file to advance as well, past the deleted
pages.)
I am not really sure I understand the behaviour you'd like to see.
Can you please explain the behaviour including more concrete use
case ?
For example - we have a logfile (opened O_APPEND) that grows
continuously. We
want to delete some old log info from the head of the file. We could use
"hole
punching" to cause a specific range of data to be freed, but that just
leaves
a sparse file. If we were to cat this file the read() would have to
advance
thru all of that empty space before arriving at actual log data. We want
both
the data to be freed and for the logical beginning of the file to be
moved
forward, to match the location of where the remaining data begins.
Freeing the space would be simplest if we just deallocate X pages from
the
file, and then the beginning of the file becomes the beginning of the
first
remaining page of the file.
Ok, now I understand. It is exactly what FALLOC_FL_COLLAPSE_RANGE is
for. It has been merged into mainline with commit v3.14-rc1-1-g00f5e61
Ah, thanks for the tip.
However it will not work with O_APPEND nor does any other fallocate
flag except pure fallocate, so not even punch hole would have
worked.
Hm, that's a little bit inconvenient. What if the process calling
There seems to be some confusion between per fd flag O_APPEND and
inode flag S_APPEND.
Collapse range can work well with O_APPEND flag but it will not work
if S_APPEND is set.
Ah ok, that's much better.
fallocate() is different from the one appending to the log (i.e., using two
separate file descriptors)? Also, when a range is collapsed, are all
processes with descriptors opened on the file updated - are their seek
positions correctly decremented? I dug up the patches and didn't see any
code to handle this but may have missed it.
You are right. Currently seek positions are not updated.
But this will not be of any problem (at least not on XFS) for your use
case if file is open with O_APPEND.
Refer to this discussion:
https://lkml.org/lkml/2014/2/26/701
Thanks, looks good.
I find it somewhat surprising that such a useful feature was implemented for
the benefit of video editing, when it has massive applicability in databases
(particularly for simplified truncation of write-ahead logs, as well as for
append-only database designs). This is such an obviously beneficial feature
but I haven't seen it talked up very much. Of course, being so specific to
very recent Linux releases will certainly impede its adoption.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
