Hi Manish, On Mon, Sep 14, 2009 at 3:12 PM, Manish Katiyar <mkatiyar@xxxxxxxxx> wrote: > On Mon, Sep 14, 2009 at 3:09 PM, SandeepKsinha <sandeepksinha@xxxxxxxxx> wrote: >> Hi Manish, >> >> On Mon, Sep 14, 2009 at 2:57 PM, Manish Katiyar <mkatiyar@xxxxxxxxx> wrote: >>> On Mon, Sep 14, 2009 at 11:59 AM, SandeepKsinha <sandeepksinha@xxxxxxxxx> wrote: >>>> Hi Greg, >>>> >>>> Thanks a lot for such nice explanation. But I still have the same >>>> query lingering... >>>> >>>> >>>> On Mon, Sep 14, 2009 at 9:36 AM, Greg Freemyer <greg.freemyer@xxxxxxxxx> wrote: >>>>> On Sun, Sep 13, 2009 at 1:49 PM, SandeepKsinha <sandeepksinha@xxxxxxxxx> wrote: >>>>>> On Sun, Sep 13, 2009 at 11:08 PM, Manish Katiyar <mkatiyar@xxxxxxxxx> wrote: >>>>>>> On Sun, Sep 13, 2009 at 10:12 PM, SandeepKsinha <sandeepksinha@xxxxxxxxx> wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Sep 13, 2009 at 10:07 PM, Manish Katiyar <mkatiyar@xxxxxxxxx> wrote: >>>>>>>>> On Sun, Sep 13, 2009 at 5:44 PM, SandeepKsinha <sandeepksinha@xxxxxxxxx> wrote: >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Looking at the flags in the extent info, Is there any specifc flags >>>>>>>>>> which indicates an extent to be a HOLE?? >>>>>>>>> >>>>>>>>> I am not sure if I understand the question correctly ...... why would >>>>>>>>> you need that ? Can you give an example where it should be used ?? >>>>>>>>> >>>>>>>> >>>>>>>> Look at e4defrag.c, >>>>>>>> it checks the file size and allocates the same number of blocks for >>>>>>>> donor inode. Which will eventually make a holey file into a normal >>>>>>>> one. >>>>>>>> Any tool/application should make sure that they leave a sparse file as sparse. >>>>>>>> >>>>>>>> I think, as suggested by Greg Freemyer, we can use BMAP ioctl to get >>>>>>>> such information. >>>>>>> >>>>>>> Yes, but I think bmap would be costly if the file is large and is not >>>>>>> holey :-( .... but that would be probably same calling fiemap if the >>>>>>> file is completely fragmented such that each extent size is 1. >>>>>>> >>>>>> >>>>>> Since, ext2/ext3 did not have mutli block allocation thats why this is >>>>>> the only way that we might have. >>>>>> But generally most of the new features work on with extent based files on ext4. >>>>>> >>>>>> I am still wondering that how to we represent a hole using extents in >>>>>> a extent based file. >>>>>> Just like we had a convention of having the block number 0 in case of holes. >>>>>> >>>>>> Similarly, what do we look at to figure out if its a hole or not. BMAP >>>>>> is one way. But since, in a extent based file, we have only extents, >>>>>> there should be some flag to indicate the same. >>>>> >>>>> Sandeep, >>>>> >>>>> If you look at e4defrag, it first gets a list of all the extents. I'm >>>>> pretty sure extents only exist for allocated extents. Holes do not >>>>> have any associated extents. >>> >>> I did a extent dump of a holey file. >>> >>> Inode: 12 Type: regular Mode: 0644 Flags: 0x80000 >>> Generation: 4284390079 Version: 0x00000001 >>> User: 5572 Group: 5573 Size: 20877312000 >>> File ACL: 0 Directory ACL: 0 >>> Links: 1 Blockcount: 16000 >>> Fragment: Address: 0 Number: 0 Size: 0 >>> ctime: 0x4aae0975 -- Mon Sep 14 14:44:29 2009 >>> atime: 0x4aad34e9 -- Sun Sep 13 23:37:37 2009 >>> mtime: 0x4aae0975 -- Mon Sep 14 14:44:29 2009 >>> EXTENTS: >>> (0-3999): 10241-14240, (20384000-20387999): 14337-18336 >>> >>> Level Entries Logical Physical Length Flags >>> 0/ 0 1/ 2 0 - 3999 10241 - 14240 4000 >>> 0/ 0 2/ 2 20384000 - 20387999 14337 - 18336 4000 >>> >>> I am looking at the source, but as you can see it is not printing the >>> extents for the hole part. >>> >> >> I suspect the debugfs code masking that out for the output. > > yes... and that is the flag/method that we are looking for ... isn't it :-) ??? > read the above conversation, this answers all the question. me: How does ext4 represent holes ? Do we have a extent (unitialized for holes) ?? Kalpak: physical block no = 0 me: But we keep an extent for the same. If I do a FIEMAP Kalpak: yup me: I will also get an extent for the holes Kalpak: FIEMAP wont return hole extents me: ok Kalpak: but u can find out that if there was hole by FIEMAP output if last block != this block there was a hole in between Hope this helps everyone. Also, now we do not bother much about holes. >> If you remember, for a normal block based file also, it does something >> similar, it does not prints the values for holes. >> >> It can be a case here as well. >>> >>>>> >>>> >>>> This is what my actual question is, think of this situation... >>>> >>>> |-------------|-------------|-------------|------------------| >>>> 0 500 700 1200 3000 >>>> [Logical block numbers for an inode] >>>> >>>> >>>> In this situation you will have four extents for sure. >>>> ext_1 -> 0 -- 500 >>>> ext_2 -> 501 --700 [ This will be an initialized extent] >>>> ext_3 -> 701 -- 1200 >>>> ext_4 -> 1200 -- 3000 >>>> >>>> After looking at the sources and some comments in the ext4 source >>>> code, I could figure out that holes would be having an initialized >>>> extent. >>>> Reference: http://lxr.linux.no/#linux+v2.6.31/fs/ext4/extents.c#L2843 >>>> >>>> I think we cannot have a mixture of both a BMAP and an EMAP, it will be either. >>>> >>>>> Then it calls merge_extents to create extent groups. In e4defrag >>>>> terminology, an extent group is a collection of all the logically >>>>> contiguous extents. I don't know if the kernel uses that terminology >>>>> or not. >>>>> >>>> >>>> Hope they are not merging together any initialized and uninitialized >>>> extents together, since they can be logically contiguous. Or rather >>>> they will be. >>>> >>>> >>>>> In other words in e4defrag terminology a sparse file is a series of: >>>>> >>>>> extent group - hole - extent group - hole - extent group - etc. >>>>> >>>>> Then e4defrag creates a donor file with exactly the same allocated >>>>> block areas by calling fallocate on the donor file for each extent >>>>> group with the same starting offset and length as the extent group. >>>>> >>>> >>>> This is true and should be applicable to initialized extents as well. >>>> I fear if they are >>>> >>>> >>>>> Thus the donor file ends up have exactly the same holes as the >>>>> original file. Then the donor blocks are used to defrag the original >>>>> file by calling move_extent. In the kernel, the move_extent logic >>>>> looks for holes and only replaces blocks that are allocated in the >>>>> original file. >>>>> >>>> >>>> This is true. I am sure of the kernel logic. >>>> >>>>> Greg >>>>> >>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Sandeep. >>>> >>>> >>>> >>>> >>>> >>>> >>>> “To learn is to change. Education is a process that changes the learner.” >>>> >>> >>> >>> >>> -- >>> Thanks - >>> Manish >>> >> >> >> >> -- >> Regards, >> Sandeep. >> >> >> >> >> >> >> “To learn is to change. Education is a process that changes the learner.” >> > > > > -- > Thanks - > Manish > -- Regards, Sandeep. “To learn is to change. Education is a process that changes the learner.” -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecartis@xxxxxxxxxxxx Please read the FAQ at http://kernelnewbies.org/FAQ
