Re: VFS hot tracking: How to calculate data temperature?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, Nov 5, 2012 at 7:57 PM, Steven Whitehouse <swhiteho@xxxxxxxxxx> wrote:
> Hi,
>
> On Mon, 2012-11-05 at 19:46 +0800, Zhi Yong Wu wrote:
>> On Mon, Nov 5, 2012 at 6:33 PM, Steven Whitehouse <swhiteho@xxxxxxxxxx> wrote:
>> > Hi,
>> >
>> > On Mon, 2012-11-05 at 16:44 +0800, Zhi Yong Wu wrote:
>> >> On Mon, Nov 5, 2012 at 4:28 PM, Dave Chinner <david@xxxxxxxxxxxxx> wrote:
>> >> > On Mon, Nov 05, 2012 at 10:35:50AM +0800, Zhi Yong Wu wrote:
>> >> >> On Sat, Nov 3, 2012 at 5:27 AM, Mingming.cao <cmm@xxxxxxxxxx> wrote:
>> >> >> > On Fri, 2012-11-02 at 14:38 +0800, Zhi Yong Wu wrote:
>> >> >> >> Here also has another question.
>> >> >> >>
>> >> >> >> How to save the file temperature among the umount to be able to
>> >> >> >> preserve the file tempreture after reboot?
>> >> >> >>
>> >> >> >> This above is the requirement from DB product.
>> >> >> >> I thought that we can save file temperature in its inode struct, that
>> >> >> >> is, add one new field in struct inode, then this info will be written
>> >> >> >> to disk with inode.
>> >> >> >>
>> >> >> >> Any comments or ideas are appreciated, thanks.
>> >> >> >>
>> >> >> >>
>> >> >> >
>> >> >> > Maybe could save the last file temperature with extended attributes.
>> >> >> It seems that only ext4 has the concept of extended attributes.
>> >> >
>> >> > All major filesystems have xattr support. They are used extensively
>> >> > by the security and integrity subsystems, for example.
>> >> got it, thanks.
>> >> >
>> >> > Saving the information might be something that is useful to certian
>> >> > applications, but lets have the people that need that functionality
>> >> > spell out their requirements before discussing how or what to
>> >> > implement.  Indeed, discussion shoul dreally focus on getting the
>> >> > core, in-memory infrastructure sorted out first before trying to
>> >> > expand the functionality further...
>> >> ah, but the latest patchset need some love from experienced FS guys:).......
>> >
>> > There is one other possible issue with saving the data into the
>> > filesystem, which is that it may disturb what you are trying to measure.
>> > Some filesystems (GFS2 is one) store data for small inodes in the same
>> > block as the inode itself. So that means the accesses to the saved hot
>> > tracking info may potentially affect the data access times too. Also
>> > there is a very limited amount of space to expand the number of fields
>> > in the inode, so xattr may be the only solution, depending on how much
>> > data needs to be stored in each case.
>> Very good analysis, two possible issues are very meanful, thanks.
>> >
>> > In the GFS2 case (I don't think it is unique in this) xattrs are stored
>> > out of line and having to access them in every open means an extra block
>> > read per inode, which again has performance implications.
>> >
>> > So that is not an insurmountable problem, but something to take into
>> > account in selecting a solution,
>> In summary, you look like preferring to xattr as its solution.
>>
>
> Well, that depends on exactly how large the data to be stored is, and
> other factors. It will add overhead to the storage/retrieval but at
> least it is fairly generic (wrt on-disk format) so likely to be easier
> to retrofit to existing filesystems.
Do you have some idea with more details about how to retrofit to existing FS?:)
>
> I suspect this may be one of those cases where there is no obvious right
> answer and it is a case of selecting the least worst option, if that
> makes sense?
Then we can only check which solution is better via large scale
performance test.
>
> Steve.
>
>



-- 
Regards,

Zhi Yong Wu
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [Samba]     [Device Mapper]     [CEPH Development]
  Powered by Linux