any other comments? On Fri, Nov 2, 2012 at 12:43 PM, Ram Pai <linuxram@xxxxxxxxxx> wrote: > On Fri, Nov 02, 2012 at 12:04:23PM +0800, Zhi Yong Wu wrote: >> HI, guys >> >> VFS hot tracking currently show result as below, and it is very >> strange and not nice. >> >> inode #279, reads 0, writes 1, avg read time 18446744073709551615, >> avg write time 5251566408153596, temp 109 >> >> Do anyone know if there is one simpler but effective way to calculate >> data temperature? > > Intuitively, data gets hot when it is accessed frequently, and cools down as > the frequency of its access decreases. > > It is like heating water, the longer you heat it, the hotter it gets. The more > intense the flame the faster it gets heated. So it is a function of both > intensity and continuity. In the case of data, intensity can be mapped to > the access frequency within a discrete interval. And the continuity can be > mapped to sustainence of the same access frequency across further discrete > intervals. Any reduction of intensity cools down the data. Any break in > continuity cools down the data. > > We need to define the size of a 'discrete interval', and keep collecting the > frequency of access in each discrete interval and take weighted > average across all these discrete intervals. That should give us the > temperature of the data. > > Here is an example to make my thoughts clear. > > Lets say we define discrete interval as 1sec. > > If a given peice of data is accessed 100 times in this 1sec, then the > temperature of the data at the end of the 1sec will become (0+100)/2=50, where > 0 is the temperature of the data at the beginning of the 1sec. In the next 1 > sec, if the data is accessed 100 times again, then the temperature of the data > at the end of the 2nd sec becomes (50+100)/2=75 and so on a so forth. If > during any interval if the data is not accessed at all, the temperature goes > down by half. This way of calculation biases the temperature towards most > recent access. In other words a high amount of recent access significantly > rises the temperature of the data compared to the same high amount of > non-recent access. > > This technique will require us to capture *only* two information; the access > frequency and the latest temperature of the data. > > A crude approach is to schedule a kernel daemon approximately every second to > update the latest temperature using the access frequency since the last time > the temperature got updated. But this approach is not scalable. The other > approach is to decouple the frequency of temperature calculation from the size of > the discrete interval. The kernel daemon thread can take its own sweet time to > calculate the temperature. Whenever the daemon gets to calculate the new > temperature of a given inode it can evenly spread the load, since the last > temperature calculation, across all the discrete intervals. > > The formula for calculation will be (Ty+((2^y)-1)x)/(2^y). Where 'x' is the > number of accesses in the last 'y' discrete intervals. 'T' is the previous > temperature. Offcourse; I am approximating that there were exactly 'x/y' > accesses in each of these 'y' discrete interval. > > This way of calculation will require one more field to be added in the > datastructure, which is the timestamp when the last temperature got updated. > > > RP > -- Regards, Zhi Yong Wu -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
