On Tue, Nov 12, 2013 at 1:47 AM, Andreas Joachim Peters <Andreas.Joachim.Peters@xxxxxxx> wrote: > I think you need to support the following functionality to support HSM (file not block based): > > 1 implement a trigger on file creation/modification/deletion > > 2 store the additional HSM identifier for recall as a file attribute > > 3 policy based purging of file related blocks (LRU cache etc.) > > 4 implement an optional trigger to recall a purged file and block the IO (our experience is that automatic recalls are problematic for huge installations if the aggregation window for desired recalls is short since they create inefficient and chaotic access on tapes) > > 5 either snapshot a file before migration, do an exclusive lock or freeze it to avoid modifications during migration (you need to have a unique enough identifier for a file, either inode/path + checksum or also inode/path + modification time works) DMAPI seems to be the natural choice for items 1 & 4 above. > FYI: there was a paper about migration policy scanning performance by IBM two years ago: > http://domino.watson.ibm.com/library/CyberDig.nsf/papers/4A50C2D66A1F90F7852578E3005A2034/$File/rj10484.pdf An important omission in that paper is the exact ILM policy that was used to scan the file system. I strongly suspect that it was a catch-all policy that matches every file without examining any metadata. When you add conditions that check file metadata, scan time would increase, probably by a few orders of magnitude. -- Dmitry Borodaenko -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
