Looking at the potential applications of HSM, an implementation based around the object store/radosgw may also be highly desirable. Spectralogic recently announced their DS3 API which adds bring online/offline operations to S3. In particular, - People using S3/SWIFT think in terms of bulk per-file operations rather than POSIX semantics. Updates are a problem for tape based systems. It can be done but recovering from a bad migration of active data is not easy. - Containers group related files together. The expensive operations on tape are mount and seek. Thus, the aim should be to maximise the number of related files on a tape for ease of recall. When moving files around from one media for another (repack operations), container level grouping ensures that the number of recall mounts is minimised. - Keeping the tape drives busy js always difficult… tape drives are now regularly exceeding 250MB/s on a single stream so the storage system needs to be able to maintain a high data rate. Tape drive performance drops rapidly when the drives have to stop and then restart as the buffers fill up again. Tim On 9 Nov 2013, at 09:33, Sage Weil <sage@xxxxxxxxxxx> wrote: > The latest Lustre just added HSM support: > > http://archive.hpcwire.com/hpcwire/2013-11-06/lustre_scores_business_class_upgrade_with_hsm.html > > Here is a slide deck with some high-level detail: > > https://jira.hpdd.intel.com/secure/attachment/13185/Lustre_HSM_Design.pdf > > Is anyone familiar with the interfaces and requirements of the file system > itself? I don't know much about how these systems are implemented, but I > would guess there are relatively lightweight requirements on the fs (ceph > mds in our case) to keep track of file state (online or archived > elsewhere). And some hooks to trigger migrations? > > If anyone is interested in this area, I would be happy to help figure out > how to integrate things cleanly! > > sage > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
