Hello Robert! On Mon 12-09-22 21:10:24, Plaster, Robert wrote: > HI Jan – Our team has been using fanotify for HSM as a DMAPI replacement > for a while now. We came from StorageTek/Sun/Oracle HSM product line > development teams. We have been working on this for about 5 years and > just this month are supporting end-users. DMAPI was a huge stumbling > block for us. We figured out what minimum set of api calls were needed to > make it work. > > Our experience with fanotify has been fantastic. Not much overhead CPU > load but for boot volumes we do filter out events for swap and other > (there are so many) OS temp files that are really of no concern to HSM. > We can create as many files as the file system on NVMe can without any > back-pressure and the HSM process will go as fast as the target media > supports. I'm glad to hear fanotify is useful for you. > We have tested close to 600M files per HSM client and we keep adding > client files as time permits, we have no coded limits for the number of > HSM clients or max number of files in the repository. Also, the > repository for HSM clients is heterogenous so it allows us to push files > from one client type to another without any transcoding. I asked the guys > doing the actual fanotify part to comment but they said it would be a > couple days as they are heads down on a fix for a customer. > > Currently we have HSM and punch-hole running on xfs and tested it on zfs > (works but client isn’t finalized) and we have Lustre and SpectrumScale > on our to-do list. Basically any FS with extended attributes should work > for HSM and some (not all) will work with punch-hole capabilities. > > We have developed a HSM target for certain object stores (Ceph librados > and we have our own in-house object store) that support stream-IO and of > course any tape technology. We have a replication tool for making an S3 > target look like the source FS but its just replication, not HSM. Until > we get a S3 io-streaming we can’t use it for HSM. Our implementation only > works with our open-source catalog, archive platform. We tried to > announce this capability to the ceph community but we could never get > past their gatekeepers so only people we actually talk to know about it. > > Check out our site (kinda sucks and a little markety) but it’s a good > primer. In it are links to the code and manuals we have done. We have not > put out on github yet but will very soon. We are getting ready to post > some big updates to really simplify installation and configuration and > some bug fixes for some weird edge-cases. Thanks for info and the links! It is interesting to learn something about how users are actually using our code :). Honza -- Jan Kara <jack@xxxxxxxx> SUSE Labs, CR
