[LSF/MM TOPIC] [ATTEND] HSM, DMAPI, and external media filesystems

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I would like to discuss how to get DMAPI accepted into the mainline
kernel.  DMAPI is or was supported by a number of filesystems which are
included in linux (including XFS) and also is supported by a number of
proprietary filesystems.  However, the DMAPI implementation has not yet
been accepted into the main line.

- What are the barriers to inclusion and how can they be resolved?

- What FOSS HSM projects are available to use the interface?
	* xfstests/dmapi/sample_hsm
	* openhsm (opensms)
	* tinyhsm (unfinished, unreleased)

- Who are the interested parties (if any), and how could we split up the
  work?

In the end I'd like to come up with a plan that seems reasonable toward
getting DMAPI included mainline.  I know that Alex Elder and others have
done work in this area and I would like to see it bear fruit.

---

Secondly, I'd enjoy a discussion of an old idea from back in the day
when I worked as a sysadmin and became a storage software hobbyist.
Having recently lost a great deal of data due to disk failure and failed
backups, I struck upon the idea of turning the backup paradigm on it's
head:

I'd rather have the authoritative copy of my data on an external medium
and only cache it on spinning rust.  The filesystem would log every
transaction (WORM style) to removable media and the set of discs or
tapes would be replayable onto new hard disks when I had a failure.
Checkpoints would ensure that the number of discs/tapes necessary to
replay the filesystem remained manageable, and the write performance
problem would be mitigated by maintaining a temporary log on spinning
disk to spool up enough data to keep the tape from having to rewind.
One could choose to recycle media from before a checkpoint, or save it
off for posterity.

My choice of medium at the time was cd-rom since I couldn't afford a
tape drive.  I went about trying to implement such a thing using DMAPI,
which isn't suitable (you can't set an event mask on a file quickly
enough to log the first writes).  Then I tried a stacking filesystem,
fuse, got a job, and lost interest.

This is almost certainly not a new idea.  If there is already something
like this implemented, I'd like to hear about it.  If there isn't, maybe
we could come up with a rough design on a cocktail napkin and pique
enough interest to make a start.

Regards,
Ben
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [Samba]     [Device Mapper]     [CEPH Development]
  Powered by Linux