Using snapshots for transactions

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



My company is working on a system for storing a lot of data, for a long time, at low cost. It will involve thousands of hard drives and error correction codes.

One piece we need in this system is a way to impose transaction semantics on block devices. I've been looking everywhere for a good way to do this. I was happy to discover today that the snapshot functionality in Linux's device mapper code comes very close to what I need. Now I'm looking for assistance on figuring out how to get the rest of the way there.

Using LVM, I'd like to create a logical volume that consumes most of the space on a hard drive. I will create a filesystem on the volume, but I will not mount it. Then I'd like to create a snapshot of that volume and mount the snapshot read-write. LVM and device-mapper do all of this well already.

Now here are the parts I'm not sure how to do:

1) Just before committing, I need to freeze the snapshot and read the "before" and "after" versions of all blocks changed. The delta will be used to update error correction codes stored on other servers.
2) I need to atomically commit the changes I've made to the snapshot back to the original volume.
3) If something goes wrong during commit (power failure, etc.), I need to retry the commit. (If the retry fails, I need to disable the volume.)


It's OK that this strategy results in extra I/Os. Most of the work of committing a transaction can probably be done in user space if I can read the internal structures of the snapshot. Also, I'd like a way to clear the snapshot's exception list once the transaction commits; that way I don't have to destroy and create snapshots for each transaction.

So, am I going in a good direction? Does anyone have ideas, hints, or advice? I can hack the kernel if I need to.

Shane

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[Index of Archives]     [Gluster Users]     [Kernel Development]     [Linux Clusters]     [Device Mapper]     [Security]     [Bugtraq]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]

  Powered by Linux