On November 21, 2014 5:15:43 AM EST, Anshuman Aggarwal <anshuman.aggarwal@xxxxxxxxx> wrote: >I'd a appreciate any help/pointers in implementing the proposal below >including the right path to get this into the kernel itself. >---------------------------------- >I'm outlining below a proposal for a RAID device mapper virtual block >device for the kernel which adds "split raid" functionality on an >incremental batch basis for a home media server/archived content which >is rarely accessed. > >Given a set of N+X block devices (of the same size but smallest common >size wins) > >the SplitRAID device mapper device generates virtual devices which are >passthrough for N devices and write a Batched/Delayed checksum into >the X devices so as to allow offline recovery of block on the N >devices in case of a single disk failure. > >Advantages over conventional RAID: > >- Disks can be spun down reducing wear and tear over MD RAID Levels >(such as 1, 10, 5,6) in the case of rarely accessed archival content > >- Prevent catastrophic data loss for multiple device failure since >each block device is independent and hence unlike MD RAID will only >lose data incrementally. > >- Performance degradation for writes can be achieved by keeping the >checksum update asynchronous and delaying the fsync to the checksum >block device. > >In the event of improper shutdown the checksum may not have all the >updated data but will be mostly up to date which is often acceptable >for home media server requirements. A flag can be set in case the >checksum block device was shutdown properly indicating that a full >checksum rebuild is not required. > >Existing solutions considered: > >- SnapRAID (http://snapraid.sourceforge.net/) which is a snapshot >based scheme (Its advantages are that its in user space and has cross >platform support but has the huge disadvantage of every checksum being >done from scratch slowing the system, causing immense wear and tear on >every snapshot and also losing any information updates upto the >snapshot point etc) > >I'd like to get opinions on the pros and cons of this proposal from >more experienced people on the list to redirect suitably on the >following questions: > >- Maybe this can already be done using the block devices available in >the kernel? > >- If not, Device mapper the right API to use? (I think so) > >- What would be the best block devices code to look at to implement? > > >Regards, > >Anshuman Aggarwal > >_______________________________________________ >Kernelnewbies mailing list >Kernelnewbies@xxxxxxxxxxxxxxxxx >http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies I think I understand the proposal. You say N pass-through drives. I assume concatenated? If the N drives were instead in a Raid-0 stripe set and your X drives was just a single parity drive, then you would have described Raid-4. There are use cases for raid 4 and you have described a good one (rarely used data where random w/o performance is not key). I don't know if mdraid supports raid-4 or not. If not I suggest adding raid-4 support is something else you might want to look at. Anyway, at a minimum add raid-4 to the existing solutions considered section. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. _______________________________________________ Kernelnewbies mailing list Kernelnewbies@xxxxxxxxxxxxxxxxx http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
