Thanks everyone for the detailed technical feedback and clarifications - they've been extremely valuable in understanding the fundamental challenges and existing solutions. I appreciate the points about md-cluster and DRBD's network RAID capabilities. While these are robust solutions for network-based replication, I'm particularly interested in the point-in-time recovery capability for scenarios like ransomware recovery, where being able to roll back to a specific point before encryption occurred would be valuable. Regarding blk_filter - I've been exploring it since it was mentioned, and it indeed seems to be the right approach for what we're trying to achieve. However, I've found that many of our current requirements can actually be implemented using eBPF without additional kernel modules. I plan to create a detailed demonstration video to share my findings with this thread. Additionally, I'll be cleaning up and open-sourcing our replicator utility implementation for community feedback. I would very much like to attend the LSF/MM/BPF summit to discuss these ideas in person and learn more about blk_filter and proper block layer fundamentals. Would it be possible for someone to help me with an invitation? Thanks again to everyone who took the time to explain the intricacies of write caching, sector tracking limitations, and data persistence guarantees. This discussion has been incredibly educational. Thanks and regards, Vishnu KS On Tue, 7 Jan 2025 at 03:48, Dan Williams <dan.j.williams@xxxxxxxxx> wrote: > > Song Liu via Lsf-pc wrote: > > On Sat, Jan 4, 2025 at 9:52 AM Vishnu ks <ksvishnu56@xxxxxxxxx> wrote: > > > > > [...] > > > > > > @Song: Our approach fundamentally differs from md/raid in several ways: > > > > > > 1. Network-based vs Local: > > > - Our system operates over network, allowing replication across > > > geographically distributed systems > > > - md/raid works only with locally attached storage devices > > > > md-cluster (https://docs.kernel.org/driver-api/md/md-cluster.html) > > does support RAID in a cluster. > > Also, > > https://docs.kernel.org/admin-guide/blockdev/drbd/index.html -- Vishnu KS, Opensource contributor and researcher, https://iamvishnuks.com
