On Thu, Sep 3, 2020 at 1:07 AM Bart Van Assche <bvanassche@xxxxxxx> wrote: > > On 2020-09-02 01:04, Danil Kipnis wrote: > > RTRS allows for reliable transmission of sg lists between two hosts > > over rdma. It is optimised for block io. One can implement a client > > and a server module on top of RTRS which would allow for reliable > > transmission to a group of hosts. > > > > In the networking world this is called reliable multicast. I think one > > can say that reliable multicast is an equivalent to what is called > > "mirror" in the storage world. There is something called XoR network > > coding which seems to be an equivalent of raid5. There is also Reed > > Solomon network coding. > > > > Having a reliable multicast with coding rdma-based transport layer > > would allow for very flexible and scalable designs of distributed > > replication solutions based on different in-kernel transport, block > > and replication drivers. > > > > What do you think? > > How will the resulting software differ from DRBD (other than that it > uses RDMA)? DRBD replicates disks (a local one with a remote one). The idea here would be to replicate on the level of hosts, i.e. to support replication inside the transport layer. A reliable mc on top of rtrs would be more similar to a stack where an md-raid<x> sits on a client on top of two or more imported srp/nvmeof/rnbd devices, but with an ability to resync between remote machines directly. > How will it be guaranteed that the resulting software does > not suffer from the problems that have been solved by the introduction > of the DRBD activity log > (https://www.linbit.com/drbd-user-guide/users-guide-drbd-8-4/#s-activity-log)? The above would require some kind of activity log also, I'm afraid. > > Thanks, > > Bart.
