Hi Haris,
Thanks for pushing RMR forward! Lutz Pogrell and Alexei and you did a
huge amount of work. I'm honestly convinced RMR will be a very useful
extension for the kernel. It is currently used with a block client and a
block server. Haris already mentioned rbd and dm-thin as a different
potential pair of users. I'm still of the opinion that sched and mm
could be the most interesting users of rmr.
Looking forward to the github link,
Cheers,
Danil.
P.S. BTW, where is my beloved libtbl?!
On 21.11.2022 14:57, Haris Iqbal wrote:
Hey there,
We've got a prototype version working in-house. We've also implemented
a block device client and server. Another client and server could
probably be rbd and dm-thin. Will update as soon as we have a github
link.
Some technical details
- Ability to perform sync across storage nodes without the involvement
of the client.
- This helps performing sync, extending/adding legs/members without
the help of the client.
Candidate users
- We've implemented a stand-alone replicating block device. The client
is similar to the rnbd-clt and our own corresponding "store"/server
does linear mapping on the server side.
- rbd could be a client of rmr-clt. The rmr-srv (store) would talk to
lvm. rbd would provide the block device (over multiple objects) on the
client side. Lvm would function as the store on the server side. One
object would be stored on one dm-thin volume. rmr would provide for
the replication in the network.
Setups: RMR vs MD-RAID vs DRBD
- Active-active
- RMR as means of replication over network differs from the md-raid
configuration because sync traffic goes directly between servers. The
difference to the drbd setup is that the IO traffic goes to both legs
in a single hop.
How does RMR solve the activity log issue
- Synchronous replication; much like Protocol C of DRBD.
- RMR tracks all successful queue_depth (max number of IOs that can be
inflight at any moment) worth of last IOs on each storage node.
Best,
Haris
Signed-off: alexv.marov@xxxxxxxxx
Reviewed-by: danil.kipnis@xxxxxxxxxx
On Sun, Nov 22, 2020 at 5:20 PM Danil Kipnis
<danil.kipnis@xxxxxxxxxxxxxxx> wrote:
On Fri, Sep 4, 2020 at 5:33 PM Bart Van Assche <bvanassche@xxxxxxx>
wrote:
>
> On 2020-09-04 04:35, Danil Kipnis wrote:
> > On Thu, Sep 3, 2020 at 1:07 AM Bart Van Assche <bvanassche@xxxxxxx> wrote:
> >> How will it be guaranteed that the resulting software does
> >> not suffer from the problems that have been solved by the introduction
> >> of the DRBD activity log
> >> (https://www.linbit.com/drbd-user-guide/users-guide-drbd-8-4/#s-activity-log)?
> >
> > The above would require some kind of activity log also, I'm afraid.
>
> How about collaborating with the DRBD team? My concern is that otherwise
> we will end up with two drivers in the kernel that implement block device
> replication between servers connected over a network.
Will take a closer look at drbd,
Thank you,
Danil.
>
> Thanks,
>
> Bart.
