On Tue, Dec 4, 2018 at 9:03 AM Ilya Dryomov <idryomov@xxxxxxxxx> wrote:
>
> On Mon, Dec 3, 2018 at 1:50 PM Dongsheng Yang
> <dongsheng.yang@xxxxxxxxxxxx> wrote:
> >
> > Hi all,
> > This is V1 to implement the journaling feature in kernel rbd, which makes mirroring in kubernetes possible.
> > It passed the /ceph/ceph/qa/workunits/rbd/rbd_mirror.sh, with a little change as below:
> >
> > ```
> > [root@atest-guest build]# git diff /ceph/ceph/qa/workunits/rbd/rbd_mirror_helpers.sh
> > diff --git a/qa/workunits/rbd/rbd_mirror_helpers.sh b/qa/workunits/rbd/rbd_mirror_helpers.sh
> > index e019de5..9d00d3e 100755
> > --- a/qa/workunits/rbd/rbd_mirror_helpers.sh
> > +++ b/qa/workunits/rbd/rbd_mirror_helpers.sh
> > @@ -854,9 +854,9 @@ write_image()
> >
> > test -n "${size}" || size=4096
> >
> > - rbd --cluster ${cluster} -p ${pool} bench ${image} --io-type write \
> > - --io-size ${size} --io-threads 1 --io-total $((size * count)) \
> > - --io-pattern rand
> > + rbd --cluster ${cluster} -p ${pool} map ${image}
> > + fio --name=test --rw=randwrite --bs=${size} --runtime=60 --ioengine=libaio --iodepth=1 --numjobs=1 --filename=/dev/rbd0 --direct=1 --group_reporting --size $((size * count)) --group_reporting --eta-newline 1
> > + rbd --cluster ${cluster} -p ${pool} unmap ${image}
> > }
> >
> > stress_write_image()
> > ```
> >
> > Changelog from RFC:
> > 1. error out if there is some unsupported event type in replaying
>
> So the journal is still replayed in the kernel. Was there a design
> discussion about this?
>
> Like I said in one of my replies to the RFC, I think we should avoid
> replaying the journal in the kernel and try to come up with a design
> where it's done by librbd.
+1 to this. If "rbd [device] map" first replays the journal (by just
acquiring the exclusive lock via the API), then krbd would only need
to check that there are no events to replay. If there are one or more
events that it needs to replay for some reason, it implies that it
lost the exclusive lock to another client that changed the image *and*
failed to commit the entries to the journal. I think it seems
reasonable to just move the volume to R/O in that case since something
odd was occurring.
> The fundamental problem with replaying the journal in the kernel and
> therefore supporting only a couple of event types is that the journal
> has to be replayed not only at "rbd map" time, but also every time the
> exclusive lock is reacquired. Whereas we can safely error out at "rbd
> map" time, I don't see a sane way to handle a case where an unsupported
> event type is encountered after reacquiring the lock when the image is
> already mapped.
>
> Thanks,
>
> Ilya
--
Jason
