On Fri, Jan 28, 2022 at 4:22 PM Manuel Holtgrewe <zyklenfrei@xxxxxxxxx> wrote:
>
> OK, so there is a different in semantics of the kernel and the user
> space driver?
Right.
>
> Which one would you consider to be desired?
The kernel driver is probably doing the right thing.
>
> From what I can see, the kernel semantics (apparently: root can do
> everything) would allow to sync between file systems no matter what.
> With the current user space semantics, users could `chmod a=` folders
> in their $HOME and stop the sync from working. Is my interpretation
> correct?
Correct.
I haven't root caused the issue with the user space driver yet. This
blocks using the cephfs-mirror daemon with read-only source
directories.
I'll file a tracker for this. Thanks for your help.
>
> Best wishes,
> Manuel
>
> On Fri, Jan 28, 2022 at 11:43 AM Venky Shankar <vshankar@xxxxxxxxxx> wrote:
> >
> > On Fri, Jan 28, 2022 at 3:42 PM Manuel Holtgrewe <zyklenfrei@xxxxxxxxx> wrote:
> > >
> > > I'm running rsync "-Wa", see below for a reproduction from scratch
> > > that actually syncs as root when no permissions are given on the
> > > directories.
> > >
> > > -- full mount options --
> > >
> > > 172.16.62.10,172.16.62.11,172.16.62.11,172.16.62.12,172.16.62.13,172.16.62.30:/
> > > on /data/cephfs-2 type ceph
> > > (rw,noatime,name=samba,secret=<hidden>,acl)
> > > 172.16.62.22,172.16.62.23,172.16.62.23,172.16.62.24,172.16.62.25,172.16.62.32:/
> > > on /data/cephfs-3 type ceph
> > > (rw,noatime,name=gateway,secret=<hidden>,rbytes,acl)
> > >
> > > -- example --
> > >
> > > 0|0[root@gw-1 ~]# mkdir -p /data/cephfs-2/test2/x/y
> > > 0|0[root@gw-1 ~]# touch !$z
> > > touch /data/cephfs-2/test2/x/yz
> > > 0|0[root@gw-1 ~]# chmod a= -R /data/cephfs-2/test2
> > > 0|0[root@gw-1 ~]# mkdir /data/cephfs-3/test2
> > > 0|0[root@gw-1 ~]# rsync -va /data/cephfs-2/test2/. /data/cephfs-3/test2/.
> > > sending incremental file list
> > > ./
> > > x/
> > > x/yz
> > > x/y/
> >
> > Try running this from a ceph-fuse mount - it would fail. It's probably
> > related to the way how permission checks are done (we may want to fix
> > that in the user-space driver).
> >
> > Since the mirror daemon uses the user-space library, it would be
> > running into the same permission related constraints as ceph-fuse.
> >
> > >
> > > sent 165 bytes received 50 bytes 430.00 bytes/sec
> > > total size is 0 speedup is 0.00
> > > 0|0[root@gw-1 ~]# find /data/cephfs-3/test2 | xargs stat
> > > File: /data/cephfs-3/test2
> > > Size: 0 Blocks: 0 IO Block: 65536 directory
> > > Device: 2dh/45d Inode: 1099522341053 Links: 3
> > > Access: (0000/d---------) Uid: ( 0/ root) Gid: ( 0/ root)
> > > Access: 2022-01-28 11:10:31.436380533 +0100
> > > Modify: 2022-01-28 11:09:47.666606846 +0100
> > > Change: 2022-01-28 11:10:31.436380533 +0100
> > > Birth: -
> > > File: /data/cephfs-3/test2/x
> > > Size: 0 Blocks: 0 IO Block: 65536 directory
> > > Device: 2dh/45d Inode: 1099522341054 Links: 3
> > > Access: (0000/d---------) Uid: ( 0/ root) Gid: ( 0/ root)
> > > Access: 2022-01-28 11:10:31.462380399 +0100
> > > Modify: 2022-01-28 11:09:49.258598614 +0100
> > > Change: 2022-01-28 11:10:31.462380399 +0100
> > > Birth: -
> > > File: /data/cephfs-3/test2/x/yz
> > > Size: 0 Blocks: 0 IO Block: 4194304 regular empty file
> > > Device: 2dh/45d Inode: 1099522341056 Links: 1
> > > Access: (0000/----------) Uid: ( 0/ root) Gid: ( 0/ root)
> > > Access: 2022-01-28 11:10:31.447380476 +0100
> > > Modify: 2022-01-28 11:09:49.265598578 +0100
> > > Change: 2022-01-28 11:10:31.447380476 +0100
> > > Birth: -
> > > File: /data/cephfs-3/test2/x/y
> > > Size: 0 Blocks: 0 IO Block: 65536 directory
> > > Device: 2dh/45d Inode: 1099522341055 Links: 2
> > > Access: (0000/d---------) Uid: ( 0/ root) Gid: ( 0/ root)
> > > Access: 2022-01-28 11:10:31.439380518 +0100
> > > Modify: 2022-01-28 11:09:47.669606830 +0100
> > > Change: 2022-01-28 11:10:31.439380518 +0100
> > > Birth: -
> > >
> > > On Fri, Jan 28, 2022 at 11:06 AM Venky Shankar <vshankar@xxxxxxxxxx> wrote:
> > > >
> > > > On Fri, Jan 28, 2022 at 3:20 PM Manuel Holtgrewe <zyklenfrei@xxxxxxxxx> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > thanks for the reply.
> > > > >
> > > > > Actually, mounting the source and remote fs on linux with kernel
> > > > > driver (Rocky Linux 8.5 default kernel), I can `rsync`.
> > > >
> > > > You are probably running rsync with --no-perms or a custom --chmod (or
> > > > one of --no-o, --no-g) I guess?
> > > >
> > > > >
> > > > > Is this to be expected?
> > > > >
> > > > > Cheers,
> > > > >
> > > > > On Fri, Jan 28, 2022 at 10:44 AM Venky Shankar <vshankar@xxxxxxxxxx> wrote:
> > > > > >
> > > > > > Hey Manuel,
> > > > > >
> > > > > > On Thu, Jan 27, 2022 at 8:57 PM Manuel Holtgrewe <zyklenfrei@xxxxxxxxx> wrote:
> > > > > > >
> > > > > > > OK, reconstructed with another example:
> > > > > > >
> > > > > > > -- source file system --
> > > > > > >
> > > > > > > 0|0[root@gw-1 ~]# find /data/cephfs-2/test/x2 | xargs stat
> > > > > > > File: /data/cephfs-2/test/x2
> > > > > > > Size: 1 Blocks: 0 IO Block: 65536 directory
> > > > > > > Device: 2ch/44d Inode: 1099840816759 Links: 3
> > > > > > > Access: (2440/dr--r-S---) Uid: ( 0/ root) Gid: ( 0/ root)
> > > > > > > Access: 2022-01-27 16:24:15.627783470 +0100
> > > > > > > Modify: 2022-01-27 16:24:22.001750514 +0100
> > > > > > > Change: 2022-01-27 16:24:51.294599055 +0100
> > > > > > > Birth: -
> > > > > > > File: /data/cephfs-2/test/x2/y2
> > > > > > > Size: 1 Blocks: 0 IO Block: 65536 directory
> > > > > > > Device: 2ch/44d Inode: 1099840816760 Links: 2
> > > > > > > Access: (2440/dr--r-S---) Uid: ( 0/ root) Gid: ( 0/ root)
> > > > > > > Access: 2022-01-27 16:24:22.001750514 +0100
> > > > > > > Modify: 2022-01-27 16:24:27.712720985 +0100
> > > > > > > Change: 2022-01-27 16:24:51.307598988 +0100
> > > > > > > Birth: -
> > > > > > > File: /data/cephfs-2/test/x2/y2/z
> > > > > > > Size: 0 Blocks: 0 IO Block: 4194304 regular empty file
> > > > > > > Device: 2ch/44d Inode: 1099840816761 Links: 1
> > > > > > > Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
> > > > > > > Access: 2022-01-27 16:24:27.713720980 +0100
> > > > > > > Modify: 2022-01-27 16:24:27.713720980 +0100
> > > > > > > Change: 2022-01-27 16:24:27.713720980 +0100
> > > > > > > Birth: -
> > > > > > >
> > > > > > > -- resulting remote file system --
> > > > > > >
> > > > > > > 0|0[root@gw-1 ~]# find /data/cephfs-3/test/x2 | xargs stat
> > > > > > > File: /data/cephfs-3/test/x2
> > > > > > > Size: 0 Blocks: 0 IO Block: 65536 directory
> > > > > > > Device: 2dh/45d Inode: 1099521812568 Links: 2
> > > > > > > Access: (2440/dr--r-S---) Uid: ( 0/ root) Gid: ( 0/ root)
> > > > > > > Access: 2022-01-27 16:24:15.627783470 +0100
> > > > > > > Modify: 2022-01-27 16:24:22.001750514 +0100
> > > > > > > Change: 2022-01-27 16:25:53.638392179 +0100
> > > > > > > Birth: -
> > > > > >
> > > > > > The mirror daemon requires write access to a directory to update
> > > > > > entries (it uses libcephfs with uid/gid 0:0). The mode/ownership
> > > > > > changes are applied after creating the entry on the other cluster.
> > > > > >
> > > > > > There's probably no "quick" workarounds for this, I'm afraid.
> > > > > >
> > > > > > >
> > > > > > > -- log excerpt --
> > > > > > >
> > > > > > > debug 2022-01-27T15:25:42.476+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > register_directory: dir_root=/test
> > > > > > > debug 2022-01-27T15:25:42.476+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > try_lock_directory: dir_root=/test
> > > > > > > debug 2022-01-27T15:25:42.477+0000 7fe0ffbf0700 10
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > try_lock_directory: dir_root=/test locked
> > > > > > > debug 2022-01-27T15:25:42.477+0000 7fe0ffbf0700 5
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > register_directory: dir_root=/test registered with
> > > > > > > replayer=0x56173a70a680
> > > > > > > debug 2022-01-27T15:25:42.477+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > sync_snaps: dir_root=/test
> > > > > > > debug 2022-01-27T15:25:42.477+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > do_sync_snaps: dir_root=/test
> > > > > > > debug 2022-01-27T15:25:42.477+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > build_snap_map: dir_root=/test, snap_dir=/test/.snap, is_remote=0
> > > > > > > debug 2022-01-27T15:25:42.477+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > build_snap_map: entry=.
> > > > > > > debug 2022-01-27T15:25:42.478+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > build_snap_map: entry=..
> > > > > > > debug 2022-01-27T15:25:42.478+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > build_snap_map: entry=initial
> > > > > > > debug 2022-01-27T15:25:42.478+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > build_snap_map: entry=second
> > > > > > > debug 2022-01-27T15:25:42.478+0000 7fe0ffbf0700 10
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > build_snap_map: local snap_map={1384=initial,1385=second}
> > > > > > > debug 2022-01-27T15:25:42.478+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > build_snap_map: dir_root=/test, snap_dir=/test/.snap, is_remote=1
> > > > > > > debug 2022-01-27T15:25:42.479+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > build_snap_map: entry=.
> > > > > > > debug 2022-01-27T15:25:42.479+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > build_snap_map: entry=..
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > build_snap_map: entry=initial
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > build_snap_map: snap_path=/test/.snap/initial,
> > > > > > > metadata={primary_snap_id=1384}
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 10
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > build_snap_map: remote snap_map={1384=initial}
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 5
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > propagate_snap_deletes: dir_root=/test, deleted snapshots=
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 10
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > propagate_snap_renames: dir_root=/test, renamed snapshots=
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 5
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > do_sync_snaps: last snap-id transferred=1384
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 10
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > do_sync_snaps: synchronizing from snap-id=1385
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > synchronize: dir_root=/test, current=second,1385
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > synchronize: prev= initial,1384
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > synchronize: dirty_snap_id: 1384 vs (1385,1384)
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 5
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > synchronize: match -- using incremental sync with local scan
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > do_synchronize: dir_root=/test, current=second,1385
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > do_synchronize: incremental sync check from prev= initial,1384
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > pre_sync_check_and_open_handles: dir_root=/test, current=second,1385
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > pre_sync_check_and_open_handles: prev= initial,1384
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > open_dir: dir_path=/test/.snap/second
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > open_dir: expected snapshot id=1385
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > open_dir: dir_path=/test/.snap/initial
> > > > > > > debug 2022-01-27T15:25:42.480+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > open_dir: expected snapshot id=1384
> > > > > > > debug 2022-01-27T15:25:42.481+0000 7fe0ffbf0700 5
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > pre_sync_check_and_open_handles: using local (previous) snapshot for
> > > > > > > incremental transfer
> > > > > > > debug 2022-01-27T15:25:42.485+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > do_synchronize: 1 entries in stack
> > > > > > > debug 2022-01-27T15:25:42.485+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > do_synchronize: top of stack path=.
> > > > > > > debug 2022-01-27T15:25:42.485+0000 7fe0ffbf0700 10
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > propagate_deleted_entries: dir_root=/test, epath=.
> > > > > > > debug 2022-01-27T15:25:42.487+0000 7fe0ffbf0700 5
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > propagate_deleted_entries: mode matches for entry=x
> > > > > > > debug 2022-01-27T15:25:42.487+0000 7fe0ffbf0700 10
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > propagate_deleted_entries: reached EOD
> > > > > > > debug 2022-01-27T15:25:42.487+0000 7fe0ffbf0700 10
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > remote_mkdir: remote epath=./x2
> > > > > > > debug 2022-01-27T15:25:42.513+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > do_synchronize: 2 entries in stack
> > > > > > > debug 2022-01-27T15:25:42.513+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > do_synchronize: top of stack path=./x2
> > > > > > > debug 2022-01-27T15:25:42.513+0000 7fe0ffbf0700 10
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > propagate_deleted_entries: dir_root=/test, epath=./x2
> > > > > > > debug 2022-01-27T15:25:42.513+0000 7fe0ffbf0700 5
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > propagate_deleted_entries: epath=./x2 missing in previous-snap/remote
> > > > > > > dir-root
> > > > > > > debug 2022-01-27T15:25:42.514+0000 7fe0ffbf0700 10
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > remote_mkdir: remote epath=./x2/y2
> > > > > > > debug 2022-01-27T15:25:42.514+0000 7fe0ffbf0700 -1
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > remote_mkdir: failed to create remote directory=./x2/y2: (13)
> > > > > > > Permission denied
> > > > > > > debug 2022-01-27T15:25:42.514+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > do_synchronize: closing local directory=./x2
> > > > > > > debug 2022-01-27T15:25:42.514+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > do_synchronize: closing local directory=.
> > > > > > > debug 2022-01-27T15:25:42.514+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > post_sync_close_handles
> > > > > > > debug 2022-01-27T15:25:42.514+0000 7fe0ffbf0700 -1
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > do_sync_snaps: failed to synchronize dir_root=/test, snapshot=second
> > > > > > > debug 2022-01-27T15:25:42.514+0000 7fe0ffbf0700 -1
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > sync_snaps: failed to sync snapshots for dir_root=/test
> > > > > > > debug 2022-01-27T15:25:42.514+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > unregister_directory: dir_root=/test
> > > > > > > debug 2022-01-27T15:25:42.514+0000 7fe0ffbf0700 20
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > unlock_directory: dir_root=/test
> > > > > > > debug 2022-01-27T15:25:42.515+0000 7fe0ffbf0700 10
> > > > > > > cephfs::mirror::PeerReplayer(f477cfed-6270-4beb-aaa1-a41df7b58955)
> > > > > > > unlock_directory: dir_root=/test unlocked
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Jan 27, 2022 at 3:00 PM Venky Shankar <vshankar@xxxxxxxxxx> wrote:
> > > > > > > >
> > > > > > > > On Wed, Jan 26, 2022 at 2:44 PM Manuel Holtgrewe <zyklenfrei@xxxxxxxxx> wrote:
> > > > > > > > >
> > > > > > > > > Dear all,
> > > > > > > > >
> > > > > > > > > I want to mirror a snapshot in Ceph v16.2.6 deployed with cephadm
> > > > > > > > > using the stock quay.io images. My source file system has a folder
> > > > > > > > > "/src/folder/x" where "/src/folder" has mode "ug=r,o=", in other words
> > > > > > > > > no write permissions for the owner (root).
> > > > > > > >
> > > > > > > > What mode does /src/folder get created on the other cluster?
> > > > > > > >
> > > > > > > > >
> > > > > > > > > The sync of a snapshot "initial" now fails with the following log excerpt.
> > > > > > > > >
> > > > > > > > > remote_mkdir: remote epath=./src/folder/x
> > > > > > > > > remote_mkdir: failed to create remote directory=./src/folder/x: (13)
> > > > > > > > > Permission denied
> > > > > > > > > do_synchronize: closing local directory=./src/folder
> > > > > > > > > do_synchronize: closing local directory=./src/
> > > > > > > > > do_synchronize: closing local directory=.
> > > > > > > > > post_sync_close_handles
> > > > > > > > > do_sync_snaps: failed to synchronize dir_root=/src/folder, snapshot=initial
> > > > > > > > > sync_snaps: failed to sync snapshots for dir_root=/src/folder
> > > > > > > > >
> > > > > > > > > The capabilities on the remote site are:
> > > > > > > > >
> > > > > > > > > client.mirror-tier-2-remote
> > > > > > > > > key: REDACTED
> > > > > > > > > caps: [mds] allow * fsname=cephfs
> > > > > > > > > caps: [mon] allow r fsname=cephfs
> > > > > > > > > caps: [osd] allow * tag cephfs data=cephfs
> > > > > > > > >
> > > > > > > > > I also just reported this in the tracker [1]. Can anyone think of a
> > > > > > > > > workaround (in the lines of "sudo make me a sandwich ;-)?"
> > > > > > > > >
> > > > > > > > > Best wishes,
> > > > > > > > > Manuel
> > > > > > > > >
> > > > > > > > > [1] https://tracker.ceph.com/issues/54017
> > > > > > > > > _______________________________________________
> > > > > > > > > ceph-users mailing list -- ceph-users@xxxxxxx
> > > > > > > > > To unsubscribe send an email to ceph-users-leave@xxxxxxx
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Cheers,
> > > > > > > > Venky
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Cheers,
> > > > > > Venky
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Cheers,
> > > > Venky
> > > >
> > >
> >
> >
> > --
> > Cheers,
> > Venky
> >
>
--
Cheers,
Venky
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
