Re: MDS internal op exportdir despite ephemeral pinning

[Frank Schilder <frans@xxxxxx>]

Hi Patrick,

thanks for your super fast answer.

> I assume you mean "distributed ephemeral pinning"?

Yes. Just to remove any potential for a misunderstanding from my side, I enabled it with (copy-paste from the command history, /mnt/admin/cephfs/ is the mount point of "/" with all possible client permissions granted):

setfattr -n ceph.dir.pin.distributed -v 1 /mnt/admin/cephfs/hpc/home
setfattr -n ceph.dir.pin.distributed -v 1 /mnt/admin/cephfs/hpc/groups
setfattr -n ceph.dir.pin.distributed -v 1 /mnt/admin/cephfs/shares

# ceph versions
{
    "mon": {
        "ceph version 15.2.17 (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable)": 5
    },
    "mgr": {
        "ceph version 15.2.17 (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable)": 5
    },
    "osd": {
        "ceph version 15.2.17 (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable)": 1048
    },
    "mds": {
        "ceph version 15.2.17 (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable)": 12
    },
    "overall": {
        "ceph version 15.2.17 (8a82819d84cf884bd39c17e3236e0632ac146dc4) octopus (stable)": 1070
    }
}

I will collect the other output you ask for and send it to you privately. Unless you state otherwise, I will attach a gz-file to an e-mail.

Thanks for your help!

Best regards,
=================
Frank Schilder
AIT Risø Campus
Bygning 109, rum S14

________________________________________
From: Patrick Donnelly <pdonnell@xxxxxxxxxx>
Sent: 18 November 2022 19:07:30
To: Frank Schilder
Cc: ceph-users@xxxxxxx
Subject: Re:  MDS internal op exportdir despite ephemeral pinning

On Fri, Nov 18, 2022 at 12:51 PM Frank Schilder <frans@xxxxxx> wrote:
>
> Hi Patrick,
>
> thanks! I did the following but don't know how to interpret the result. The three directories we have ephemeral pinning set are:
>
> /shares
> /hpc/home
> /hpc/groups

I assume you mean "distributed ephemeral pinning"?

> If I understand the documentation correctly, everything under /hpc/home/user should be on the same MDS. Trying it out I get (user-name obscured):
>
> # for mds in $(bin/active_mds); do
>   echo -n "${mds}: "
>   ceph tell mds.$mds get subtrees | grep '"/hpc/home/user' | wc -l
> done 2>/dev/null
> ceph-13: 14
> ceph-16: 2
> ceph-14: 2
> ceph-08: 14
> ceph-17: 0
> ceph-11: 6
> ceph-12: 14
> ceph-10: 14
>
> Its all over the place. Could you please help me with how I should interpret this?

Please share the version you're using. "/hpc/home/user" should not
show up in the subtree output. If possible, can you privately share
with me the output of:

- `ceph versions`
- `ceph fs dump`
- `get subtrees` on each active MDS
- `dump tree /hpc/home 0` on each active MDS

Feel free to anonymize path strings as desired.


--
Patrick Donnelly, Ph.D.
He / Him / His
Principal Software Engineer
Red Hat, Inc.
GPG: 19F28A586F808C2402351B93C3301A3E258DD79D

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx