Hi Paul, thanks for the fast reply. When you say "bit 21", do you mean "(feature_map & 2^21) == true" (i.e., counting from 0 starting at the right-hand end)? Then it would be set for all connected ceph fs clients. Assuming upmap is supported by all clients. If I understand correctly, to use the upmap mode with balancer, I need to set ceph osd set-require-min-compat-client luminous Which I would guess will not allow the jewel clients to reconnect. I would be grateful if you could clarify these points to me: 1) Can I use up-map mode without setting this? 2) If so, what happens if a jewel client without this feature bit set tries to connect? 3) I guess that in case that as soon as an up-map table is created, only clients with this bit set can connect. In case we run into problems, is there a way to roll back? Many thanks and best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Paul Emmerich <paul.emmerich@xxxxxxxx> Sent: 13 April 2020 13:32:40 To: Frank Schilder Cc: ceph-users Subject: Re: Check if upmap is supported by client? bit 21 in the features bitmap is upmap support Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 On Mon, Apr 13, 2020 at 11:53 AM Frank Schilder <frans@xxxxxx> wrote: > > Dear all, > > I would like to enable the balancer on a mimic 13.2.8 cluster in upmap mode. Unfortunately, I have a lot of ceph fs kernel clients that report their version as jewel, but might already support upmap. The ceph client kernel module received already a lot of back-ports and supports features of later ceph versions, for example, quotas. I guess they report back jewel, because not all luminous/mimic features are back-ported yet. Is there a way to check if a client supports upmap? > > Here some info: > > [root@gnosis ~]# ceph features > { > [...] > "client": [ > { > "features": "0x27018fb86aa42ada", > "release": "jewel", > "num": 1676 > }, > { > "features": "0x2f018fb86aa42ada", > "release": "luminous", > "num": 1 > }, > { > "features": "0x3ffddff8ffacfffb", > "release": "luminous", > "num": 167 > } > ], > > The fs clients are the top two entries, the third entry is rbd clients. Note that the feature key for the fs clients is almost identical. Here a snippet from mds session ls for one such jewel client: > > { > "id": 25641514, > "num_leases": 0, > "num_caps": 1, > "state": "open", > "request_load_avg": 0, > "uptime": 588563.550276, > "replay_requests": 0, > "completed_requests": 0, > "reconnecting": false, > "inst": "client.25641514 192.168.57.124:0/3398308464", > "client_metadata": { > "features": "00000000000000ff", > "entity_id": "con-fs2-hpc", > "hostname": "sn253.hpc.ait.dtu.dk", > "kernel_version": "3.10.0-957.12.2.el7.x86_64", > "root": "/hpc/groups" > } > } > > Since I would like to use upmap right from the beginning, my alternative is to re-weight a few of the really bad outliers manually to simplify changing back. > > What would you suggest? > > Thanks and best regards, > > ================= > Frank Schilder > AIT Risø Campus > Bygning 109, rum S14 > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |