Hi, On Mon, Apr 13, 2020 at 3:08 PM Frank Schilder <frans@xxxxxx> wrote: > > Hi Paul, > > thanks for the fast reply. When you say "bit 21", do you mean "(feature_map & 2^21) == true" (i.e., counting from 0 starting at the right-hand end)? yes > Assuming upmap is supported by all clients. If I understand correctly, to use the upmap mode with balancer, I need to set > > ceph osd set-require-min-compat-client luminous > > Which I would guess will not allow the jewel clients to reconnect. I would be grateful if you could clarify these points to me: yes > 1) Can I use up-map mode without setting this? no > 2) If so, what happens if a jewel client without this feature bit set tries to connect? it'll error out with a message about feature mismatch; it checks actually relevant feature flags not the reverse mapping to a release which is usually wrong for kernel clients > 3) I guess that in case that as soon as an up-map table is created, only clients with this bit set can connect. In case we run into problems, is there a way to roll back? yes, you can remove the upmap items manually and change the client requirement; I don't know the command to do this off the top of my head Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 > > Many thanks and best regards, > > ================= > Frank Schilder > AIT Risø Campus > Bygning 109, rum S14 > > ________________________________________ > From: Paul Emmerich <paul.emmerich@xxxxxxxx> > Sent: 13 April 2020 13:32:40 > To: Frank Schilder > Cc: ceph-users > Subject: Re: Check if upmap is supported by client? > > bit 21 in the features bitmap is upmap support > > Paul > > -- > Paul Emmerich > > Looking for help with your Ceph cluster? Contact us at https://croit.io > > croit GmbH > Freseniusstr. 31h > 81247 München > www.croit.io > Tel: +49 89 1896585 90 > > On Mon, Apr 13, 2020 at 11:53 AM Frank Schilder <frans@xxxxxx> wrote: > > > > Dear all, > > > > I would like to enable the balancer on a mimic 13.2.8 cluster in upmap mode. Unfortunately, I have a lot of ceph fs kernel clients that report their version as jewel, but might already support upmap. The ceph client kernel module received already a lot of back-ports and supports features of later ceph versions, for example, quotas. I guess they report back jewel, because not all luminous/mimic features are back-ported yet. Is there a way to check if a client supports upmap? > > > > Here some info: > > > > [root@gnosis ~]# ceph features > > { > > [...] > > "client": [ > > { > > "features": "0x27018fb86aa42ada", > > "release": "jewel", > > "num": 1676 > > }, > > { > > "features": "0x2f018fb86aa42ada", > > "release": "luminous", > > "num": 1 > > }, > > { > > "features": "0x3ffddff8ffacfffb", > > "release": "luminous", > > "num": 167 > > } > > ], > > > > The fs clients are the top two entries, the third entry is rbd clients. Note that the feature key for the fs clients is almost identical. Here a snippet from mds session ls for one such jewel client: > > > > { > > "id": 25641514, > > "num_leases": 0, > > "num_caps": 1, > > "state": "open", > > "request_load_avg": 0, > > "uptime": 588563.550276, > > "replay_requests": 0, > > "completed_requests": 0, > > "reconnecting": false, > > "inst": "client.25641514 192.168.57.124:0/3398308464", > > "client_metadata": { > > "features": "00000000000000ff", > > "entity_id": "con-fs2-hpc", > > "hostname": "sn253.hpc.ait.dtu.dk", > > "kernel_version": "3.10.0-957.12.2.el7.x86_64", > > "root": "/hpc/groups" > > } > > } > > > > Since I would like to use upmap right from the beginning, my alternative is to re-weight a few of the really bad outliers manually to simplify changing back. > > > > What would you suggest? > > > > Thanks and best regards, > > > > ================= > > Frank Schilder > > AIT Risø Campus > > Bygning 109, rum S14 > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx > > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |