On Tue, Jul 11, 2017 at 12:46 PM, Ashley Merrick <ashley@xxxxxxxxxxxxxx> wrote: > Hello, > > Perfect thanks that fixed my issue! > > Still seems to be a bug on the ceph pg dump unless it has been moved out of the PG and directly into the OSD? I am looking into this issue and have been trying to bisect it but it has proved challenging. Please feel free to open a tracker as we have discussed or I will open one once I have more information. Edward, if you feel this could be improved please also open a separate tracker for the ratio ordering issue outlining the steps involved and noting the issues encountered. > > ,Ashley > > -----Original Message----- > From: Edward R Huyer [mailto:erhvks@xxxxxxx] > Sent: Tuesday, 11 July 2017 7:53 AM > To: Ashley Merrick <ashley@xxxxxxxxxxxxxx> > Cc: ceph-users@xxxxxxxxxxxxxx > Subject: OSD Full Ratio Luminous - Unset > > I just now ran into the same problem you did, though I managed to get it straightened out. > > It looks to me like the "ceph osd set-{full,nearfull,backfillfull}-ratio" commands *do* work, with two caveats. > > Caveat 1: "ceph pg dump" doesn't reflect the change for some reason. > Caveat 2: There doesn't seem to be much of a safety measure to prevent you from entering incoherent values. In particular, if backfillfull-ratio is smaller than nearfull-ratio, you will continue to have the ERR health message. Backfillfull must be greater than (or equal to?) nearfull. So, for instance, my cluster at the moment has a full ratio of .9, a backfillfull ratio of 0.85, and a nearfull ratio of 0.8. > > So there appears to be a bug when upgrading from Jewel where the full ratios don't carry over properly, a display bug of some kind with "ceph pg dump", and a general issue with lack of information about full ratios being out of order. Oh, and the set ratio commands allow incoherent values, though that might be intentional. > > ----- > Edward Huyer > School of Interactive Games and Media > Rochester Institute of Technology > Golisano 70-2373 > 152 Lomb Memorial Drive > Rochester, NY 14623 > 585-475-6651 > mailto:erhvks@xxxxxxx > > Obligatory Legalese: > The information transmitted, including attachments, is intended only for the person(s) or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and destroy any copies of this information. > > So using the commands given I checked all the mon's and a couple of OSD's that are not backfilling due to saying they are backfill_full. > > However when running the command the value's I have been trying to set via the set commands are correctly set and reporting via the admin socket. > > However "ceph pg dump | head" still shows 0 for them all and the PG's still aren't back filling, so apart from ceph pg dump showing incorrect values the admin socket shows correct however cluster is still stuck in non-backfilling state / ERR_health due to messages. > > ,Ashley > > ________________________________ > From: Brad Hubbard <bhubbard at redhat.com> > Sent: 07 July 2017 10:31:01 > To: Ashley Merrick > Cc: ceph-users at ceph.com > Subject: Re: OSD Full Ratio Luminous - Unset > > On Fri, Jul 7, 2017 at 4:49 PM, Ashley Merrick <ashley at amerrick.co.uk> wrote: >> After looking into this further it seem's none of the : >> >> >> ceph osd set-{full,nearfull,backfillfull}-ratio >> >> >> Commands seem to be taking any effect on the cluster including the >> backfillfull ratio, this command looks to have been added/changed >> since Jewel, and a different way of setting the above. However does >> not seem to be giving the expected results. > > Hi Ashley, > > Please do open a tracker for this including commands used and resulting output so we can investigate this fully. > > In the meantime you should be able to view and manipulate individual full ratio values via the admin socket. > > ceph daemon osd.0 config show|grep ratio ceph daemon mon.a config show|grep ratio ceph daemon osd.0 help|grep "config set " > >> >> >> ,Ashley >> >> ________________________________ >> From: ceph-users <ceph-users-bounces at lists.ceph.com> on behalf of >> Ashley Merrick <ashley at amerrick.co.uk> >> Sent: 06 July 2017 12:44:09 >> To: ceph-users at ceph.com >> Subject: Re: OSD Full Ratio Luminous - Unset >> >> Anyone have some feedback on this? Happy to log a bug ticket if it is >> one, but want to make sure not missing something Luminous change related. >> >> ,Ashley >> >> Sent from my iPhone >> >> On 4 Jul 2017, at 3:30 PM, Ashley Merrick <ashley at amerrick.co.uk> wrote: >> >> Okie noticed their is a new command to set these. >> >> >> Tried these and still showing as 0 and error on full ratio out of >> order "ceph osd set-{full,nearfull,backfillfull}-ratio" >> >> >> ,Ashley >> >> ________________________________ >> From: ceph-users <ceph-users-bounces at lists.ceph.com> on behalf of >> Ashley Merrick <ashley at amerrick.co.uk> >> Sent: 04 July 2017 05:55:10 >> To: ceph-users at ceph.com >> Subject: OSD Full Ratio Luminous - Unset >> >> >> Hello, >> >> >> On a Luminous upgraded from Jewel I am seeing the following in ceph -s : >> "Full ratio(s) out of order" >> >> >> and >> >> >> ceph pg dump | head >> dumped all >> version 44281 >> stamp 2017-07-04 05:52:08.337258 >> last_osdmap_epoch 0 >> last_pg_scan 0 >> full_ratio 0 >> nearfull_ratio 0 >> >> I have tried to inject the values however makes no effect, these where >> previously non 0 values and the issue only showed after running "ceph >> osd require-osd-release luminous" >> >> >> Thanks, >> >> Ashley >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users at lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users at lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Cheers, Brad _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |