I never tried "prio" out, but the reports I have seen claim that prio is inferior. However, as far as I know it is safe to change these settings. Unfortunately, you need to restart services to apply the changes. Before you do, check if *all* daemons are using the same setting. Contrary to the naming (osd_*), this setting applies to all daemons. I added it to the global options and, most notably, performance of the MDS was improved a lot. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Francois Legrand <fleg@xxxxxxxxxxxxxx> Sent: 26 June 2020 15:03:23 To: Frank Schilder; ceph-users@xxxxxxx Subject: Re: Re: Removing pool in nautilus is incredibly slow I changed osd_op_queue_cut_off to high and rebooted all the osds. But the result is more or less the same (storage is still extremely slow, 2h30 to rdb extract a 64GB image !). The only improvement is that it seems that degraded pgs have disapeared (which is at least a good point). It seems that there is a problem in priority of operations. Thus do you think (and also others on the list) that changing the osd_op_queue setting could help (change to prio or mclock_client). What are the risks or secondary effects of trying mclock_client on a production cluster (is it safe) ? F. Le 26/06/2020 à 09:46, Frank Schilder a écrit : > I'm using > > osd_op_queue = wpq > osd_op_queue_cut_off = high > > and these settings are recommended. > > Best regards, > ================= > Frank Schilder > AIT Risø Campus > Bygning 109, rum S14 > > ________________________________________ > From: Francois Legrand <fleg@xxxxxxxxxxxxxx> > Sent: 26 June 2020 09:44:00 > To: Frank Schilder; ceph-users@xxxxxxx > Subject: Re: Re: Removing pool in nautilus is incredibly slow > > We are now using osd_op_queue = wpq. Maybe returning to prio should help ? > What are you using on your mimic custer ? > F. > > Le 25/06/2020 à 19:28, Frank Schilder a écrit : >> OK, this *does* sound bad. I would consider this a show stopper for upgrade from mimic. >> >> Best regards, >> ================= >> Frank Schilder >> AIT Risø Campus >> Bygning 109, rum S14 >> >> ________________________________________ >> From: Francois Legrand <fleg@xxxxxxxxxxxxxx> >> Sent: 25 June 2020 19:25:14 >> To: ceph-users@xxxxxxx >> Subject: Re: Removing pool in nautilus is incredibly slow >> >> I also had this kind of symptoms with nautilus. >> Replacing a failed disk (from cluster ok) generates degraded objects. >> Also, we have a proxmox cluster accessing vm images stored in our ceph storage with rbd. >> Each time I had some operation on the ceph cluster like adding or removing a pool, most of our proxmox vms lost contact with their system disk in ceph and crashed (or remount system storage in read-only mode). At first I thought it was a network problem, but now I am sure that it's related to ceph becoming unresponsive during background operations. >> For now, proxmox cannot even access ceph storage using rbd (it fails with timeout). >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |