Re: OSD spend too much time on "waiting for readable" -> slow ops -> laggy pg -> rgw stop -> worst case osd restart

[Manuel Lausch <manuel.lausch@xxxxxxxx>]

This is the patch I made. I think this is the wrong place to do this. but in the first place in worked.


diff --git a/src/osd/PrimaryLogPG.cc b/src/osd/PrimaryLogPG.cc
index 9fb22e0f9ee..69341840153 100644
--- a/src/osd/PrimaryLogPG.cc
+++ b/src/osd/PrimaryLogPG.cc
@@ -798,6 +798,10 @@ void PrimaryLogPG::maybe_force_recovery()
 
 bool PrimaryLogPG::check_laggy(OpRequestRef& op)
 {
+  if (!cct->_conf->osd_read_lease_enabled) {
+    // possibility to deactivate this feature.
+    return true;
+  }
   if (!HAVE_FEATURE(recovery_state.get_min_upacting_features(),
                    SERVER_OCTOPUS)) {
     dout(20) << __func__ << " not all upacting has SERVER_OCTOPUS" << dendl;
@@ -833,6 +837,10 @@ bool PrimaryLogPG::check_laggy(OpRequestRef& op)
 
 bool PrimaryLogPG::check_laggy_requeue(OpRequestRef& op)
 {
+  if (!cct->_conf->osd_read_lease_enabled) {
+    // possibility to deactivate this feature.
+    return true;
+  }
   if (!HAVE_FEATURE(recovery_state.get_min_upacting_features(),
                    SERVER_OCTOPUS)) {
     return true;


________________________________________
Von: Peter Lieven <pl@xxxxxxx>
Gesendet: Mittwoch, 10. November 2021 11:37
An: Manuel Lausch; Sage Weil
Cc: ceph-users@xxxxxxx
Betreff: Re:  Re: OSD spend too much time on "waiting for readable" -> slow ops -> laggy pg -> rgw stop -> worst case osd restart

Am 10.11.21 um 11:35 schrieb Manuel Lausch:
> oh shit,
>
> I patched in a switch to deactivate the read_lease feature. This is only a hack to test a bit around. But accidentally I had this switch enabled for my last tests done here in this mail-thread.
>
> The bad news. The require_osd_release doesn't fix the slow op problematic, only the increasing of the osdmap epochs are fixed.
> Unfortunately, even reduceing the paxos_prpopose_interval changes anything. My last tests with it was wrong due to my hack :-(


Would it be an option to make this hack a switch for all those who don't require the read lease feature and are happy with reading from just the primary?


Peter



_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx