i don't suppose you're using sts roles with AssumeRole? https://tracker.ceph.com/issues/59495 tracks a bug where each AssumeRole request was writing to the user metadata unnecessarily, which would race with your admin api requests On Tue, Oct 24, 2023 at 9:56 AM mahnoosh shahidi <mahnooosh.shd@xxxxxxxxx> wrote: > > Thanks Casey for your explanation, > > Yes it succeeded eventually. Sometimes after about 100 retries. It's odd that it stays in racing condition for that much time. > > Best Regards, > Mahnoosh > > On Tue, Oct 24, 2023 at 5:17 PM Casey Bodley <cbodley@xxxxxxxxxx> wrote: >> >> errno 125 is ECANCELED, which is the code we use when we detect a >> racing write. so it sounds like something else is modifying that user >> at the same time. does it eventually succeed if you retry? >> >> On Tue, Oct 24, 2023 at 9:21 AM mahnoosh shahidi >> <mahnooosh.shd@xxxxxxxxx> wrote: >> > >> > Hi all, >> > >> > I couldn't understand what does the status -125 mean from the docs. I'm >> > getting 500 response status code when I call rgw admin APIs and the only >> > log in the rgw log files is as follows. >> > >> > s3:get_obj recalculating target >> > initializing for trans_id = >> > tx00000aa90f570fb8281cf-006537bf9e-84395fa-default >> > s3:get_obj reading permissions >> > getting op 1 >> > s3:put_obj verifying requester >> > s3:put_obj normalizing buckets and tenants >> > s3:put_obj init permissions >> > s3:put_obj recalculating target >> > s3:put_obj reading permissions >> > s3:put_obj init op >> > s3:put_obj verifying op mask >> > s3:put_obj verifying op permissions >> > s3:put_obj verifying op params >> > s3:put_obj pre-executing >> > s3:put_obj executing >> > :modify_user completing >> > WARNING: set_req_state_err err_no=125 resorting to 500 >> > :modify_user op status=-125 >> > :modify_user http status=500 >> > ====== req done req=0x7f3f85a78620 op status=-125 http_status=500 >> > latency=0.076000459s ====== >> > >> > Can anyone explain what this error means and why it's happening? >> > >> > Best Regards, >> > Mahnoosh >> > _______________________________________________ >> > ceph-users mailing list -- ceph-users@xxxxxxx >> > To unsubscribe send an email to ceph-users-leave@xxxxxxx >> > >> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |