Re: crashes after upgrade from octopus to pacific

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



 ceph version 16.2.9 (4c3647a322c0ff5a1dd2344e039859dcbd28c830) pacific (stable)
 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f4558e24ce0]
 2: (RGWHandler_REST_S3Website::retarget(RGWOp*, RGWOp**, optional_yield)+0x174) [0x7f4563e02684]
 3: (rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, optional_yield, bool)+0xf0a) [0x7f456399e6fa]
 4: (process_request(rgw::sal::RGWRadosStore*, RGWREST*, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rgw::auth::StrategyRegistry const&, RGWRestfulIO*, OpsLogSink*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*, int*)+0x2891) [0x7f45639a21c1]
 5: /lib64/libradosgw.so.2(+0x43d640) [0x7f4563921640]
 6: /lib64/libradosgw.so.2(+0x43ef6a) [0x7f4563922f6a]
 7: make_fcontext()
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.


> On Jul 19, 2022, at 2:16 PM, Ramin Najjarbashi <ramin.najarbashi@xxxxxxxxx> wrote:
> 
> Hi
> My account in "tracker.ceph.com <http://tracker.ceph.com/>" was not approved after 5 days but anyway I have some problems in Ceph v16.2.7.
> Accourding this issue [rgw: s3website crashes after upgrade from octopus to pacific](https://tracker.ceph.com/issues/53913 <http://tracker.ceph.com/issues/53913>), RGW crashed whene s3website calls without subdomain (`s->object=<NULL> s->bucket=`)
> 
> ---- RGW LOG ----
>   -638> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 HTTP_ACCEPT=text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
>   -637> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 HTTP_ACCEPT_ENCODING=gzip, deflate
>   -636> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 HTTP_ACCEPT_LANGUAGE=en-US,en;q=0.9,fr;q=0.8
>   -630> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 HTTP_CACHE_CONTROL=max-age=0
>   -628> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 HTTP_HOST=s3-website.XXX.com
>   -627> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 HTTP_UPGRADE_INSECURE_REQUESTS=1
>   -626> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 HTTP_USER_AGENT=Mozlila/5.0 (Linux; Android 7.0; SM-G892A Bulid/NRD90M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/60.0.3112.107 Mobile Safari/537.36
>   -625> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 HTTP_VERSION=1.1
>   -621> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 HTTP_X_SCHEME=http
>   -619> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 REQUEST_METHOD=GET
>   -618> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 REQUEST_URI=/
>   -617> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 SCRIPT_URI=/
>   -616> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 SERVER_PORT=1234
>   -615> 2022-07-16T06:05:24.159+0430 7fbd26b71700  1 ====== starting new request req=0x7fbc4a034620 =====
>   -614> 2022-07-16T06:05:24.159+0430 7fbd26b71700  2 req 14431852651046008579 0.000000000s initializing for trans_id = tx00000c8483d85dc607703-0062d215dc-15359a41-default
>   -613> 2022-07-16T06:05:24.159+0430 7fbd26b71700 10 req 14431852651046008579 0.000000000s rgw api priority: s3=8 s3website=7
>   -612> 2022-07-16T06:05:24.159+0430 7fbd26b71700 10 req 14431852651046008579 0.000000000s host=s3-website.XXX.com
>   -611> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 req 14431852651046008579 0.000000000s subdomain= domain=s3-website.XXX.com in_hosted_domain=1 in_hosted_domain_s3website=1
>   -610> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 req 14431852651046008579 0.000000000s final domain/bucket subdomain= domain=s3-website.XXX.com in_hosted_domain=1 in_hosted_domain_s3website=1 s->info.domain=s3-website.XXX.com s->info.request_uri=/
>   -609> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 req 14431852651046008579 0.000000000s get_handler handler=33RGWHandler_REST_Service_S3Website
>   -608> 2022-07-16T06:05:24.159+0430 7fbd26b71700 10 req 14431852651046008579 0.000000000s handler=33RGWHandler_REST_Service_S3Website
>   -607> 2022-07-16T06:05:24.159+0430 7fbd26b71700  2 req 14431852651046008579 0.000000000s getting op 0
>   -606> 2022-07-16T06:05:24.159+0430 7fbd26b71700 10 req 14431852651046008579 0.000000000s s3:get_obj scheduling with throttler client=2 cost=1
>   -605> 2022-07-16T06:05:24.159+0430 7fbd26b71700 10 req 14431852651046008579 0.000000000s s3:get_obj op=28RGWGetObj_ObjStore_S3Website
>   -604> 2022-07-16T06:05:24.159+0430 7fbd26b71700  2 req 14431852651046008579 0.000000000s s3:get_obj verifying requester
>   -603> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 req 14431852651046008579 0.000000000s s3:get_obj rgw::auth::StrategyRegistry::s3_main_strategy_t: trying rgw::auth::s3::AWSAuthStrategy
>   -602> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 req 14431852651046008579 0.000000000s s3:get_obj rgw::auth::s3::AWSAuthStrategy: trying rgw::auth::s3::S3AnonymousEngine
>   -601> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 req 14431852651046008579 0.000000000s s3:get_obj rgw::auth::s3::S3AnonymousEngine granted access
>   -600> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 req 14431852651046008579 0.000000000s s3:get_obj rgw::auth::s3::AWSAuthStrategy granted access
>   -599> 2022-07-16T06:05:24.159+0430 7fbd26b71700  2 req 14431852651046008579 0.000000000s s3:get_obj normalizing buckets and tenants
>   -598> 2022-07-16T06:05:24.159+0430 7fbd26b71700 10 req 14431852651046008579 0.000000000s s->object=<NULL> s->bucket=
>   -597> 2022-07-16T06:05:24.159+0430 7fbd26b71700  2 req 14431852651046008579 0.000000000s s3:get_obj init permissions
>   -596> 2022-07-16T06:05:24.159+0430 7fbd26b71700 20 req 14431852651046008579 0.000000000s s3:get_obj RGWSI_User_RADOS::read_user_info(): anonymous user
>   -595> 2022-07-16T06:05:24.159+0430 7fbd26b71700  2 req 14431852651046008579 0.000000000s s3:get_obj recalculating target
>   -594> 2022-07-16T06:05:24.159+0430 7fbd26b71700 10 req 14431852651046008579 0.000000000s retarget Starting retarget
>   -291> 2022-07-16T06:05:24.191+0430 7fbd26b71700 -1 *** Caught signal (Segmentation fault) 
>   ---- END OF LOG ----
> 
>   Crash Info:
> 
>   {
>     "archived": "2022-07-15 12:56:38.444301",
>     "backtrace": [
>         "/lib64/libpthread.so.0(+0x12c20) [0x7efce9409c20]",
>         "(RGWHandler_REST_S3Website::retarget(RGWOp*, RGWOp**, optional_yield)+0x174) [0x7efcf4478c24]",
>         "(rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, optional_yield, bool)+0xf0a) [0x7efcf401735a]",
>         "(process_request(rgw::sal::RGWRadosStore*, RGWREST*, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rgw::auth::StrategyRegistry const&, RGWRestfulIO*, OpsLogSink*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*, int*)+0x2891) [0x7efcf401ae21]",
>         "/lib64/libradosgw.so.2(+0x4b1b63) [0x7efcf3f6ab63]",
>         "/lib64/libradosgw.so.2(+0x4b3604) [0x7efcf3f6c604]",
>         "/lib64/libradosgw.so.2(+0x4b386e) [0x7efcf3f6c86e]",
>         "make_fcontext()"
>     ],
>     "ceph_version": "16.2.7",
>     "crash_id": "2022-07-14T09:01:24.464923Z_e2aad000-0cc2-4cf3-b3ec-71e74bb040f9",
>     "entity_name": "client.rgw.cstorage4012.rgw0",
>     "os_id": "centos",
>     "os_name": "CentOS Linux",
>     "os_version": "8",
>     "os_version_id": "8",
>     "process_name": "radosgw",
>     "stack_sig": "310d2f225070c57edbbd5ed9c5155e0b9d68ad26119e6cfc6f1f4c845140d5e5",
>     "timestamp": "2022-07-14T09:01:24.464923Z",
>     "utsname_hostname": "cstorage4012",
>     "utsname_machine": "x86_64",
>     "utsname_release": "5.4.0-120-generic",
>     "utsname_sysname": "Linux",
>     "utsname_version": "#136-Ubuntu SMP Fri Jun 10 13:40:48 UTC 2022"
> }
> 
> 
> We have the same problem with `get Object`:
> 
> {
>     "archived": "2022-07-17 11:51:59.826423",
>     "backtrace": [
>         "/lib64/libpthread.so.0(+0x12c20) [0x7efe00733c20]",
>         "(rgw_bucket::rgw_bucket(rgw_bucket const&)+0x23) [0x7efe0b362023]",
>         "(rgw::sal::RGWObject::get_obj() const+0x20) [0x7efe0b391c30]",
>         "(RGWPostObj::execute(optional_yield)+0xa4) [0x7efe0b6c25e4]",
>         "(rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, optional_yield, bool)+0xb36) [0x7efe0b340f86]",
>         "(process_request(rgw::sal::RGWRadosStore*, RGWREST*, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rgw::auth::StrategyRegistry const&, RGWRestfulIO*, OpsLogSink*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*, int*)+0x2891) [0x7efe0b344e21]",
>         "/lib64/libradosgw.so.2(+0x4b1b63) [0x7efe0b294b63]",
>         "/lib64/libradosgw.so.2(+0x4b3604) [0x7efe0b296604]",
>         "/lib64/libradosgw.so.2(+0x4b386e) [0x7efe0b29686e]",
>         "make_fcontext()"
>     ],
>     "ceph_version": "16.2.7",
>     "crash_id": "2022-07-17T10:07:54.781936Z_aac370aa-48a9-414a-bf53-60f5414cbae6",
>     "entity_name": "client.rgw.cstorage4009.rgw0",
>     "os_id": "centos",
>     "os_name": "CentOS Linux",
>     "os_version": "8",
>     "os_version_id": "8",
>     "process_name": "radosgw",
>     "stack_sig": "b61cb061c4a28ad8d10e37dd95d61781f935632545c6e6efa4ec946bf0d81315",
>     "timestamp": "2022-07-17T10:07:54.781936Z",
>     "utsname_hostname": "cstorage4009",
>     "utsname_machine": "x86_64",
>     "utsname_release": "5.4.0-120-generic",
>     "utsname_sysname": "Linux",
>     "utsname_version": "#136-Ubuntu SMP Fri Jun 10 13:40:48 UTC 2022"
> }

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux