RGW Metadata Search - Elasticserver

Amardeep Singh <amardeep@xxxxxxxxxxxxxx> · Wed, 14 Feb 2018 16:24:49 +0530



    Hi,

      
      I am trying to setup RGW Metadata Search with Elastic server tier
      type as per blog post here.
      https://ceph.com/rgw/new-luminous-rgw-metadata-search/

      
      The environment setup is done using ceph-ansible  docker
      containers. 

      
      Containers running on Node 1 - rgw, mds, mgr, mon , 5 osds

      
      Containers running on Node 2 - rgw, mon

      
      The steps performed are for setting up according to blog post are:

      
      docker exec ceph-rgw radosgw-admin --cluster test realm create
      --rgw-realm=gold --default

      docker exec ceph-rgw radosgw-admin --cluster test zonegroup delete
      --rgw-zonegroup=default

      docker exec ceph-rgw radosgw-admin --cluster test zonegroup create
      --rgw-zonegroup=uk --endpoints=http://rgw1:8080 --master --default

      docker exec ceph-rgw radosgw-admin --cluster test zone create
      --rgw-zonegroup=uk --rgw-zone=uk-west --endpoints=http://rgw1:8080
      --access-key=xxxxxxx --secret=xxxxxxxx --default --master

      docker exec ceph-rgw radosgw-admin --cluster test user create
      --uid=zone.user --display-name="Zone User" --access-key=xxxxxx
      --secret=xxxxxx --system

      docker exec ceph-rgw radosgw-admin --cluster test period update
      --commit

      
      Restart Docker Container and then perform the following :

      
      docker exec ceph-rgw radosgw-admin --cluster test zone create
      --rgw-zonegroup=uk --rgw-zone=uk-west-es
      --endpoints=http://rgw2:8080 --access-key=xxxxxxx --secret=xxxxxx
      

      docker exec ceph-rgw radosgw-admin --cluster test zone modify
      --rgw-zone=uk-west-es --tier-type=elasticsearch
--tier-config=endpoint=http://elasticserver:9200,num_shards=10,num_replicas=1

      docker exec ceph-rgw radosgw-admin --cluster test period update
      --commit

      
      After completing the above I can add documents but not able to
      search metadata using obo tool.

      
      Checking sync status I get 

      
      realm 281ba7e8-3bd1-47de-981a-c94914bdf54f (gold)

      zonegroup 0d9efa23-09d6-4adf-a486-0858f3261d7b (uk)

      zone 3c1aae3e-4252-47cc-8e66-ae1cb6275158 (uk-west)

      metadata sync no sync (zone is master)

      data sync source: 13ee3cfa-10ba-45ef-aec0-e42d8f55e3b6
        (uk-west-es)

      not syncing from zone

      
      Enabled Debug on RGW 

      
      Tue Feb 13 11:51:49 2018

      /admin/realm/period

      2018-02-13 17:21:49.500561 7fd4c5989700 15 generated auth
        header: AWS HCTJCLF1F1E857N09X1Y:Yn03Gr+aJJf+CqQxgaWeLqskTH8=

      2018-02-13 17:21:49.500576 7fd4c5989700 20 sending request
        to
http://rgw2:8080/admin/realm/period?period=74d8f220-1165-4092-8042-14734c27364c&epoch=2&rgwx-zonegroup=94f8340b-9fdc-4277-8e55-3dd6fe878f48

      2018-02-13 17:21:49.500587 7fd4c5989700 20 register_request
        mgr=0x56386f2d5770 req_data->id=47,
        easy_handle=0x563870068000

      2018-02-13 17:21:49.500620 7fd4c5989700 20 run:
        stack=0x56386f37b750 is io blocked

      2018-02-13 17:21:49.500874 7fd4c618a700 20 link_request
        req_data=0x56386f5d7cc0 req_data->id=47,
        easy_handle=0x563870068000

      2018-02-13 17:21:50.342999 7fd518c36700 2
        RGWDataChangesLog::ChangesRenewThread: start

      2018-02-13 17:21:50.503796 7fd4c618a700 10
        receive_http_header

      2018-02-13 17:21:50.503809 7fd4c618a700 10 received
        header:HTTP/1.1 400 Bad Request

      2018-02-13 17:21:50.503814 7fd4c618a700 10
        receive_http_header

      2018-02-13 17:21:50.503815 7fd4c618a700 10 received
        header:Content-Length: 115

      2018-02-13 17:21:50.503818 7fd4c618a700 10
        receive_http_header

      2018-02-13 17:21:50.503819 7fd4c618a700 10 received
        header:x-amz-request-id:
        tx00000000000000000004b-005a82d155-3729-uk-west

      2018-02-13 17:21:50.503821 7fd4c618a700 10
        receive_http_header

      2018-02-13 17:21:50.503821 7fd4c618a700 10 received
        header:Accept-Ranges: bytes

      2018-02-13 17:21:50.503823 7fd4c618a700 10
        receive_http_header

      2018-02-13 17:21:50.503823 7fd4c618a700 10 received
        header:Content-Type: application/json

      2018-02-13 17:21:50.503825 7fd4c618a700 10
        receive_http_header

      2018-02-13 17:21:50.503825 7fd4c618a700 10 received
        header:Date: Tue, 13 Feb 2018 11:51:50 GMT

      2018-02-13 17:21:50.503826 7fd4c618a700 10
        receive_http_header

      2018-02-13 17:21:50.503827 7fd4c618a700 10 received header:

      2018-02-13 17:21:50.504056 7fd4c5989700 20
cr:s=0x56386f37b750:op=0x563870044900:21RGWPostRESTResourceCRI9RGWPeriodiE:
        operate()

      2018-02-13 17:21:50.504075 7fd4c5989700 5 failed to wait
        for op, ret=-22: POST
http://rgw2:8080/admin/realm/period?period=74d8f220-1165-4092-8042-14734c27364c&epoch=2&rgwx-zonegroup=94f8340b-9fdc-4277-8e55-3dd6fe878f48

      2018-02-13 17:21:50.504173 7fd4c5989700 20
cr:s=0x56386f37b750:op=0x563870044900:21RGWPostRESTResourceCRI9RGWPeriodiE:
        operate() returned r=-22

      2018-02-13 17:21:50.504190 7fd4c5989700 20
        cr:s=0x56386f37b750:op=0x56386f588e00:14PushAndRetryCR:
        operate()

      2018-02-13 17:21:50.504192 7fd4c5989700 10 rgw period
        pusher: waiting 30.000000s for retry..

      2018-02-13 17:21:50.504205 7fd4c5989700 20 run:
        stack=0x56386f37b750 is io blocked

      2018-02-13 17:22:12.343154 7fd518c36700 2
        RGWDataChangesLog::ChangesRenewThread: start

      
      I have tried it few times now and can't understand the reason. Is
      it due to docker setup? As per blog post this should work straight
      in luminous.

       
      Thanks,

      Amar

    
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com