adding --name to regionmap update command has allowed me to update the regionmap: > radosgw-admin regionmap update --name client.radosgw.us-master-1 so now i have reloaded zone and region and updated region map on the gateway in each zone, then restarted whole clusters, then restarted apahce and radosgw, same problem. I cannot see how this can be anything other than an issue inside radosgw-agent as it is not hitting the gateway due to the botched > DEBUG:boto:url = 'https://example.comhttps://example.com/admin/config' Im out of ideas. Should i submit this as a bug? On 22/07/14 15:25, Bachelder, Kurt wrote: > > It certainly doesn?t hurt to reload your zone and region > configurations on your RGWs and re-run the regionmap update for the > instances tied to each zone, just to ensure consistency. > > *From:*Peter [mailto:ptiernan at tchpc.tcd.ie] > *Sent:* Tuesday, July 22, 2014 10:20 AM > *To:* Bachelder, Kurt; Craig Lewis > *Cc:* Ceph Users > *Subject:* Re: [ceph-users] radosgw-agent failed to parse > > thanks for the suggestion. ive attempted a regionmap update but im > hitting this error: > > > failed to list regions: (2) No such file or directory > 2014-07-22 14:13:04.096601 7ff825ac77c0 -1 failed to list objects > pool_iterate_begin() returned r=-2 > > > so perhaps i do have some issue with my configuration. Although i > would have thought that if the gateway is outputting the correct > regionmap at /admin/config path, then all should be well with regionmap. > > > > On 22/07/14 14:13, Bachelder, Kurt wrote: > > I?m sure you?ve already tried this, but we?ve gotten burned a few > times by not running radosgw-admin regionmap update after making > region/zone changes. Bouncing the RGW?s probably wouldn?t hurt > either. > > *From:*ceph-users [mailto:ceph-users-bounces at lists.ceph.com] *On > Behalf Of *Peter > *Sent:* Tuesday, July 22, 2014 4:51 AM > *To:* Craig Lewis > *Cc:* Ceph Users > *Subject:* Re: [ceph-users] radosgw-agent failed to parse > > yes, im scratching my head over this too. It doesn't seem to be an > authentication issue as the radosgw-agent never reaches the > us-secondary gateway (i've kept an eye on us-secondary logs as i > execute radosgw-agent on us-master). > > > On 22/07/14 03:51, Craig Lewis wrote: > > I was hoping for some easy fixes :-P > > I created two system users, in both zones. Each user has > different access and secret, but I copied the access and > secret from the primary to the secondary. I can't imaging > that this would cause the problem you're seeing, but it is > something different from the examples. > > Sorry, I'm out of ideas. > > On Mon, Jul 21, 2014 at 7:13 AM, Peter <ptiernan at tchpc.tcd.ie > <mailto:ptiernan at tchpc.tcd.ie>> wrote: > > hello again, > > i couldn't find 'http://us-secondary.example.comhttp:// > <http://us-secondary.example.comhttp:/>us-secondary.example.com/ > <http://us-secondary.example.com/>' in any zone or regions > config files. How could it be getting the URL from > someplace else if i am specifying as command line option > after radosgw-agent ? > > > Here is region config: > > > > { "name": "us", > "api_name": "us", > "is_master": "True", > "endpoints": [ > "http:\/\/us-master.example.com:80\/" > <http://%5C/%5C/us-master.example.com:80%5C/>], > "master_zone": "us-master", > "zones": [ > { "name": "us-master", > "endpoints": [ > "http:\/\/us-master.example.com:80\/" > <http://%5C/%5C/us-master.example.com:80%5C/>], > "log_meta": "true", > "log_data": "true"}, > { "name": "us-secondary", > "endpoints": [ > "http:\/\/us-master.example.com:80\/" > <http://%5C/%5C/us-master.example.com:80%5C/>], > "log_meta": "true", > "log_data": "true"} > ], > "placement_targets": [ > { > "name": "default-placement", > "tags": [] > } > ], > "default_placement": "default-placement"} > > > I also get the above when i navigate to > http://us-master.example.com/admin/config > <http://us-master.example.com/admin/config> and > http://us-secondary.example.com/admin/config > <http://us-secondary.example.com/admin/config> . > > us-master zone looks like this: > > > > { "domain_root": ".us-master.domain.rgw", > "control_pool": ".us-master.rgw.control", > "gc_pool": ".us-master.rgw.gc", > "log_pool": ".us-master.log", > "intent_log_pool": ".us-master.intent-log", > "usage_log_pool": ".us-master.usage", > "user_keys_pool": ".us-master.users", > "user_email_pool": ".us-master.users.email", > "user_swift_pool": ".us-master.users.swift", > "user_uid_pool": ".us-master.users.uid", > "system_key": { "access_key": > "EA02UO07DA8JJJX7ZIPJ", "secret_key": > "InmPlbQhsj7dqYYYYjdNabqkZaqR8ShWC6fS0XVo"}, > "placement_pools": [ > { "key": "default-placement", > "val": { "index_pool": > ".us-master.rgw.buckets.index", > "data_pool": ".us-master.rgw.buckets"} > } > ] > } > > > us-secondary zone: > > > > { "domain_root": ".us-secondary.domain.rgw", > "control_pool": ".us-secondary.rgw.control", > "gc_pool": ".us-secondary.rgw.gc", > "log_pool": ".us-secondary.log", > "intent_log_pool": ".us-secondary.intent-log", > "usage_log_pool": ".us-secondary.usage", > "user_keys_pool": ".us-secondary.users", > "user_email_pool": ".us-secondary.users.email", > "user_swift_pool": ".us-secondary.users.swift", > "user_uid_pool": ".us-secondary.users.uid", > "system_key": { "access_key": > "EA02UO07DA8JJJX7ZIPJ", "secret_key": > "InmPlbQhsj7dqYYYYjdNabqkZaqR8ShWC6fS0XVo"}, > "placement_pools": [ > { "key": "default-placement", > "val": { "index_pool": > ".us-secondary.rgw.buckets.index", > "data_pool": ".us-secondary.rgw.buckets"} > } > ] > } > > > us-master user exists on us-master cluster gateway, > us-secondary user exists on us-secondary cluster gateway. > both us-master and us-secondary gateway users have same > access and secret key. should us-master and us-secondary > users exist on both clusters? > > i can resolve us-master.example.com > <http://us-master.example.com> and > us-secondary.example.com <http://us-secondary.example.com> > from both gateways. > > > Thanks > > On 09/07/14 22:20, Craig Lewis wrote: > > Just to ask a couple obvious questions... > > You didn't accidentally > put 'http://us-secondary.example.comhttp:// > <http://us-secondary.example.comhttp:/>us-secondary.example.com/ > <http://us-secondary.example.com/>' in any of your > region or zone configuration files? The fact that > it's missing the :80 makes me think it's getting that > URL from someplace that isn't the command line. > > You do have both system users on both clusters, with > the same access and secret keys? > > You can resolve us-secondary.example.com > <http://us-secondary.example.com>. from this host? > > I tested URLs of the form > http://us-secondary.example.com/ > <http://us-secondary.example.com/> and > http://us-secondary.example.com:80 > <http://us-secondary.example.com:80> in my setup, and > both work fine. > > On Wed, Jul 9, 2014 at 3:56 AM, Peter > <ptiernan at tchpc.tcd.ie <mailto:ptiernan at tchpc.tcd.ie>> > wrote: > > thank you for your reply. I am running ceph > 0.80.1, radosgw-agent 1.2 on Ubuntu 14.04 LTS > (GNU/Linux 3.13.0-24-generic x86_64) . I also ran > into this same issue with ubuntu 12.04 previously. > There are no special characters in the access or > secret key (ive had issues with this before so i > make sure of this). > > here is the output python interpreter: > > Python 2.7.6 (default, Mar 22 2014, 22:59:56) > [GCC 4.8.2] on linux2 > Type "help", "copyright", "credits" or > "license" for more information. > > > >>> import urlparse > >>> result = > urlparse.urlparse('http://us-secondary.example.com:80') > >>> print result.hostname, result.port > > us-secondary.example.com > <http://us-secondary.example.com> 80 > > > that looks ok to me. > > > > > On 07/07/14 22:57, Josh Durgin wrote: > > On 07/04/2014 08:36 AM, Peter wrote: > > i am having issues running radosgw-agent > to sync data between two > radosgw zones. As far as i can tell both > zones are running correctly. > > My issue is when i run the radosgw-agent > command: > > > radosgw-agent -v --src-access-key > <access_key> --src-secret-key > <secret_key> --dest-access-key > <access_key> --dest-secret-key > <secret_key> --src-zone us-master > http://us-secondary.example.com:80 > > > i get the following error: > > |DEBUG:boto:Using access key provided by > client.|| > ||DEBUG:boto:Using secret key provided by > client.|| > ||DEBUG:boto:StringToSign:|| > ||GET|| > || > ||Fri, 04 Jul 2014 15:25:53 GMT|| > ||/admin/config|| > ||DEBUG:boto:Signature:|| > ||AWS > EA20YO07DA8JJJX7ZIPJ:WbykwyXu5m5IlbEsBzo8bKEGIzg=|| > ||DEBUG:boto:url = > 'http://us-secondary.example.comhttp://us-secondary.example.com/admin/config'|| > <http://us-secondary.example.com/admin/config%27%7C%7C> > > ||params={}|| > ||headers={'Date': 'Fri, 04 Jul 2014 > 15:25:53 GMT', 'Content-Length': > '0', 'Authorization': 'AWS > EA20YO07DA8JJJX7ZIPJ:WbykwyXu5m5IlbEsBzo8bKEGIzg=', > 'User-Agent': > 'Boto/2.20.1 Python/2.7.6 > Linux/3.13.0-24-generic'}|| > ||data=None|| > ||ERROR:root:Could not retrieve region map > from destination|| > ||Traceback (most recent call last):|| > || File > "/usr/lib/python2.7/dist-packages/radosgw_agent/cli.py", > line > 269, in main|| > || region_map = > client.get_region_map(dest_conn)|| > || File > "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", > line 391, in get_region_map|| > || region_map = request(connection, > 'get', 'admin/config')|| > || File > "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", > line 153, in request|| > || result = handler(url, params=params, > headers=request.headers, > data=data)|| > || File > "/usr/lib/python2.7/dist-packages/requests/api.py", > line 55, in > get|| > || return request('get', url, **kwargs)|| > || File > "/usr/lib/python2.7/dist-packages/requests/api.py", > line 44, in > request|| > || return > session.request(method=method, url=url, > **kwargs)|| > || File > "/usr/lib/python2.7/dist-packages/requests/sessions.py", > line > 349, in request|| > || prep = self.prepare_request(req)|| > || File > "/usr/lib/python2.7/dist-packages/requests/sessions.py", > line > 287, in prepare_request|| > || hooks=merge_hooks(request.hooks, > self.hooks),|| > || File > "/usr/lib/python2.7/dist-packages/requests/models.py", > line > 287, in prepare|| > || self.prepare_url(url, params)|| > || File > "/usr/lib/python2.7/dist-packages/requests/models.py", > line > 334, in prepare_url|| > || scheme, auth, host, port, path, > query, fragment = parse_url(url)|| > || File > "/usr/lib/python2.7/dist-packages/urllib3/util.py", > line 390, > in parse_url|| > || raise LocationParseError("Failed to > parse: %s" % url)|| > ||LocationParseError: Failed to parse: > Failed to parse: > us-secondary.example.comhttp: > > > |||Is this a bug? or is my setup wrong? i > can navigate to > http://us-secondary.example.com/admin/config > and it correctly outputs > zone details. at the output above > > > It seems like an issue with your environment. > What version of > radosgw-agent and which distro is this running on? > > Are there any special characters in the access > or secret keys that > might need to be escaped on the command line? > > |DEBUG:boto:url = > 'http://us-secondary.example.comhttp://us-secondary.example.com/admin/config'|| > <http://us-secondary.example.com/admin/config%27%7C%7C> > > > > |should the url be repeated like that? > > > No, and it's rather strange since it should be > the url passed on the > command line, parsed, and with /admin/config > added. > > Could post the result of this run in a python > interpreter: > > import urlparse > result = > urlparse.urlparse('http://us-secondary.example.com:80') > print result.hostname, result.port > > Josh > > > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > <mailto:ceph-users at lists.ceph.com> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140722/65332c30/attachment.htm>
 |