hello again, i have reviewed my deployment, and --name argument is used everywhere with radosgw-admin command. as i create the pools during deploy, there is no rgw.root pool, so i cannot make changes to it. I still think this is an issue with radosgw-agent, as if i change the destination on the command line, it also changes in the botched up URL it tries to hit: "radosgw-agent http://example2.net"; this appears in the error: DEBUG:boto:url = 'http://example2.nethttp://example2.net/admin/config <https://example.comhttps://example.com/admin/config>' so it is definitely coming from input from command line. On 22/07/14 20:44, Craig Lewis wrote: > You should use the --name argument with every radosgw-admin command. > If you don't, you'll end up making changes to .rgw.root, not > .us.rgw.root. > > I'd run through the federation setup again, making sure to include the > appropriate --name. As Kurt said, it's safe to reload and reapply the > configs. Make sure you restart radosgw when it says. > > One of my problems during setup was that I had a bad config loaded in > .rgw.root, but the correct one in .us.rgw.root. It caused all sorts > of problems when I forgot the --name arg. > > Setting up federation is somewhat sensitive to order of operations. > When I was testing it, I frequently messed something up. Several > times it was faster to delete all the pools and start over, rather > than figuring out what I broke. > > > > On Tue, Jul 22, 2014 at 7:46 AM, Peter <ptiernan at tchpc.tcd.ie > <mailto:ptiernan at tchpc.tcd.ie>> wrote: > > adding --name to regionmap update command has allowed me to update > the regionmap: > >> radosgw-admin regionmap update --name client.radosgw.us-master-1 > > so now i have reloaded zone and region and updated region map on > the gateway in each zone, then restarted whole clusters, then > restarted apahce and radosgw, same problem. > > I cannot see how this can be anything other than an issue inside > radosgw-agent as it is not hitting the gateway due to the botched > >> DEBUG:boto:url = >> 'https://example.comhttps://example.com/admin/config' > > Im out of ideas. Should i submit this as a bug? > > > > On 22/07/14 15:25, Bachelder, Kurt wrote: >> >> It certainly doesn?t hurt to reload your zone and region >> configurations on your RGWs and re-run the regionmap update for >> the instances tied to each zone, just to ensure consistency. >> >> *From:*Peter [mailto:ptiernan at tchpc.tcd.ie] >> *Sent:* Tuesday, July 22, 2014 10:20 AM >> *To:* Bachelder, Kurt; Craig Lewis >> *Cc:* Ceph Users >> *Subject:* Re: [ceph-users] radosgw-agent failed to parse >> >> thanks for the suggestion. ive attempted a regionmap update but >> im hitting this error: >> >> >> failed to list regions: (2) No such file or directory >> 2014-07-22 14:13:04.096601 7ff825ac77c0 -1 failed to list >> objects pool_iterate_begin() returned r=-2 >> >> >> so perhaps i do have some issue with my configuration. Although i >> would have thought that if the gateway is outputting the correct >> regionmap at /admin/config path, then all should be well with >> regionmap. >> >> >> >> On 22/07/14 14:13, Bachelder, Kurt wrote: >> >> I?m sure you?ve already tried this, but we?ve gotten burned a >> few times by not running radosgw-admin regionmap update after >> making region/zone changes. Bouncing the RGW?s probably >> wouldn?t hurt either. >> >> *From:*ceph-users [mailto:ceph-users-bounces at lists.ceph.com] >> *On Behalf Of *Peter >> *Sent:* Tuesday, July 22, 2014 4:51 AM >> *To:* Craig Lewis >> *Cc:* Ceph Users >> *Subject:* Re: [ceph-users] radosgw-agent failed to parse >> >> yes, im scratching my head over this too. It doesn't seem to >> be an authentication issue as the radosgw-agent never reaches >> the us-secondary gateway (i've kept an eye on us-secondary >> logs as i execute radosgw-agent on us-master). >> >> >> On 22/07/14 03:51, Craig Lewis wrote: >> >> I was hoping for some easy fixes :-P >> >> I created two system users, in both zones. Each user has >> different access and secret, but I copied the access and >> secret from the primary to the secondary. I can't >> imaging that this would cause the problem you're seeing, >> but it is something different from the examples. >> >> Sorry, I'm out of ideas. >> >> On Mon, Jul 21, 2014 at 7:13 AM, Peter >> <ptiernan at tchpc.tcd.ie <mailto:ptiernan at tchpc.tcd.ie>> wrote: >> >> hello again, >> >> i couldn't find >> 'http://us-secondary.example.comhttp:// >> <http://us-secondary.example.comhttp:/>us-secondary.example.com/ >> <http://us-secondary.example.com/>' in any zone or >> regions config files. How could it be getting the URL >> from someplace else if i am specifying as command >> line option after radosgw-agent ? >> >> >> Here is region config: >> >> >> >> { "name": "us", >> "api_name": "us", >> "is_master": "True", >> "endpoints": [ >> "http:\/\/us-master.example.com:80\/" >> <http://%5C/%5C/us-master.example.com:80%5C/>], >> "master_zone": "us-master", >> "zones": [ >> { "name": "us-master", >> "endpoints": [ >> "http:\/\/us-master.example.com:80\/" >> <http://%5C/%5C/us-master.example.com:80%5C/>], >> "log_meta": "true", >> "log_data": "true"}, >> { "name": "us-secondary", >> "endpoints": [ >> "http:\/\/us-master.example.com:80\/" >> <http://%5C/%5C/us-master.example.com:80%5C/>], >> "log_meta": "true", >> "log_data": "true"} >> ], >> "placement_targets": [ >> { >> "name": "default-placement", >> "tags": [] >> } >> ], >> "default_placement": "default-placement"} >> >> >> I also get the above when i navigate to >> http://us-master.example.com/admin/config >> <http://us-master.example.com/admin/config> and >> http://us-secondary.example.com/admin/config >> <http://us-secondary.example.com/admin/config> . >> >> us-master zone looks like this: >> >> >> >> { "domain_root": ".us-master.domain.rgw", >> "control_pool": ".us-master.rgw.control", >> "gc_pool": ".us-master.rgw.gc", >> "log_pool": ".us-master.log", >> "intent_log_pool": ".us-master.intent-log", >> "usage_log_pool": ".us-master.usage", >> "user_keys_pool": ".us-master.users", >> "user_email_pool": ".us-master.users.email", >> "user_swift_pool": ".us-master.users.swift", >> "user_uid_pool": ".us-master.users.uid", >> "system_key": { "access_key": >> "EA02UO07DA8JJJX7ZIPJ", "secret_key": >> "InmPlbQhsj7dqYYYYjdNabqkZaqR8ShWC6fS0XVo"}, >> "placement_pools": [ >> { "key": "default-placement", >> "val": { "index_pool": >> ".us-master.rgw.buckets.index", >> "data_pool": ".us-master.rgw.buckets"} >> } >> ] >> } >> >> >> us-secondary zone: >> >> >> >> { "domain_root": ".us-secondary.domain.rgw", >> "control_pool": ".us-secondary.rgw.control", >> "gc_pool": ".us-secondary.rgw.gc", >> "log_pool": ".us-secondary.log", >> "intent_log_pool": ".us-secondary.intent-log", >> "usage_log_pool": ".us-secondary.usage", >> "user_keys_pool": ".us-secondary.users", >> "user_email_pool": ".us-secondary.users.email", >> "user_swift_pool": ".us-secondary.users.swift", >> "user_uid_pool": ".us-secondary.users.uid", >> "system_key": { "access_key": >> "EA02UO07DA8JJJX7ZIPJ", "secret_key": >> "InmPlbQhsj7dqYYYYjdNabqkZaqR8ShWC6fS0XVo"}, >> "placement_pools": [ >> { "key": "default-placement", >> "val": { "index_pool": >> ".us-secondary.rgw.buckets.index", >> "data_pool": >> ".us-secondary.rgw.buckets"} >> } >> ] >> } >> >> >> us-master user exists on us-master cluster gateway, >> us-secondary user exists on us-secondary cluster >> gateway. both us-master and us-secondary gateway >> users have same access and secret key. should >> us-master and us-secondary users exist on both clusters? >> >> i can resolve us-master.example.com >> <http://us-master.example.com> and >> us-secondary.example.com >> <http://us-secondary.example.com> from both gateways. >> >> >> Thanks >> >> On 09/07/14 22:20, Craig Lewis wrote: >> >> Just to ask a couple obvious questions... >> >> You didn't accidentally >> put 'http://us-secondary.example.comhttp:// >> <http://us-secondary.example.comhttp:/>us-secondary.example.com/ >> <http://us-secondary.example.com/>' in any of >> your region or zone configuration files? The >> fact that it's missing the :80 makes me think >> it's getting that URL from someplace that isn't >> the command line. >> >> You do have both system users on both clusters, >> with the same access and secret keys? >> >> You can resolve us-secondary.example.com >> <http://us-secondary.example.com>. from this host? >> >> I tested URLs of the form >> http://us-secondary.example.com/ >> <http://us-secondary.example.com/> and >> http://us-secondary.example.com:80 >> <http://us-secondary.example.com:80> in my setup, >> and both work fine. >> >> On Wed, Jul 9, 2014 at 3:56 AM, Peter >> <ptiernan at tchpc.tcd.ie >> <mailto:ptiernan at tchpc.tcd.ie>> wrote: >> >> thank you for your reply. I am running ceph >> 0.80.1, radosgw-agent 1.2 on Ubuntu 14.04 LTS >> (GNU/Linux 3.13.0-24-generic x86_64) . I also >> ran into this same issue with ubuntu 12.04 >> previously. >> There are no special characters in the access >> or secret key (ive had issues with this >> before so i make sure of this). >> >> here is the output python interpreter: >> >> Python 2.7.6 (default, Mar 22 2014, 22:59:56) >> [GCC 4.8.2] on linux2 >> Type "help", "copyright", "credits" or >> "license" for more information. >> >> >> >>> import urlparse >> >>> result = >> urlparse.urlparse('http://us-secondary.example.com:80') >> >>> print result.hostname, result.port >> >> us-secondary.example.com >> <http://us-secondary.example.com> 80 >> >> >> that looks ok to me. >> >> >> >> >> On 07/07/14 22:57, Josh Durgin wrote: >> >> On 07/04/2014 08:36 AM, Peter wrote: >> >> i am having issues running >> radosgw-agent to sync data between two >> radosgw zones. As far as i can tell >> both zones are running correctly. >> >> My issue is when i run the >> radosgw-agent command: >> >> >> radosgw-agent -v --src-access-key >> <access_key> --src-secret-key >> <secret_key> --dest-access-key >> <access_key> --dest-secret-key >> <secret_key> --src-zone us-master >> http://us-secondary.example.com:80 >> >> >> i get the following error: >> >> |DEBUG:boto:Using access key provided >> by client.|| >> ||DEBUG:boto:Using secret key >> provided by client.|| >> ||DEBUG:boto:StringToSign:|| >> ||GET|| >> || >> ||Fri, 04 Jul 2014 15:25:53 GMT|| >> ||/admin/config|| >> ||DEBUG:boto:Signature:|| >> ||AWS >> EA20YO07DA8JJJX7ZIPJ:WbykwyXu5m5IlbEsBzo8bKEGIzg=|| >> ||DEBUG:boto:url = >> 'http://us-secondary.example.comhttp://us-secondary.example.com/admin/config'|| >> <http://us-secondary.example.com/admin/config%27%7C%7C> >> >> ||params={}|| >> ||headers={'Date': 'Fri, 04 Jul 2014 >> 15:25:53 GMT', 'Content-Length': >> '0', 'Authorization': 'AWS >> EA20YO07DA8JJJX7ZIPJ:WbykwyXu5m5IlbEsBzo8bKEGIzg=', >> 'User-Agent': >> 'Boto/2.20.1 Python/2.7.6 >> Linux/3.13.0-24-generic'}|| >> ||data=None|| >> ||ERROR:root:Could not retrieve >> region map from destination|| >> ||Traceback (most recent call last):|| >> || File >> "/usr/lib/python2.7/dist-packages/radosgw_agent/cli.py", >> line >> 269, in main|| >> || region_map = >> client.get_region_map(dest_conn)|| >> || File >> "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", >> line 391, in get_region_map|| >> || region_map = >> request(connection, 'get', >> 'admin/config')|| >> || File >> "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", >> line 153, in request|| >> || result = handler(url, >> params=params, headers=request.headers, >> data=data)|| >> || File >> "/usr/lib/python2.7/dist-packages/requests/api.py", >> line 55, in >> get|| >> || return request('get', url, >> **kwargs)|| >> || File >> "/usr/lib/python2.7/dist-packages/requests/api.py", >> line 44, in >> request|| >> || return >> session.request(method=method, >> url=url, **kwargs)|| >> || File >> "/usr/lib/python2.7/dist-packages/requests/sessions.py", >> line >> 349, in request|| >> || prep = self.prepare_request(req)|| >> || File >> "/usr/lib/python2.7/dist-packages/requests/sessions.py", >> line >> 287, in prepare_request|| >> || hooks=merge_hooks(request.hooks, >> self.hooks),|| >> || File >> "/usr/lib/python2.7/dist-packages/requests/models.py", >> line >> 287, in prepare|| >> || self.prepare_url(url, params)|| >> || File >> "/usr/lib/python2.7/dist-packages/requests/models.py", >> line >> 334, in prepare_url|| >> || scheme, auth, host, port, path, >> query, fragment = parse_url(url)|| >> || File >> "/usr/lib/python2.7/dist-packages/urllib3/util.py", >> line 390, >> in parse_url|| >> || raise >> LocationParseError("Failed to parse: >> %s" % url)|| >> ||LocationParseError: Failed to >> parse: Failed to parse: >> us-secondary.example.comhttp: >> >> >> |||Is this a bug? or is my setup >> wrong? i can navigate to >> http://us-secondary.example.com/admin/config >> and it correctly outputs >> zone details. at the output above >> >> >> It seems like an issue with your >> environment. What version of >> radosgw-agent and which distro is this >> running on? >> >> Are there any special characters in the >> access or secret keys that >> might need to be escaped on the command line? >> >> |DEBUG:boto:url = >> 'http://us-secondary.example.comhttp://us-secondary.example.com/admin/config'|| >> <http://us-secondary.example.com/admin/config%27%7C%7C> >> >> >> >> |should the url be repeated like that? >> >> >> No, and it's rather strange since it >> should be the url passed on the >> command line, parsed, and with >> /admin/config added. >> >> Could post the result of this run in a >> python interpreter: >> >> import urlparse >> result = >> urlparse.urlparse('http://us-secondary.example.com:80') >> print result.hostname, result.port >> >> Josh >> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users at lists.ceph.com >> <mailto:ceph-users at lists.ceph.com> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140723/5e68190a/attachment.htm>
 |