This tells me that you are not using the latest released version of the agent because we dropped the requirement for the `requests` library that is showing up in your tracebacks. You will need to make sure that the agent is completely removed, including the `requests` library and then install the latest agent available from the Python package Index (PyPI) with a Python installer (e.g. `pip install radosgw-agent`) or if you feel comfortable use the latest tip from the master branch (I advice to use the former). On Tue, Feb 3, 2015 at 5:12 AM, <ghislain.chevalier@xxxxxxxxxx> wrote: > Hi Alfredo, > > Here are the logs you requested using the original client.py python script. > > ---------------------------------------------------------------------- > DIRECT LAUNCH USING CLI:KO > radosgw-agent -v -c /etc/ceph/radosgw-agent/fr-rennes-radosgw1-sync.conf_direct > the standard output is also written in the log file configured in the synchro file > > DEBUG:boto:Using access key provided by client. > DEBUG:boto:Using secret key provided by client. > DEBUG:boto:StringToSign: > GET > > > Tue, 03 Feb 2015 09:37:54 GMT > /admin/config > DEBUG:boto:Signature: > AWS SBF7DQZ1ESR6Y34225H8:9i6/4do7tDpViolN1XdxgefMs3U= > DEBUG:boto:url = 'http://openstorage.ushttp://openstorage.us/admin/config' > params={} > headers={'Date': 'Tue, 03 Feb 2015 09:37:54 GMT', 'Content-Length': '0', 'Authorization': 'AWS SBF7DQZ1ESR6Y34225H8:9i6/4do7tDpViolN1XdxgefMs3U=', 'User-Agent': 'Boto/2.20.1 Python/2.7.6 Linux/3.11.0-26-generic'} > data=None > ERROR:root:Could not retrieve region map from destination > Traceback (most recent call last): > File "/usr/lib/python2.7/dist-packages/radosgw_agent/cli.py", line 269, in main > region_map = client.get_region_map(dest_conn) > File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 391, in get_region_map > region_map = request(connection, 'get', 'admin/config') > File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 153, in request > result = handler(url, params=params, headers=request.headers, data=data) > File "/usr/lib/python2.7/dist-packages/requests/api.py", line 55, in get > return request('get', url, **kwargs) > File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request > return session.request(method=method, url=url, **kwargs) > File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 421, in request > prep = self.prepare_request(req) > File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 359, in prepare_request > hooks=merge_hooks(request.hooks, self.hooks), > File "/usr/lib/python2.7/dist-packages/requests/models.py", line 287, in prepare > self.prepare_url(url, params) > File "/usr/lib/python2.7/dist-packages/requests/models.py", line 334, in prepare_url > scheme, auth, host, port, path, query, fragment = parse_url(url) > File "/usr/lib/python2.7/dist-packages/urllib3/util.py", line 390, in parse_url > raise LocationParseError("Failed to parse: %s" % url) > LocationParseError: Failed to parse: Failed to parse: openstorage.ushttp: > > -------------------------------------------------------------------------------- > LAUNCH USING /etc/init.d/radosgw-agent shell script:KO > /etc/init.d/radosgw-agent start /etc/ceph/radosgw-agent/fr-rennes-radosgw1-sync.conf_direct > Standard output: > Starting radosgw-agent fr-rennes-radosgw1-sync.conf_direct > > Log File : > 2015-02-03T10:51:00.794 4735:ERROR:root:Could not retrieve region map from destination > Traceback (most recent call last): > File "/usr/lib/python2.7/dist-packages/radosgw_agent/cli.py", line 269, in main > region_map = client.get_region_map(dest_conn) > File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 391, in get_region_map > region_map = request(connection, 'get', 'admin/config') > File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 153, in request > result = handler(url, params=params, headers=request.headers, data=data) > File "/usr/lib/python2.7/dist-packages/requests/api.py", line 55, in get > return request('get', url, **kwargs) > File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request > return session.request(method=method, url=url, **kwargs) > File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 421, in request > prep = self.prepare_request(req) > File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 359, in prepare_request > hooks=merge_hooks(request.hooks, self.hooks), > File "/usr/lib/python2.7/dist-packages/requests/models.py", line 287, in prepare > self.prepare_url(url, params) > File "/usr/lib/python2.7/dist-packages/requests/models.py", line 334, in prepare_url > scheme, auth, host, port, path, query, fragment = parse_url(url) > File "/usr/lib/python2.7/dist-packages/urllib3/util.py", line 390, in parse_url > raise LocationParseError("Failed to parse: %s" % url) > LocationParseError: Failed to parse: Failed to parse: openstorage.ushttp: > > ------------------------------------------------------------------------------------------- > LAUNCH AS A SERVICE: OK > service radosgw-agent start /etc/ceph/radosgw-agent/fr-rennes-radosgw1-sync.conf_direct > standard output: > Starting radosgw-agent fr-rennes-radosgw1-sync.conf_direct > > Log file: > 2015-02-03T10:42:02.878 3162:INFO:urllib3.connectionpool:Starting new HTTP connection (1): openstorage.us > > ---------------------------------------------------------------------------------------------------------------------------------------------------- > ---------------------------------------------------------------------------------------------------------------------------------------------------- > Here are the logs you requested using the altered client.py python script. > > DIRECT LAUNCH USING CLI:OK > radosgw-agent -v -c /etc/ceph/radosgw-agent/fr-rennes-radosgw1-sync.conf_direct > the standard output is also witten in the log file configured in the synchro file > > 2015-02-03T11:00:40.447 6158:DEBUG:boto:Using access key provided by client. > 2015-02-03T11:00:40.447 6158:DEBUG:boto:Using secret key provided by client. > 2015-02-03T11:00:40.448 6158:DEBUG:boto:StringToSign: > GET > > > Tue, 03 Feb 2015 10:00:40 GMT > /admin/config > 2015-02-03T11:00:40.448 6158:DEBUG:boto:Signature: > AWS SBF7DQZ1ESR6Y34225H8:LUEBUrR9z1Bgbrj6F/XGrGF+8BM= > 2015-02-03T11:00:40.448 6158:DEBUG:boto:url = 'http://openstorage.us/admin/config' > params={} > headers={'Date': 'Tue, 03 Feb 2015 10:00:40 GMT', 'Content-Length': '0', 'Authorization': 'AWS SBF7DQZ1ESR6Y34225H8:LUEBUrR9z1Bgbrj6F/XGrGF+8BM=', 'User-Agent': 'Boto/2.20.1 Python/2.7.6 Linux/3.11.0-26-generic'} > data=None > 2015-02-03T11:00:40.457 6158:INFO:urllib3.connectionpool:Starting new HTTP connection (1): proxy > 2015-02-03T11:00:40.467 6158:DEBUG:urllib3.connectionpool:Setting read timeout to None > 2015-02-03T11:00:40.848 6158:DEBUG:urllib3.connectionpool:"GET http://openstorage.us/admin/config HTTP/1.1" 200 None > > -------------------------------------------------------------------------------- > LAUNCH USING /etc/init.d/radosgw-agent shell script:KO > /etc/init.d/radosgw-agent start /etc/ceph/radosgw-agent/fr-rennes-radosgw1-sync.conf_direct > Standard output: > Starting radosgw-agent fr-rennes-radosgw1-sync.conf_direct > > Log File : > 2015-02-03T11:07:49.271 7513:INFO:urllib3.connectionpool:Starting new HTTP connection (1): proxy > 2015-02-03T11:07:49.654 7513:INFO:urllib3.connectionpool:Starting new HTTP connection (1): proxy > 2015-02-03T11:07:50.469 7513:ERROR:radosgw_agent.sync:finding number of shards failed > 2015-02-03T11:07:50.470 7513:WARNING:radosgw_agent.sync:error preparing for sync, will retry. Traceback: > Traceback (most recent call last): > File "/usr/lib/python2.7/dist-packages/radosgw_agent/sync.py", line 26, in prepare_sync > syncer.prepare() > File "/usr/lib/python2.7/dist-packages/radosgw_agent/sync.py", line 202, in prepare > self.init_num_shards() > File "/usr/lib/python2.7/dist-packages/radosgw_agent/sync.py", line 73, in init_num_shards > self.num_shards = client.num_log_shards(self.src_conn, self.type) > File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 285, in num_log_shards > out = request(connection, 'get', 'admin/log', dict(type=shard_type)) > File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 160, in request > check_result_status(result) > File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 116, in check_result_status > HttpError)(result.status_code, result.content) > NotFound: Http error code 404 content <HTML><HEAD> > > > ------------------------------------------------------------------------------------------- > LAUNCH AS A SERVICE:KO > service radosgw-agent start /etc/ceph/radosgw-agent/fr-rennes-radosgw1-sync.conf_direct > standard output: > Starting radosgw-agent fr-rennes-radosgw1-sync.conf_direct > > Log file: > 2015-02-03T11:04:25.435 6872:ERROR:root:Could not retrieve region map from destination > Traceback (most recent call last): > File "/usr/lib/python2.7/dist-packages/radosgw_agent/cli.py", line 269, in main > region_map = client.get_region_map(dest_conn) > File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 396, in get_region_map > region_map = request(connection, 'get', 'admin/config') > File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 158, in request > result = handler(url, params=params, headers=request.headers, data=data) > File "/usr/lib/python2.7/dist-packages/requests/api.py", line 55, in get > return request('get', url, **kwargs) > File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request > return session.request(method=method, url=url, **kwargs) > File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 421, in request > prep = self.prepare_request(req) > File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 359, in prepare_request > hooks=merge_hooks(request.hooks, self.hooks), > File "/usr/lib/python2.7/dist-packages/requests/models.py", line 287, in prepare > self.prepare_url(url, params) > File "/usr/lib/python2.7/dist-packages/requests/models.py", line 338, in prepare_url > "Perhaps you meant http://{0}?".format(url)) > MissingSchema: Invalid URL u'/admin/config': No schema supplied. Perhaps you meant http:///admin/config? > > > > Best regards > > -----Message d'origine----- > De : alfredo.deza@xxxxxxxxxxx [mailto:alfredo.deza@xxxxxxxxxxx] De la part de Alfredo Deza > Envoyé : lundi 2 février 2015 19:42 > À : CHEVALIER Ghislain IMT/OLPS > Cc : ceph-devel@xxxxxxxxxxxxxxx > Objet : Re: [Ceph-devel] radosgw-agent failed to parse > > If you use the `-v` flag you will get very verbose output. > > On Mon, Feb 2, 2015 at 12:12 PM, <ghislain.chevalier@xxxxxxxxxx> wrote: >> OK I will send them asap >> >> Logs are not very verbose.. Can I set a debug mode? >> >> -----Message d'origine----- >> De : alfredo.deza@xxxxxxxxxxx [mailto:alfredo.deza@xxxxxxxxxxx] De la >> part de Alfredo Deza Envoyé : lundi 2 février 2015 16:59 À : CHEVALIER >> Ghislain IMT/OLPS Cc : >> Objet : Re: [Ceph-devel] radosgw-agent failed to parse >> >> That sounds suspicious because we haven't packaged a 1.2.1 release. >> >> The latest on PyPI is 1.2, which is the same in the master branch. >> >> Would you mind sending logs that show *how* the url is malformed (both when it is OK and when it is not) ? >> >> On Mon, Feb 2, 2015 at 9:53 AM, <ghislain.chevalier@xxxxxxxxxx> wrote: >>> HI, >>> >>> Thx for replying >>> >>> According to dpkg -l, it's 1.2.1. >>> >>> I noticed that the URL is malformed when launching using directly >>> radosgw-agent -c <syncfile> but well formed when launching using >>> service radosgw-agent start <syncfile> >>> >>> best regards >>> >>> -----Message d'origine----- >>> De : alfredo.deza@xxxxxxxxxxx [mailto:alfredo.deza@xxxxxxxxxxx] De la >>> part de Alfredo Deza Envoyé : lundi 2 février 2015 15:18 À : >>> CHEVALIER Ghislain IMT/OLPS Cc : ceph-devel@xxxxxxxxxxxxxxx Objet : Re: >>> [Ceph-devel] radosgw-agent failed to parse >>> >>> What version of the agent are you using? And when it fails, how does the log output look? >>> >>> On Mon, Feb 2, 2015 at 8:56 AM, <ghislain.chevalier@xxxxxxxxxx> wrote: >>>> Hi all, >>>> >>>> Context : Ubuntu 14.04 TLS firefly 0.80.8 >>>> >>>> I sent this post in ceph-users (identical subject) because I recently encountered the same issue. >>>> Maybe I missed something between July and January... >>>> >>>> I found that the http request wasn't correctly formed by >>>> /usr/lib/python2.7/dist-packages/radosgw_agent/client.py >>>> >>>> I did the changes below >>>> # url = '{protocol}://{host}{path}'.format(protocol=request.protocol, >>>> # host=request.host, >>>> # path=request.path) >>>> url = '{path}'.format(protocol="", host="", path=request.path) >>>> >>>> The request is then correctly formed and sent. >>>> >>>> I still have problems setting the federation between 2 zones of 2 ceph clusters in 2 regions. >>>> >>>> I go on investigating. >>>> >>>> Best regards >>>> >>>> >>>> ____________________________________________________________________ >>>> _ _ ___________________________________________________ >>>> >>>> Ce message et ses pieces jointes peuvent contenir des informations >>>> confidentielles ou privilegiees et ne doivent donc pas etre >>>> diffuses, exploites ou copies sans autorisation. Si vous avez recu >>>> ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. >>>> >>>> This message and its attachments may contain confidential or >>>> privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. >>>> If you have received this email in error, please notify the sender and delete this message and its attachments. >>>> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. >>>> Thank you. >>>> >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" >>>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo >>>> info at http://vger.kernel.org/majordomo-info.html >>> >>> _____________________________________________________________________ >>> _ ___________________________________________________ >>> >>> Ce message et ses pieces jointes peuvent contenir des informations >>> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, >>> exploites ou copies sans autorisation. Si vous avez recu ce message >>> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. >>> >>> This message and its attachments may contain confidential or >>> privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. >>> If you have received this email in error, please notify the sender and delete this message and its attachments. >>> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. >>> Thank you. >>> >> >> ______________________________________________________________________ >> ___________________________________________________ >> >> Ce message et ses pieces jointes peuvent contenir des informations >> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, >> exploites ou copies sans autorisation. Si vous avez recu ce message >> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. >> >> This message and its attachments may contain confidential or >> privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. >> If you have received this email in error, please notify the sender and delete this message and its attachments. >> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. >> Thank you. >> > > _________________________________________________________________________________________________________________________ > > Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc > pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler > a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, > Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. > > This message and its attachments may contain confidential or privileged information that may be protected by law; > they should not be distributed, used or copied without authorisation. > If you have received this email in error, please notify the sender and delete this message and its attachments. > As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. > Thank you. > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
