Re: [Ceph-devel] radosgw-agent failed to parse

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



1.2.1 was just released bringing parity for both RPM/DEB and the PyPI versions.

You should now be able to get it either way. Let me know if you are still seeing the same issues.



----- Original Message -----
From: "ghislain chevalier" <ghislain.chevalier@xxxxxxxxxx>
To: "Alfredo Deza" <adeza@xxxxxxxxxx>
Cc: ceph-devel@xxxxxxxxxxxxxxx
Sent: Thursday, February 5, 2015 9:42:09 AM
Subject: RE: [Ceph-devel] radosgw-agent failed to parse

HI

Thanks a lot

I installed the last version using (pip install radosgw-agent) as recommended after removing the current version (apt-get purge radosgw-agent)
The python scripts are now installed in /usr/local/bin/...../radosgw_agent instead of /usr/bin/....../radosgw_agent

Can you tell me what is required about the 'requests' library?

So, I will test it asap

Did you plan to update the package in order to be compliant with apt-get install radosgw-agent?

Best regards 

-----Message d'origine-----
De : alfredo.deza@xxxxxxxxxxx [mailto:alfredo.deza@xxxxxxxxxxx] De la part de Alfredo Deza
Envoyé : mardi 3 février 2015 14:30
À : CHEVALIER Ghislain IMT/OLPS
Cc : ceph-devel@xxxxxxxxxxxxxxx
Objet : Re: [Ceph-devel] radosgw-agent failed to parse

This tells me that you are not using the latest released version of the agent because we dropped the requirement for the `requests` library that is showing up in your tracebacks.

You will need to make sure that the agent is completely removed, including the `requests` library and then install the latest agent available from the Python package Index (PyPI) with a Python installer (e.g.
`pip install radosgw-agent`) or if you feel comfortable use the latest tip from the master branch (I advice to use the former).


On Tue, Feb 3, 2015 at 5:12 AM,  <ghislain.chevalier@xxxxxxxxxx> wrote:
> Hi Alfredo,
>
> Here are the logs you requested using the original client.py python script.
>
> ----------------------------------------------------------------------
> DIRECT LAUNCH USING CLI:KO
> radosgw-agent -v -c 
> /etc/ceph/radosgw-agent/fr-rennes-radosgw1-sync.conf_direct
> the standard output is also written in the log file configured in the 
> synchro file
>
> DEBUG:boto:Using access key provided by client.
> DEBUG:boto:Using secret key provided by client.
> DEBUG:boto:StringToSign:
> GET
>
>
> Tue, 03 Feb 2015 09:37:54 GMT
> /admin/config
> DEBUG:boto:Signature:
> AWS SBF7DQZ1ESR6Y34225H8:9i6/4do7tDpViolN1XdxgefMs3U=
> DEBUG:boto:url = 'http://openstorage.ushttp://openstorage.us/admin/config'
> params={}
> headers={'Date': 'Tue, 03 Feb 2015 09:37:54 GMT', 'Content-Length': 
> '0', 'Authorization': 'AWS 
> SBF7DQZ1ESR6Y34225H8:9i6/4do7tDpViolN1XdxgefMs3U=', 'User-Agent': 
> 'Boto/2.20.1 Python/2.7.6 Linux/3.11.0-26-generic'} data=None ERROR:root:Could not retrieve region map from destination Traceback (most recent call last):
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/cli.py", line 269, in main
>     region_map = client.get_region_map(dest_conn)
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 391, in get_region_map
>     region_map = request(connection, 'get', 'admin/config')
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 153, in request
>     result = handler(url, params=params, headers=request.headers, data=data)
>   File "/usr/lib/python2.7/dist-packages/requests/api.py", line 55, in get
>     return request('get', url, **kwargs)
>   File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request
>     return session.request(method=method, url=url, **kwargs)
>   File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 421, in request
>     prep = self.prepare_request(req)
>   File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 359, in prepare_request
>     hooks=merge_hooks(request.hooks, self.hooks),
>   File "/usr/lib/python2.7/dist-packages/requests/models.py", line 287, in prepare
>     self.prepare_url(url, params)
>   File "/usr/lib/python2.7/dist-packages/requests/models.py", line 334, in prepare_url
>     scheme, auth, host, port, path, query, fragment = parse_url(url)
>   File "/usr/lib/python2.7/dist-packages/urllib3/util.py", line 390, in parse_url
>     raise LocationParseError("Failed to parse: %s" % url)
> LocationParseError: Failed to parse: Failed to parse: openstorage.ushttp:
>
> ----------------------------------------------------------------------
> ---------- LAUNCH USING /etc/init.d/radosgw-agent shell script:KO 
> /etc/init.d/radosgw-agent start 
> /etc/ceph/radosgw-agent/fr-rennes-radosgw1-sync.conf_direct
> Standard output:
> Starting radosgw-agent fr-rennes-radosgw1-sync.conf_direct
>
> Log File :
> 2015-02-03T10:51:00.794 4735:ERROR:root:Could not retrieve region map 
> from destination Traceback (most recent call last):
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/cli.py", line 269, in main
>     region_map = client.get_region_map(dest_conn)
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 391, in get_region_map
>     region_map = request(connection, 'get', 'admin/config')
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 153, in request
>     result = handler(url, params=params, headers=request.headers, data=data)
>   File "/usr/lib/python2.7/dist-packages/requests/api.py", line 55, in get
>     return request('get', url, **kwargs)
>   File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request
>     return session.request(method=method, url=url, **kwargs)
>   File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 421, in request
>     prep = self.prepare_request(req)
>   File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 359, in prepare_request
>     hooks=merge_hooks(request.hooks, self.hooks),
>   File "/usr/lib/python2.7/dist-packages/requests/models.py", line 287, in prepare
>     self.prepare_url(url, params)
>   File "/usr/lib/python2.7/dist-packages/requests/models.py", line 334, in prepare_url
>     scheme, auth, host, port, path, query, fragment = parse_url(url)
>   File "/usr/lib/python2.7/dist-packages/urllib3/util.py", line 390, in parse_url
>     raise LocationParseError("Failed to parse: %s" % url)
> LocationParseError: Failed to parse: Failed to parse: openstorage.ushttp:
>
> ----------------------------------------------------------------------
> ---------------------
> LAUNCH AS A SERVICE: OK
> service radosgw-agent start 
> /etc/ceph/radosgw-agent/fr-rennes-radosgw1-sync.conf_direct
> standard output:
> Starting radosgw-agent fr-rennes-radosgw1-sync.conf_direct
>
> Log file:
> 2015-02-03T10:42:02.878 3162:INFO:urllib3.connectionpool:Starting new 
> HTTP connection (1): openstorage.us
>
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> --------
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> -------- Here are the logs you requested using the altered client.py 
> python script.
>
> DIRECT LAUNCH USING CLI:OK
> radosgw-agent -v -c 
> /etc/ceph/radosgw-agent/fr-rennes-radosgw1-sync.conf_direct
> the standard output is also witten in the log file configured in the 
> synchro file
>
> 2015-02-03T11:00:40.447 6158:DEBUG:boto:Using access key provided by client.
> 2015-02-03T11:00:40.447 6158:DEBUG:boto:Using secret key provided by client.
> 2015-02-03T11:00:40.448 6158:DEBUG:boto:StringToSign:
> GET
>
>
> Tue, 03 Feb 2015 10:00:40 GMT
> /admin/config
> 2015-02-03T11:00:40.448 6158:DEBUG:boto:Signature:
> AWS SBF7DQZ1ESR6Y34225H8:LUEBUrR9z1Bgbrj6F/XGrGF+8BM=
> 2015-02-03T11:00:40.448 6158:DEBUG:boto:url = 'http://openstorage.us/admin/config'
> params={}
> headers={'Date': 'Tue, 03 Feb 2015 10:00:40 GMT', 'Content-Length': 
> '0', 'Authorization': 'AWS 
> SBF7DQZ1ESR6Y34225H8:LUEBUrR9z1Bgbrj6F/XGrGF+8BM=', 'User-Agent': 
> 'Boto/2.20.1 Python/2.7.6 Linux/3.11.0-26-generic'} data=None
> 2015-02-03T11:00:40.457 6158:INFO:urllib3.connectionpool:Starting new 
> HTTP connection (1): proxy
> 2015-02-03T11:00:40.467 6158:DEBUG:urllib3.connectionpool:Setting read 
> timeout to None
> 2015-02-03T11:00:40.848 6158:DEBUG:urllib3.connectionpool:"GET 
> http://openstorage.us/admin/config HTTP/1.1" 200 None
>
> ----------------------------------------------------------------------
> ---------- LAUNCH USING /etc/init.d/radosgw-agent shell script:KO 
> /etc/init.d/radosgw-agent start 
> /etc/ceph/radosgw-agent/fr-rennes-radosgw1-sync.conf_direct
> Standard output:
> Starting radosgw-agent fr-rennes-radosgw1-sync.conf_direct
>
> Log File :
> 2015-02-03T11:07:49.271 7513:INFO:urllib3.connectionpool:Starting new 
> HTTP connection (1): proxy
> 2015-02-03T11:07:49.654 7513:INFO:urllib3.connectionpool:Starting new 
> HTTP connection (1): proxy
> 2015-02-03T11:07:50.469 7513:ERROR:radosgw_agent.sync:finding number 
> of shards failed
> 2015-02-03T11:07:50.470 7513:WARNING:radosgw_agent.sync:error preparing for sync, will retry. Traceback:
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/sync.py", line 26, in prepare_sync
>     syncer.prepare()
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/sync.py", line 202, in prepare
>     self.init_num_shards()
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/sync.py", line 73, in init_num_shards
>     self.num_shards = client.num_log_shards(self.src_conn, self.type)
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 285, in num_log_shards
>     out = request(connection, 'get', 'admin/log', dict(type=shard_type))
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 160, in request
>     check_result_status(result)
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 116, in check_result_status
>     HttpError)(result.status_code, result.content)
> NotFound: Http error code 404 content <HTML><HEAD>
>
>
> ----------------------------------------------------------------------
> ---------------------
> LAUNCH AS A SERVICE:KO
> service radosgw-agent start 
> /etc/ceph/radosgw-agent/fr-rennes-radosgw1-sync.conf_direct
> standard output:
> Starting radosgw-agent fr-rennes-radosgw1-sync.conf_direct
>
> Log file:
> 2015-02-03T11:04:25.435 6872:ERROR:root:Could not retrieve region map 
> from destination Traceback (most recent call last):
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/cli.py", line 269, in main
>     region_map = client.get_region_map(dest_conn)
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 396, in get_region_map
>     region_map = request(connection, 'get', 'admin/config')
>   File "/usr/lib/python2.7/dist-packages/radosgw_agent/client.py", line 158, in request
>     result = handler(url, params=params, headers=request.headers, data=data)
>   File "/usr/lib/python2.7/dist-packages/requests/api.py", line 55, in get
>     return request('get', url, **kwargs)
>   File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request
>     return session.request(method=method, url=url, **kwargs)
>   File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 421, in request
>     prep = self.prepare_request(req)
>   File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 359, in prepare_request
>     hooks=merge_hooks(request.hooks, self.hooks),
>   File "/usr/lib/python2.7/dist-packages/requests/models.py", line 287, in prepare
>     self.prepare_url(url, params)
>   File "/usr/lib/python2.7/dist-packages/requests/models.py", line 338, in prepare_url
>     "Perhaps you meant http://{0}?".format(url))
> MissingSchema: Invalid URL u'/admin/config': No schema supplied. Perhaps you meant http:///admin/config?
>
>
>
> Best regards
>
> -----Message d'origine-----
> De : alfredo.deza@xxxxxxxxxxx [mailto:alfredo.deza@xxxxxxxxxxx] De la 
> part de Alfredo Deza Envoyé : lundi 2 février 2015 19:42 À : CHEVALIER 
> Ghislain IMT/OLPS Cc : ceph-devel@xxxxxxxxxxxxxxx Objet : Re: 
> [Ceph-devel] radosgw-agent failed to parse
>
> If you use the `-v` flag you will get very verbose output.
>
> On Mon, Feb 2, 2015 at 12:12 PM,  <ghislain.chevalier@xxxxxxxxxx> wrote:
>> OK I will send them asap
>>
>> Logs are not very verbose.. Can I set a debug mode?
>>
>> -----Message d'origine-----
>> De : alfredo.deza@xxxxxxxxxxx [mailto:alfredo.deza@xxxxxxxxxxx] De la 
>> part de Alfredo Deza Envoyé : lundi 2 février 2015 16:59 À : 
>> CHEVALIER Ghislain IMT/OLPS Cc :
>> Objet : Re: [Ceph-devel] radosgw-agent failed to parse
>>
>> That sounds suspicious because we haven't packaged a 1.2.1 release.
>>
>> The latest on PyPI is 1.2, which is the same in the master branch.
>>
>> Would you mind sending logs that show *how* the url is malformed (both when it is OK and when it is not) ?
>>
>> On Mon, Feb 2, 2015 at 9:53 AM,  <ghislain.chevalier@xxxxxxxxxx> wrote:
>>> HI,
>>>
>>> Thx for replying
>>>
>>> According to dpkg -l, it's 1.2.1.
>>>
>>> I noticed that the URL is malformed when launching using directly 
>>> radosgw-agent -c <syncfile> but well formed when launching using 
>>> service radosgw-agent start <syncfile>
>>>
>>> best regards
>>>
>>> -----Message d'origine-----
>>> De : alfredo.deza@xxxxxxxxxxx [mailto:alfredo.deza@xxxxxxxxxxx] De 
>>> la part de Alfredo Deza Envoyé : lundi 2 février 2015 15:18 À :
>>> CHEVALIER Ghislain IMT/OLPS Cc : ceph-devel@xxxxxxxxxxxxxxx Objet : Re:
>>> [Ceph-devel] radosgw-agent failed to parse
>>>
>>> What version of the agent are you using? And when it fails, how does the log output look?
>>>
>>> On Mon, Feb 2, 2015 at 8:56 AM,  <ghislain.chevalier@xxxxxxxxxx> wrote:
>>>> Hi all,
>>>>
>>>> Context : Ubuntu 14.04 TLS firefly 0.80.8
>>>>
>>>> I sent this post in ceph-users (identical subject) because I recently encountered the same issue.
>>>> Maybe I missed something between July and January...
>>>>
>>>> I found that the http request wasn't correctly formed by 
>>>> /usr/lib/python2.7/dist-packages/radosgw_agent/client.py
>>>>
>>>> I did the changes below
>>>> #    url = '{protocol}://{host}{path}'.format(protocol=request.protocol,
>>>> #                                             host=request.host,
>>>> #                                             path=request.path)
>>>>      url = '{path}'.format(protocol="", host="", path=request.path)
>>>>
>>>> The request is then correctly formed and sent.
>>>>
>>>> I still have problems setting the federation between 2 zones of 2 ceph clusters in 2 regions.
>>>>
>>>> I go on investigating.
>>>>
>>>> Best regards
>>>>
>>>>
>>>> ___________________________________________________________________
>>>> _ _ _ ___________________________________________________
>>>>
>>>> Ce message et ses pieces jointes peuvent contenir des informations 
>>>> confidentielles ou privilegiees et ne doivent donc pas etre 
>>>> diffuses, exploites ou copies sans autorisation. Si vous avez recu 
>>>> ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
>>>>
>>>> This message and its attachments may contain confidential or 
>>>> privileged information that may be protected by law; they should not be distributed, used or copied without authorisation.
>>>> If you have received this email in error, please notify the sender and delete this message and its attachments.
>>>> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
>>>> Thank you.
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
>>>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More 
>>>> majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>> ____________________________________________________________________
>>> _ _ ___________________________________________________
>>>
>>> Ce message et ses pieces jointes peuvent contenir des informations 
>>> confidentielles ou privilegiees et ne doivent donc pas etre 
>>> diffuses, exploites ou copies sans autorisation. Si vous avez recu 
>>> ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
>>>
>>> This message and its attachments may contain confidential or 
>>> privileged information that may be protected by law; they should not be distributed, used or copied without authorisation.
>>> If you have received this email in error, please notify the sender and delete this message and its attachments.
>>> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
>>> Thank you.
>>>
>>
>> _____________________________________________________________________
>> _ ___________________________________________________
>>
>> Ce message et ses pieces jointes peuvent contenir des informations 
>> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, 
>> exploites ou copies sans autorisation. Si vous avez recu ce message 
>> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
>>
>> This message and its attachments may contain confidential or 
>> privileged information that may be protected by law; they should not be distributed, used or copied without authorisation.
>> If you have received this email in error, please notify the sender and delete this message and its attachments.
>> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
>> Thank you.
>>
>
> ______________________________________________________________________
> ___________________________________________________
>
> Ce message et ses pieces jointes peuvent contenir des informations 
> confidentielles ou privilegiees et ne doivent donc pas etre diffuses, 
> exploites ou copies sans autorisation. Si vous avez recu ce message 
> par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
>
> This message and its attachments may contain confidential or 
> privileged information that may be protected by law; they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and delete this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
> Thank you.
>

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux