radosgw_agent sync issues

[ceph new <cephnewuser@xxxxxxxxx>]

HI

i setup 2 clusters and in using radosgw_agent to sync them last week the sync stop working if on runinig the agent from command line i see its stuck on 2 files in the console im geting :
2016-03-17 21:11:57,391 14323 [radosgw_agent.worker][DEBUG ] op state is []
2016-03-17 21:11:57,391 14323 [radosgw_agent.worker][DEBUG ] error geting op state: list index out of range
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/radosgw_agent/worker.py", line 275, in wait_for_object
    state = state[0]['state']

and in the log i see :

2016-03-17 21:38:53,221 30848 [boto][DEBUG ] Signature:
AWS WOCV3FJ0KFG4E5CVHF46:b+kB03QMTXlVIAhSfkM2aW4sSmk=
2016-03-17 21:38:53,221 30848 [boto][DEBUG ] url = "" href="http://s3-us-west.test.com/admin/opstate">http://s3-us-west.test.com/admin/opstate'
params={'client-id': 'radosgw-agent', 'object': u'test/Kenny-Wormald-photo-premiere2-56b2b75d5f9b58def9c8ed52.jpg', 'op-id': 'nyprceph1.ops.test.com:30568:135'}
headers={'Date': 'Thu, 17 Mar 2016 21:38:53 GMT', 'Content-Length': '0', 'Authorization': u'AWS WOCV3FJ0KFG4E5CVHF46:b+kB03QMTXlVIAhSfkM2aW4sSmk=', 'User-Agent': 'Boto/2.38.0 Python/2.6.6 Linux/2.6.32-504.8.1.el6.x86_64'}
data="">2016-03-17 21:38:53,222 30848 [boto][DEBUG ] Method: GET
2016-03-17 21:38:53,222 30848 [boto][DEBUG ] Path: /admin/opstate?client-id=radosgw-agent&object=test/Kenny-Wormald-photo-premiere2-56b2b75d5f9b58def9c8ed52.jpg&op-id=nyprceph1.ops.test.com%3A30568%3A135
2016-03-17 21:38:53,222 30848 [boto][DEBUG ] Data:
2016-03-17 21:38:53,222 30848 [boto][DEBUG ] Headers: {}
2016-03-17 21:38:53,222 30848 [boto][DEBUG ] Host: s3-us-west.test.com
2016-03-17 21:38:53,222 30848 [boto][DEBUG ] Port: 80
2016-03-17 21:38:53,223 30848 [boto][DEBUG ] Params: {'client-id': 'radosgw-agent', 'object': 'test/Kenny-Wormald-photo-premiere2-56b2b75d5f9b58def9c8ed52.jpg', 'op-id': 'nyprceph1.ops.test.com%3A30568%3A135'}
2016-03-17 21:38:53,223 30848 [boto][DEBUG ] Token: None
2016-03-17 21:38:53,223 30848 [boto][DEBUG ] StringToSign:
GET


Thu, 17 Mar 2016 21:38:53 GMT
/admin/opstate
2016-03-17 21:38:53,223 30848 [boto][DEBUG ] Signature:
AWS WOCV3FJ0KFG4E5CVHF46:b+kB03QMTXlVIAhSfkM2aW4sSmk=
2016-03-17 21:38:53,223 30848 [boto][DEBUG ] Final headers: {'Date': 'Thu, 17 Mar 2016 21:38:53 GMT', 'Content-Length': '0', 'Authorization': u'AWS WOCV3FJ0KFG4E5CVHF46:b+kB03QMTXlVIAhSfkM2aW4sSmk=', 'User-Agent': 'Boto/2.38.0 Python/2.6.6 Linux/2.6.32-504.8.1.el6.x86_64'}
2016-03-17 21:38:53,298 30848 [boto][DEBUG ] Response headers: [('date', 'Thu, 17 Mar 2016 21:38:53 GMT'), ('content-length', '2'), ('x-amz-request-id', 'tx00000000000000019c09c-0056eb23ed-f149c-us-west')]
2016-03-17 21:38:53,369 30848 [radosgw_agent.worker][DEBUG ] op state is []
2016-03-17 21:38:53,369 30848 [radosgw_agent.worker][DEBUG ] error geting op state: list index out of range
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/radosgw_agent/worker.py", line 275, in wait_for_object
    state = state[0]['state']
IndexError: list index out of range

i can download the file from from the master i upload it to the slave and rerun the sync but still didnt work

any way to skip the file and get the sync done ? (IE just remove it and re-upload it under new name ? )

is it need to be fix from the master side or slave ?

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com