Hi everyone,
We faced with strange issue on RadosGW (0.94.5-1precise, civet frontend behind nginx). nginx's access.log:
17/Dec/2015:20:34:55 "PUT /ZZZ?uploadId=XXX&partNumber=37 HTTP/1.1" 200
17/Dec/2015:20:34:57 "PUT /ZZZ?uploadId=XXX&partNumber=39 HTTP/1.1" 200
17/Dec/2015:20:35:47 "POST /ZZZ?uploadId=XXX HTTP/1.1" 499
17/Dec/2015:20:36:37 "POST /ZZZ?uploadId=XXX HTTP/1.1" 499
We successfully uploaded 39 parts of this object and hitting read timeout from CompleteMultipart request (POST) and our library retries it one more time (Should we retry MultipartComplete request?).
After that we can read entire object for 5 minutes or less (seems like GC schedule in rgw), afterwards we start receiving 404 NoSuchKey. Interesting thing is that head object is not deleted and we can fetch object metadata (using HEAD request). I've scanned all OSD dirs and found no content on this objects. Only reference is head object with rgw.manifest in xattrs.
I've tried to search related issues in tracker, but didn't found nothing similiar.
Unfortunatelly we have no rgw logs for this period at all.
I've enable 30/30 logging and collecting logs, but now we've more acceptable response time and no timeouts at all.
Any ideas?
Thanks!
Best regards,
Gleb M Borisov
Gleb M Borisov
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
