----- Original Message ----- > From: "Neville" <neville.taylor@xxxxxxxxxxxxx> > To: "Yehuda Sadeh-Weinraub" <yehuda@xxxxxxxxxx> > Cc: ceph-users@xxxxxxxxxxxxxx > Sent: Wednesday, April 1, 2015 11:45:09 AM > Subject: Re: Radosgw authorization failed > > > > > On 31 Mar 2015, at 11:38, Neville <neville.taylor@xxxxxxxxxxxxx> wrote: > > > > > > > > > Date: Mon, 30 Mar 2015 12:17:48 -0400 > > > From: yehuda@xxxxxxxxxx > > > To: neville.taylor@xxxxxxxxxxxxx > > > CC: ceph-users@xxxxxxxxxxxxxx > > > Subject: Re: Radosgw authorization failed > > > > > > > > > > > > ----- Original Message ----- > > > > From: "Neville" <neville.taylor@xxxxxxxxxxxxx> > > > > To: "Yehuda Sadeh-Weinraub" <yehuda@xxxxxxxxxx> > > > > Cc: ceph-users@xxxxxxxxxxxxxx > > > > Sent: Monday, March 30, 2015 6:49:29 AM > > > > Subject: Re: Radosgw authorization failed > > > > > > > > > > > > > Date: Wed, 25 Mar 2015 11:43:44 -0400 > > > > > From: yehuda@xxxxxxxxxx > > > > > To: neville.taylor@xxxxxxxxxxxxx > > > > > CC: ceph-users@xxxxxxxxxxxxxx > > > > > Subject: Re: Radosgw authorization failed > > > > > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > > From: "Neville" <neville.taylor@xxxxxxxxxxxxx> > > > > > > To: ceph-users@xxxxxxxxxxxxxx > > > > > > Sent: Wednesday, March 25, 2015 8:16:39 AM > > > > > > Subject: Radosgw authorization failed > > > > > > > > > > > > Hi all, > > > > > > > > > > > > I'm testing backup product which supports Amazon S3 as target for > > > > > > Archive > > > > > > storage and I'm trying to setup a Ceph cluster configured with the > > > > > > S3 API > > > > > > to > > > > > > use as an internal target for backup archives instead of AWS. > > > > > > > > > > > > I've followed the online guide for setting up Radosgw and created a > > > > > > default > > > > > > region and zone based on the AWS naming convention US-East-1. I'm > > > > > > not > > > > > > sure > > > > > > if this is relevant but since I was having issues I thought it > > > > > > might need > > > > > > to > > > > > > be the same. > > > > > > > > > > > > I've tested the radosgw using boto.s3 and it seems to work ok i.e. > > > > > > I can > > > > > > create a bucket, create a folder, list buckets etc. The problem is > > > > > > when > > > > > > the > > > > > > backup software tries to create an object I get an authorization > > > > > > failure. > > > > > > It's using the same user/access/secret as I'm using from boto.s3 > > > > > > and I'm > > > > > > sure the creds are right as it lets me create the initial > > > > > > connection, it > > > > > > just fails when trying to create an object (backup folder). > > > > > > > > > > > > Here's the extract from the radosgw log: > > > > > > > > > > > > --------------------------------------------------------------------------------------------------------------------------------------------- > > > > > > 2015-03-25 15:07:26.449227 7f1050dc7700 2 req 5:0.000419:s3:GET > > > > > > /:list_bucket:init op > > > > > > 2015-03-25 15:07:26.449232 7f1050dc7700 2 req 5:0.000424:s3:GET > > > > > > /:list_bucket:verifying op mask > > > > > > 2015-03-25 15:07:26.449234 7f1050dc7700 20 required_mask= 1 > > > > > > user.op_mask=7 > > > > > > 2015-03-25 15:07:26.449235 7f1050dc7700 2 req 5:0.000427:s3:GET > > > > > > /:list_bucket:verifying op permissions > > > > > > 2015-03-25 15:07:26.449237 7f1050dc7700 5 Searching permissions for > > > > > > uid=test > > > > > > mask=49 > > > > > > 2015-03-25 15:07:26.449238 7f1050dc7700 5 Found permission: 15 > > > > > > 2015-03-25 15:07:26.449239 7f1050dc7700 5 Searching permissions for > > > > > > group=1 > > > > > > mask=49 > > > > > > 2015-03-25 15:07:26.449240 7f1050dc7700 5 Found permission: 15 > > > > > > 2015-03-25 15:07:26.449241 7f1050dc7700 5 Searching permissions for > > > > > > group=2 > > > > > > mask=49 > > > > > > 2015-03-25 15:07:26.449242 7f1050dc7700 5 Found permission: 15 > > > > > > 2015-03-25 15:07:26.449243 7f1050dc7700 5 Getting permissions > > > > > > id=test > > > > > > owner=test perm=1 > > > > > > 2015-03-25 15:07:26.449244 7f1050dc7700 10 uid=test requested perm > > > > > > (type)=1, > > > > > > policy perm=1, user_perm_mask=1, acl perm=1 > > > > > > 2015-03-25 15:07:26.449245 7f1050dc7700 2 req 5:0.000437:s3:GET > > > > > > /:list_bucket:verifying op params > > > > > > 2015-03-25 15:07:26.449247 7f1050dc7700 2 req 5:0.000439:s3:GET > > > > > > /:list_bucket:executing > > > > > > 2015-03-25 15:07:26.449252 7f1050dc7700 10 cls_bucket_list > > > > > > test1(@{i=.us-east.rgw.buckets.index}.us-east.rgw.buckets[us-east.280959.2]) > > > > > > start num 1001 > > > > > > 2015-03-25 15:07:26.450828 7f1050dc7700 2 req 5:0.002020:s3:GET > > > > > > /:list_bucket:http status=200 > > > > > > 2015-03-25 15:07:26.450832 7f1050dc7700 1 ====== req done > > > > > > req=0x7f107000e2e0 > > > > > > http_status=200 ====== > > > > > > 2015-03-25 15:07:26.516999 7f1069df9700 20 enqueued request > > > > > > req=0x7f107000f0e0 > > > > > > 2015-03-25 15:07:26.517006 7f1069df9700 20 RGWWQ: > > > > > > 2015-03-25 15:07:26.517007 7f1069df9700 20 req: 0x7f107000f0e0 > > > > > > 2015-03-25 15:07:26.517010 7f1069df9700 10 allocated request > > > > > > req=0x7f107000f6b0 > > > > > > 2015-03-25 15:07:26.517021 7f1058dd7700 20 dequeued request > > > > > > req=0x7f107000f0e0 > > > > > > 2015-03-25 15:07:26.517023 7f1058dd7700 20 RGWWQ: empty > > > > > > 2015-03-25 15:07:26.517081 7f1058dd7700 20 CONTENT_LENGTH=88 > > > > > > 2015-03-25 15:07:26.517084 7f1058dd7700 20 > > > > > > CONTENT_TYPE=application/octet-stream > > > > > > 2015-03-25 15:07:26.517085 7f1058dd7700 20 > > > > > > CONTEXT_DOCUMENT_ROOT=/var/www > > > > > > 2015-03-25 15:07:26.517086 7f1058dd7700 20 CONTEXT_PREFIX= > > > > > > 2015-03-25 15:07:26.517087 7f1058dd7700 20 DOCUMENT_ROOT=/var/www > > > > > > 2015-03-25 15:07:26.517088 7f1058dd7700 20 FCGI_ROLE=RESPONDER > > > > > > 2015-03-25 15:07:26.517089 7f1058dd7700 20 > > > > > > GATEWAY_INTERFACE=CGI/1.1 > > > > > > 2015-03-25 15:07:26.517090 7f1058dd7700 20 HTTP_AUTHORIZATION=AWS > > > > > > F79L68W19B3GCLOSE3F8:AcXqtvlBzBMpwdL+WuhDRoLT/Bs= > > > > > > 2015-03-25 15:07:26.517091 7f1058dd7700 20 > > > > > > HTTP_CONNECTION=Keep-Alive > > > > > > 2015-03-25 15:07:26.517092 7f1058dd7700 20 HTTP_DATE=Wed, 25 Mar > > > > > > 2015 > > > > > > 15:07:26 GMT > > > > > > 2015-03-25 15:07:26.517092 7f1058dd7700 20 HTTP_EXPECT=100-continue > > > > > > 2015-03-25 15:07:26.517093 7f1058dd7700 20 > > > > > > HTTP_HOST=test1.devops-os-cog01.devops.local > > > > > > 2015-03-25 15:07:26.517094 7f1058dd7700 20 > > > > > > HTTP_USER_AGENT=aws-sdk-java/unknown-version > > > > > > Windows_Server_2008_R2/6.1 > > > > > > Java_HotSpot(TM)_Client_VM/24.55-b03 > > > > > > 2015-03-25 15:07:26.517096 7f1058dd7700 20 > > > > > > HTTP_X_AMZ_META_CREATIONTIME=2015-03-25T15:07:26 > > > > > > 2015-03-25 15:07:26.517097 7f1058dd7700 20 HTTP_X_AMZ_META_SIZE=88 > > > > > > 2015-03-25 15:07:26.517098 7f1058dd7700 20 > > > > > > HTTP_X_AMZ_STORAGE_CLASS=STANDARD > > > > > > 2015-03-25 15:07:26.517099 7f1058dd7700 20 HTTPS=on > > > > > > 2015-03-25 15:07:26.517100 7f1058dd7700 20 > > > > > > PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin > > > > > > 2015-03-25 15:07:26.517100 7f1058dd7700 20 QUERY_STRING= > > > > > > 2015-03-25 15:07:26.517101 7f1058dd7700 20 REMOTE_ADDR=10.40.41.106 > > > > > > 2015-03-25 15:07:26.517102 7f1058dd7700 20 REMOTE_PORT=55439 > > > > > > 2015-03-25 15:07:26.517103 7f1058dd7700 20 REQUEST_METHOD=PUT > > > > > > 2015-03-25 15:07:26.517104 7f1058dd7700 20 REQUEST_SCHEME=https > > > > > > 2015-03-25 15:07:26.517105 7f1058dd7700 20 > > > > > > REQUEST_URI=/ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3 > > > > > > 2015-03-25 15:07:26.517106 7f1058dd7700 20 > > > > > > SCRIPT_FILENAME=/var/www/s3gw.fcgi > > > > > > 2015-03-25 15:07:26.517107 7f1058dd7700 20 > > > > > > SCRIPT_NAME=/ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3 > > > > > > 2015-03-25 15:07:26.517108 7f1058dd7700 20 > > > > > > SCRIPT_URI=https://test1.devops-os-cog01.devops.local/ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3 > > > > > > 2015-03-25 15:07:26.517109 7f1058dd7700 20 > > > > > > SCRIPT_URL=/ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3 > > > > > > 2015-03-25 15:07:26.517110 7f1058dd7700 20 SERVER_ADDR=10.40.41.64 > > > > > > 2015-03-25 15:07:26.517111 7f1058dd7700 20 > > > > > > SERVER_ADMIN=no-one@devops.local > > > > > > 2015-03-25 15:07:26.517112 7f1058dd7700 20 > > > > > > SERVER_NAME=test1.devops-os-cog01.devops.local > > > > > > 2015-03-25 15:07:26.517113 7f1058dd7700 20 SERVER_PORT=443 > > > > > > 2015-03-25 15:07:26.517114 7f1058dd7700 20 SERVER_PORT_SECURE=443 > > > > > > 2015-03-25 15:07:26.517115 7f1058dd7700 20 SERVER_PROTOCOL=HTTP/1.1 > > > > > > 2015-03-25 15:07:26.517116 7f1058dd7700 20 SERVER_SIGNATURE= > > > > > > 2015-03-25 15:07:26.517117 7f1058dd7700 20 > > > > > > SERVER_SOFTWARE=Apache/2.4.7 > > > > > > (Ubuntu) > > > > > > 2015-03-25 15:07:26.517119 7f1058dd7700 1 ====== starting new > > > > > > request > > > > > > req=0x7f107000f0e0 ===== > > > > > > 2015-03-25 15:07:26.517129 7f1058dd7700 2 req 6:0.000010::PUT > > > > > > /ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3::initializing > > > > > > 2015-03-25 15:07:26.517132 7f1058dd7700 10 > > > > > > host=test1.devops-os-cog01.devops.local > > > > > > rgw_dns_name=devops-os-cog01.devops.local > > > > > > 2015-03-25 15:07:26.517143 7f1058dd7700 10 meta>> > > > > > > HTTP_X_AMZ_META_CREATIONTIME > > > > > > 2015-03-25 15:07:26.517147 7f1058dd7700 10 meta>> > > > > > > HTTP_X_AMZ_META_SIZE > > > > > > 2015-03-25 15:07:26.517150 7f1058dd7700 10 meta>> > > > > > > HTTP_X_AMZ_STORAGE_CLASS > > > > > > 2015-03-25 15:07:26.517155 7f1058dd7700 10 x>> > > > > > > x-amz-meta-creationtime:2015-03-25T15:07:26 > > > > > > 2015-03-25 15:07:26.517156 7f1058dd7700 10 x>> x-amz-meta-size:88 > > > > > > 2015-03-25 15:07:26.517157 7f1058dd7700 10 x>> > > > > > > x-amz-storage-class:STANDARD > > > > > > 2015-03-25 15:07:26.517167 7f1058dd7700 10 > > > > > > s->object=ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3 > > > > > > s->bucket=test1 > > > > > > 2015-03-25 15:07:26.517171 7f1058dd7700 2 req 6:0.000052:s3:PUT > > > > > > /ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3::getting op > > > > > > 2015-03-25 15:07:26.517175 7f1058dd7700 2 req 6:0.000055:s3:PUT > > > > > > /ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3:put_obj:authorizing > > > > > > 2015-03-25 15:07:26.517209 7f1058dd7700 20 get_obj_state: > > > > > > rctx=0x7f106c025ce0 > > > > > > obj=.us-east.users:F79L68W19B3GCLOSE3F8 state=0x7f106c0260b8 > > > > > > s->prefetch_data=0 > > > > > > 2015-03-25 15:07:26.517215 7f1058dd7700 10 cache get: > > > > > > name=.us-east.users+F79L68W19B3GCLOSE3F8 : hit > > > > > > 2015-03-25 15:07:26.517219 7f1058dd7700 20 get_obj_state: > > > > > > s->obj_tag was > > > > > > set > > > > > > empty > > > > > > 2015-03-25 15:07:26.517224 7f1058dd7700 10 cache get: > > > > > > name=.us-east.users+F79L68W19B3GCLOSE3F8 : hit > > > > > > 2015-03-25 15:07:26.517248 7f1058dd7700 20 get_obj_state: > > > > > > rctx=0x7f106c025ce0 > > > > > > obj=.us-east.users.uid:test state=0x7f106c0267c8 s->prefetch_data=0 > > > > > > 2015-03-25 15:07:26.517252 7f1058dd7700 10 cache get: > > > > > > name=.us-east.users.uid+test : hit > > > > > > 2015-03-25 15:07:26.517256 7f1058dd7700 20 get_obj_state: > > > > > > s->obj_tag was > > > > > > set > > > > > > empty > > > > > > 2015-03-25 15:07:26.517261 7f1058dd7700 10 cache get: > > > > > > name=.us-east.users.uid+test : hit > > > > > > 2015-03-25 15:07:26.517296 7f1058dd7700 10 get_canon_resource(): > > > > > > dest=/test1/ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3 > > > > > > 2015-03-25 15:07:26.517298 7f1058dd7700 10 auth_hdr: > > > > > > PUT > > > > > > application/octet-stream > > > > > > Wed, 25 Mar 2015 15:07:26 GMT > > > > > > x-amz-meta-creationtime:2015-03-25T15:07:26 > > > > > > x-amz-meta-size:88 > > > > > > x-amz-storage-class:STANDARD > > > > > > /test1/ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3 > > > > > > 2015-03-25 15:07:26.517341 7f1058dd7700 15 calculated > > > > > > digest=coYY2IBRECc42IXE7HQlYxBcqfU= > > > > > > 2015-03-25 15:07:26.517343 7f1058dd7700 15 > > > > > > auth_sign=AcXqtvlBzBMpwdL+WuhDRoLT/Bs= > > > > > > 2015-03-25 15:07:26.517344 7f1058dd7700 15 compare=-34 > > > > > > 2015-03-25 15:07:26.517346 7f1058dd7700 10 failed to authorize > > > > > > request > > > > > > 2015-03-25 15:07:26.517363 7f1058dd7700 2 req 6:0.000244:s3:PUT > > > > > > /ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3:put_obj:http > > > > > > status=403 > > > > > > 2015-03-25 15:07:26.517367 7f1058dd7700 1 ====== req done > > > > > > req=0x7f107000f0e0 > > > > > > http_status=403 ====== > > > > > > 2015-03-25 15:07:26.517374 7f1058dd7700 20 process_request() > > > > > > returned -1 > > > > > > 2015-03-25 15:07:28.058030 7f1088ff9700 2 > > > > > > RGWDataChangesLog::ChangesRenewThread: start > > > > > > > > > > Nothing pops up as being wrong. Assuming the first and second > > > > > requests were > > > > > done using the same credentials, I would guess that it might be the > > > > > client > > > > > incorrectly creating the canonical representation of the request. > > > > > Maybe it > > > > > doesn't use the x-amz-storage-class correctly there?. Can you try > > > > > verifying what is the plain text signature string that it's using? > > > > > Also, > > > > > try disabling the storage class param, see if it makes any > > > > > difference. > > > > > > > > > > Yehuda > > > > I've looked through the application logs but it doesn't appear to > > > > capture > > > > what the request string is that it's using. I also can't find out any > > > > way of > > > > disabling the x-amz-storage-class unfortunately. I did test the > > > > application > > > > against AWS S3 and the functionality I'm trying to use seems to work > > > > fine so > > > > it suggests it is a problem with my Ceph implementation. To help > > > > troubleshoot I've tried to capture some different requests for > > > > comparison: > > > > GET request from backup application (Successful): > > > > 2015-03-30 11:55:05.331145 7f103f5a4700 10 get_canon_resource(): > > > > dest=/test1/ > > > > 2015-03-30 11:55:05.331148 7f103f5a4700 10 auth_hdr: > > > > GET > > > > application/x-www-form-urlencoded; charset=utf-8 > > > > Mon, 30 Mar 2015 10:55:05 GMT > > > > /test1/ > > > > 2015-03-30 11:55:05.331195 7f103f5a4700 15 calculated > > > > digest=4tEUx7ERu+joNwYWrQboPhy/+Y4= > > > > 2015-03-30 11:55:05.331197 7f103f5a4700 15 > > > > auth_sign=4tEUx7ERu+joNwYWrQboPhy/+Y4= > > > > 2015-03-30 11:55:05.331198 7f103f5a4700 15 compare=0 > > > > 2015-03-30 11:55:05.331200 7f103f5a4700 2 req 167:0.000252:s3:GET > > > > /:list_bucket:reading permissions > > > > 2015-03-30 11:55:05.331239 7f103f5a4700 20 get_obj_state: > > > > rctx=0x7f103f5a3230 > > > > obj=.us-east.domain.rgw:test1 state=0x7f109d479098 s->prefetch_data=0 > > > > 2015-03-30 11:55:05.331245 7f103f5a4700 10 cache get: > > > > name=.us-east.domain.rgw+test1 : hit > > > > PUT request from backup application (Fails): > > > > 2015-03-30 11:55:05.414753 7f105cddf700 10 get_canon_resource(): > > > > dest=/test1/ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3 > > > > 2015-03-30 11:55:05.414755 7f105cddf700 10 auth_hdr: > > > > PUT > > > > application/octet-stream > > > > Mon, 30 Mar 2015 10:55:05 GMT > > > > x-amz-meta-creationtime:2015-03-30T10:55:05 > > > > x-amz-meta-size:88 > > > > x-amz-storage-class:STANDARD > > > > /test1/ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3 > > > > 2015-03-30 11:55:05.414800 7f105cddf700 15 calculated > > > > digest=DtPnePmBb00mfJgAr/z+uW70Unc= > > > > 2015-03-30 11:55:05.414802 7f105cddf700 15 > > > > auth_sign=iK1Wkt0uoIif9udGTf1P1fqF6Bk= > > > > 2015-03-30 11:55:05.414803 7f105cddf700 15 compare=37 > > > > 2015-03-30 11:55:05.414805 7f105cddf700 10 failed to authorize request > > > > PUT request from "S3 Browser" application using S3 compatible > > > > connection > > > > (Successful): > > > > 2015-03-30 13:07:47.433691 7f10445ae700 10 get_canon_resource(): > > > > dest=/test1/february/?acl > > > > 2015-03-30 13:07:47.433694 7f10445ae700 10 auth_hdr: > > > > PUT > > > > x-amz-date:Mon, 30 Mar 2015 12:07:47 GMT > > > > /test1/february/?acl > > > > 2015-03-30 13:07:47.433740 7f10445ae700 15 calculated > > > > digest=9g91r6RZ3YCZawt+H4YuGaSlffQ= > > > > 2015-03-30 13:07:47.433742 7f10445ae700 15 > > > > auth_sign=9g91r6RZ3YCZawt+H4YuGaSlffQ= > > > > 2015-03-30 13:07:47.433743 7f10445ae700 15 compare=0 > > > > 2015-03-30 13:07:47.433745 7f10445ae700 2 req 183:0.000227:s3:PUT > > > > /test1/february/:put_acls:reading permissions > > > > PUT request using boto S3 (Successful): > > > > 2015-03-30 13:26:29.394718 7f103f5a4700 10 get_canon_resource(): > > > > dest=/test1/ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3/ > > > > 2015-03-30 13:26:29.394721 7f103f5a4700 10 auth_hdr: > > > > PUT > > > > 1B2M2Y8AsgTpgAmY7PhCfg== > > > > application/octet-stream > > > > Mon, 30 Mar 2015 12:26:02 GMT > > > > /test1/ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3/ > > > > 2015-03-30 13:26:29.394821 7f103f5a4700 15 calculated > > > > digest=ljvCTLAqySSfcJ3baWfK5776kJM= > > > > 2015-03-30 13:26:29.394823 7f103f5a4700 15 > > > > auth_sign=ljvCTLAqySSfcJ3baWfK5776kJM= > > > > 2015-03-30 13:26:29.394824 7f103f5a4700 15 compare=0 > > > > 2015-03-30 13:26:29.394828 7f103f5a4700 2 req 188:0.000414:s3:PUT > > > > /test1/ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3/:put_obj:reading > > > > permissions > > > > Does this help at all? Is it fair to assume it's not the application if > > > > it > > > > works ok with AWS S3? I'm wondering if there is a bug in the S3 > > > > implementation on Ceph which mean it handles one of these headers > > > > incorrectly: > > > > x-amz-meta-creationtime > > > > x-amz-meta-size > > > > x-amz-storage-class > > > > > > The x-amz-meta-* headers are standard user defined headers that we > > > support and test regularly. As for the storage-class header, we still > > > accept it as a valid input, and we sign it as required (just verified it > > > again). > > > With regard to it working against S3, but not against rgw, it is possible > > > that the library is using a different method of authentication against > > > the former (AWS4 vs AWS). It's hard for me to say whether the culprit is > > > the storage-class, it could be something else that we don't see in these > > > logs. In order to be sure we'll need to compare the plain text > > > canonicalized resource string that the library signs. Maybe you could > > > modify its source code and add a debug printing where needed? > > > > > > Yehuda > > > > > > I've managed to enable debugging in the application and have captured the > > plain text string as follows: > > > > 2015-03-31 11:15:17,906 [DEBUG] S3Signer.sign - Calculated string to sign: > > "PUT > > application/octet-stream > > Tue, 31 Mar 2015 10:15:17 GMT > > x-amz-meta-ca_root:CA_CCIFS_C6DCCF63-EC57-45b2-87E7-D9B14B971CA3 > > x-amz-meta-creationtime:2015-03-31T10:15:17 > > x-amz-meta-size:88 > > x-amz-storage-class:STANDARD > > /test1/ca_ccifs_c6dccf63-ec57-45b2-87e7-d9b14b971ca3" > > > > So it looks like we're dropping the > > "x-amz-meta-ca_root:CA_CCIFS_C6DCCF63-EC57-45b2-87E7-D9B14B971CA3" part on > > the rgw side before calculating the auth digest hence why it's failing. > > Any ideas why it would be dropping that part? > > It's the underscore. Apache (2.4+) filters it out. Can you modify it to 'ca-root', or 'caroot' instead? Yehuda _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com