Do you have your admin keyring in the /etc/ceph directory of your radosgw host? That sounds like step 1 here: http://ceph.com/docs/master/start/quick-rgw/#generate-a-keyring-and-key I think I encountered an issue there myself, and did a sudo chmod 644 on the keyring. On Wed, May 29, 2013 at 1:17 PM, Daniel Curran <danielcurran90@xxxxxxxxx> wrote: > Unfortunately it seems like I messed up yesterday. I didn't have the > client.radosgw.gateway section in my ceph.conf. I don't get the apache > errors now but I still don't have access since the secret_key is still not > being created or at least not showing up. I can try to auth but it just says > 'Auth GET failed: http://192.168.1.100:80/auth/ 403 Forbidden' with > everything I send it. > > This is what I have at the moment in the files you requested. > ceph.conf: > ------------------------------ > -------------------------------------------------------------- > [global] > fsid = 1ec4438a-3f59-4cfd-86b8-a89607401d81 > mon_initial_members = ceph0 > mon_host = 192.168.1.100 > auth_supported = cephx > osd_journal_size = 1024 > filestore_xattr_use_omap = true > > [client.radosgw.gateway] > host = ceph0 > keyring = /etc/ceph/keyring.radosgw.gateway > rgw socket path = /tmp/radosgw.sock > log file = /var/log/ceph/radosgw.log > rgw dns name = ceph0 > -------------------------------------------------------------------------------------------- > > rgw.conf: > -------------------------------------------------------------------------------------------- > FastCgiExternalServer /var/www/s3gw.fcgi -socket /tmp/radosgw.sock > > > <VirtualHost *:80> > ServerName ceph0 > ServerAdmin admin@localhost > DocumentRoot /var/www > > > RewriteEngine On > RewriteRule ^/([a-zA-Z0-9-_.]*)([/]?.*) > /s3gw.fcgi?page=$1¶ms=$2&%{QUERY_STRING} > [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L] > > <IfModule mod_fastcgi.c> > <Directory /var/www> > Options +ExecCGI > AllowOverride All > SetHandler fastcgi-script > Order allow,deny > Allow from all > AuthBasicAuthoritative Off > </Directory> > </IfModule> > > AllowEncodedSlashes On > ErrorLog /var/log/apache2/error.log > CustomLog /var/log/apache2/access.log combined > ServerSignature Off > > </VirtualHost> > -------------------------------------------------------------------------------------------- > > s3gw.fcgi > -------------------------------------------------------------------------------------------- > > #!/bin/sh > exec /usr/bin/radosgw -c /etc/ceph/ceph.conf -n client.radosgw.gateway > -------------------------------------------------------------------------------------------- > > Here's what the /var/log/ceph/radosgw.log says when it denies me: > 2013-05-29 16:00:29.118234 7f5f60cf6700 2 req 11:0.000093:swift-auth:GET > /auth/::getting op > 2013-05-29 16:00:29.118237 7f5f60cf6700 2 req 11:0.000096:swift-auth:GET > /auth/:swift_auth_get:authorizing > 2013-05-29 16:00:29.118239 7f5f60cf6700 2 req 11:0.000098:swift-auth:GET > /auth/:swift_auth_get:reading permissions > 2013-05-29 16:00:29.118243 7f5f60cf6700 2 req 11:0.000102:swift-auth:GET > /auth/:swift_auth_get:reading the cors attr > 2013-05-29 16:00:29.118246 7f5f60cf6700 10 Going to read cors from attrs > 2013-05-29 16:00:29.118248 7f5f60cf6700 2 req 11:0.000107:swift-auth:GET > /auth/:swift_auth_get:verifying op permissions > 2013-05-29 16:00:29.118250 7f5f60cf6700 2 req 11:0.000109:swift-auth:GET > /auth/:swift_auth_get:verifying op params > 2013-05-29 16:00:29.118252 7f5f60cf6700 2 req 11:0.000111:swift-auth:GET > /auth/:swift_auth_get:executing > 2013-05-29 16:00:29.118273 7f5f60cf6700 20 get_obj_state: > rctx=0x7f5efc007630 obj=.users.swift:johndoe:swift state=0x7f5efc00c378 > s->prefetch_data=0 > 2013-05-29 16:00:29.118284 7f5f60cf6700 10 moving .users.swift+johndoe:swift > to cache LRU end > 2013-05-29 16:00:29.118286 7f5f60cf6700 10 cache get: > name=.users.swift+johndoe:swift : hit > 2013-05-29 16:00:29.118292 7f5f60cf6700 20 get_obj_state: s->obj_tag was set > empty > 2013-05-29 16:00:29.118298 7f5f60cf6700 10 moving .users.swift+johndoe:swift > to cache LRU end > 2013-05-29 16:00:29.118300 7f5f60cf6700 10 cache get: > name=.users.swift+johndoe:swift : hit > 2013-05-29 16:00:29.118316 7f5f60cf6700 20 get_obj_state: > rctx=0x7f5efc0071f0 obj=.users.uid:johndoe state=0x7f5efc00c9f8 > s->prefetch_data=0 > 2013-05-29 16:00:29.118321 7f5f60cf6700 10 moving .users.uid+johndoe to > cache LRU end > 2013-05-29 16:00:29.118323 7f5f60cf6700 10 cache get: > name=.users.uid+johndoe : hit > 2013-05-29 16:00:29.118326 7f5f60cf6700 20 get_obj_state: s->obj_tag was set > empty > 2013-05-29 16:00:29.118330 7f5f60cf6700 10 moving .users.uid+johndoe to > cache LRU end > 2013-05-29 16:00:29.118332 7f5f60cf6700 10 cache get: > name=.users.uid+johndoe : hit > 2013-05-29 16:00:29.118358 7f5f60cf6700 0 NOTICE: > RGW_SWIFT_Auth_Get::execute(): bad swift key > 2013-05-29 16:00:29.118380 7f5f60cf6700 2 req 11:0.000238:swift-auth:GET > /auth/:swift_auth_get:http status=403 > 2013-05-29 16:00:29.118504 7f5f60cf6700 1 ====== req done req=0x14d3b30 > http_status=403 ====== > > > Which looks reasonable since I don't have the proper key to send along to > it. Was there a specific log file you wanted to see? > > Thanks, > Daniel > > > On Tue, May 28, 2013 at 6:50 PM, John Wilkins <john.wilkins@xxxxxxxxxxx> > wrote: >> >> Can you paste your ceph.conf file, rgw.conf file, s3gw.fcgi file and >> your log file? >> >> I'm trying to work out a reliable procedure for the S3 subdomain, and >> ran into a a few issues. >> >> On Tue, May 28, 2013 at 4:45 PM, Daniel Curran <danielcurran90@xxxxxxxxx> >> wrote: >> > Hmm....it looks like maybe my issue is bigger than just the swift >> > secret_key >> > not showing up. I started using the other key that's provided under >> > "keys" >> > in the user map just to see if it works but I get 500's back. In my >> > apache >> > error log I get >> > >> > [Tue May 28 15:19:59 2013] [notice] Apache/2.2.22 (Ubuntu) >> > mod_fastcgi/mod_fastcgi-SNAP-0910052141 configured -- resuming normal >> > operations >> > [Tue May 28 15:24:04 2013] [error] [client 192.168.1.100] (2)No such >> > file or >> > directory: FastCGI: failed to connect to server "/var/www/s3gw.fcgi": >> > connect() failed >> > [Tue May 28 15:24:04 2013] [error] [client 192.168.1.100] FastCGI: >> > incomplete headers (0 bytes) received from server "/var/www/s3gw.fcgi" >> > >> > The errors pop up when I try to hit that endpoint. It's strange because >> > it's >> > owned by www-data which apache is using and has r/x permissions for >> > everyone. I'm thinking maybe there's something wrong in my rewrite rule >> > or >> > with the s3gw.fcgi itself. >> > >> > rewrite rules: >> > RewriteEngine On >> > RewriteRule ^/([a-zA-Z0-9-_.]*)([/]?.*) >> > /s3gw.fcgi?page=$1¶ms=$2&%{QUERY_STRING} >> > [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L] >> > >> > contents of /var/www/s3gw.fcgi: >> > #!/bin/sh >> > exec /usr/bin/radosgw -c /etc/ceph/ceph.conf -n client.radosgw.gateway >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > On Tue, May 28, 2013 at 11:39 AM, Daniel Curran >> > <danielcurran90@xxxxxxxxx> >> > wrote: >> >> >> >> Hey, >> >> >> >> I'll check see if I can get it going right now. Thanks for taking the >> >> time >> >> to help out >> >> >> >> Daniel >> >> >> >> >> >> On Sat, May 25, 2013 at 7:01 PM, John Wilkins >> >> <john.wilkins@xxxxxxxxxxx> >> >> wrote: >> >>> >> >>> Daniel, >> >>> >> >>> I finished up the procedure this morning and put it up on the web. It >> >>> seems to be working fine for me. >> >>> >> >>> On Fri, May 24, 2013 at 3:35 PM, John Wilkins >> >>> <john.wilkins@xxxxxxxxxxx> >> >>> wrote: >> >>> > Daniel, >> >>> > >> >>> > I'm working on it. It did need a bit of a rewrite for ceph-deploy, >> >>> > even though the steps are conceptually the same. In the step here: >> >>> > >> >>> > >> >>> > >> >>> > http://ceph.com/docs/master/start/quick-rgw/#modify-the-ceph-configuration-file >> >>> > >> >>> > 1. The rewrite--not published yet, as I'm working out a few more >> >>> > kinks--is to use the ceph-deploy version of the ceph conf file on >> >>> > the >> >>> > admin node. So you make the modification to the configuration file >> >>> > there; then, use "ceph-deploy config push" to push it to the server >> >>> > host. >> >>> > >> >>> > 2. Create a directory is on the server node, as before. >> >>> > >> >>> > 3. The rewrite rule there is outside of the <VirtualHost> tags, and >> >>> > there are two instances of <VirtualHost>. Remove the first end, and >> >>> > second beginning of the </VirtualHost><VirtualHost *:80>, but leave >> >>> > the rewrite rule. That was a bug. Sorry. >> >>> > >> >>> > 4. Then, create the keyring here: >> >>> > >> >>> > http://ceph.com/docs/master/start/quick-rgw/#generate-a-keyring-and-key >> >>> > It wouldn't have worked before, because you didn't have the right >> >>> > keyring in /etc/ceph. The third step in that procedure says >> >>> > ceph.keyring. That's no longer correct. By convention with >> >>> > ceph-deploy, the key ring is ceph.client.admin.keyring. >> >>> > >> >>> > 5. http://ceph.com/docs/master/start/quick-rgw/#enable-ssl Do this >> >>> > before creating the users if you want to use SSL. >> >>> > >> >>> > 6. Restart ceph, apache, and radosgw as the root user. Then, >> >>> > proceed >> >>> > to creating a user. >> >>> > >> >>> > I'll add some examples when I get all the quirks ironed out so you >> >>> > can >> >>> > just cut and paste. >> >>> > >> >>> > Regards, >> >>> > >> >>> > >> >>> > John >> >>> > >> >>> > >> >>> > On Fri, May 24, 2013 at 2:22 PM, Daniel Curran >> >>> > <danielcurran90@xxxxxxxxx> wrote: >> >>> >> >> >>> >> >> >>> >> ---------- Forwarded message ---------- >> >>> >> From: Daniel Curran <danielcurran90@xxxxxxxxx> >> >>> >> Date: Fri, May 24, 2013 at 4:21 PM >> >>> >> Subject: Re: RADOS Gateway Configuration >> >>> >> To: John Wilkins <john.wilkins@xxxxxxxxxxx> >> >>> >> >> >>> >> >> >>> >> Hey, >> >>> >> >> >>> >> I just finished the 5-minute quick start for rados that was given >> >>> >> earlier >> >>> >> but I get the same issue. >> >>> >> >> >>> >> >> >>> >> { "user_id": "johndoe", >> >>> >> "display_name": "John Doe", >> >>> >> "email": "john@xxxxxxxxxxx", >> >>> >> "suspended": 0, >> >>> >> "max_buckets": 1000, >> >>> >> "auid": 0, >> >>> >> "subusers": [ >> >>> >> { "id": "johndoe:swift", >> >>> >> "permissions": "full-control"}], >> >>> >> "keys": [ >> >>> >> { "user": "johndoe", >> >>> >> "access_key": "ZM3S10OX6SPBDPGZC19J", >> >>> >> "secret_key": >> >>> >> "AhZlWa1cLkff\/v0NLnMb7HGIAcPu3lMgM8CTKxey"}], >> >>> >> >> >>> >> "swift_keys": [ >> >>> >> { "user": "johndoe:swift", >> >>> >> "secret_key": ""}], >> >>> >> "caps": []} >> >>> >> >> >>> >> with no secret_key under "swift_keys" for me to pass along. >> >>> >> Regenerating the >> >>> >> keys doesn't do anything different. I've also tried using the >> >>> >> secret_key >> >>> >> under "keys" but that doesn't work either. Any thoughts? >> >>> >> >> >>> >> >> >>> >> Thanks your help, >> >>> >> Daniel >> >>> >> >> >>> >> >> >>> >> >> >>> >> >> >>> >> On Thu, May 23, 2013 at 1:28 PM, John Wilkins >> >>> >> <john.wilkins@xxxxxxxxxxx> >> >>> >> wrote: >> >>> >>> >> >>> >>> It puts it in the same directory where you executed ceph-deploy. >> >>> >>> >> >>> >>> On Thu, May 23, 2013 at 10:57 AM, Daniel Curran >> >>> >>> <danielcurran90@xxxxxxxxx> wrote: >> >>> >>> > Hey John, >> >>> >>> > >> >>> >>> > Thanks for the reply. I'll check out that other doc you have >> >>> >>> > there. >> >>> >>> > Just >> >>> >>> > for >> >>> >>> > future reference do you know where ceph-deploy puts the ceph >> >>> >>> > keyring? >> >>> >>> > >> >>> >>> > Daniel >> >>> >>> > >> >>> >>> > >> >>> >>> > On Wed, May 22, 2013 at 7:19 PM, John Wilkins >> >>> >>> > <john.wilkins@xxxxxxxxxxx> >> >>> >>> > wrote: >> >>> >>> >> >> >>> >>> >> Daniel, >> >>> >>> >> >> >>> >>> >> It looks like I need to update that portion of the docs too, as >> >>> >>> >> it >> >>> >>> >> links back to the 5-minute quick start. Once you are up and >> >>> >>> >> running >> >>> >>> >> with "HEALTH OK" on either the 5-minute Quick Start or Quick >> >>> >>> >> Ceph >> >>> >>> >> Deploy, your storage cluster is running fine. The remaining >> >>> >>> >> issues >> >>> >>> >> would likely be with authentication, chmod on the files, or >> >>> >>> >> with >> >>> >>> >> the >> >>> >>> >> RGW setup. There's a quick start for RGW, which I had verified >> >>> >>> >> here: >> >>> >>> >> http://ceph.com/docs/master/start/quick-rgw/. Someone else had >> >>> >>> >> a >> >>> >>> >> problem with the Rewrite rule on that example reported here: >> >>> >>> >> http://tracker.ceph.com/issues/4608. It's likely I need to run >> >>> >>> >> through >> >>> >>> >> with specific Ceph and Apache versions. There are also a few >> >>> >>> >> additional tips in the configuration section. >> >>> >>> >> http://ceph.com/docs/master/radosgw/config/ >> >>> >>> >> >> >>> >>> >> There is an issue in some cases where keys have forward or >> >>> >>> >> backslash >> >>> >>> >> characters, and you may need to regenerate the keys. >> >>> >>> >> >> >>> >>> >> >> >>> >>> >> >> >>> >>> >> On Wed, May 22, 2013 at 4:42 PM, Daniel Curran >> >>> >>> >> <danielcurran90@xxxxxxxxx> >> >>> >>> >> wrote: >> >>> >>> >> > >> >>> >>> >> > Hello, >> >>> >>> >> > >> >>> >>> >> > I just started using ceph recently and was trying to get the >> >>> >>> >> > RADOS >> >>> >>> >> > Gateway >> >>> >>> >> > working in order to use the Swift compatible API. I followed >> >>> >>> >> > the >> >>> >>> >> > install >> >>> >>> >> > instructions found here (http://ceph.com/docs/master >> >>> >>> >> > /start/quick-ceph-deploy/) and got to a point where "ceph >> >>> >>> >> > health" >> >>> >>> >> > give >> >>> >>> >> > me >> >>> >>> >> > HEALTH_OK. This is all well and good but near the end of the >> >>> >>> >> > rados gw >> >>> >>> >> > setup >> >>> >>> >> > (found here >> >>> >>> >> > http://ceph.com/docs/master/radosgw/manual-install/) >> >>> >>> >> > I >> >>> >>> >> > need >> >>> >>> >> > to >> >>> >>> >> > execute the following line: >> >>> >>> >> > >> >>> >>> >> > sudo ceph -k /etc/ceph/ceph.keyring auth add >> >>> >>> >> > client.radosgw.gateway >> >>> >>> >> > -i >> >>> >>> >> > /etc/ceph/keyring.radosgw.gateway >> >>> >>> >> > >> >>> >>> >> > Unfortunately, I don't believe ceph-deploy places the keyring >> >>> >>> >> > at >> >>> >>> >> > /etc/ceph/ceph.keyring. I tried to use the one from >> >>> >>> >> > /var/lib/ceph/bootstrap-osd/ceph.keyring but it was unable to >> >>> >>> >> > authenticate >> >>> >>> >> > as client.admin. Is there another location that the keyring >> >>> >>> >> > needs to >> >>> >>> >> > be >> >>> >>> >> > copied from or am I doing something totally wrong? >> >>> >>> >> > >> >>> >>> >> > I didn't want to be held back so I restarted and did the >> >>> >>> >> > manual >> >>> >>> >> > install >> >>> >>> >> > from >> >>> >>> >> > the 5-minute quick start where I was able to find the ring. I >> >>> >>> >> > had >> >>> >>> >> > more >> >>> >>> >> > issues almost immediately. I have to execute the following >> >>> >>> >> > steps >> >>> >>> >> > to >> >>> >>> >> > create >> >>> >>> >> > some users for swift: >> >>> >>> >> > >> >>> >>> >> > radosgw-admin user create --uid=johndoe --display-name="John >> >>> >>> >> > Doe" >> >>> >>> >> > --email=john@xxxxxxxxxxx >> >>> >>> >> > sudo radosgw-admin subuser create --uid=johndoe >> >>> >>> >> > --subuser=johndoe:swift >> >>> >>> >> > --access=full >> >>> >>> >> > >> >>> >>> >> > sudo radosgw-admin key create --subuser=johndoe:swift >> >>> >>> >> > --key-type=swift >> >>> >>> >> > >> >>> >>> >> > The first two gave me output I was expecting but the very >> >>> >>> >> > last >> >>> >>> >> > line >> >>> >>> >> > had >> >>> >>> >> > some >> >>> >>> >> > weirdness that essentially made swift unusable. The expected >> >>> >>> >> > output >> >>> >>> >> > is >> >>> >>> >> > something along these lines: >> >>> >>> >> > >> >>> >>> >> > { "user_id": "johndoe", >> >>> >>> >> > "rados_uid": 0, >> >>> >>> >> > "display_name": "John Doe", >> >>> >>> >> > "email": "john@xxxxxxxxxxx", >> >>> >>> >> > "suspended": 0, >> >>> >>> >> > "subusers": [ >> >>> >>> >> > { "id": "johndoe:swift", >> >>> >>> >> > "permissions": "full-control"}], >> >>> >>> >> > "keys": [ >> >>> >>> >> > { "user": "johndoe", >> >>> >>> >> > "access_key": "QFAMEDSJP5DEKJO0DDXY", >> >>> >>> >> > "secret_key": >> >>> >>> >> > "iaSFLDVvDdQt6lkNzHyW4fPLZugBAI1g17LO0+87"}], >> >>> >>> >> > "swift_keys": [ >> >>> >>> >> > { "user": "johndoe:swift", >> >>> >>> >> > "secret_key": >> >>> >>> >> > "E9T2rUZNu2gxUjcwUBO8n\/Ev4KX6\/GprEuH4qhu1"}]} >> >>> >>> >> > >> >>> >>> >> > Where that last secret key is what we hand the swift CLI as >> >>> >>> >> > seen >> >>> >>> >> > here: >> >>> >>> >> > >> >>> >>> >> > swift -V 1.0 -A http://radosgw.example.com/auth -U >> >>> >>> >> > johndoe:swift >> >>> >>> >> > -K >> >>> >>> >> > E9T2rUZNu2gxUjcwUBO8n\/Ev4KX6\/GprEuH4qhu1 post test >> >>> >>> >> > >> >>> >>> >> > However, my output came out like this: >> >>> >>> >> > >> >>> >>> >> > { "user_id": "johndoe", >> >>> >>> >> > "display_name": "John Doe", >> >>> >>> >> > "email": "john@xxxxxxxxxxx", >> >>> >>> >> > "suspended": 0, >> >>> >>> >> > "max_buckets: 1000, >> >>> >>> >> > "auid": 0, >> >>> >>> >> > "subusers": [ >> >>> >>> >> > { "id": "johndoe:swift", >> >>> >>> >> > "permissions": "full-control"}], >> >>> >>> >> > "keys": [ >> >>> >>> >> > { "user": "johndoe", >> >>> >>> >> > "access_key": "SUEXWVL3WB2Z64CRAG97", >> >>> >>> >> > "secret_key": >> >>> >>> >> > "C\/jHFJ3wdPv4iJ+aq4JeZ52LEC3OdnhsYEnVkhBP"}], >> >>> >>> >> > "swift_keys": [ >> >>> >>> >> > { "user": "johndoe:swift", >> >>> >>> >> > "secret_key": ""}], >> >>> >>> >> > "caps": []} >> >>> >>> >> > >> >>> >>> >> > >> >>> >>> >> > Giving me no swift key to use. I don't believe the key is >> >>> >>> >> > supposed to >> >>> >>> >> > be >> >>> >>> >> > blank because I tried that and received auth errors (to the >> >>> >>> >> > best >> >>> >>> >> > of >> >>> >>> >> > my >> >>> >>> >> > ability). I can't tell if this is my fault since I'm new nor >> >>> >>> >> > am >> >>> >>> >> > I >> >>> >>> >> > able >> >>> >>> >> > to >> >>> >>> >> > find a way around it. It looks like there are definitely >> >>> >>> >> > changes >> >>> >>> >> > between >> >>> >>> >> > the >> >>> >>> >> > version used in the doc and mine so maybe it's all working as >> >>> >>> >> > it >> >>> >>> >> > should >> >>> >>> >> > but >> >>> >>> >> > the secret_key for swift lives somewhere else. If anyone >> >>> >>> >> > knows >> >>> >>> >> > anything >> >>> >>> >> > I'd >> >>> >>> >> > appreciate it a lot. >> >>> >>> >> > >> >>> >>> >> > Thank you, >> >>> >>> >> > Daniel >> >>> >>> >> > >> >>> >>> >> > >> >>> >>> >> > >> >>> >>> >> > _______________________________________________ >> >>> >>> >> > ceph-users mailing list >> >>> >>> >> > ceph-users@xxxxxxxxxxxxxx >> >>> >>> >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> >>> >>> >> > >> >>> >>> >> >> >>> >>> >> >> >>> >>> >> >> >>> >>> >> -- >> >>> >>> >> John Wilkins >> >>> >>> >> Senior Technical Writer >> >>> >>> >> Intank >> >>> >>> >> john.wilkins@xxxxxxxxxxx >> >>> >>> >> (415) 425-9599 >> >>> >>> >> http://inktank.com >> >>> >>> > >> >>> >>> > >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> -- >> >>> >>> John Wilkins >> >>> >>> Senior Technical Writer >> >>> >>> Intank >> >>> >>> john.wilkins@xxxxxxxxxxx >> >>> >>> (415) 425-9599 >> >>> >>> http://inktank.com >> >>> >> >> >>> >> >> >>> >> >> >>> > >> >>> > >> >>> > >> >>> > -- >> >>> > John Wilkins >> >>> > Senior Technical Writer >> >>> > Intank >> >>> > john.wilkins@xxxxxxxxxxx >> >>> > (415) 425-9599 >> >>> > http://inktank.com >> >>> >> >>> >> >>> >> >>> -- >> >>> John Wilkins >> >>> Senior Technical Writer >> >>> Intank >> >>> john.wilkins@xxxxxxxxxxx >> >>> (415) 425-9599 >> >>> http://inktank.com >> >> >> >> >> > >> >> >> >> -- >> John Wilkins >> Senior Technical Writer >> Intank >> john.wilkins@xxxxxxxxxxx >> (415) 425-9599 >> http://inktank.com > > -- John Wilkins Senior Technical Writer Intank john.wilkins@xxxxxxxxxxx (415) 425-9599 http://inktank.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |