Yea my first thought was a rewrite rule issue. Unfortunately the one you just gave me also ends up with the same result. Here is a log snippet when trying to list containers and create a new container from the openstack folsom dashboard: Mar 4 17:16:02 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:02.965606 7fd895016780 20 enqueued request req=0x7fd8968ca010 Mar 4 17:16:02 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:02.965624 7fd895016780 20 RGWWQ: Mar 4 17:16:02 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:02.965626 7fd895016780 20 req: 0x7fd8968ca010 Mar 4 17:16:02 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:02.965628 7fd895016780 10 allocated request req=0x7fd8968e4c40 Mar 4 17:16:02 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:02.965669 7fd8417fa700 20 dequeued request req=0x7fd8968ca010 Mar 4 17:16:02 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:02.965677 7fd8417fa700 20 RGWWQ: empty Mar 4 17:16:02 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:02.965681 7fd8417fa700 1 ====== starting new request req=0x7fd8968ca010 ===== Mar 4 17:16:02 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:02.965704 7fd8417fa700 2 req 13:0.000023::::initializing Mar 4 17:16:02 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:02.965730 7fd8417fa700 10 s->object=AUTH_2c5cfd34f2af41919847c0f4e0a28e04 s->bucket=v1 Mar 4 17:16:02 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:02.965749 7fd8417fa700 2 req 13:0.000068::GET /v1/AUTH_2c5cfd34f2af41919847c0f4e0a28e04::http status=400 Mar 4 17:16:02 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:02.965802 7fd8417fa700 1 ====== req done req=0x7fd8968ca010 http_status=400 ====== Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.770745 7fd895016780 20 enqueued request req=0x7fd8968e4c40 Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.770755 7fd895016780 20 RGWWQ: Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.770756 7fd895016780 20 req: 0x7fd8968e4c40 Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.770759 7fd895016780 10 allocated request req=0x7fd8968ca010 Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.770790 7fd835fe3700 20 dequeued request req=0x7fd8968e4c40 Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.770796 7fd835fe3700 20 RGWWQ: empty Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.770800 7fd835fe3700 1 ====== starting new request req=0x7fd8968e4c40 ===== Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.770820 7fd835fe3700 2 req 14:0.000021::::initializing Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.770840 7fd835fe3700 10 s->object=AUTH_2c5cfd34f2af41919847c0f4e0a28e04/test s->bucket=v1 Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.770859 7fd835fe3700 2 req 14:0.000060::HEAD /v1/AUTH_2c5cfd34f2af41919847c0f4e0a28e04/test::http status=400 Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.771207 7fd835fe3700 1 ====== req done req=0x7fd8968e4c40 http_status=400 ====== Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.773224 7fd895016780 20 enqueued request req=0x7fd8968ca010 Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.773229 7fd895016780 20 RGWWQ: Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.773231 7fd895016780 20 req: 0x7fd8968ca010 Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.773234 7fd895016780 10 allocated request req=0x7fd8968e77c0 Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.773250 7fd832fdd700 20 dequeued request req=0x7fd8968ca010 Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.773256 7fd832fdd700 20 RGWWQ: empty Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.773259 7fd832fdd700 1 ====== starting new request req=0x7fd8968ca010 ===== Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.773277 7fd832fdd700 2 req 15:0.000019::::initializing Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.773290 7fd832fdd700 10 s->object=AUTH_2c5cfd34f2af41919847c0f4e0a28e04/test s->bucket=v1 Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.773305 7fd832fdd700 2 req 15:0.000047::PUT /v1/AUTH_2c5cfd34f2af41919847c0f4e0a28e04/test::http status=400 Mar 4 17:16:11 node-172-17-0-13 radosgw.log: 2013-03-04 17:16:11.773336 7fd832fdd700 1 ====== req done req=0x7fd8968ca010 http_status=400 ====== On Mon, Mar 4, 2013 at 9:11 AM, Yehuda Sadeh <yehuda@xxxxxxxxxxx> wrote: > On Mon, Mar 4, 2013 at 8:49 AM, Nick Bartos <nick@xxxxxxxxxxxxxxx> wrote: >> I'm trying to figure out what's wrong with my radosgw setup. I am >> getting HTTP 400 responses when running some basic stat/upload tests, >> which work fine when using openstack (folsom) swift. Additionally, I >> get HTTP 400 responses when trying to use the openstack (folsom) >> dashboard to list or create containers. >> >> I'm using the ceph-modified mod_fastcgi, and apache 2.2.23 with the worker mpm. >> >> Here is a snip of the log which starts after I start apache and radosgw: >> >> Mar 4 16:32:48 node-172-17-0-13 fcgi-pm[28606]: [notice] FastCGI: >> process manager initialized (pid 28606) >> Mar 4 16:32:48 node-172-17-0-13 apache2[28605]: [notice] >> Apache/2.2.23 (Unix) mod_fastcgi/mod_fastcgi-SNAP-0910052141 >> configured -- resuming normal operations >> Mar 4 16:32:48 node-172-17-0-13 apache2[28605]: [info] Server built: >> Mar 2 2013 21:17:00 >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:32:53.788234 7fd895016780 0 ceph version 0.56.3 >> (6eb7e15a4783b122e9b0c85ea9ba064145958aa5), process radosgw, pid 28746 >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:32:53.788382 7fd895016780 0 ext_mime_map_init(): failed to open >> file=/etc/mime.types ret=-2 >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:32:53.791035 7fd87f7fe700 2 garbage collection: start >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:32:53.811325 7fd87ebf9700 2 keystone revoke thread: start >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:32:53.811377 7fd87ebf9700 20 sending request to >> http://172.17.2.3:35357/v2.0/tokens/revoked >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:32:53.812816 7fd895016780 10 allocated request req=0x7fd8968df5f0 >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:32:53.824017 7fd87ebf9700 10 request returned {"signed": >> "-----BEGIN CMS-----\nMIIBQwYJKoZIhvcNAQcCoIIBNDCCATACAQExCTAHBgUrDgMCGjAeBgkqhkiG9w0B\nBwGgEQQPeyJyZXZva2VkIjogW119MYH/MIH8AgEBMFwwVzELMAkGA1UEBhMCVVMx\nDjAMBgNVBAgTBVVuc2V0MQ4wDAYDVQQHEwVVbnNldDEOMAwGA1UEChMFVW5zZXQx\nGDAWBgNVBAMTD3d3dy5leGFtcGxlLmNvbQIBATAHBgUrDgMCGjANBgkqhkiG9w0B\nAQEFAASBgC7gYTOTf9SQEZKrk82UJVUUnPHyecpEI2kHb3IZ3iTmnm/IVHMAlNs7\n9Kmz6S9z0jFDWMqHxlnLAVuqzjGJMZFChuDTe4f/lRnHqd3I/ZlckVmAUw0ECAX5\nDmsQ+uziQYqHOiUGxeV0rwOu+x+khrxlbPv9x+2GcnIuEkG11ytr\n-----END >> CMS-----\n"} >> Mar 4 16:32:53 node-172-17-0-13 keystone: wsgi DEBUG 172.17.2.3 - - >> [04/Mar/2013 16:32:53] "GET /v2.0/tokens/revoked HTTP/1.1" 200 633 >> 0.011981 >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: add_child: <- signed >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:32:53.824115 7fd87ebf9700 10 signed=-----BEGIN CMS----- >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: >> MIIBQwYJKoZIhvcNAQcCoIIBNDCCATACAQExCTAHBgUrDgMCGjAeBgkqhkiG9w0B >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: >> BwGgEQQPeyJyZXZva2VkIjogW119MYH/MIH8AgEBMFwwVzELMAkGA1UEBhMCVVMx >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: >> DjAMBgNVBAgTBVVuc2V0MQ4wDAYDVQQHEwVVbnNldDEOMAwGA1UEChMFVW5zZXQx >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: >> GDAWBgNVBAMTD3d3dy5leGFtcGxlLmNvbQIBATAHBgUrDgMCGjANBgkqhkiG9w0B >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: >> AQEFAASBgC7gYTOTf9SQEZKrk82UJVUUnPHyecpEI2kHb3IZ3iTmnm/IVHMAlNs7 >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: >> 9Kmz6S9z0jFDWMqHxlnLAVuqzjGJMZFChuDTe4f/lRnHqd3I/ZlckVmAUw0ECAX5 >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: >> DmsQ+uziQYqHOiUGxeV0rwOu+x+khrxlbPv9x+2GcnIuEkG11ytr >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: -----END CMS----- >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:32:53.824124 7fd87ebf9700 10 >> content=MIIBQwYJKoZIhvcNAQcCoIIBNDCCATACAQExCTAHBgUrDgMCGjAeBgkqhkiG9w0BBwGgEQQPeyJyZXZva2VkIjogW119MYH/MIH8AgEBMFwwVzELMAkGA1UEBhMCVVMxDjAMBgNVBAgTBVVuc2V0MQ4wDAYDVQQHEwVVbnNldDEOMAwGA1UEChMFVW5zZXQxGDAWBgNVBAMTD3d3dy5leGFtcGxlLmNvbQIBATAHBgUrDgMCGjANBgkqhkiG9w0BAQEFAASBgC7gYTOTf9SQEZKrk82UJVUUnPHyecpEI2kHb3IZ3iTmnm/IVHMAlNs79Kmz6S9z0jFDWMqHxlnLAVuqzjGJMZFChuDTe4f/lRnHqd3I/ZlckVmAUw0ECAX5DmsQ+uziQYqHOiUGxeV0rwOu+x+khrxlbPv9x+2GcnIuEkG11ytr >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:32:53.824132 7fd87ebf9700 0 ceph_decode_cms returned -95 >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:32:53.824138 7fd87ebf9700 0 ERROR: keystone revocation processing >> returned error r=-95 >> Mar 4 16:32:53 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:32:53.906447 7fd87f7fe700 2 garbage collection: stop >> Mar 4 16:33:46 node-172-17-0-13 keystone: wsgi DEBUG 10.1.10.12 - - >> [04/Mar/2013 16:33:46] "POST /v2.0/tokens HTTP/1.1" 200 2711 0.084738 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.891625 7fd895016780 20 enqueued request req=0x7fd8968df5f0 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.891641 7fd895016780 20 RGWWQ: >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.891643 7fd895016780 20 req: 0x7fd8968df5f0 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.891646 7fd895016780 10 allocated request req=0x7fd8968e4a50 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.891700 7fd8237be700 20 dequeued request req=0x7fd8968df5f0 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.891709 7fd8237be700 20 RGWWQ: empty >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.891716 7fd8237be700 1 ====== starting new request >> req=0x7fd8968df5f0 ===== >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.891751 7fd8237be700 2 req 1:0.000035::::initializing >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.891786 7fd8237be700 10 >> s->object=AUTH_2c5cfd34f2af41919847c0f4e0a28e04 s->bucket=v1 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.891812 7fd8237be700 2 req 1:0.000096::HEAD >> /v1/AUTH_2c5cfd34f2af41919847c0f4e0a28e04::http status=400 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.891877 7fd8237be700 1 ====== req done req=0x7fd8968df5f0 >> http_status=400 ====== >> Mar 4 16:33:46 node-172-17-0-13 keystone: wsgi DEBUG 10.1.10.12 - - >> [04/Mar/2013 16:33:46] "POST /v2.0/tokens HTTP/1.1" 200 2711 0.078482 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.978717 7fd895016780 20 enqueued request req=0x7fd8968e4a50 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.978726 7fd895016780 20 RGWWQ: >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.978727 7fd895016780 20 req: 0x7fd8968e4a50 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.978730 7fd895016780 10 allocated request req=0x7fd8968e4e80 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.978763 7fd8387e8700 20 dequeued request req=0x7fd8968e4a50 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.978769 7fd8387e8700 20 RGWWQ: empty >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.978773 7fd8387e8700 1 ====== starting new request >> req=0x7fd8968e4a50 ===== >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.978800 7fd8387e8700 2 req 2:0.000028::::initializing >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.978826 7fd8387e8700 10 >> s->object=AUTH_2c5cfd34f2af41919847c0f4e0a28e04/container-33771340493 >> s->bucket=v1 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.978843 7fd8387e8700 2 req 2:0.000071::PUT > > That's wrong. Missing the path, maybe rewrite rule is broken? > > >> /v1/AUTH_2c5cfd34f2af41919847c0f4e0a28e04/container-33771340493::http >> status=400 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.978880 7fd8387e8700 1 ====== req done req=0x7fd8968e4a50 >> http_status=400 ====== >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.982832 7fd895016780 20 enqueued request req=0x7fd8968e4e80 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.982839 7fd895016780 20 RGWWQ: >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.982841 7fd895016780 20 req: 0x7fd8968e4e80 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.982844 7fd895016780 10 allocated request req=0x7fd8968e22c0 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.982881 7fd81cfb1700 20 dequeued request req=0x7fd8968e4e80 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.982891 7fd81cfb1700 20 RGWWQ: empty >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.982894 7fd81cfb1700 1 ====== starting new request >> req=0x7fd8968e4e80 ===== >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.982923 7fd81cfb1700 2 req 3:0.000029::::initializing >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.982941 7fd81cfb1700 10 >> s->object=AUTH_2c5cfd34f2af41919847c0f4e0a28e04/container-33771340493/file-43854197694 >> s->bucket=v1 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.982960 7fd81cfb1700 2 req 3:0.000066::PUT >> /v1/AUTH_2c5cfd34f2af41919847c0f4e0a28e04/container-33771340493/file-43854197694::http >> status=400 >> Mar 4 16:33:46 node-172-17-0-13 radosgw.log: 2013-03-04 >> 16:33:46.982995 7fd81cfb1700 1 ====== req done req=0x7fd8968e4e80 >> http_status=400 ====== >> >> Radosgw is started via upstart using this command: >> setuidgid apache radosgw -n client.radosgw.gateway -f -d >> >> Here is the relevant addition to ceph.conf: >> [client.radosgw.gateway] >> rgw socket path = /var/run/radosgw/radosgw.sock >> rgw keystone url = http://172.17.2.3:35357 >> rgw keystone admin token = vEPetvbcJl5V >> >> Here is the apache config: >> # Defaults. >> ServerRoot /usr/lib64/apache2 >> PidFile /var/run/apache/apache.pid >> HostnameLookups Off >> # This shouldn't need to be anything valid, if we get redirects here then there >> # is an error in our config, as we shouldn't be sending redirects with a >> # different hostname than was provided by the client >> ServerName localhost >> ServerSignature Off >> ServerTokens ProductOnly >> Timeout 60 >> KeepAlive On >> KeepAliveTimeout 5 >> StartServers 2 >> ServerLimit 2 >> ThreadsPerChild 64 >> MinSpareThreads 128 >> MaxSpareThreads 128 >> # For worker MPM, set MaxClients = ServerLimit * ThreadsPerChild >> MaxClients 128 >> MaxRequestsPerChild 10000 >> ErrorLog syslog:daemon >> TraceEnable Off >> LoadModule authz_host_module modules/mod_authz_host.so >> LoadModule rewrite_module modules/mod_rewrite.so >> DocumentRoot /var/www/empty >> LogLevel info >> <Directory /> >> Options None >> AllowOverride None >> Order deny,allow >> Deny from all >> </Directory> >> >> # Rados gateway. >> Listen 8888 >> LoadModule fastcgi_module modules/mod_fastcgi.so >> FastCgiIpcDir /var/run/fastcgi >> # Note: The fcgi file here doesn't have to exist since we start the application >> # externally, but the directory structure should or apache will complain about >> # the document root not existing. >> FastCgiExternalServer /var/www/empty/radosgw.fcgi -socket >> /var/run/radosgw/radosgw.sock >> <Directory /var/www/empty> >> Options +ExecCGI >> SetHandler fastcgi-script >> Order allow,deny >> Allow from all >> </Directory> >> <VirtualHost *:8888> >> RewriteEngine On >> #RewriteRule ^(.*) /radosgw.fcgi/$1 >> RewriteRule ^/([a-zA-Z0-9-_.]*)([/]?.*) >> /radosgw.fcgi?page=$1¶ms=$2&%{QUERY_STRING} >> [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L] >> #RewriteRule ^/(.*) /radosgw.fcgi?params=$1&%{QUERY_STRING} >> [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L] > > This is the one that you should use. Actually, you can try this: > > RewriteRule ^/(.*) /radosgw.fcgi?%{QUERY_STRING} > [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L] > >> AllowEncodedSlashes On > >> </VirtualHost> >> >> I've tried all the rewrite rules above. >> >> We would like to support both the swift and s3 APIs on this service, >> using separate ports if we have to. > > You can have it either way. > > > Yehuda _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |