On Fri, Nov 1, 2013 at 11:12 PM, Sage Weil <sage@xxxxxxxxxxx> wrote: > On Sat, 2 Nov 2013, Trivedi, Narendra wrote: >> >> Hi Sage, >> >> I believe I issued a "ceph-deploy install..." from the admin node as per the >> documentation and that was almost ok as per the output of the command below >> except sometimes there?s an error followed by an ?OK? message (see the >> highlighted item in the red below). I eventually ran into some permission >> issues but seems things went okay: Maybe what can be confusing here is that ceph-deploy interprets stderr as ERROR logging level. Unfortunately, some tools will output normal informative data to stderr when they are clearly not errors. stdout, on the other hand, is interpreted by ceph-deploy as DEBUG level, so you will see logging at that level too. There is no way for ceph-deploy to tell if you are actually seeing errors because the tool is in fact sending error messages or because it decided to use stderr to send information that should go to stdout. > > Hmm, the below output makes it look like it was successfully installed on > node1 node2 and node3. Can you confirm that /etc/ceph exists on all three > of those hosts? > > Oh, looking back at your original message, it looks like you are trying to > create OSDs on /tmp/osd*. I would create directories like /ceph/osdo, > /ceph/osd1, or similar. I believe you need to create the directories > beforehand, too. (In a normal deployment, you are either feeding ceph raw > disks (/dev/XXX) or an existing mount point on a dedicated disk you > already configured and mounted.) > > sage > > > > >> >> >> [ceph@ceph-admin-node-centos-6-4 my-cluster]$ ceph-deploy install >> ceph-node1-mon-centos-6-4 ceph-node2-osd0-centos-6-4 >> ceph-node3-osd1-centos-6-4 >> >> [ceph_deploy.cli][INFO ] Invoked (1.3): /usr/bin/ceph-deploy install >> ceph-node1-mon-centos-6-4 ceph-node2-osd0-centos-6-4 >> ceph-node3-osd1-centos-6-4 >> >> [ceph_deploy.install][DEBUG ] Installing stable version dumpling on cluster >> ceph hosts ceph-node1-mon-centos-6-4 ceph-node2-osd0-centos-6-4 >> ceph-node3-osd1-centos-6-4 >> >> [ceph_deploy.install][DEBUG ] Detecting platform for host >> ceph-node1-mon-centos-6-4 ... >> >> [ceph-node1-mon-centos-6-4][DEBUG ] connected to host: >> ceph-node1-mon-centos-6-4 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] detect platform information from remote >> host >> >> [ceph-node1-mon-centos-6-4][DEBUG ] detect machine type >> >> [ceph_deploy.install][INFO ] Distro info: CentOS 6.4 Final >> >> [ceph-node1-mon-centos-6-4][INFO ] installing ceph on >> ceph-node1-mon-centos-6-4 >> >> [ceph-node1-mon-centos-6-4][INFO ] adding EPEL repository >> >> [ceph-node1-mon-centos-6-4][INFO ] Running command: sudo wget >> http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm >> >> [ceph-node1-mon-centos-6-4][ERROR ] --2013-11-01 19:51:20-- >> http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm >> >> [ceph-node1-mon-centos-6-4][ERROR ] Connecting to 10.12.132.208:8080... >> connected. >> >> [ceph-node1-mon-centos-6-4][ERROR ] Proxy request sent, awaiting response... >> 200 OK >> >> [ceph-node1-mon-centos-6-4][ERROR ] Length: 14540 (14K) [application/x-rpm] >> >> [ceph-node1-mon-centos-6-4][ERROR ] Saving to: >> `epel-release-6-8.noarch.rpm.2' >> >> [ceph-node1-mon-centos-6-4][ERROR ] >> >> [ceph-node1-mon-centos-6-4][ERROR ] 0K .......... >> .... 100% 4.79M=0.003s >> >> [ceph-node1-mon-centos-6-4][ERROR ] >> >> [ceph-node1-mon-centos-6-4][ERROR ] Last-modified header invalid -- >> time-stamp ignored. >> >> [ceph-node1-mon-centos-6-4][ERROR ] 2013-11-01 19:52:20 (4.79 MB/s) - >> `epel-release-6-8.noarch.rpm.2' saved [14540/14540] >> >> [ceph-node1-mon-centos-6-4][ERROR ] >> >> [ceph-node1-mon-centos-6-4][INFO ] Running command: sudo rpm -Uvh >> --replacepkgs epel-release-6*.rpm >> >> [ceph-node1-mon-centos-6-4][DEBUG ] Preparing... >> ################################################## >> >> [ceph-node1-mon-centos-6-4][DEBUG ] epel-release >> ################################################## >> >> [ceph-node1-mon-centos-6-4][INFO ] Running command: sudo rpm --import >> https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc >> >> [ceph-node1-mon-centos-6-4][INFO ] Running command: sudo rpm -Uvh >> --replacepkgs >> http://ceph.com/rpm-dumpling/el6/noarch/ceph-release-1-0.el6.noarch.rpm >> >> [ceph-node1-mon-centos-6-4][DEBUG ] Retrieving >> http://ceph.com/rpm-dumpling/el6/noarch/ceph-release-1-0.el6.noarch.rpm >> >> [ceph-node1-mon-centos-6-4][DEBUG ] Preparing... >> ################################################## >> >> [ceph-node1-mon-centos-6-4][DEBUG ] ceph-release >> ################################################## >> >> [ceph-node1-mon-centos-6-4][INFO ] Running command: sudo yum -y -q install >> ceph >> >> [ceph-node1-mon-centos-6-4][DEBUG ] Package ceph-0.67.4-0.el6.x86_64 already >> installed and latest version >> >> [ceph-node1-mon-centos-6-4][INFO ] Running command: sudo ceph --version >> >> [ceph-node1-mon-centos-6-4][DEBUG ] ceph version 0.67.4 >> (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7) >> >> [ceph_deploy.install][DEBUG ] Detecting platform for host >> ceph-node2-osd0-centos-6-4 ... >> >> [ceph-node2-osd0-centos-6-4][DEBUG ] connected to host: >> ceph-node2-osd0-centos-6-4 >> >> [ceph-node2-osd0-centos-6-4][DEBUG ] detect platform information from remote >> host >> >> [ceph-node2-osd0-centos-6-4][DEBUG ] detect machine type >> >> [ceph_deploy.install][INFO ] Distro info: CentOS 6.4 Final >> >> [ceph-node2-osd0-centos-6-4][INFO ] installing ceph on >> ceph-node2-osd0-centos-6-4 >> >> [ceph-node2-osd0-centos-6-4][INFO ] adding EPEL repository >> >> [ceph-node2-osd0-centos-6-4][INFO ] Running command: sudo wget >> http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm >> >> [ceph-node2-osd0-centos-6-4][ERROR ] --2013-11-01 19:52:54-- >> http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm >> >> [ceph-node2-osd0-centos-6-4][ERROR ] Connecting to 10.12.132.208:8080... >> connected. >> >> [ceph-node2-osd0-centos-6-4][ERROR ] Proxy request sent, awaiting >> response... 200 OK >> >> [ceph-node2-osd0-centos-6-4][ERROR ] Length: 14540 (14K) [application/x-rpm] >> >> [ceph-node2-osd0-centos-6-4][ERROR ] Saving to: >> `epel-release-6-8.noarch.rpm.1' >> >> [ceph-node2-osd0-centos-6-4][ERROR ] >> >> [ceph-node2-osd0-centos-6-4][ERROR ] 0K .......... >> .... 100% 5.56M=0.002s >> >> [ceph-node2-osd0-centos-6-4][ERROR ] >> >> [ceph-node2-osd0-centos-6-4][ERROR ] Last-modified header invalid -- >> time-stamp ignored. >> >> [ceph-node2-osd0-centos-6-4][ERROR ] 2013-11-01 19:53:54 (5.56 MB/s) - >> `epel-release-6-8.noarch.rpm.1' saved [14540/14540] >> >> [ceph-node2-osd0-centos-6-4][ERROR ] >> >> [ceph-node2-osd0-centos-6-4][INFO ] Running command: sudo rpm -Uvh >> --replacepkgs epel-release-6*.rpm >> >> [ceph-node2-osd0-centos-6-4][DEBUG ] Preparing... >> ################################################## >> >> [ceph-node2-osd0-centos-6-4][DEBUG ] epel-release >> ################################################## >> >> [ceph-node2-osd0-centos-6-4][INFO ] Running command: sudo rpm --import >> https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc >> >> [ceph-node2-osd0-centos-6-4][INFO ] Running command: sudo rpm -Uvh >> --replacepkgs >> http://ceph.com/rpm-dumpling/el6/noarch/ceph-release-1-0.el6.noarch.rpm >> >> [ceph-node2-osd0-centos-6-4][DEBUG ] Retrieving >> http://ceph.com/rpm-dumpling/el6/noarch/ceph-release-1-0.el6.noarch.rpm >> >> [ceph-node2-osd0-centos-6-4][DEBUG ] Preparing... >> ################################################## >> >> [ceph-node2-osd0-centos-6-4][DEBUG ] ceph-release >> ################################################## >> >> [ceph-node2-osd0-centos-6-4][INFO ] Running command: sudo yum -y -q install >> ceph >> >> [ceph-node2-osd0-centos-6-4][DEBUG ] Package ceph-0.67.4-0.el6.x86_64 >> already installed and latest version >> >> [ceph-node2-osd0-centos-6-4][INFO ] Running command: sudo ceph --version >> >> [ceph-node2-osd0-centos-6-4][DEBUG ] ceph version 0.67.4 >> (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7) >> >> [ceph_deploy.install][DEBUG ] Detecting platform for host >> ceph-node3-osd1-centos-6-4 ... >> >> [ceph-node3-osd1-centos-6-4][DEBUG ] connected to host: >> ceph-node3-osd1-centos-6-4 >> >> [ceph-node3-osd1-centos-6-4][DEBUG ] detect platform information from remote >> host >> >> [ceph-node3-osd1-centos-6-4][DEBUG ] detect machine type >> >> [ceph_deploy.install][INFO ] Distro info: CentOS 6.4 Final >> >> [ceph-node3-osd1-centos-6-4][INFO ] installing ceph on >> ceph-node3-osd1-centos-6-4 >> >> [ceph-node3-osd1-centos-6-4][INFO ] adding EPEL repository >> >> [ceph-node3-osd1-centos-6-4][INFO ] Running command: sudo wget >> http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm >> >> [ceph-node3-osd1-centos-6-4][ERROR ] --2013-11-01 19:54:17-- >> http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm >> >> [ceph-node3-osd1-centos-6-4][ERROR ] Connecting to 10.12.132.208:8080... >> connected. >> >> [ceph-node3-osd1-centos-6-4][ERROR ] Proxy request sent, awaiting >> response... 200 OK >> >> [ceph-node3-osd1-centos-6-4][ERROR ] Length: 14540 (14K) [application/x-rpm] >> >> [ceph-node3-osd1-centos-6-4][ERROR ] Saving to: >> `epel-release-6-8.noarch.rpm.1' >> >> [ceph-node3-osd1-centos-6-4][ERROR ] >> >> [ceph-node3-osd1-centos-6-4][ERROR ] 0K .......... >> .... 100% 5.37M=0.003s >> >> [ceph-node3-osd1-centos-6-4][ERROR ] >> >> [ceph-node3-osd1-centos-6-4][ERROR ] Last-modified header invalid -- >> time-stamp ignored. >> >> [ceph-node3-osd1-centos-6-4][ERROR ] 2013-11-01 19:55:16 (5.37 MB/s) - >> `epel-release-6-8.noarch.rpm.1' saved [14540/14540] >> >> [ceph-node3-osd1-centos-6-4][ERROR ] >> >> [ceph-node3-osd1-centos-6-4][INFO ] Running command: sudo rpm -Uvh >> --replacepkgs epel-release-6*.rpm >> >> [ceph-node3-osd1-centos-6-4][DEBUG ] Preparing... >> ################################################## >> >> [ceph-node3-osd1-centos-6-4][DEBUG ] epel-release >> ################################################## >> >> [ceph-node3-osd1-centos-6-4][INFO ] Running command: sudo rpm --import >> https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc >> >> [ceph-node3-osd1-centos-6-4][INFO ] Running command: sudo rpm -Uvh >> --replacepkgs >> http://ceph.com/rpm-dumpling/el6/noarch/ceph-release-1-0.el6.noarch.rpm >> >> [ceph-node3-osd1-centos-6-4][DEBUG ] Retrieving >> http://ceph.com/rpm-dumpling/el6/noarch/ceph-release-1-0.el6.noarch.rpm >> >> [ceph-node3-osd1-centos-6-4][DEBUG ] Preparing... >> ################################################## >> >> [ceph-node3-osd1-centos-6-4][DEBUG ] ceph-release >> ################################################## >> >> [ceph-node3-osd1-centos-6-4][INFO ] Running command: sudo yum -y -q install >> ceph >> >> [ceph-node3-osd1-centos-6-4][DEBUG ] Package ceph-0.67.4-0.el6.x86_64 >> already installed and latest version >> >> [ceph-node3-osd1-centos-6-4][INFO ] Running command: sudo ceph --version >> >> [ceph-node3-osd1-centos-6-4][DEBUG ] ceph version 0.67.4 >> (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7) >> >> [ceph@ceph-admin-node-centos-6-4 my-cluster]$ ceph-deploy mon create >> ceph-node1-mon-centos-6-4 >> >> [ceph_deploy.cli][INFO ] Invoked (1.3): /usr/bin/ceph-deploy mon create >> ceph-node1-mon-centos-6-4 >> >> [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts >> ceph-node1-mon-centos-6-4 >> >> [ceph_deploy.mon][DEBUG ] detecting platform for host >> ceph-node1-mon-centos-6-4 ... >> >> [ceph-node1-mon-centos-6-4][DEBUG ] connected to host: >> ceph-node1-mon-centos-6-4 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] detect platform information from remote >> host >> >> [ceph-node1-mon-centos-6-4][DEBUG ] detect machine type >> >> [ceph_deploy.mon][INFO ] distro info: CentOS 6.4 Final >> >> [ceph-node1-mon-centos-6-4][DEBUG ] determining if provided host has same >> hostname in remote >> >> [ceph-node1-mon-centos-6-4][DEBUG ] get remote short hostname >> >> [ceph-node1-mon-centos-6-4][DEBUG ] deploying mon to >> ceph-node1-mon-centos-6-4 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] get remote short hostname >> >> [ceph-node1-mon-centos-6-4][DEBUG ] remote hostname: >> ceph-node1-mon-centos-6-4 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] write cluster configuration to >> /etc/ceph/{cluster}.conf >> >> [ceph_deploy.mon][ERROR ] RuntimeError: config file /etc/ceph/ceph.conf >> exists with different content; use --overwrite-conf to overwrite >> >> [ceph_deploy][ERROR ] GenericError: Failed to create 1 monitors >> >> [ceph@ceph-admin-node-centos-6-4 my-cluster]$ ceph-deploy mon create >> ceph-node1-mon-centos-6-4 >> >> [ceph_deploy.cli][INFO ] Invoked (1.3): /usr/bin/ceph-deploy mon create >> ceph-node1-mon-centos-6-4 >> >> [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts >> ceph-node1-mon-centos-6-4 >> >> [ceph_deploy.mon][DEBUG ] detecting platform for host >> ceph-node1-mon-centos-6-4 ... >> >> [ceph-node1-mon-centos-6-4][DEBUG ] connected to host: >> ceph-node1-mon-centos-6-4 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] detect platform information from remote >> host >> >> [ceph-node1-mon-centos-6-4][DEBUG ] detect machine type >> >> [ceph_deploy.mon][INFO ] distro info: CentOS 6.4 Final >> >> [ceph-node1-mon-centos-6-4][DEBUG ] determining if provided host has same >> hostname in remote >> >> [ceph-node1-mon-centos-6-4][DEBUG ] get remote short hostname >> >> [ceph-node1-mon-centos-6-4][DEBUG ] deploying mon to >> ceph-node1-mon-centos-6-4 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] get remote short hostname >> >> [ceph-node1-mon-centos-6-4][DEBUG ] remote hostname: >> ceph-node1-mon-centos-6-4 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] write cluster configuration to >> /etc/ceph/{cluster}.conf >> >> [ceph-node1-mon-centos-6-4][DEBUG ] create the mon path if it does not exist >> >> [ceph-node1-mon-centos-6-4][DEBUG ] checking for done path: >> /var/lib/ceph/mon/ceph-ceph-node1-mon-centos-6-4/done >> >> [ceph-node1-mon-centos-6-4][DEBUG ] done path does not exist: >> /var/lib/ceph/mon/ceph-ceph-node1-mon-centos-6-4/done >> >> [ceph-node1-mon-centos-6-4][INFO ] creating tmp path: /var/lib/ceph/tmp >> >> [ceph-node1-mon-centos-6-4][INFO ] creating keyring file: >> /var/lib/ceph/tmp/ceph-ceph-node1-mon-centos-6-4.mon.keyring >> >> [ceph-node1-mon-centos-6-4][DEBUG ] create the monitor keyring file >> >> [ceph-node1-mon-centos-6-4][INFO ] Running command: sudo ceph-mon --cluster >> ceph --mkfs -i ceph-node1-mon-centos-6-4 --keyring >> /var/lib/ceph/tmp/ceph-ceph-node1-mon-centos-6-4.mon.keyring >> >> [ceph-node1-mon-centos-6-4][DEBUG ] ceph-mon: mon.noname-a 10.12.0.70:6789/0 >> is local, renaming to mon.ceph-node1-mon-centos-6-4 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] ceph-mon: set fsid to >> c732fc5f-a656-401a-a8e5-4bfed1f89d20 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] ceph-mon: created monfs at >> /var/lib/ceph/mon/ceph-ceph-node1-mon-centos-6-4 for >> mon.ceph-node1-mon-centos-6-4 >> >> [ceph-node1-mon-centos-6-4][INFO ] unlinking keyring file >> /var/lib/ceph/tmp/ceph-ceph-node1-mon-centos-6-4.mon.keyring >> >> [ceph-node1-mon-centos-6-4][DEBUG ] create a done file to avoid re-doing the >> mon deployment >> >> [ceph-node1-mon-centos-6-4][DEBUG ] create the init path if it does not >> exist >> >> [ceph-node1-mon-centos-6-4][DEBUG ] locating the `service` executable... >> >> [ceph-node1-mon-centos-6-4][INFO ] Running command: sudo /sbin/service ceph >> -c /etc/ceph/ceph.conf start mon.ceph-node1-mon-centos-6-4 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] === mon.ceph-node1-mon-centos-6-4 === >> >> [ceph-node1-mon-centos-6-4][DEBUG ] Starting Ceph >> mon.ceph-node1-mon-centos-6-4 on ceph-node1-mon-centos-6-4... >> >> [ceph-node1-mon-centos-6-4][DEBUG ] Starting ceph-create-keys on >> ceph-node1-mon-centos-6-4... >> >> [ceph-node1-mon-centos-6-4][INFO ] Running command: sudo ceph >> --cluster=ceph --admin-daemon >> /var/run/ceph/ceph-mon.ceph-node1-mon-centos-6-4.asok mon_status >> >> [ceph-node1-mon-centos-6-4][DEBUG ]*************************************************************************** >> ***** >> >> [ceph-node1-mon-centos-6-4][DEBUG ] status for monitor: >> mon.ceph-node1-mon-centos-6-4 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] { >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "election_epoch": 2, >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "extra_probe_peers": [], >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "monmap": { >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "created": "0.000000", >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "epoch": 1, >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "fsid": >> "c732fc5f-a656-401a-a8e5-4bfed1f89d20", >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "modified": "0.000000", >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "mons": [ >> >> [ceph-node1-mon-centos-6-4][DEBUG ] { >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "addr": "10.12.0.70:6789/0", >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "name": >> "ceph-node1-mon-centos-6-4", >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "rank": 0 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] } >> >> [ceph-node1-mon-centos-6-4][DEBUG ] ] >> >> [ceph-node1-mon-centos-6-4][DEBUG ] }, >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "name": "ceph-node1-mon-centos-6-4", >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "outside_quorum": [], >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "quorum": [ >> >> [ceph-node1-mon-centos-6-4][DEBUG ] 0 >> >> [ceph-node1-mon-centos-6-4][DEBUG ] ], >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "rank": 0, >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "state": "leader", >> >> [ceph-node1-mon-centos-6-4][DEBUG ] "sync_provider": [] >> >> [ceph-node1-mon-centos-6-4][DEBUG ] } >> >> [ceph-node1-mon-centos-6-4][DEBUG ]*************************************************************************** >> ***** >> >> [ceph-node1-mon-centos-6-4][INFO ] monitor: mon.ceph-node1-mon-centos-6-4 >> is running >> >> [ceph-node1-mon-centos-6-4][INFO ] Running command: sudo ceph >> --cluster=ceph --admin-daemon >> /var/run/ceph/ceph-mon.ceph-node1-mon-centos-6-4.asok mon_status >> >> >> >> Thanks! >> >> Narendra >> >> -----Original Message----- >> From: Sage Weil [mailto:sage@xxxxxxxxxxx] >> Sent: Friday, November 01, 2013 8:37 PM >> To: Trivedi, Narendra >> Cc: ceph-users@xxxxxxxxxxxxxx >> Subject: Re: Very frustrated with Ceph! >> >> >> >> On Sat, 2 Nov 2013, Trivedi, Narendra wrote: >> >> > [ceph-node2-osd0-centos-6-4][WARNIN] osd keyring does not exist yet, >> >> > creating one >> >> > >> >> > [ceph-node2-osd0-centos-6-4][DEBUG ] create a keyring file >> >> > >> >> > [ceph_deploy.osd][ERROR ] OSError: [Errno 2] No such file or directory >> >> >> >> Did you do 'ceph-deploy install ...' on these hosts? >> >> >> >> sage >> >> >> This message contains information which may be confidential and/or >> privileged. Unless you are the intended recipient (or authorized to receive >> for the intended recipient), you may not read, use, copy or disclose to >> anyone the message or any information contained in the message. If you have >> received the message in error, please advise the sender by reply e-mail and >> delete the message and any attachment(s) thereto without retaining any >> copies. >> >> > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |