run
/usr/bin/radosgw -c /etc/ceph/ceph.conf -n client.radosgw.gateway
error:
2013-06-19 09:19:55.148536 7f120aa0d820 0 librados: client.radosgw.gateway authentication error (95) Operation not supported
2013-06-19 09:19:55.148923 7f120aa0d820 -1 Couldn't init storage provider (RADOS)
How to solve the
problem!!
yangpengtao.slyt
2013-06-19
发件人：
ceph-users-request
发送时间：
2013-05-07 06:41:25
收件人：
ceph-users
抄送：
主题： ceph-users Digest, Vol
4, Issue 24
Send ceph-users mailing list submissions to
ceph-users@xxxxxxxxxxxxxx
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
or, via email, send a message with subject or body 'help' to
ceph-users-request@xxxxxxxxxxxxxx
You can reach the person managing the list at
ceph-users-owner@xxxxxxxxxxxxxx
When replying, please edit your Subject line so it is more specific
than "Re: Contents of ceph-users digest..."
Today's Topics:
1. Ubuntu 13.04 Packages (Stephen Street)
2. Re: EPEL packages for QEMU-KVM with rbd support? (Josh Durgin)
3. Cluster unable to finish balancing (Berant Lemmenes)
4. Re: EPEL packages for QEMU-KVM with rbd support? (w sun)
----------------------------------------------------------------------
Message: 1
Date: Mon, 6 May 2013 12:37:40 -0700
From: Stephen Street <sgs@xxxxxxxxxxxxxxxxxxxxxx>
To: ceph-users@xxxxxxxxxxxxxx
Subject: [ceph-users] Ubuntu 13.04 Packages
Message-ID:
<BB40F5B0-4394-4185-9D1B-38CB3462AF31@xxxxxxxxxxxxxxxxxxxxxx>
Content-Type: text/plain; charset=us-ascii
Hi All,
I'm upgrading my test systems to Ubuntu 13.04 Raring Ringtail and I have noticed that http://ceph.com/debian does not contain build for Raring. What are the plans for releasing Raring packages?
Thanks
Stephen Street
------------------------------
Message: 2
Date: Mon, 06 May 2013 12:54:57 -0700
From: Josh Durgin <josh.durgin@xxxxxxxxxxx>
To: ceph-users@xxxxxxxxxxxxxx
Subject: Re: [ceph-users] EPEL packages for QEMU-KVM with rbd support?
Message-ID: <51880A91.6000705@xxxxxxxxxxx>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
On 05/06/2013 11:56 AM, Barry O'Rourke wrote:
> The version currently shipping with RHEL is qemu-kvm-0.12.1.2-2.355
> which doesn't work with ceph authentication, so you'll need to go for
> greater than 1.0.1.
The RHEL version number is pretty meaningless, since there are >2800
patches on top of it, including a lot of newer stuff like coroutines.
0.12 didn't include rbd at all, but it's not too hard to put up-to-date
rbd on top of the RHEL package.
Josh
------------------------------
Message: 3
Date: Mon, 6 May 2013 16:43:48 -0400
From: Berant Lemmenes <berant@xxxxxxxxxxxx>
To: ceph-users@xxxxxxxxxxxxxx
Subject: [ceph-users] Cluster unable to finish balancing
Message-ID:
<CAGJeadSRLAKUryt9x3VPdm86u7f1vz+xarpkNNa_AU9HSrbpQA@xxxxxxxxxxxxxx>
Content-Type: text/plain; charset="iso-8859-1"
TL;DR
bobtail Ceph cluster unable to finish rebalance after drive failure, usage
increasing even with no clients connected.....
I've been running a test bobtail cluster for a couple of months and it's
been working great. Last week I had a drive die and rebalance; durring that
time another OSD crashed. All was still well, however as the second osd had
just crashed I restarted made sure that it re-entered properly and
rebalancing continued and then I went to bed.
Waking up in the morning I found 2 OSDs were 100% full and two more were
almost full. To get out of the situation I decreased the replication size
from 3 to 2, and then also carefully (I believe carefully enough) remove
some PGs in order to start things up again.
I got things going again and things appeared to be rebalancing correctly;
however it got to the point were it stopped at 1420 PGs active+clean and
the rest were stuck backfilling.
Looking at the PG dump, all of the PGs that were having issues were on
osd.1. So I stopped it, verified things were continuing to rebalance after
it was down/out and then formated osd.1's disk and put it back in.
Since then I've not been able to get the cluster back to HEALTHY, due to a
combination of OSDs dying while recovering (not due to disk failure, just
crashes) as well as the used space in the cluster increasing abnormally.
Right now I have all the clients disconnected and just the cluster
rebalancing and the usage is increasing to the point where I have 12TB used
when I have only < 3TB in cephfs and 2TB in a single RBD image
(replication size 2). I've since shutdown the cluster so I don't fill it up.
My crushmap is the default, here is the usual suspects. I'm happy to
provide additional information.
pg dump: http://pastebin.com/LUyu6Z09
ceph osd tree:
osd.8 is the failed drive (I will be replacing tonight), weight on osd.1
and osd.6 was done via reweight-by-utilization
# id weight type name up/down reweight
-1 19.5 root default
-3 19.5 rack unknownrack
-2 19.5 host ceph-test
0 1.5 osd.0 up 1
1 1.5 osd.1 up 0.6027
2 1.5 osd.2 up 1
3 1.5 osd.3 up 1
4 1.5 osd.4 up 1
5 2 osd.5 up 1
6 2 osd.6 up 0.6676
7 2 osd.7 up 1
8 2 osd.8 down 0
9 2 osd.9 up 1
10 2 osd.10 up 1
ceph -s:
health HEALTH_WARN 24 pgs backfill; 85 pgs backfill_toofull; 29 pgs
backfilling; 40 pgs degraded; 1 pgs recovery_wait; 121 pgs stuck unclean;
recovery 109306/2091318 degraded (5.227%); recovering 3 o/s, 43344KB/s; 2
near full osd(s); noout flag(s) set
monmap e2: 1 mons at {a=10.200.200.21:6789/0}, election epoch 1, quorum
0 a
osdmap e16251: 11 osds: 10 up, 10 in
pgmap v3145187: 1536 pgs: 1414 active+clean, 6
active+remapped+wait_backfill, 10
active+remapped+wait_backfill+backfill_toofull, 4
active+degraded+wait_backfill+backfill_toofull, 22
active+remapped+backfilling, 42 active+remapped+backfill_toofull, 7
active+degraded+backfilling, 17 active+degraded+backfill_toofull, 1
active+recovery_wait+remapped, 4
active+degraded+remapped+wait_backfill+backfill_toofull, 8
active+degraded+remapped+backfill_toofull, 1 active+clean+scrubbing+deep;
31607 GB data, 12251 GB used, 4042 GB / 16293 GB avail; 109306/2091318
degraded (5.227%); recovering 3 o/s, 43344KB/s
mdsmap e3363: 1/1/1 up {0=a=up:active}
rep size:
pool 0 'data' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 384
pgp_num 384 last_change 897 owner 0 crash_replay_interval 45
pool 1 'metadata' rep size 2 crush_ruleset 1 object_hash rjenkins pg_num
384 pgp_num 384 last_change 13364 owner 0
pool 2 'rbd' rep size 2 crush_ruleset 2 object_hash rjenkins pg_num 384
pgp_num 384 last_change 13208 owner 0
pool 4 'media_video' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num
384 pgp_num 384 last_change 890 owner 0
ceph.conf:
[global]
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
osd pool default size = 3
osd pool default min size = 1
osd pool default pg num = 366
osd pool default pgp num = 366
[osd]
osd journal size = 1000
journal_aio = true
#osd recovery max active = 10
osd mkfs type = xfs
osd mkfs options xfs = -f -i size=2048
osd mount options xfs = inode64,noatime
[mon.a]
host = ceph01
mon addr = 10.200.200.21:6789
[osd.0]
# 1.5 TB SATA
host = ceph01
devs = /dev/sdc
weight = 1.5
[osd.1]
# 1.5 TB SATA
host = ceph01
devs = /dev/sdd
weight = 1.5
[osd.2]
# 1.5 TB SATA
host = ceph01
devs = /dev/sdg
weight = 1.5
[osd.3]
# 1.5 TB SATA
host = ceph01
devs = /dev/sdj
weight = 1.5
[osd.4]
# 1.5 TB SATA
host = ceph01
devs = /dev/sdk
weight = 1.5
[osd.5]
# 2 TB SAS
host = ceph01
devs = /dev/sdf
weight = 2
[osd.6]
# 2 TB SAS
host = ceph01
devs = /dev/sdh
weight = 2
[osd.7]
# 2 TB SAS
host = ceph01
devs = /dev/sda
weight = 2
[osd.8]
# 2 TB SAS
host = ceph01
devs = /dev/sdb
weight = 2
[osd.9]
# 2 TB SAS
host = ceph01
devs = /dev/sdi
weight = 2
[osd.10]
# 2 TB SAS
host = ceph01
devs = /dev/sde
weight = 2
[mds.a]
host = ceph01
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20130506/7d1862a0/attachment-0001.htm>
------------------------------
Message: 4
Date: Mon, 6 May 2013 22:41:19 +0000
From: w sun <wsun2@xxxxxxxxxxx>
To: "ceph-users@xxxxxxxxxxxxxx" <ceph-users@xxxxxxxxxxxxxx>
Subject: Re: [ceph-users] EPEL packages for QEMU-KVM with rbd support?
Message-ID: <BAY178-W38C3EA1BDCE30F18F3911AE2B90@xxxxxxx>
Content-Type: text/plain; charset="iso-8859-1"
Hi Josh,
I assume by "put up-to-date rbd on top of the RHEL package", you mean that the latest "asynchronous flush" fix (QEMU portion) can be back-ported and included in the RPMs? Or not?
Thx. --weiguo
> Date: Mon, 6 May 2013 12:54:57 -0700
> From: josh.durgin@xxxxxxxxxxx
> To: ceph-users@xxxxxxxxxxxxxx
> Subject: Re: [ceph-users] EPEL packages for QEMU-KVM with rbd support?
>
> On 05/06/2013 11:56 AM, Barry O'Rourke wrote:
> > The version currently shipping with RHEL is qemu-kvm-0.12.1.2-2.355
> > which doesn't work with ceph authentication, so you'll need to go for
> > greater than 1.0.1.
>
> The RHEL version number is pretty meaningless, since there are >2800
> patches on top of it, including a lot of newer stuff like coroutines.
>
> 0.12 didn't include rbd at all, but it's not too hard to put up-to-date
> rbd on top of the RHEL package.
>
> Josh
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20130506/eed9705d/attachment.htm>
------------------------------
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
End of ceph-users Digest, Vol 4, Issue 24
*****************************************
.
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
