On 13/04/15 15:33, Mark Kirkwood wrote:
Hi,
I've been experimenting with the new rgw creation in ceph-deploy, using
version 1.5.23 together with ceph 0.94 (-948-gd77de49).
If simply run it without any args, then it works fine. e.g:
$ ceph-deploy rgw create ceph1
However if I try to set a name for the daemon:
$ ceph-deploy rgw create ceph1:radosgw.ceph1
then it fails to create and start a radosgw daemon at all. Poking
through the logs I see:
$ tail ceph-client.radosgw.ceph1.log
2015-04-13 14:13:20.513860 7f7e36c04840 0 librados:
client.radosgw.ceph1 authentication error (1) Operation not permitted
Not terribly informative. Looking at the audit log:
$ tail ceph.audit.log
015-04-13 15:10:37.880085 mon.0 192.168.122.21:6789/0 147 : audit [INF]
from='client.? 192.168.122.22:0/1004669' entity='client.bootstrap-rgw'
cmd=[{"prefix": "auth get-or-create", "entity": "client.radosgw.ceph2",
"caps": ["osd", "allow rwx", "mon", "allow rw"]}]: access denied
Looking at what ceph-deploy is doing I see that the failing command is:
$ sudo ceph --cluster ceph --name client.bootstrap-rgw --keyring
/var/lib/ceph/bootstrap-rgw/ceph.keyring auth get-or-create
client.radosgw.ceph2 osd 'allow rwx' mon 'allow rw' -o
/var/lib/ceph/radosgw/ceph-radosgw.ceph2/keyring
This is failing whereas the equivalent command with radosgw replaced by
rgw works. Looking at what privileges client.bootstrap-rgw has I see:
$ sudo ceph auth get client.bootstrap-rgw
[client.bootstrap-rgw]
key = XXXXXXXXXXXXXXXXXX==
caps mon = "allow profile bootstrap-rgw"
Ah, so need to look at how the profile bootstrap-rgw is setup:
$ more src/mon/MonCap.cc
...
if (profile == "bootstrap-rgw") {
profile_grants.push_back(MonCapGrant("mon", MON_CAP_R)); // read
monmap
profile_grants.push_back(MonCapGrant("osd", MON_CAP_R)); // read
osdmap
profile_grants.push_back(MonCapGrant("mon getmap"));
profile_grants.push_back(MonCapGrant("auth get-or-create")); //
FIXME: this can expose other mds keys
profile_grants.back().command_args["entity"] = StringConstraint("",
"client.rgw."); <======
profile_grants.back().command_args["caps_mon"] =
StringConstraint("allow rw", "");
profile_grants.back().command_args["caps_osd"] =
StringConstraint("allow rwx", "");
}
...
I see - the ["entity"] check requires the name to match "client.rgw.".
Double checking back on the deployed host (ceph1), I see that:
$ ceph --cluster ceph --name client.bootstrap-rgw --keyring
/var/lib/ceph/bootstrap-rgw/ceph.keyring auth get-or-create
client.rgw.ceph3 osd 'allow rwx' mon 'allow rw'
[client.rgw.ceph3]
key = AQCrNytVp9bqFBAAgTJVQDVQFJqiSkFqD68XsQ==
$ ceph --cluster ceph --name client.bootstrap-rgw --keyring
/var/lib/ceph/bootstrap-rgw/ceph.keyring auth get-or-create
client.radosgw.ceph3 osd 'allow rwx' mon 'allow rw'
Error EACCES: access denied
Ok so that is definitely the problem.
Is this restriction intended? If so we really should make ceph-deploy
aware of this...as it is a bit of a foot gun (and not as easy as I made
it seem above to figure out *why* the deploy didn't work).
Cheers
Mark
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
ping?
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
