Yeah, makes sense.
Cheers
Mark
On 08/05/15 03:04, Travis Rhoden wrote:
Hello again Mark,
Thanks again for pointing this out. After some discussion, we would
like to keep the 'client.rgw.*' naming convention, and we do need to
update ceph-deploy to make sure people do not shoot themselves in the
foot with this.
It will still be totally fine to have custom names, but more along the
lines of client.rgw.foo, client.rgw.bar, etc.
I've filed http://tracker.ceph.com/issues/11561 to track this.
- Travis
On Wed, Apr 22, 2015 at 9:24 AM, Travis Rhoden <trhoden@xxxxxxxxx> wrote:
Hi Mark,
Thanks for the detective work! I'll confirm whether the constraint on
the client name is really intended -- I believe that it is. Once
confirmed, I completely agree that ceph-deploy should not allow you to
create rgw daemons with names that we know Ceph is going to reject.
- Travis
On Wed, Apr 22, 2015 at 4:42 AM, Mark Kirkwood
<mark.kirkwood@xxxxxxxxxxxxxxx> wrote:
On 13/04/15 15:33, Mark Kirkwood wrote:
Hi,
I've been experimenting with the new rgw creation in ceph-deploy, using
version 1.5.23 together with ceph 0.94 (-948-gd77de49).
If simply run it without any args, then it works fine. e.g:
$ ceph-deploy rgw create ceph1
However if I try to set a name for the daemon:
$ ceph-deploy rgw create ceph1:radosgw.ceph1
then it fails to create and start a radosgw daemon at all. Poking
through the logs I see:
$ tail ceph-client.radosgw.ceph1.log
2015-04-13 14:13:20.513860 7f7e36c04840 0 librados:
client.radosgw.ceph1 authentication error (1) Operation not permitted
Not terribly informative. Looking at the audit log:
$ tail ceph.audit.log
015-04-13 15:10:37.880085 mon.0 192.168.122.21:6789/0 147 : audit [INF]
from='client.? 192.168.122.22:0/1004669' entity='client.bootstrap-rgw'
cmd=[{"prefix": "auth get-or-create", "entity": "client.radosgw.ceph2",
"caps": ["osd", "allow rwx", "mon", "allow rw"]}]: access denied
Looking at what ceph-deploy is doing I see that the failing command is:
$ sudo ceph --cluster ceph --name client.bootstrap-rgw --keyring
/var/lib/ceph/bootstrap-rgw/ceph.keyring auth get-or-create
client.radosgw.ceph2 osd 'allow rwx' mon 'allow rw' -o
/var/lib/ceph/radosgw/ceph-radosgw.ceph2/keyring
This is failing whereas the equivalent command with radosgw replaced by
rgw works. Looking at what privileges client.bootstrap-rgw has I see:
$ sudo ceph auth get client.bootstrap-rgw
[client.bootstrap-rgw]
key = XXXXXXXXXXXXXXXXXX==
caps mon = "allow profile bootstrap-rgw"
Ah, so need to look at how the profile bootstrap-rgw is setup:
$ more src/mon/MonCap.cc
...
if (profile == "bootstrap-rgw") {
profile_grants.push_back(MonCapGrant("mon", MON_CAP_R)); // read
monmap
profile_grants.push_back(MonCapGrant("osd", MON_CAP_R)); // read
osdmap
profile_grants.push_back(MonCapGrant("mon getmap"));
profile_grants.push_back(MonCapGrant("auth get-or-create")); //
FIXME: this can expose other mds keys
profile_grants.back().command_args["entity"] = StringConstraint("",
"client.rgw."); <======
profile_grants.back().command_args["caps_mon"] =
StringConstraint("allow rw", "");
profile_grants.back().command_args["caps_osd"] =
StringConstraint("allow rwx", "");
}
...
I see - the ["entity"] check requires the name to match "client.rgw.".
Double checking back on the deployed host (ceph1), I see that:
$ ceph --cluster ceph --name client.bootstrap-rgw --keyring
/var/lib/ceph/bootstrap-rgw/ceph.keyring auth get-or-create
client.rgw.ceph3 osd 'allow rwx' mon 'allow rw'
[client.rgw.ceph3]
key = AQCrNytVp9bqFBAAgTJVQDVQFJqiSkFqD68XsQ==
$ ceph --cluster ceph --name client.bootstrap-rgw --keyring
/var/lib/ceph/bootstrap-rgw/ceph.keyring auth get-or-create
client.radosgw.ceph3 osd 'allow rwx' mon 'allow rw'
Error EACCES: access denied
Ok so that is definitely the problem.
Is this restriction intended? If so we really should make ceph-deploy
aware of this...as it is a bit of a foot gun (and not as easy as I made
it seem above to figure out *why* the deploy didn't work).
Cheers
Mark
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
ping?
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
