Hi again,
[root@xen-blade05 ~]# virsh pool-info rbd
Name: rbd
UUID: ebc61120-527e-6e0a-efdc-4522a183877e
State: running
Persistent: no
Autostart: no
Capacity: 5.28 TiB
Allocation: 16.99 GiB
Available: 5.24 TiB
I managed to get it running. How ? I rebooted the xenserver host and
tried again... It worked...
Some mystery here :)
Cheers,
Sébastien
On 21.07.2013 14:01, Sébastien RICCIO wrote:
Hi,
thanks a lot for your answer. I was successfully able to create the
storage pool with virsh.
However it is not listed when I issue a virsh pool-list:
Name State Autostart
-----------------------------------------
If I try to add it again:
[root@xen-blade05 ~]# virsh pool-create ceph.xml
error: Failed to create pool from ceph.xml
error: operation failed: pool 'rbd' already exists with uuid
34609207-3ea0-28c5-fee9-92e6b493e6cb
When i try to get info about it:
virsh # pool-info rbd
Name: rbd
UUID: 34609207-3ea0-28c5-fee9-92e6b493e6cb
State: inactive
Persistent: yes
Autostart: no
When I try to start it:
virsh# pool-start rbd
error: Failed to start pool rbd
error: An error occurred, but the cause is unknown
dumping the config
virsh # pool-dumpxml rbd
<pool type='rbd'>
<name>rbd</name>
<uuid>34609207-3ea0-28c5-fee9-92e6b493e6cb</uuid>
<capacity unit='bytes'>0</capacity>
<allocation unit='bytes'>0</allocation>
<available unit='bytes'>0</available>
<source>
<host name='10.111.80.1' port='6789'/>
<name>rbd</name>
<auth username='admin' type='ceph'>
<secret uuid='4cae707d-e049-4ec1-ac51-2cf999969dd3'/>
</auth>
</source>
</pool>
virsh # secret-dumpxml 4cae707d-e049-4ec1-ac51-2cf999969dd3
<secret ephemeral='no' private='no'>
<uuid>4cae707d-e049-4ec1-ac51-2cf999969dd3</uuid>
<usage type='ceph'>
<name>client.admin</name>
</usage>
</secret>
Ceph is working on the box
[root@xen-blade05 ~]# ceph status
cluster f58c1405-8891-46cb-adf0-c6ab2da4050a
health HEALTH_OK
monmap e1: 3 mons at
{ceph01=10.111.80.1:6789/0,ceph02=10.111.80.2:6789/0,ceph03=10.111.80.3:6789/0},
election epoch 18, quorum 0,1,2 ceph01,ceph02,ceph03
osdmap e96: 6 osds: 6 up, 6 in
pgmap v1317: 192 pgs: 192 active+clean; 17400 MB data, 41192 MB
used, 5367 GB / 5407 GB avail
mdsmap e1: 0/0/1 up
[root@xen-blade05 ~]# ceph osd lspools
0 data,1 metadata,2 rbd,
Any idea where to look now ? :)
Thanks for your help.
Cheers,
Sébastien
On 21.07.2013 11:42, Wido den Hollander wrote:
Hi,
On 07/21/2013 08:14 AM, Sébastien RICCIO wrote:
Hi !
I'm currently trying to get the xenserver on centos 6.4 tech preview
working against a test ceph cluster and having the same issue.
Some infos: the cluster is named "ceph", the pool is named "rbd".
ceph.xml:
<pool type='rbd'>
<name>rbd</name>
<source>
<name>ceph</name>
<host name='10.111.80.1' port='6789'/>
</source>
</pool>
You need a secret section inside the <source> like this:
<pool type='rbd'>
<name>MyCephPool</name>
<source>
<host name='10.111.80.1' port='6789'/>
<name>rbd</name>
<auth username='admin' type='ceph'>
<secret uuid='4cae707d-e049-4ec1-ac51-2cf999969dd3'/>
</auth>
</source>
</pool>
The secret "uuid" should reference back to the UUID of the secret
you'll define.
secret.xml:
<secret ephemeral='no' private='no'>
<usage type='ceph'>
<name>client.admin <key_from_ceph_auth_list></name>
</usage>
</secret>
Don't put all that information in the <name> section, the XML should
be like this:
<secret ephemeral='no' private='no'>
<uuid>4cae707d-e049-4ec1-ac51-2cf999969dd3</uuid>
<usage type='ceph'>
<name>client.admin</name>
</usage>
</secret>
[root@xen-blade05 ~]# virsh pool-create ceph.xml
This should be the procedure to follow:
$ virsh secret-define secret.xml
$ virsh secret-set-value <uuid> <cephx key>
$ virsh pool-define ceph.xml
error: Failed to create pool from ceph.xml
error: Invalid secret: virSecretFree
You get this error due to cephx being disabled since you didn't
define a <auth> section. This error is a bug in libvirt, but that
will go away as soon as you use cephx.
Wido
same error :/
Any ideas ?
Don't hesitate to ask if you need more infos.
Cheers,
Sébastien
Hi John,
Could you try without the cat'ing and such?
Could you also try this:
$ virsh secret define secret.xml
$ virsh secret-set-value <uuid> <secret>
$ virsh pool-create ceph.xml
Could you post both XML files and not use any Xen commands like 'xe'?
I want to verify where this problem is.
Wido
On 07/11/2013 10:34 PM, John Shen wrote:
>/ Wido, Thanks! I tried again with your command syntax but the
result is
/>/ the same.
/>/
/>/ [root at xen02
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> ~]# virsh
secret-set-value $(cat uuid) $(cat client.admin.key)
/>/ Secret value set
/>/
/>/ [root at xen02
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> ~]# xe
sr-create type=libvirt name-label=ceph
/>/ device-config:xml-filename=ceph.xml
/>/ Error code: libvirt
/>/ Error parameters: libvirt: VIR_ERR_65: VIR_FROM_30: Invalid
secret:
/>/ virSecretFree
/>/ [root at xen02
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> ~]# virsh
pool-create ceph.xml
/>/ error: Failed to create pool from ceph.xml
/>/ error: Invalid secret: virSecretFree
/>/
/>/ [root at xen02
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> ~]#
/>/
/>/
/>/
/>/ On Thu, Jul 11, 2013 at 1:14 PM, Wido den Hollander <wido at
42on.com <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>
/>/ <mailto:wido at 42on.com
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>>> wrote:
/>/
/>/ Hi.
/>/
/>/ So, the problem here is a couple of things.
/>/
/>/ First: libvirt doesn't handle RBD storage pools without
auth. That's
/>/ my bad, but I never resolved that bug:
/>/ http://tracker.ceph.com/__issues/3493
/>/ <http://tracker.ceph.com/issues/3493>
/>/
/>/ For now, make sure cephx is enabled.
/>/
/>/ Also, the commands you are using don't seem to be right.
/>/
/>/ It should be:
/>/
/>/ $ virsh secret-set-value $(cat uuid) <base64 secret key>
/>/
/>/ Could you try again with cephx enabled and setting the
secret value
/>/ like mentioned above?
/>/
/>/ Wido
/>/
/>/
/>/ On 07/11/2013 06:00 PM, John Shen wrote:
/>/
/>/ Hi Dave, Thank you so much for getting back to me.
/>/
/>/ the command returns the same errors:
/>/
/>/ [root at xen02
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> ~]# virsh
pool-create ceph.xml
/>/ error: Failed to create pool from ceph.xml
/>/ error: Invalid secret: virSecretFree
/>/
/>/ [root at xen02
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> ~]#
/>/
/>/ the secret was precreated for the user admin that I use
/>/ elsewhere with
/>/ no issues (rbd mount, cephfs etc.), and per the ceph
/>/ documentation, i
/>/ just set the secret value with this command
/>/
/>/ virsh secret-set-value $(cat uuid) --base64 $(cat
/>/ client.admin.key)
/>/
/>/ where the key is obtained from
/>/
/>/ ceph auth list
/>/
/>/ and uuid is generated by
/>/
/>/ virsh secret-define --file secret.xml
/>/
/>/ # cat secret.xml
/>/ <secret ephemeral='no' private='no'>
/>/ <usage type='ceph'>
/>/ <name>client.admin $(cat
client.admin.key)</name>
/>/ </usage>
/>/ </secret>
/>/
/>/
/>/
/>/ On Thu, Jul 11, 2013 at 7:22 AM, Dave Scott
/>/ <Dave.Scott at eu.citrix.com
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>
<mailto:Dave.Scott at eu.citrix.com
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>>
/>/ <mailto:Dave.Scott at eu.citrix.__com
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>
/>/ <mailto:Dave.Scott at eu.citrix.com
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>>>> wrote:
/>/
/>/ [sorry I didn't manage to reply to the original
message; I
/>/ only just
/>/ joined this list.
/>/ Sorry if this breaks your threading!]
/>/
/>/ On 10 Jul 2013 at 16:01 John Shen wrote:
/>/
/>/ > I was following the tech preview of libvirt/ceph
/>/ integration in
/>/ xenserver, but ran
/>/ > into an issue with ceph auth in setting up the
SR. any
/>/ help would
/>/ be greatly
/>/ > appreciated.
/>/
/>/ I must confess that I've disabled auth in my test
environment.
/>/ Clearly I should
/>/ go back and enable it again :-)
/>/
/>/ > uuid was generated per:
/>/ http://eu.ceph.com/docs/wip-__dump/rbd/libvirt/
/>/ <http://eu.ceph.com/docs/wip-dump/rbd/libvirt/>
/>/ >
/>/ > according to inktank, storage pool auth syntax
differs
/>/ slightly
/>/ from block device
/>/ > attachment. I tried both format but got the
same error.
/>/ >
/>/ > Ref:
/>/ >
/>/ >
/>/
http://xenserver.org/blog/__entry/tech-preview-of-__xenserver-libvirt-ceph.html
/>/
<http://xenserver.org/blog/entry/tech-preview-of-xenserver-libvirt-ceph.html>
/>/ >
/>/ > [root <at> xen01 ~]# xe sr-create type=libvirt
/>/ name-label=ceph
/>/ device-config:xml-filename=__ceph.xml
/>/ > Error code: libvirt
/>/ > Error parameters: libvirt: VIR_ERR_65:
VIR_FROM_30: Invalid
/>/ secret: virSecretFree
/>/
/>/ The "xe sr-create" call is handled by "xapi"
which calls
/>/ "xapi-libvirt-storage" which
/>/ uses the libvirt API directly to create the pool. It
/>/ _should_ do the
/>/ same as running
/>/
/>/ virsh pool-create ceph.xml
/>/
/>/ Could you try the "virsh pool-create" and see if
that
/>/ works? If it
/>/ does, then we need
/>/ to figure out what the "virsh" CLI is doing that
my Pool.create
/>/ function call isn't. If
/>/ it doesn't then there might be some other missing
step. Did
/>/ you have
/>/ to pre-create
/>/ a secret (is that "virsh secret-create"?)
/>/
/>/ Cheers,
/>/ Dave Scott
/>/
/>/
/>/
/>/
/>/ --
/>/ --John Shen
/>/
/>/
/>/ _________________________________________________
/>/ ceph-users mailing list
/>/ ceph-users at lists.ceph.com
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>
<mailto:ceph-users at lists.ceph.com
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>>
/>/ http://lists.ceph.com/__listinfo.cgi/ceph-users-ceph.__com
/>/ <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>
/>/
/>/
/>/
/>/ --
/>/ Wido den Hollander
/>/ 42on B.V.
/>/
/>/ Phone: +31 (0)20 700 9902<tel:%2B31%20%280%2920%20700%209902>
/>/ Skype: contact42on
/>/ _________________________________________________
/>/ ceph-users mailing list
/>/ ceph-users at lists.ceph.com
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>
<mailto:ceph-users at lists.ceph.com
<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>>
/>/ http://lists.ceph.com/__listinfo.cgi/ceph-users-ceph.__com
/>/ <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com>
/>/
/>/
/>/
/>/
/>/ --
/>/ --John Shen
/
--
Wido den Hollander
42on B.V.
Phone: +31 (0)20 700 9902
Skype: contact42on
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
