Hi All,
This seems to be a weird issue. Firstly all deployment is done with "ceph-deploy" and 3 host machines acting as MON and OSD using the Hammer release on Ubuntu 14.04.3 and running KVM (libvirt).
When using vanilla CEPH, single rbd pool no log device or cache tiering, the virtual machine will start without any problem. I can see CEPH doing data work and the virtual machine runs the OS installer fine.
However when I enable cache tiering so that I have a separate RBD and SSD pool, libvirt is unable to start and virtual machines at all with an error of a feature I think CEPH disabled (not sure). This is entirely repeatable as I did re-installs of software and even the operating system.
I'm not sure if it is CEPH in this case but it only seems to happen when doing cache tiering which leads me to believe it has some of the blame.
My setup is:
3 Host Servers (doing both KVM and CEPH)
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty
Each server has 2x OSD devices with a total of 6 across the cluster. 1x 500GB HDD and 1x 120GB SSD in each host. All JBOD. (Operating system disks excluded from this)
My error received from libvirt: (libvirt and ceph secrets omitted)
internal error: process exited while connecting to monitor: libust[19895/19895]: Warning: HOME environment variable not set. Disabling LTTng-UST per-user tracing. (in setup_local_apps() at lttng-ust-comm.c:305) libust[19895/19896]: Error: Error opening shm /lttng-ust-wait-5 (in get_wait_shm() at lttng-ust-comm.c:886) libust[19895/19896]: Error: Error opening shm /lttng-ust-wait-5 (in get_wait_shm() at lttng-ust-comm.c:886) qemu-system-x86_64: -drive file=rbd:rbd/osc-maas-1:id=libvirt:key=XXXXXXXXXXXXXX:auth_supported=cephx\;none:mon_host=osc-mgmt-1\:6789\;osc-mgmt-2\:6789\;osc-mgmt-3\:6789,if=none,id=drive-virtio-disk0,format=raw: error reading header from osc-maas-1 qemu-system-x86_64: -drive file=rbd:rbd/osc-maas-1:id=libvirt:key=XXXXXXXXXXXXXX:auth_supported=cephx\;none:mon_host=osc-mgmt-1\:6789\;osc-mgmt-2\:6789\;osc-mgmt-3\:6789,if=none,id=drive-virtio-disk0,format=raw: could not open disk image rbd:rbd/osc-maas-1:id=libvirt:key=XXXXXXXXXXXXXX
I have followed these guides to split pools onto SSD and do cache tiering.
Thanks
Pieter
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
