Re: gluster and kvm livemigration

[Andrew Lau <andrew@xxxxxxxxxxxxxx>]

Have you tried this?

Work Around until libvirtd fixes the port conflict (http://review.gluster.org/#/c/6147/)

Modify /etc/glusterfs/glusterd.vol

    option base-port 50152

service glusterd restart

On Sat, Jan 25, 2014 at 1:44 AM, Bernhard Glomm <bernhard.glomm@xxxxxxxxxxx> wrote:

I submitted 

Bug 1057645 

https://bugzilla.redhat.com/show_bug.cgi?id=1057645

Bernhard

On 24.01.2014 11:07:49, Bernhard Glomm wrote:

samuli wrote: 

Can you try to set storage.owner-uid and storage.owner-gid to 
libvirt-qemu? To do that you have to stop volume.

hi samuli, hi all 

I tried setting storage.owner-uid and storage.owner-gid to 

libvirt-qemu, as suggested, but with the same effect,

during livemigration the ownership of the imagefile changes from libvirt-qemu/kvm to root/root

root@pong[/5]:~ # gluster volume info glfs_atom01

 

Volume Name: glfs_atom01

Type: Replicate

Volume ID: f28f0f62-37b3-4b10-8e86-9b373f4c0e75

Status: Started

Number of Bricks: 1 x 2 = 2

Transport-type: tcp

Bricks:

Brick1: 172.24.1.11:/ecopool/fs_atom01

Brick2: 172.24.1.13:/ecopool/fs_atom01

Options Reconfigured:

storage.owner-gid: 104

storage.owner-uid: 107

network.remote-dio: enable

this is tree -pfungiA <path to where my images live> : atom01 is running

[-rw------- libvirt- kvm     ]  /srv/vms/mnt_atom01/atom01.img

[drwxr-xr-x libvirt- kvm     ]  /srv/vms/mnt_atom02

[-rw------- root     root    ]  /srv/vms/mnt_atom02/atom02.img

[drwxr-xr-x libvirt- kvm     ]  /srv/vms/mnt_atom03

Now I migrate through "VirtualMachineManager" and watching tree

I see the permission changing to:

[drwxr-xr-x libvirt- kvm     ]  /srv/vms/mnt_atom01

[-rw------- root     root    ]  /srv/vms/mnt_atom01/atom01.img

[drwxr-xr-x libvirt- kvm     ]  /srv/vms/mnt_atom02

[-rw------- root     root    ]  /srv/vms/mnt_atom02/atom02.img

From inside the atom01 (the VM) the filesystem becomes readonly.

But in contrast to

http://epboven.home.xs4all.nl/gluster-migrate.html

I can still read all file, can checksum them, just no write access

from outside the image file behaves as Paul described,

as long as the machine is running I can't read the file

root@pong[/5]:~ # virsh list

 Id    Name                           State

----------------------------------------------------

 6     atom01                         running

root@pong[/5]:~ # l /srv/vms/mnt_atom01/atom01.img

-rw------- 1 root root 10G Jan 24 10:20 /srv/vms/mnt_atom01/atom01.img

root@pong[/5]:~ # file /srv/vms/mnt_atom01/atom01.img

/srv/vms/mnt_atom01/atom01.img: writable, regular file, no read permission

root@pong[/5]:~ # md5sum /srv/vms/mnt_atom01/atom01.img

md5sum: /srv/vms/mnt_atom01/atom01.img: Permission denied

root@pong[/5]:~ # virsh destroy atom01

Domain atom01 destroyed

root@pong[/5]:~ # l /srv/vms/mnt_atom01/atom01.img

-rw------- 1 root root 10G Jan 24 10:20 /srv/vms/mnt_atom01/atom01.img

root@pong[/5]:~ # file /srv/vms/mnt_atom01/atom01.img

/srv/vms/mnt_atom01/atom01.img: x86 boot sector; partition 1: ID=0x83, starthead 1, startsector 63, 16777165 sectors; partition 2: ID=0xf, starthead 254, startsector 16777228, 1677718 sectors, code offset 0x63

root@pong[/5]:~ # md5sum /srv/vms/mnt_atom01/atom01.img

9d048558deb46fef7b24e8895711c554  /srv/vms/mnt_atom01/atom01.img
root@pong[/5]:~ # 

But interestingly the source of the migration can access the file after migration completed

like so: start atom01 on host "ping", migrate it to "pong"

root@pong[/8]:~ # file /srv/vms/mnt_atom01/atom01.img

/srv/vms/mnt_atom01/atom01.img: writable, regular file, no read permission 

root@ping[/5]:~ # file /srv/vms/mnt_atom01/atom01.img

/srv/vms/mnt_atom01/atom01.img: x86 boot sector; partition 1: ID=0x83, starthead 1, startsector 63, 16777165 sectors; partition 2: ID=0xf, starthead 254, startsector 16777228, 1677718 sectors, code offset 0x63

100% reproducible 

Regards

Bernhard

_______________________________________________

Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users

--

---

Bernhard Glomm
IT Administration

Phone:	+49 (30) 86880 134
Fax:	+49 (30) 86880 100
Skype:	bernhard.glomm.ecologic

Ecologic Institut gemeinnützige GmbH | Pfalzburger Str. 43/44 | 10717 Berlin | Germany
GF: R. Andreas Kraemer | AG: Charlottenburg HRB 57947 | USt/VAT-IdNr.: DE811963464
Ecologic™ is a Trade Mark (TM) of Ecologic Institut gemeinnützige GmbH

---

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users