Yes, no problem. This is an export domain in ovirt: gluster v info export Volume Name: export Type: Replicate Volume ID: 0b50341b-9c9a-4868-9c53-7f0ecdb44162 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: glusternode1:/tank2/export/brick1 Brick2: glusternode2:/tank2/export/brick1 Options Reconfigured: performance.write-behind: on performance.cache-size: 1GB server.event-threads: 8 client.event-threads: 8 client.bind-insecure: on cluster.eager-lock: enable performance.stat-prefetch: off performance.io-cache: off performance.read-ahead: off performance.quick-read: off server.allow-insecure: on network.remote-dio: enable cluster.quorum-type: auto storage.owner-gid: 36 storage.owner-uid: 36 auth.allow: * user.cifs: enable nfs.disable: ON cluster.server-quorum-type: serverFirst problem in ovirt is to init the sanlock... Am Dienstag, den 12.07.2016, 15:50 +0530 schrieb Krutika Dhananjay: > Frank, > > Could you share your volume configuration (`gluster volume info > <VOLNAME>`)? > > -Krutika > > On Tue, Jul 12, 2016 at 3:44 PM, David Gossage <dgossage@carouselchec > ks.com> wrote: > > > > On Tue, Jul 12, 2016 at 5:08 AM, Frank Rothenstein <f.rothenstein@b > > odden-kliniken.de> wrote: > > > Hi David, > > > > > > As I'm on gluster I use > > > > > > glusterfs.x86_64 3.7.12-2.el7 @centos-gluster37 > > > > > > I already tried to revert back to 3.7.11 but somehow this didn't > > > help. > > > > > > Do you have more info on aio/caching? > > > > Not at the moment I really haven't had time to dig into it this > > week yet. I'll quote Lindsay Mathieson 's response and maybe she > > can chime in if she sees this chain. > > > > > > -------------------------------------------------------------- > > Hi David, I did a quick test this morning with Proxmox and 3.7.13 > > and was able to get it working with the fuse mount *and* libgfapi. > > > > > > One caveat - you *have* to enable qemu caching, either write-back > > or write-through. 3.7.12 & 13 seem to now disable aio support, and > > qemu requires that when caching is turned off. > > > > > > There are setting for aio in gluster that I haven't played with > > yet. > > ----------------------------------------------------------------- > > > > Did you revert client and server's to 3.7.11? It could be also > > that we had 2 completely different issues. The only similarity > > being the recent upgrades and Operation not permitted messages. > > > > > FrankAm Dienstag, den 12.07.2016, 04:25 -0500 schrieb David > > > Gossage: > > > > > > > > On Tue, Jul 12, 2016 at 4:06 AM, Frank Rothenstein <f.rothenste > > > in@bod > > > > den-kliniken.de> wrote: > > > > > Hi all, > > > > > > > > > > I am stuck with a strange problem using ovirt/gluster on > > > > > ZFSonLinux. > > > > > > > > > > I detached a data-volume form my Cluster an reimported it to > > > change > > > > > from POSIX-type/glusterfs to GlusterFS-Storage. When I try to > > > > > attach it > > > > > to the Cluster I always get a Sanlock error. I checked the > > > logs and > > > > > what I found were some "operation not permitted" > > > > > errors in the brick logs: > > > > > > > > > > [2016-07-08 09:19:24.542046] E [MSGID: 113040] [posix- > > > > > helpers.c:1684:__posix_fd_ctx_get] 0-datadom-posix: Failed to > > > get > > > > > anonymous real_path: > > > > > /tank2/datadom/brick1/br/.glusterfs/c1/43/c1433bab-cd72-4395- > > > ac62- > > > > > b278670c333a _fd = -1 [No such file or directory] > > > > > [2016-07-08 09:19:24.542103] W [MSGID: 113006] > > > > > [posix.c:3010:posix_readv] 0-datadom-posix: pfd is NULL from > > > > > fd=0x7f15f0003b48 [Operation not permitted] > > > > > [2016-07-08 09:19:24.542142] E [MSGID: 115068] [server-rpc- > > > > > fops.c:1486:server_readv_cbk] 0-datadom-server: 4836: READV > > > -2 > > > > > (c1433bab-cd72-4395-ac62-b278670c333a) ==> (Operation not > > > > > permitted) > > > > > [Operation not permitted] > > > > > > > > > > Anyone ever expierienced such problem? > > > > > > > > > > In fact I am able to create/change/delete files in the brick > > > path > > > > > itself and the mount points (NFS/Fuse). > > > > > Ovirt itself seems to test the volumes with /usr/bin/dd, but > > > uses > > > > > iflag=direct. Using this parameter I'm also unable to create > > > any > > > > > file - > > > > > is there any gluster-option that changed recently? Or is it > > > ZFS, > > > > > that > > > > > introduced something that blocks now? > > > > > > > > What version of Gluster are you on? > > > > > > > > My errors aren't quite the same as yours, but I have a similar > > > setup > > > > > > > > Ovirt 3.6.6/Gluster3.7.11 on ZFSonLinux > > > > > > > > Updating from 3.7.11 -> 12 or 13 this last weekend I had these > > > issues > > > > pop up for me which while not exactly the same did result > > > > in [Operation not permitted] > > > > > > > > [2016-07-09 15:27:46.935694] I [fuse-bridge.c:4083:fuse_init] > > > 0- > > > > glusterfs-fuse: FUSE inited with protocol versions: glusterfs > > > 7.22 > > > > kernel 7.22 > > > > [2016-07-09 15:27:49.555466] W [MSGID: 114031] [client-rpc- > > > > fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote > > > > operation failed [Operation not permitted] > > > > [2016-07-09 15:27:49.556574] W [MSGID: 114031] [client-rpc- > > > > fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote > > > > operation failed [Operation not permitted] > > > > [2016-07-09 15:27:49.556659] W [fuse- > > > bridge.c:2227:fuse_readv_cbk] 0- > > > > glusterfs-fuse: 80: READ => -1 gfid=deb61291-5176-4b81-8315- > > > > 3f1cf8e3534d fd=0x7f5224002f68 (Operation not permitted) > > > > [2016-07-09 15:27:59.612477] W [MSGID: 114031] [client-rpc- > > > > fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-1: remote > > > > operation failed [Operation not permitted] > > > > [2016-07-09 15:27:59.613700] W [MSGID: 114031] [client-rpc- > > > > fops.c:3050:client3_3_readv_cbk] 0-GLUSTER1-client-0: remote > > > > operation failed [Operation not permitted] > > > > [2016-07-09 15:27:59.613781] W [fuse- > > > bridge.c:2227:fuse_readv_cbk] 0- > > > > glusterfs-fuse: 168: READ => -1 gfid=deb61291-5176-4b81-8315- > > > > 3f1cf8e3534d fd=0x7f5224002f68 (Operation not permitted) > > > > > > > > Another gluster user suggested that in 3.7.12/13 aio support > > > was > > > > disabled and I may need to change disk caching method. So far > > > I have > > > > just stayed at 3.7.11 until I have time to get a better test > > > setup. > > > > > > > > > Thanks in advance, Frank > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _________________________________________________________________ > > > __ > > > > > ___________ > > > > > BODDEN-KLINIKEN Ribnitz-Damgarten GmbH > > > > > Sandhufe 2 > > > > > 18311 Ribnitz-Damgarten > > > > > > > > > > Telefon: 03821-700-0 > > > > > Fax: 03821-700-240 > > > > > > > > > > E-Mail: info@xxxxxxxxxxxxxxxxxx Internet: http://www.bodden > > > -klini > > > > > ken.de > > > > > > > > > > Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, > > > Steuer- > > > > > Nr.: 079/133/40188 > > > > > Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: > > > Dr. > > > > > Falko Milski > > > > > > > > > > Der Inhalt dieser E-Mail ist ausschließlich für den > > > bezeichneten > > > > > Adressaten bestimmt. Wenn Sie nicht der vorge- > > > > > sehene Adressat dieser E-Mail oder dessen Vertreter sein > > > sollten, > > > > > beachten Sie bitte, dass jede Form der Veröf- > > > > > fentlichung, Vervielfältigung oder Weitergabe des Inhalts > > > dieser E- > > > > > Mail unzulässig ist. Wir bitten Sie, sofort den > > > > > Absender zu informieren und die E-Mail zu löschen. > > > > > > > > > > > > > > > Bodden-Kliniken Ribnitz-Damgarten GmbH 2016 > > > > > *** Virenfrei durch Kerio Mail Server und Sophos Antivirus > > > *** > > > > > > > > > > _______________________________________________ > > > > > Gluster-users mailing list > > > > > Gluster-users@xxxxxxxxxxx > > > > > http://www.gluster.org/mailman/listinfo/gluster-users > > > > > > > > > > > > > > > > > > > > > > _________________________________________________________________ > > > _____________ > > > BODDEN-KLINIKEN Ribnitz-Damgarten GmbH > > > Sandhufe 2 > > > 18311 Ribnitz-Damgarten > > > > > > Telefon: 03821-700-0 > > > Fax: 03821-700-240 > > > > > > E-Mail: info@xxxxxxxxxxxxxxxxxx Internet: http://www.bodden-kli > > > niken.de > > > > > > Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, > > > Steuer-Nr.: 079/133/40188 > > > Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr. > > > Falko Milski > > > > > > Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten > > > Adressaten bestimmt. Wenn Sie nicht der vorge- > > > sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, > > > beachten Sie bitte, dass jede Form der Veröf- > > > fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser > > > E-Mail unzulässig ist. Wir bitten Sie, sofort den > > > Absender zu informieren und die E-Mail zu löschen. > > > > > > > > > Bodden-Kliniken Ribnitz-Damgarten GmbH 2016 > > > *** Virenfrei durch Kerio Mail Server und Sophos Antivirus *** > > > > > > > > > > > > _______________________________________________ > > Gluster-users mailing list > > Gluster-users@xxxxxxxxxxx > > http://www.gluster.org/mailman/listinfo/gluster-users > ______________________________________________________________________________ BODDEN-KLINIKEN Ribnitz-Damgarten GmbH Sandhufe 2 18311 Ribnitz-Damgarten Telefon: 03821-700-0 Fax: 03821-700-240 E-Mail: info@xxxxxxxxxxxxxxxxxx Internet: http://www.bodden-kliniken.de Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer-Nr.: 079/133/40188 Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr. Falko Milski Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten Adressaten bestimmt. Wenn Sie nicht der vorge- sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, beachten Sie bitte, dass jede Form der Veröf- fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser E-Mail unzulässig ist. Wir bitten Sie, sofort den Absender zu informieren und die E-Mail zu löschen. Bodden-Kliniken Ribnitz-Damgarten GmbH 2016 *** Virenfrei durch Kerio Mail Server und Sophos Antivirus *** _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
