Here is a quick way how to test this: GlusterFS 3.7.13 volume with default settings with brick on ZFS dataset. gluster-test1 is server and gluster-test2 is client mounting with FUSE. Writing file with oflag=direct is not ok: [root@gluster-test2 gluster]# dd if=/dev/zero of=file oflag=direct count=1 bs=1024000 dd: failed to open ‘file’: Invalid argument Enable network.remote-dio on Gluster Volume: [root@gluster-test1 gluster]# gluster volume set gluster network.remote-dio enable volume set: success Writing small file with oflag=direct is ok: [root@gluster-test2 gluster]# dd if=/dev/zero of=file oflag=direct count=1 bs=1024000 1+0 records in 1+0 records out 1024000 bytes (1.0 MB) copied, 0.0103793 s, 98.7 MB/s Writing bigger file with oflag=direct is ok: [root@gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct count=100 bs=1M 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.10583 s, 94.8 MB/s Enable Sharding on Gluster Volume: [root@gluster-test1 gluster]# gluster volume set gluster features.shard enable volume set: success Writing small file with oflag=direct is ok: [root@gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct count=1 bs=1M 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115247 s, 91.0 MB/s Writing bigger file with oflag=direct is not ok: [root@gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct count=100 bs=1M dd: error writing ‘file3’: Operation not permitted dd: closing output file ‘file3’: Operation not permitted -samuli > On 22 Jul 2016, at 16:12, Vijay Bellur <vbellur@xxxxxxxxxx> wrote: > > 2016-07-22 1:54 GMT-04:00 Frank Rothenstein <f.rothenstein@xxxxxxxxxxxxxxxxxx>: >> The point is that even if all other backend storage filesystems do correctly >> untill 3.7.11 there was no error on ZFS. Something happened nobody ever >> could explain in the release of 3.7.12 that makes FUSE-mount _in ovirt_ (it >> partly uses dd with iflag=direct , using iflag=direct yourself gives also >> errors on the FUSE-mounts ) unusable. >> >> So 3.7.11 is the last usable version when using ZFS on bricks, afaik. >> > > Can you please share the exact dd command that causes this problem? > > Thanks, > Vijay > _______________________________________________ > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx > http://www.gluster.org/mailman/listinfo/gluster-users _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
