Kaushal M wrote:
On Thu, Apr 26, 2018 at 9:06 PM Mark Staudinger <mark.staudinger@xxxxxxx>
wrote:
Hi Folks,
I'm trying to debug an issue that I've found while attempting to qualify
GlusterFS for potential distributed storage projects on the FreeBSD-11.1
server platform - using the existing package of GlusterFS v3.11.1_4
The main issue I've encountered is that I cannot add new bricks while
setting/increasing the replica count.
If I create a replicated volume "poc" on two hosts, say s1:/gluster/1/poc
and s2:/gluster/1/poc, the volume is created properly and shows replicated
status, files are written to both volumes.
If I create a single volume: s1:/gluster/1/poc as a single / distributed
brick, and then try to run
gluster volume add-brick poc replica 2 s2:/gluster/1/poc
it will always fail (sometimes after a pause, sometimes not.) The only
error I'm seeing on the server hosting the new brick, aside from the
generic "Unable to add bricks" message, is like so:
I [MSGID: 106578]
[glusterd-brick-ops.c:1352:glusterd_op_perform_add_bricks] 0-management:
replica-count is set 2
I [MSGID: 106578]
[glusterd-brick-ops.c:1362:glusterd_op_perform_add_bricks] 0-management:
type is set 2, need to change it
E [MSGID: 106054]
[glusterd-utils.c:12974:glusterd_handle_replicate_brick_ops] 0-management:
Failed to set extended attribute trusted.add-brick : Operation not
supported [Operation not supported]
The log here seems to indicate the filesystem on the new brick being added
doesn't seem to support setting xattrs.
Maybe check the new brick again?
E [MSGID: 106074] [glusterd-brick-ops.c:2565:glusterd_op_add_brick]
0-glusterd: Unable to add bricks
E [MSGID: 106123] [glusterd-mgmt.c:311:gd_mgmt_v3_commit_fn] 0-management:
Add-brick commit failed.
I was initially using ZFS and noted that ZFS on FreeBSD does not support
xattr, so I reverted to using UFS as the storage type for the brick, and
still encounter this behavior.
I also recompiled the port (again, GlusterFS v3.11.1) with the patch from
https://bugzilla.redhat.com/show_bug.cgi?id=1484246 as this deals
specifically with xattr handling in FreeBSD.
To recap - I'm able to create any type of volume (2 or 3-way replicated or
distributed), but I'm unable to add replicated bricks to a volume.
I was, however, able to add a second distributed brick ( gluster volume
add-brick poc s2:/gluster/1/poc ) - so the issue seems specific to adding
and/or changing the replica count while adding a new brick.
Please let me know if there are any other issues in addition to bug
#1452961 I should be aware of, or additional log or debug info I can
provide.
Best Regards,
Mark
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users
Hi Kaushal,
That's what the error message would indicate, but UFS2 does support
xattr so that's a bit unexpected. Since I'm able to add a distributed
brick, I can rule out basic setup issues.
All the same, I can see via 'truss'
extattr_set_link(0x8081ff890,0x1,0x803b27886,0x805258c40,0x400) ERR#45
'Operation not supported’
Is anyone else using a similar setup (FreeBSD >= 11.1-RELEASE with the
official pkg) that has tested adding bricks in this way? Are there any
changes to the newfs or mount settings required for this to work that
aren't obvious?
Cheers,
-=Mark
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users
