What happens when you read the file? Do you see right contents? These
look like linked files created in order to locate the files in the
right server. Did you recently upgrade or add/remove bricks?
Can you also look at the gfid on these files from server side?
Run
getfattr -dm - <file name>
On Mon, May 16, 2011 at 2:19 AM, Martin Schenker
<martin.schenker at profitbricks.com> wrote:
> Hi all!
>
> Here we have another mismatch between the client "view" and the server
> mounts:
>
> From the server site everything seems well, the 20G file is visible and the
> attributes seem to match:
>
> ??????? 0 root at pserver5:~ # getfattr -R -d -e hex -m "trusted.afr."
> /mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f-8542864da6ef/hdd-images/
>
> ??????? # file:
> mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f-8542864da6ef/hdd-images//20964
> ??????? trusted.afr.storage0-client-2=0x000000000000000000000000
> ??????? trusted.afr.storage0-client-3=0x000000000000000000000000
>
> ??????? 0 root at pserver5:~ # find /mnt/gluster/ -name 20964 | xargs -i ls -al
> {}
> ??????? -rwxrwx--- 1 libvirt-qemu vcb 21474836480 May 13 11:21
> /mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f-8542864da6ef/hdd-images/20964
>
> But the client view shows 2!! files with 0 byte size!! And these aren't any
> link files created by Gluster? ( with the T on the end)
>
> ??????? 0 root at pserver5:~ # find /opt/profitbricks/storage/ -name 20964 |
> xargs -i ls -al {}
> ??????? -rwxrwx--- 1 libvirt-qemu kvm 0 May 13 11:24
> /opt/profitbricks/storage/images/2078/ebb83b05-3a83-9d18-ad8f-8542864da6ef/hdd-images/20964
>
> ??????? -rwxrwx--- 1 libvirt-qemu kvm 0 May 13 11:24
> /opt/profitbricks/storage/images/2078/ebb83b05-3a83-9d18-ad8f-8542864da6ef/hdd-images/20964
>
> I'm a bit stumped that we seem to have so many weird errors cropping up. Any
> ideas? I've checked the ext4 filesystem on all boxes, no real problems. We
> run a distributed cluster with 4 servers offering 2 bricks each.
>
> Best, Martin
>
>
>> -----Original Message-----
>> From: Mohit Anchlia [mailto:mohitanchlia at gmail.com]
>> Sent: Monday, May 16, 2011 2:24 AM
>> To: Martin Schenker
>> Cc: gluster-users at gluster.org
>> Subject: Re: Brick pair file mismatch,
>> self-heal problems?
>>
>>
>> Try this to trigger self heal:
>>
>> find <gluster-mount> -noleaf -print0 -name <file name>| xargs
>> --null stat >/dev/null
>>
>>
>>
>> On Sun, May 15, 2011 at 11:20 AM, Martin Schenker
>> <martin.schenker at profitbricks.com> wrote:
>> > Can someone enlighten me what's going on here? We have a two peers,
>> > the file 21313 is shown through the client mountpoint as
>> "1Jan1970",
>> > attribs on server pserver3 don't match but NO self-heal or
>> repair can
>> > be triggered through "ls -alR"?!?
>> >
>> > Checking the files through the server mounts show that two versions
>> > are on the system. But the wrong one (as with the
>> "1Jan1970") seems to
>> > be the preferred one by the client?!?
>> >
>> > Do I need to use setattr or what in order to get the client
>> to see the
>> > RIGHT version?!? This is not the ONLY file displaying this
>> problematic
>> > behaviour!
>> >
>> > Thanks for any feedback.
>> >
>> > Martin
>> >
>> > pserver5:
>> >
>> > 0 root at pserver5:~ # ls -al
>> >
>> /mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f-854286
>> > 4da6ef
>> > /hdd-images
>> >
>> > -rwxrwx--- 1 libvirt-qemu vcb ?483183820800 May 13 13:41 21313
>> >
>> > 0 root at pserver5:~ # getfattr -R -d -e hex -m "trusted.afr."
>> >
>> /mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f-854286
>> > 4da6ef
>> > /hdd-images/21313
>> > getfattr: Removing leading '/' from absolute path names
>> > # file:
>> >
>> mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f
>> -8542864da6ef/
>> > hdd-images/21313
>> > trusted.afr.storage0-client-2=0x000000000000000000000000
>> > trusted.afr.storage0-client-3=0x000000000000000000000000
>> >
>> > 0 root at pserver5:~ # ls -alR
>> >
>> /opt/profitbricks/storage/images/2078/ebb83b05-3a83-9d18-ad8f-8542864d
>> > a6ef/h
>> > dd-images/21313
>> > -rwxrwx--- 1 libvirt-qemu kvm 483183820800 Jan ?1 ?1970
>> >
>> /opt/profitbricks/storage/images/2078/ebb83b05-3a83-9d18-ad8f-
>> 8542864da6ef/h
>> > dd-images/21313
>> >
>> > pserver3:
>> >
>> > 0 root at pserver3:~ # ls -al
>> >
>> /mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f-854286
>> > 4da6ef
>> > /hdd-images
>> >
>> > -rwxrwx--- 1 libvirt-qemu kvm ?483183820800 Jan ?1 ?1970 21313
>> >
>> > 0 root at pserver3:~ # ls -alR
>> >
>> /opt/profitbricks/storage/images/2078/ebb83b05-3a83-9d18-ad8f-8542864d
>> > a6ef/h
>> > dd-images/21313
>> > -rwxrwx--- 1 libvirt-qemu kvm 483183820800 Jan ?1 ?1970
>> >
>> /opt/profitbricks/storage/images/2078/ebb83b05-3a83-9d18-ad8f-
>> 8542864da6ef/h
>> > dd-images/21313
>> >
>> > 0 root at pserver3:~ # getfattr -R -d -e hex -m "trusted.afr."
>> > /mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-
>> > ad8f-8542864da6ef/hdd-images/21313
>> > getfattr: Removing leading '/' from absolute path names
>> > # file:
>> >
>> mnt/gluster/brick1/storage/images/2078/ebb83b05-3a83-9d18-ad8f-8542864
>> > da6ef/
>> > hdd-images/21313
>> > trusted.afr.storage0-client-2=0x000000000000000000000000
>> > trusted.afr.storage0-client-3=0x0b0000090900000000000000
>> <- mismatch,
>> > should be targeted for self-heal/repair? Why is there a
>> difference in the
>> > views?
>> >
>> >
>> > From the volfile:
>> >
>> > volume storage0-client-2
>> > ? ?type protocol/client
>> > ? ?option remote-host de-dc1-c1-pserver3
>> > ? ?option remote-subvolume /mnt/gluster/brick1/storage
>> > ? ?option transport-type rdma
>> > ? ?option ping-timeout 5
>> > end-volume
>> >
>> > volume storage0-client-3
>> > ? ?type protocol/client
>> > ? ?option remote-host de-dc1-c1-pserver5
>> > ? ?option remote-subvolume /mnt/gluster/brick1/storage
>> > ? ?option transport-type rdma
>> > ? ?option ping-timeout 5
>> > end-volume
>> >
>> >
>> >
>> > _______________________________________________
>> > Gluster-users mailing list
>> > Gluster-users at gluster.org
>> > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>> >
>>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
>
