Issue for replicate translator

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi, All:

I've met a problem when doing unit test for replicate translator. And this
essential problem has bothered me for 2 weeks.

My setup is 4 copies in 4 machines (A,B,C,D), all A,B,C,D are acting both
server and client(the mount point is /home/),
and there is another machine E, which doesn't has any copy, just act as
purely client and utilize all A,B,C,D's disks

my simulate failure strategy is:

Step 1. randomly choose 1 machine, fail the NIC glusterfs is listen on(there
are still 3 copies on-line)
Step 2. sleep for a while (like 60 seconds)
Step 3. bring up the NIC I failed before (there are 4 copies on-line right
now)
Step 4. do "ls -laR /home" on the failed machine before
Step 5. goto step 1

Simultaneously,  I'm also doing `ls -laR /home > test_result.txt` on the
machine E for times.

I've found the problems like:
1. missing files in directory or duplicate name in the same directory in the
ls output, like:
   see the small part of different output of `ls -laR`, this is the vimdiff
output.
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file84                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file83
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file84                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file84
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file85                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file85

----------------------------------------------------------------------------------|
-rw-r--r--   1 root root 5004454 2009-10-16 19:21
file86

----------------------------------------------------------------------------------|
-rw-r--r--   1 root root 5004454 2009-10-16 19:21
file87
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file88                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file88
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file89                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file89
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file89                          |
----------------------------------------------------------------------------------
  -rw-r--r--   1 root root 5004454 2009-10-16 19:20
file9                           |  -rw-r--r--   1 root root 5004454
2009-10-16 19:20 file9
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file90                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file90
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file91                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file91
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file91                          |
----------------------------------------------------------------------------------
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file92                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file92
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file93                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file93
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file94                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file94
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file95                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file95
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file97                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file96
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file97                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file97
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file98                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file98
  -rw-r--r--   1 root root 5004454 2009-10-16 19:21
file99                          |  -rw-r--r--   1 root root 5004454
2009-10-16 19:21 file99
    There are apparently some files are missing and duplicate name, all are
in the same directory.

2. occasionally, the ls reports:
    ls: reading directory /home/dir1/dir21: File descriptor in bad state

I really want guys can solve this basic and essential problem

The glusterfsd.vol I'm using for all 5 machines is:
=========================================================================================
# THIS IS THE SERVER-END CONFIGURATION
# Brick 1
volume posix
        type storage/posix
        option directory /mnt/disk1
end-volume

volume locks
        type features/locks
        subvolumes posix
end-volume

volume brick
        type performance/io-threads
        option thread-count 16
        subvolumes locks
end-volume

# Server
volume server
        type protocol/server
        option transport-type tcp/server
        option transport.socket.bind-address `ifconfig -a | grep
"10.106.105." | awk '{print $2}' | awk 'BEGIN {FS=":"};{print $2}'`
        option transport.socket.listen-port 6996
        subvolumes brick
        option auth.addr.brick.allow *
end-volume

# SERVER-END CONFIGURATION ENDS

# THIS IS THE CLIENT-END CONFIGURATION
# 3 Disks Machines
# Machine 1
volume cbrick1
        type protocol/client
        option transport-type tcp
        option remote-host 10.106.105.150
        option remote-port 6996
        option remote-subvolume brick
end-volume

# Machine 2
volume cbrick4
        type protocol/client
        option transport-type tcp
        option remote-host 10.106.105.151
        option remote-port 6996
        option remote-subvolume brick
end-volume

# Machine 3
volume cbrick7
        type protocol/client
        option transport-type tcp
        option remote-host 10.106.105.152
        option remote-port 6996
        option remote-subvolume brick
end-volume

# Machine 4
volume cbrick10
        type protocol/client
        option transport-type tcp
        option remote-host 10.106.105.153
        option remote-port 6996
        option remote-subvolume brick
end-volume

# All the bricks delare finished

# Replicate part
volume rep1
        type cluster/replicate
        subvolumes cbrick1 cbrick4 cbrick7 cbrick10
end-volume

# CLIENT END CONFIGURATION ENDS
========================================================================================================



Regards,
Zhuo Yin (917)215-8740
Gentoo Linux Fan - int (*(*(*pFile)())[10])();


[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux