Re: GlusterFS + ZFS Error

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



We have two servers each with ZFS running in raidz on three hard drives of the same size and glusterFS between the two servers.  The ZFS has two filesystems tank and tank/home each created using zfs create and GlusterFS is mounted based on the tank/home filesystem to /mnt.  At this point everything works great but when I then run zfs create tank/home/virtual and then cd to /mnt/virtual and run ls I get the errors i reported to printed to the command line.

Volume files:
glusterfsd.vol
## file auto generated by /bin/glusterfs-volgen
# Cmd line:
# $ /bin/glusterfs-volgen --name home --raid 1 mx.server.edu:/tank/home mx2.server.edu:/tank/home

volume posix1
    type storage/posix
#   option o-direct enable # (default: disable) boolean type only
#   option export-statfs-size no # (default: yes) boolean type only
#   option mandate-attribute off # (default: on) boolean type only
#   option span-devices 8 # (default: 0) integer value
#   option background-unlink yes # (default: no) boolean type
    option directory /tank/home
end-volume

volume locks1
    type features/locks
#   option mandatory on # Default off, used in specific applications
    subvolumes posix1
end-volume

volume brick1
    type performance/io-threads
    option thread-count 8
#   option autoscaling yes # Heuristic for autoscaling threads on demand
#   option min-threads 2 # min count for thread pool
#   option max-threads 64 # max count for thread pool
    subvolumes locks1
end-volume

volume server-tcp
    type protocol/server
    option transport-type tcp
    option auth.addr.brick1.allow *
    option transport.socket.listen-port 6996
    option transport.socket.nodelay on
    subvolumes brick1
end-volume

glusterfs.vol
## file auto generated by /bin/glusterfs-volgen
# Cmd line:
# $ /bin/glusterfs-volgen --name home --raid 1 mx.server.edu:/tank/home mx2.server.edu:/tank/home

# RAID 1
# TRANSPORT-TYPE tcp
volume mx.server.edu-1
    type protocol/client
    option transport-type tcp
    option remote-host mx.server.edu
    option transport.socket.nodelay on
    option transport.remote-port 6996
    option remote-subvolume brick1
end-volume

volume mx2.server.edu-1
    type protocol/client
    option transport-type tcp
    option remote-host mx2.server.edu
    option transport.socket.nodelay on
    option transport.remote-port 6996
    option remote-subvolume brick1
end-volume

volume mirror-0
    type cluster/replicate
    subvolumes mx.server.edu-1 mx2.server.edu-1
end-volume

volume writebehind
    type performance/write-behind
    option cache-size 4MB
#   option enable-trickling-writes yes # Flush final write calls when network is free
#   option enable-O_SYNC yes # Enable O_SYNC for write-behind
#   option disable-for-first-nbytes 1 # Disable first nbytes with very small initial writes
    subvolumes mirror-0
end-volume

volume readahead
    type performance/read-ahead
    option page-count 4
#   option force-atime-update yes # force updating atimes, default off
    subvolumes writebehind
end-volume

volume iocache
    type performance/io-cache
    option cache-size `echo $(( $(grep 'MemTotal' /proc/meminfo | sed 's/[^0-9]//g') / 5120 ))`MB
    option cache-timeout 1
#   option priority *.html:1,abc*:2 # Priority list for iocaching files
    subvolumes readahead
end-volume

volume quickread
    type performance/quick-read
    option cache-timeout 1
    option max-file-size 64kB
    subvolumes iocache
end-volume

volume statprefetch
    type performance/stat-prefetch
    subvolumes quickread
end-volume


and how do you want the log files as they are already pretty big

James J Regan IV
COO Emeritus Consulting
B.S. Computer Science
B.A. Linguistics


On Wed, Jul 7, 2010 at 5:40 AM, Lakshmipathi <lakshmipathi@xxxxxxxxxxx> wrote:
Hi,
Could you please explain more about your setup in detail ? Are you trying to create a sub-file system in glusterfs-server using zfs create command?
Please sent us the volume and log files.Thanks.

--
----
Cheers,
Lakshmipathi.G
FOSS Programmer.

-------- Original Message --------
Subject: GlusterFS + ZFS Error
Date: Tue, 6 Jul 2010 17:10:10 -0400
From: Regan_James <jamreg@xxxxxxxxxxxxxxxxxxxxxxxx>
To: gluster-devel <gluster-devel@xxxxxxxxxx>


I set up a ZFS + glusterFS system and everything worked fine until I ran a zfs create command to create a subfile-system at which point the GlusterFS became corrupted and ls commands of the new filesystem first gave me an error "cannot access <filesystem>: Device or resource busy" and then the next and all other times I ran it I got "cannot open directory .: No such file or directory." This error happens every time I do, when only one is running or when both are running

James J Regan IV
COO Emeritus Consulting
B.S. Computer Science
B.A. Linguistics




[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux