Reliability issues with Gluster 3.10 and shard

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello all,

I'm trying to take advantage of the shard xlator, however I've found it causes a lot of issues that I hope is easily resolvable

1) large file operations work well (copy file from folder a to folder b
2) seek operations and list operations frequently fail (ls directory, read bytes xyz at offset 235567)

Turning off the shard feature resolves this issue for new files created in the volume. mounted using the gluster fuse mount

here's my volume settings, please let me know if there's some changes I can make.

Option                                  Value                                  
------                                  -----                                  
cluster.lookup-unhashed                 on                                     
cluster.lookup-optimize                 on                                     
cluster.min-free-disk                   10%                                    
cluster.min-free-inodes                 5%                                     
cluster.rebalance-stats                 off                                    
cluster.subvols-per-directory           (null)                                 
cluster.readdir-optimize                off                                    
cluster.rsync-hash-regex                (null)                                 
cluster.extra-hash-regex                (null)                                 
cluster.dht-xattr-name                  trusted.glusterfs.dht                  
cluster.randomize-hash-range-by-gfid    off                                    
cluster.rebal-throttle                  normal                                 
cluster.lock-migration                  off                                    
cluster.local-volume-name               (null)                                 
cluster.weighted-rebalance              on                                     
cluster.switch-pattern                  (null)                                 
cluster.entry-change-log                on                                     
cluster.read-subvolume                  (null)                                 
cluster.read-subvolume-index            -1                                     
cluster.read-hash-mode                  1                                      
cluster.background-self-heal-count      8                                      
cluster.metadata-self-heal              on                                     
cluster.data-self-heal                  on                                     
cluster.entry-self-heal                 on                                     
cluster.self-heal-daemon                on                                     
cluster.heal-timeout                    600                                    
cluster.self-heal-window-size           1                                      
cluster.data-change-log                 on                                     
cluster.metadata-change-log             on                                     
cluster.data-self-heal-algorithm        diff                                   
cluster.eager-lock                      enable                                 
disperse.eager-lock                     on                                     
cluster.quorum-type                     auto                                   
cluster.quorum-count                    (null)                                 
cluster.choose-local                    on                                     
cluster.self-heal-readdir-size          1KB                                    
cluster.post-op-delay-secs              1                                      
cluster.ensure-durability               on                                     
cluster.consistent-metadata             no                                     
cluster.heal-wait-queue-length          128                                    
cluster.favorite-child-policy           none                                   
cluster.stripe-block-size               128KB                                  
cluster.stripe-coalesce                 true                                   
diagnostics.latency-measurement         off                                    
diagnostics.dump-fd-stats               off                                    
diagnostics.count-fop-hits              off                                    
diagnostics.brick-log-level             INFO                                   
diagnostics.client-log-level            INFO                                   
diagnostics.brick-sys-log-level         CRITICAL                               
diagnostics.client-sys-log-level        CRITICAL                               
diagnostics.brick-logger                (null)                                 
diagnostics.client-logger               (null)                                 
diagnostics.brick-log-format            (null)                                 
diagnostics.client-log-format           (null)                                 
diagnostics.brick-log-buf-size          5                                      
diagnostics.client-log-buf-size         5                                      
diagnostics.brick-log-flush-timeout     120                                    
diagnostics.client-log-flush-timeout    120                                    
diagnostics.stats-dump-interval         0                                      
diagnostics.fop-sample-interval         0                                      
diagnostics.fop-sample-buf-size         65535                                  
diagnostics.stats-dnscache-ttl-sec      86400                                  
performance.cache-max-file-size         0                                      
performance.cache-min-file-size         0                                      
performance.cache-refresh-timeout       1                                      
performance.cache-priority                                                     
performance.cache-size                  1GB                                    
performance.io-thread-count             64                                     
performance.high-prio-threads           16                                     
performance.normal-prio-threads         16                                     
performance.low-prio-threads            32                                     
performance.least-prio-threads          1                                      
performance.enable-least-priority       on                                     
performance.cache-size                  1GB                                    
performance.flush-behind                on                                     
performance.nfs.flush-behind            on                                     
performance.write-behind-window-size    2GB                                    
performance.resync-failed-syncs-after-fsyncoff                                    
performance.nfs.write-behind-window-size1MB                                    
performance.strict-o-direct             off                                    
performance.nfs.strict-o-direct         off                                    
performance.strict-write-ordering       off                                    
performance.nfs.strict-write-ordering   off                                    
performance.lazy-open                   yes                                    
performance.read-after-open             no                                     
performance.read-ahead-page-count       4                                      
performance.md-cache-timeout            1                                      
performance.cache-swift-metadata        true                                   
performance.cache-samba-metadata        false                                  
performance.cache-capability-xattrs     true                                   
performance.cache-ima-xattrs            on                                     
features.encryption                     off                                    
encryption.master-key                   (null)                                 
encryption.data-key-size                256                                    
encryption.block-size                   4096                                   
network.frame-timeout                   1800                                   
network.ping-timeout                    42                                     
network.tcp-window-size                 (null)                                 
features.lock-heal                      off                                    
features.grace-timeout                  10                                     
network.remote-dio                      disable                                
client.event-threads                    3                                      
network.ping-timeout                    42                                     
network.tcp-window-size                 (null)                                 
network.inode-lru-limit                 90000                                  
auth.allow                              *                                      
auth.reject                             (null)                                 
transport.keepalive                     on                                     
server.allow-insecure                   on                                     
server.root-squash                      off                                    
server.anonuid                          65534                                  
server.anongid                          65534                                  
server.statedump-path                   /var/run/gluster                       
server.outstanding-rpc-limit            64                                     
features.lock-heal                      off                                    
features.grace-timeout                  10                                     
server.ssl                              (null)                                 
auth.ssl-allow                          *                                      
server.manage-gids                      off                                    
server.dynamic-auth                     on                                     
client.send-gids                        on                                     
server.gid-timeout                      300                                    
server.own-thread                       (null)                                 
server.event-threads                    3                                      
ssl.own-cert                            (null)                                 
ssl.private-key                         (null)                                 
ssl.ca-list                             (null)                                 
ssl.crl-path                            (null)                                 
ssl.certificate-depth                   (null)                                 
ssl.cipher-list                         (null)                                 
ssl.dh-param                            (null)                                 
ssl.ec-curve                            (null)                                 
transport.address-family                inet6                                  
performance.write-behind                on                                     
performance.read-ahead                  off                                    
performance.readdir-ahead               on                                     
performance.io-cache                    on                                     
performance.quick-read                  off                                    
performance.open-behind                 on                                     
performance.stat-prefetch               on                                     
performance.client-io-threads           on                                     
performance.nfs.write-behind            on                                     
performance.nfs.read-ahead              off                                    
performance.nfs.io-cache                off                                    
performance.nfs.quick-read              off                                    
performance.nfs.stat-prefetch           off                                    
performance.nfs.io-threads              off                                    
performance.force-readdirp              true                                   
performance.cache-invalidation          false                                  
features.uss                            off                                    
features.snapshot-directory             .snaps                                 
features.show-snapshot-directory        off                                    
network.compression                     off                                    
network.compression.window-size         -15                                    
network.compression.mem-level           8                                      
network.compression.min-size            0                                      
network.compression.compression-level   -1                                     
network.compression.debug               false                                  
features.limit-usage                    (null)                                 
features.quota-timeout                  0                                      
features.default-soft-limit             80%                                    
features.soft-timeout                   60                                     
features.hard-timeout                   5                                      
features.alert-time                     86400                                  
features.quota-deem-statfs              off                                    
geo-replication.indexing                off                                    
geo-replication.indexing                off

-ben
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users

[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux