Hi All, I'm trying to configure GlusterFS 3.1.2 with cluster/switch xlator. My goal is to store data on local disks, and minimize network traffic. I was planing to have a directory for every node in GlusterFS root name space, and use switch scheduler. I am getting random errors on file creating. Underlying file system is XFS. Here is the volume file and log excerpt. Any suggestions? Thank you. volume posix type storage/posix option directory /local/gluster_volumes/test-volume end-volume volume locks type features/locks subvolumes posix end-volume volume brick type performance/io-threads subvolumes locks end-volume volume server type protocol/server option transport-type rdma option auth.addr.brick.allow * subvolumes brick end-volume volume node1 type protocol/client option transport-type rdma option remote-host node1 option remote-subvolume brick end-volume volume node2 type protocol/client option transport-type rdma option remote-host node2 option remote-subvolume brick end-volume volume cluster-switch type cluster/switch option lookup-unhashed on option pattern.switch.case /node1*:node1;*:node2 subvolumes node1 node2 end-volume [2011-03-02 15:26:35.594999] T [server-resolve.c:127:resolve_loc_touchup] server: return value inode_path 1 [2011-03-02 15:26:35.595033] D [dht-linkfile.c:138:dht_linkfile_create] : dict is NULL, need to make sure gfid's are same [2011-03-02 15:26:35.595143] T [auth-glusterfs.c:185:auth_glusterfs_authenticate] rpc-service: Auth Info: pid: 24584, uid: 572, gid: 513, owner: 24584 [2011-03-02 15:26:35.595167] T [server-resolve.c:127:resolve_loc_touchup] server: return value inode_path 10 [2011-03-02 15:26:35.595276] D [dht-diskusage.c:69:dht_du_info_cbk] cluster-switch: on subvolume 'node9-009': avail_percent is: 99.00 and avail_space is: 1768715341824 [2011-03-02 15:26:35.595313] D [dht-diskusage.c:69:dht_du_info_cbk] cluster-switch: on subvolume 'node9-008': avail_percent is: 99.00 and avail_space is: 1766295830528 [2011-03-02 15:26:35.595362] D [posix.c:265:posix_lstat_with_gfid] posix: failed to get gfid [2011-03-02 15:26:35.595381] E [posix.c:1127:posix_mknod] posix: mknod on /file.test failed: No data available [2011-03-02 15:26:35.595394] T [server3_1-fops.c:511:server_mknod_cbk] server: 61: MKNOD /file.test ==> -1 (No data available) [2011-03-02 15:26:35.595471] W [fuse-bridge.c:1514:fuse_create_cbk] glusterfs-fuse: 80: /file.test => -1 (No data available) [2011-03-02 15:26:35.595532] T [fuse-bridge.c:279:fuse_lookup] glusterfs-fuse: 81: LOOKUP /file.test [2011-03-02 15:26:35.595989] T [auth-glusterfs.c:185:auth_glusterfs_authenticate] rpc-service: Auth Info: pid: 24584, uid: 572, gid: 513, owner: 24584 [2011-03-02 15:26:35.595998] T [rpcsvc.c:723:rpcsvc_program_actor] rpc-service: Actor found: GlusterFS-3.1.0 - LOOKUP [2011-03-02 15:26:35.596016] T [server-resolve.c:127:resolve_loc_touchup] server: return value inode_path 10 [2011-03-02 15:26:35.596094] T [rpc-clnt.c:631:rpc_clnt_reply_init] rpc-clnt: recieved rpc message (RPC XID: 0x109x Program: GlusterFS 3.1, ProgVers: 310, Proc: 27) from rpc-transport (node9-009) [2011-03-02 15:26:35.596122] T [rpcsvc.c:1516:rpcsvc_submit_generic] rpc-service: Tx message: 216 [2011-03-02 15:26:35.596152] T [rpcsvc.c:1151:rpcsvc_record_build_header] rpc-service: Reply fraglen 240, payload: 216, rpc hdr: 24 [2011-03-02 15:26:35.596167] T [rpcsvc.c:1555:rpcsvc_submit_generic] rpc-service: submitted reply for rpc-message (XID: 0x62x, Program: GlusterFS-3.1.0, ProgVers: 310, Proc: 27) to rpc-transport (rdma.server) [2011-03-02 15:26:35.596181] T [rpc-clnt.c:631:rpc_clnt_reply_init] rpc-clnt: recieved rpc message (RPC XID: 0x62x Program: GlusterFS 3.1, ProgVers: 310, Proc: 27) from rpc-transport (node9-008) [2011-03-02 15:26:35.596205] D [dht-common.c:566:dht_lookup_everywhere_cbk] cluster-switch: found on node9-008 linkfile /file.test (-> '') [2011-03-02 15:26:35.596216] D [dht-common.c:605:dht_lookup_everywhere_cbk] cluster-switch: deleting stale linkfile /file.test on node9-008 Gvozden Neskovic neskovic at gmail.com
