Hi J?rgen, Thanks for your advice. I'll setup some vms later and give this a try. On my prod boxes I've compiled and installed 3.0.8 this morning as had used 3.0.x previously without issue. Using glusterfs-volgen based configuration it's up and running nicely without any problems. Cheers, Phil. On 7 September 2011 11:31, J?rgen Winkler <juergen.winkler at xidras.com>wrote: > Hi Phil, > > we?d the same Problem, try to compile with debug options. > Yes this sounds strange but it help?s when u are using SLES, the glusterd > works ok and u can start to work with it. > > just put > > exportCFLAGS='-g3 -O0' > > between %build and %configure in the glusterfs spec file. > > > > But be warned don?t use it with important data especially when u are > planing to use the replication feature, this will cause in data loss sooner > or later. > > Cheers ! > > > > > > Am 07.09.2011 11:21, schrieb gluster-users-request at gluster.**org<gluster-users-request at gluster.org> > : > >> Send Gluster-users mailing list submissions to >> gluster-users at gluster.org >> >> To subscribe or unsubscribe via the World Wide Web, visit >> http://gluster.org/cgi-bin/**mailman/listinfo/gluster-users<http://gluster.org/cgi-bin/mailman/listinfo/gluster-users> >> or, via email, send a message with subject or body 'help' to >> gluster-users-request at gluster.**org<gluster-users-request at gluster.org> >> >> You can reach the person managing the list at >> gluster-users-owner at gluster.**org<gluster-users-owner at gluster.org> >> >> When replying, please edit your Subject line so it is more specific >> than "Re: Contents of Gluster-users digest..." >> >> >> Today's Topics: >> >> 1. Re: Reading directly from brick (Reinis Rozitis) >> 2. Re: NFS secondary groups not working. (Di Pe) >> 3. Inconsistent md5sum of replicated file (Anthony Delviscio) >> 4. Re: Inconsistent md5sum of replicated file (Pranith Kumar K) >> 5. Problems with SLES 11 (Phil Bayfield) >> >> >> ------------------------------**------------------------------** >> ---------- >> >> Message: 1 >> Date: Tue, 6 Sep 2011 23:24:24 +0300 >> From: "Reinis Rozitis"<r at roze.lv> >> Subject: Re: Reading directly from brick >> To:<gluster-users at gluster.org> >> Message-ID:<**F7DAC991835C44889BCDB281F977B6**92 at NeiRoze> >> Content-Type: text/plain; format=flowed; charset="utf-8"; >> reply-type=original >> >> Simple answer - no, it's not ever safe to do writes to an active Gluster >>> backend. >>> >> Question was about reads though and then the answer is it is perfectly >> fine >> (and faster) to do reads directly from the filesystem (in replicated >> setups) >> if you keep in mind that by doing so you lose the Glusters autoheal >> eature - eg if one of the gluster nodes goes down and there is a file >> written meanwhile when the server comes up if you access the file directly >> it won't show up while it would when accessing it via the gluster mount >> point (you can work arround it by manually triggering the self heal). >> >> >> I've heard that reads from glusterfs are around 20 times slower than from >>> ext3: >>> >> "20 times" might be fetched out of thin air but of course there is a >> significant overhead of serving a file from a gluster which basically >> involves network operations and additional meta data checks versus >> fetching >> the file directly from iron. >> >> >> rr >> >> >> >> ------------------------------ >> >> Message: 2 >> Date: Tue, 6 Sep 2011 14:46:28 -0700 >> From: Di Pe<dipeit at gmail.com> >> Subject: Re: NFS secondary groups not working. >> To: gluster-users at gluster.org >> Message-ID: >> <CAB9T+o+fAb+**YasVxMsUsVmMw0Scp3BLSqc0Y_** >> grusRmV11qejg at mail.gmail.com<CAB9T%2Bo%2BfAb%2BYasVxMsUsVmMw0Scp3BLSqc0Y_grusRmV11qejg at mail.gmail.com> >> > >> Content-Type: text/plain; charset=ISO-8859-1 >> >> Anand, has this issue been confirmed by gluster and is it in the pipe >> to get fixed or do you need .additional information? We are no gluster >> experts but are happy to help if we know who to provide additional >> debugging info. >> >> On Mon, Aug 29, 2011 at 9:44 AM, Mike Hanby<mhanby at uab.edu> wrote: >> >>> I just noticed the problem happening on one client in our environment >>> (clients and servers running 3.2.2), other clients work fine. >>> >>> The clients and servers are all CentOS 5.6 x86_64 >>> >>> I get the same permission denied using Gluster FUSE and Gluster NFS >>> mounts on this client. >>> >>> I'm not mounting it with ACL. >>> >>> The volume is a simple distributed volume with two servers. >>> >>> -----Original Message----- >>>> From: gluster-users-bounces at gluster.**org<gluster-users-bounces at gluster.org>[mailto: >>>> gluster-users- >>>> bounces at gluster.org] On Behalf Of Hubert-Jan Schaminee >>>> Sent: Saturday, August 27, 2011 10:10 AM >>>> To: Anand Avati >>>> Cc: gluster-users at gluster.org >>>> Subject: Re: NFS secondary groups not working. >>>> >>>> Op zaterdag 13-08-2011 om 20:22 uur [tijdzone +0530], schreef Anand >>>> Avati: >>>> >>>>> >>>>> On Sat, Aug 13, 2011 at 5:29 PM, Dipeit<dipeit at gmail.com> wrote: >>>>> ? ? ? ? We noticed this bug too using the gluster client. I'm >>>>> ? ? ? ? surprised that not more people noticed this lack of posix >>>>> ? ? ? ? compliance. This makes gluster really unusable in multiuser >>>>> ? ? ? ? environments. Is that because gluster is mostly used in large >>>>> ? ? ? ? web farms like pandora? >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> GlusterFS is POSIX compliant w.r.t user groups. We have not seen this >>>>> issue in our testing. Can you give more info about your setup? Have >>>>> you mounted with -o acl or without? Anything unusual in the logs? >>>>> >>>>> >>>>> Avati >>>>> >>>> I'm having the same problem here. >>>> >>>> I use the latest version (3.2.3 build on Aug 23 2011 19:54:51 of the >>>> download site) on a Centos 5.6 as a gluster servers, Debian squeeze >>>> (same version) as client. >>>> I'm refused access to files and directories despite having correct >>>> group permissions. >>>> >>>> So I installed a clean Centos client (also latest version) for a test >>>> and everything is working perfectly .... ? >>>> >>>> The used Debian (squeeze) and Centos are 64 bits (repository from >>>> gluster.com). >>>> Using Debian testing (64 and 32 bits) and gluster from the Debian >>>> repository also denies me access in 64 and 32 bits version. >>>> >>>> I assume the mixed environment explains why this bug is rare. >>>> >>>> The used gluster installation is a basic replicated setup one with two >>>> servers like described in the de Gluster docs. >>>> >>>> >>>> Hubert-Jan Schamin?e >>>> >>>> >>>> >>>> >>>> >>>> >>>> ______________________________**_________________ >>>> Gluster-users mailing list >>>> Gluster-users at gluster.org >>>> http://gluster.org/cgi-bin/**mailman/listinfo/gluster-users<http://gluster.org/cgi-bin/mailman/listinfo/gluster-users> >>>> >>> ______________________________**_________________ >>> Gluster-users mailing list >>> Gluster-users at gluster.org >>> http://gluster.org/cgi-bin/**mailman/listinfo/gluster-users<http://gluster.org/cgi-bin/mailman/listinfo/gluster-users> >>> >>> >> ------------------------------ >> >> Message: 3 >> Date: Tue, 6 Sep 2011 18:52:52 -0400 >> From: Anthony Delviscio<adelviscio at gmail.com**> >> Subject: Inconsistent md5sum of replicated file >> To: gluster-users at gluster.org >> Message-ID: >> <CAKE0inQy3Tjf3TB11kc+F_F-**P7kN2CJ+eG+2FaRUxOe4tnzgwQ@** >> mail.gmail.com<CAKE0inQy3Tjf3TB11kc%2BF_F-P7kN2CJ%2BeG%2B2FaRUxOe4tnzgwQ at mail.gmail.com> >> > >> Content-Type: text/plain; charset="windows-1252" >> >> I was wondering if anyone would be able to shed some light on how a file >> could end up with inconsistent md5sums on Gluster backend storage. >> >> >> >> Our configuration is running on Gluster v3.1.5 in a distribute-replicate >> setup consisting of 8 bricks. >> >> Our OS is Red Hat 5.6 x86_64. Backend storage is an ext3 RAID 5. >> >> >> >> The 8 bricks are in RR DNS and are mounted for reading/writing via NFS >> automounts. >> >> >> >> When comparing md5sums of the file from two different NFS clients, they >> were >> different. >> >> >> >> The extended attributes of the files on backend storage are identical. >> The >> file size and permissions are identical. The stat data (excluding inode >> on >> backend storage file system) is identical. >> >> However, running md5sum on the two files, results in two different >> md5sums. >> >> >> >> Copying both files to another location/server and running the md5sum also >> results in no change ? they?re still different. >> >> >> >> Gluster logs do not show anything related to the filename in question. >> Triggering >> a self-healing operation didn?t seem to do anything and it may have to do >> with the fact that the extended attributes are identical. >> >> >> >> If more information is required, let me know and I will try to >> accommodate. >> >> >> Thank you >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL:<http://gluster.org/**pipermail/gluster-users/** >> attachments/20110906/4628faa2/**attachment-0001.htm<http://gluster.org/pipermail/gluster-users/attachments/20110906/4628faa2/attachment-0001.htm> >> > >> >> ------------------------------ >> >> Message: 4 >> Date: Wed, 7 Sep 2011 14:13:56 +0530 >> From: Pranith Kumar K<pranithk at gluster.com> >> Subject: Re: Inconsistent md5sum of replicated file >> To: Anthony Delviscio<adelviscio at gmail.com**> >> Cc: gluster-users at gluster.org >> Message-ID:<4E672ECC.7050703@**gluster.com <4E672ECC.7050703 at gluster.com> >> > >> Content-Type: text/plain; charset="windows-1252"; Format="flowed" >> >> hi Anthony, >> Could you send the output of the getfattr -d -m . -e hex >> <filepath> on both the bricks and also the stat output on the both the >> backends. Give the outputs for its parent directory also. >> >> Pranith. >> >> On 09/07/2011 04:22 AM, Anthony Delviscio wrote: >> >>> I was wondering if anyone would be able to shed some light on how a >>> file could end up with inconsistent md5sums on Gluster backend storage. >>> >>> Our configuration is running on Gluster v3.1.5 in a >>> distribute-replicate setup consisting of 8 bricks. >>> >>> Our OS is Red Hat 5.6 x86_64.Backend storage is an ext3 RAID 5. >>> >>> The 8 bricks are in RR DNS and are mounted for reading/writing via NFS >>> automounts. >>> >>> When comparing md5sums of the file from two different NFS clients, >>> they were different. >>> >>> The extended attributes of the files on backend storage are >>> identical.The file size and permissions are identical.The stat data >>> (excluding inode on backend storage file system) is identical. >>> >>> However, running md5sum on the two files, results in two different >>> md5sums. >>> >>> Copying both files to another location/server and running the md5sum >>> also results in no change ? they?re still different. >>> >>> Gluster logs do not show anything related to the filename in >>> question.Triggering a self-healing operation didn?t seem to do >>> anything and it may have to do with the fact that the extended >>> attributes are identical. >>> >>> If more information is required, let me know and I will try to >>> accommodate. >>> >>> Thank you >>> >>> >>> ______________________________**_________________ >>> Gluster-users mailing list >>> Gluster-users at gluster.org >>> http://gluster.org/cgi-bin/**mailman/listinfo/gluster-users<http://gluster.org/cgi-bin/mailman/listinfo/gluster-users> >>> >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL:<http://gluster.org/**pipermail/gluster-users/** >> attachments/20110907/86d14cab/**attachment-0001.htm<http://gluster.org/pipermail/gluster-users/attachments/20110907/86d14cab/attachment-0001.htm> >> > >> >> ------------------------------ >> >> Message: 5 >> Date: Wed, 7 Sep 2011 10:15:43 +0100 >> From: Phil Bayfield<phil at techlightenment.**com <phil at techlightenment.com> >> > >> Subject: Problems with SLES 11 >> To: gluster-users at gluster.org >> Message-ID: >> <CAFXH-**fW0DBE9YomJzAtvdFAWaf5Zpq-**TfbfTPb+K7gBu-R+06Q at mail.** >> gmail.com<CAFXH-fW0DBE9YomJzAtvdFAWaf5Zpq-TfbfTPb%2BK7gBu-R%2B06Q at mail.gmail.com> >> > >> Content-Type: text/plain; charset="iso-8859-1" >> >> Hi there, >> >> I compiled and installed the latest version of Gluster on a couple of SLES >> 11 SP1 boxes, everything up to this point seemed ok. >> >> I start the daemon on both boxes, and both are listening on 24007. >> >> I issue a "gluster peer probe" command on one of the boxes and the daemon >> instantly dies, I restart it and it shows: >> >> # gluster peer status >> Number of Peers: 1 >> >> Hostname: mckalcpap02 >> Uuid: 00000000-0000-0000-0000-**000000000000 >> State: Establishing Connection (Connected) >> >> I attempted to run the probe on the other box, the daemon crashes, now as >> I >> start the daemon on each box the daemon just crashes on the other box. >> >> The log output immediately prior to the crash is as follows: >> >> [2011-06-07 08:05:10.700710] I >> [glusterd-handler.c:623:**glusterd_handle_cli_probe] 0-glusterd: Received >> CLI >> probe req mckalcpap02 24007 >> [2011-06-07 08:05:10.701058] I [glusterd-handler.c:391:** >> glusterd_friend_find] >> 0-glusterd: Unable to find hostname: mckalcpap02 >> [2011-06-07 08:05:10.701086] I >> [glusterd-handler.c:3422:**glusterd_probe_begin] 0-glusterd: Unable to >> find >> peerinfo for host: mckalcpap02 (24007) >> [2011-06-07 08:05:10.702832] I [glusterd-handler.c:3404:** >> glusterd_friend_add] >> 0-glusterd: connect returned 0 >> [2011-06-07 08:05:10.703110] I >> [glusterd-handshake.c:317:**glusterd_set_clnt_mgmt_**program] 0-: Using >> Program >> glusterd clnt mgmt, Num (1238433), Version (1) >> >> If I use the IP address the same thing happens: >> >> [2011-06-07 08:07:12.873075] I >> [glusterd-handler.c:623:**glusterd_handle_cli_probe] 0-glusterd: Received >> CLI >> probe req 10.9.54.2 24007 >> [2011-06-07 08:07:12.873410] I [glusterd-handler.c:391:** >> glusterd_friend_find] >> 0-glusterd: Unable to find hostname: 10.9.54.2 >> [2011-06-07 08:07:12.873438] I >> [glusterd-handler.c:3422:**glusterd_probe_begin] 0-glusterd: Unable to >> find >> peerinfo for host: 10.9.54.2 (24007) >> [2011-06-07 08:07:12.875046] I [glusterd-handler.c:3404:** >> glusterd_friend_add] >> 0-glusterd: connect returned 0 >> [2011-06-07 08:07:12.875280] I >> [glusterd-handshake.c:317:**glusterd_set_clnt_mgmt_**program] 0-: Using >> Program >> glusterd clnt mgmt, Num (1238433), Version (1) >> >> There is no firewall issue: >> >> # telnet mckalcpap02 24007 >> Trying 10.9.54.2... >> Connected to mckalcpap02. >> Escape character is '^]'. >> >> Following restart (which crashes the other node) the log output is as >> follows: >> >> [2011-06-07 08:10:09.616486] I [glusterd.c:564:init] 0-management: Using >> /etc/glusterd as working directory >> [2011-06-07 08:10:09.617619] C [rdma.c:3933:rdma_init] >> 0-rpc-transport/rdma: >> Failed to get IB devices >> [2011-06-07 08:10:09.617676] E [rdma.c:4812:init] 0-rdma.management: >> Failed >> to initialize IB Device >> [2011-06-07 08:10:09.617700] E [rpc-transport.c:741:rpc_**transport_load] >> 0-rpc-transport: 'rdma' initialization failed >> [2011-06-07 08:10:09.617724] W [rpcsvc.c:1288:rpcsvc_**transport_create] >> 0-rpc-service: cannot create listener, initing the transport failed >> [2011-06-07 08:10:09.617830] I [glusterd.c:88:glusterd_uuid_**init] >> 0-glusterd: retrieved UUID: 1e344f5d-6904-4d14-9be2-**8f0f44b97dd7 >> [2011-06-07 08:10:11.258098] I [glusterd-handler.c:3404:** >> glusterd_friend_add] >> 0-glusterd: connect returned 0 >> Given volfile: >> +-----------------------------**------------------------------** >> -------------------+ >> 1: volume management >> 2: type mgmt/glusterd >> 3: option working-directory /etc/glusterd >> 4: option transport-type socket,rdma >> 5: option transport.socket.keepalive-**time 10 >> 6: option transport.socket.keepalive-**interval 2 >> 7: end-volume >> 8: >> >> +-----------------------------**------------------------------** >> -------------------+ >> [2011-06-07 08:10:11.258431] I >> [glusterd-handshake.c:317:**glusterd_set_clnt_mgmt_**program] 0-: Using >> Program >> glusterd clnt mgmt, Num (1238433), Version (1) >> [2011-06-07 08:10:11.280533] W [socket.c:1494:__socket_proto_** >> state_machine] >> 0-socket.management: reading from socket failed. Error (Transport endpoint >> is not connected), peer (10.9.54.2:1023) >> [2011-06-07 08:10:11.280595] W [socket.c:1494:__socket_proto_** >> state_machine] >> 0-management: reading from socket failed. Error (Transport endpoint is not >> connected), peer (10.9.54.2:24007) >> [2011-06-07 08:10:17.256235] E [socket.c:1685:socket_connect_**finish] >> 0-management: connection to 10.9.54.2:24007 failed (Connection refused) >> >> There are no logs on the node which crashes. >> >> I've tried various possibly solutions from searching the net but got >> getting >> anywhere, can anyone advise how to proceed? >> >> Thanks, >> Phil. >> >> > ______________________________**_________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/**mailman/listinfo/gluster-users<http://gluster.org/cgi-bin/mailman/listinfo/gluster-users> > -- Phil Bayfield Development Manager Alchemy Social, part of Techlightenment, an Experian company Office 202 | 89 Worship Street | London | EC2A 2BF t: +44 (0) 207 392 2618 m: +44 (0) 7825 561 091 e: phil at techlightenment.com <phil at techlightenment.com>skype: phil.tl www.techlightenment.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://gluster.org/pipermail/gluster-users/attachments/20110907/a1935629/attachment-0001.htm>
