-Atin
Sent from one plus one
On 05-Mar-2016 11:46 am, "Ajil Abraham" <ajil95.abraham@xxxxxxxxx> wrote:
>
> Thanks for all the support. After handling the input validation in my code, Glusterd no longer crashes. I am still waiting for clearance from my superior to pass on all the details. Expecting him to revert by this Sunday.
Great to know that and we appreciate your contribution, if you happen to find any issues feel free to send patches :)
>
> - Ajil
>
> On Fri, Mar 4, 2016 at 10:20 AM, Joseph Fernandes <josferna@xxxxxxxxxx> wrote:
>>
>> Well that may not be completely correct !
>>
>> Its "gluster volume status all", unlike volume maintenance operation which are rare.
>>
>> Status can be issued multiple times in a day or might be put in a script/cron-job to check the health of the
>> cluster.
>> But anyways the fix is ready as the bug says.
>>
>> Crash is what we need to worry about.
>>
>> ~Joe
>>
>> ----- Original Message -----
>> > From: "Atin Mukherjee" <amukherj@xxxxxxxxxx>
>> > To: "Joseph Fernandes" <josferna@xxxxxxxxxx>, "Atin Mukherjee" <atin.mukherjee83@xxxxxxxxx>
>> > Cc: "Gluster Devel" <gluster-devel@xxxxxxxxxxx>, "Ajil Abraham" <ajil95.abraham@xxxxxxxxx>
>> > Sent: Friday, March 4, 2016 9:37:43 AM
>> > Subject: Re: glusterd crashing
>> >
>> >
>> >
>> > On 03/04/2016 07:10 AM, Joseph Fernandes wrote:
>> > > Might be this bug can give some context on the mem-leak (fix recently
>> > > merged on master but not on 3.7.x)
>> > >
>> > > https://bugzilla.redhat.com/show_bug.cgi?id=1287517
>> > Yes, this is what we'd be fixing in 3.7.x too, but if you refer to [1]
>> > the hike is seen when a command is run in a loop which is typically not
>> > a use case in any production setup.
>> >
>> > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1287517#c15
>> > >
>> > > ~Joe
>> > >
>> > >
>> > > ----- Original Message -----
>> > >> From: "Atin Mukherjee" <atin.mukherjee83@xxxxxxxxx>
>> > >> To: "Joseph Fernandes" <josferna@xxxxxxxxxx>
>> > >> Cc: "Gluster Devel" <gluster-devel@xxxxxxxxxxx>, "Ajil Abraham"
>> > >> <ajil95.abraham@xxxxxxxxx>
>> > >> Sent: Friday, March 4, 2016 7:01:54 AM
>> > >> Subject: Re: glusterd crashing
>> > >>
>> > >> -Atin
>> > >> Sent from one plus one
>> > >> On 04-Mar-2016 6:12 am, "Joseph Fernandes" <josferna@xxxxxxxxxx> wrote:
>> > >>>
>> > >>> Hi Ajil,
>> > >>>
>> > >>> Well few things,
>> > >>>
>> > >>> 1. Whenever you see a crash its better to send across the Backtrace(BT)
>> > >> using gdb and attach the log files (or share it via some cloud drive)
>> > >>>
>> > >>> 2. About the memory leak, What kind of tools are you using for profiling
>> > >> memory, valgrind ? if so please attach the valgrind reports.
>> > >>> $> glusterd --xlator-option *.run-with-valgrind=yes
>> > >>>
>> > >>> 3. Well I am not sure if glusterd uses any of the mempools as we do in
>> > >> client and brick processes, Atin can shed some light on this.
>> > >>> Well In that case you can used the statedump mechanism check for
>> > >> mem-leaks check the glusterfs/doc/debugging/statedump.md
>> > >> GlusterD does use mempool and it has infra for capturing statedump as
>> > >> well.
>> > >> I am aware of few bytes of memory leaks in few paths which is really not a
>> > >> huge concern but it shouldn't crash.
>> > >>>
>> > >>> Hope this helps
>> > >>>
>> > >>> ~Joe
>> > >>>
>> > >>>
>> > >>> ----- Original Message -----
>> > >>>> From: "Ajil Abraham" <ajil95.abraham@xxxxxxxxx>
>> > >>>> To: "Atin Mukherjee" <atin.mukherjee83@xxxxxxxxx>
>> > >>>> Cc: "Gluster Devel" <gluster-devel@xxxxxxxxxxx>
>> > >>>> Sent: Thursday, March 3, 2016 10:48:56 PM
>> > >>>> Subject: Re: glusterd crashing
>> > >>>>
>> > >>>> Hi Atin,
>> > >>>>
>> > >>>> The inputs I use are as per the requirements of a project I am working
>> > >> on for
>> > >>>> one of the large finance institutions in Dubai. I will try to handle the
>> > >>>> input validation within my code. I uncovered some of the issues while
>> > >> doing
>> > >>>> a thorough testing of my code.
>> > >>>>
>> > >>>> I tried with 3.7.6 and also my own build from master branch. I will
>> > >> check
>> > >>>> with my superiors before sending you backtrace and other details. So
>> > >> far, I
>> > >>>> have seen memory leak in 100s of KBs.
>> > >>>>
>> > >>>> -Ajil
>> > >>>>
>> > >>>>
>> > >>>> On Thu, Mar 3, 2016 at 10:17 PM, Atin Mukherjee <
>> > >> atin.mukherjee83@xxxxxxxxx
>> > >>>>> wrote:
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> Hi Ajil,
>> > >>>>
>> > >>>> Its good to see that you are doing a thorough testing gluster. From
>> > >> your mail
>> > >>>> it looks like your automation focuses on mostly negative tests. I need
>> > >> few
>> > >>>> additional details to get to know whether they are known:
>> > >>>>
>> > >>>> 1. Version of gluster
>> > >>>> 2. Backtrace of the crash along with reproducer
>> > >>>> 3. Amount of memory leak in terms of bytes/KB/MB?? Have you already
>> > >>>> identified them?
>> > >>>>
>> > >>>> -Atin
>> > >>>> Sent from one plus one
>> > >>>> On 03-Mar-2016 10:01 pm, "Ajil Abraham" < ajil95.abraham@xxxxxxxxx >
>> > >> wrote:
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> For my project, I am trying to do some automation using glusterd. It is
>> > >> very
>> > >>>> frustrating to see it crashing frequently. Looks like input validation
>> > >> is
>> > >>>> the culprit. I also see lot of buffer overflow and memory leak issues.
>> > >>>> Making a note of these and will try to fix them. Surprised to see such
>> > >> basic
>> > >>>> issues still existing in Gluster.
>> > >>>>
>> > >>>> -Ajil
>> > >>>>
>> > >>>> _______________________________________________
>> > >>>> Gluster-devel mailing list
>> > >>>> Gluster-devel@xxxxxxxxxxx
>> > >>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>> > >>>>
>> > >>>>
>> > >>>> _______________________________________________
>> > >>>> Gluster-devel mailing list
>> > >>>> Gluster-devel@xxxxxxxxxxx
>> > >>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>> > >>
>> > > _______________________________________________
>> > > Gluster-devel mailing list
>> > > Gluster-devel@xxxxxxxxxxx
>> > > http://www.gluster.org/mailman/listinfo/gluster-devel
>> > >
>> >
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel@xxxxxxxxxxx
> http://www.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
