I've uploaded a patch to fix this problem: https://review.gluster.org/19040
On Fri, Dec 15, 2017 at 11:33 AM, Xavi Hernandez <jahernan@xxxxxxxxxx> wrote:
I've checked the size of 'gluster volume set help' on current master and it's 51176 bytes. Only 24 bytes below the size of the buffer.I think the reason why regression tests fail is that it enables bd xlator, which adds some more options that make the help output to grow beyond the buffer size.I'll send a patch to fix the problem.XaviOn Fri, Dec 15, 2017 at 10:05 AM, Xavi Hernandez <jahernan@xxxxxxxxxx> wrote:On Fri, Dec 15, 2017 at 9:57 AM, Atin Mukherjee <amukherj@xxxxxxxxxx> wrote:But why doesn't it crash every time if this is the RCA? None of us could actually reproduce it locally.That's a good question. One of my patches has failed and it doesn't add any new option (in fact it's a very trivial change), so I'm not sure why it may or may not crash.I'll analyze it. Anyway, that function needs a patch because there's no space limit check before writing to the buffer.XaviOn Fri, Dec 15, 2017 at 2:23 PM, Xavi Hernandez <jahernan@xxxxxxxxxx> wrote:I've seen this failure in one of my local tests and I've done a quick analysis:(gdb) bt
#0 0x00007ff29e1fce07 in ?? () from /lib64/libgcc_s.so.1
#1 0x00007ff29e1fe9b8 in _Unwind_Backtrace () from /lib64/libgcc_s.so.1
#2 0x00007ff2aa9fb458 in backtrace () from /lib64/libc.so.6
#3 0x00007ff2ac14af30 in _gf_msg_backtrace_nomem (level=GF_LOG_ALERT, stacksize=200) at logging.c:1128
#4 0x00007ff2ac151170 in gf_print_trace (signum=11, ctx=0xdec260) at common-utils.c:762
#5 0x000000000040a2c6 in glusterfsd_print_trace (signum=11) at glusterfsd.c:2274
#6 <signal handler called>
#7 0x00007ff2ac466751 in _dl_close () from /lib64/ld-linux-x86-64.so.2
#8 0x00007ff2aaa304df in _dl_catch_error () from /lib64/libc.so.6
#9 0x00007ff2ab35f715 in _dlerror_run () from /lib64/libdl.so.2
#10 0x00007ff2ab35f08f in dlclose () from /lib64/libdl.so.2
#11 0x00007ff2a06af786 in glusterd_get_volopt_content (ctx=0x7ff298000d88, xml_out=false) at glusterd-utils.c:13150
#12 0x00007ff2a06a2896 in glusterd_volset_help (dict=0x70616e732d776f68, op_errstr=0x732e736572757461) at glusterd-utils.c:9199
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb) f 11
#11 0x00007ff2a06af786 in glusterd_get_volopt_content (ctx=0x7ff298000d88, xml_out=false) at glusterd-utils.c:13150
13150 dlclose (dl_handle);
(gdb) print dl_handle
$1 = (void *) 0x6978656c7069746c
(gdb) x/s &dl_handle
0x7ff294206500: "ltiplexing feature is disabled.\n\n"
(gdb)So I think the problem is a buffer overflow.Looking at the code in glusterd-utils.c, function glusterd_get_volopt_content(), I guess that we are writing too much data into output_string, which is a stack defined array of 50 KB, and we have an overflow there. Probably the number of options and its description has grown beyond this limit.I'll send a patch for this shortly.XaviOn Fri, Dec 15, 2017 at 8:31 AM, Sunny Kumar <sunkumar@xxxxxxxxxx> wrote:+1
Console log
https://build.gluster.org/job/centos6-regression/8021/consol e
Regard
Sunny
On Fri, Dec 15, 2017 at 12:32 PM, Ravishankar N <ravishankar@xxxxxxxxxx> wrote:
> ...for a lot of patches on master .The crash is in volume set; the .t just
> does a volume set help. Can the glusterd devs take a look as it is blocking
> merging patches? I have raised BZ 1526268 with the details.
>
> Thanks!
>
> Ravi
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel@xxxxxxxxxxx
> http://lists.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-devel
