Did this get handled via other channels? I think I heard a discussion about a 20GB export and restricting the size of export trees in standup but I wasn't following it closely. -Greg On Sat, Oct 21, 2017 at 2:43 AM Xiaoxi Chen <superdebuger@xxxxxxxxx> wrote: > > Hi , I hit this on one of my MDS rank, it looks to me like the > export_dir message is way much too big , the hux dump take more than > 20GB(stop by no more disk space...). > > > The only crazy thing in the FS is we have 6 mdses and 5000+ export binding.f > > > 2017-10-21 00:47:18.731219 7f27463e0700 0 -- > 10.148.245.141:6800/4152861050 >> 10.148.245.48:0/3250358670 > conn(0x558aeb4ac800 :6800 s=STATE_OPEN pgs=44 cs=1 l=0).fault server, > going to standby > 2017-10-21 00:47:44.500382 7f27463e0700 -1 failed to decode message of > type 1110 v1: buffer::end_of_buffer > 2017-10-21 00:47:44.503923 7f27463e0700 -1 *** Caught signal (Aborted) ** > in thread 7f27463e0700 thread_name:msgr-worker-1 > > ceph version 12.2.0 (32ce2a3ae5239ee33d6150705cdb24d43bab910c) luminous (rc) > 1: (()+0x59b344) [0x558ad5d94344] > 2: (()+0x11390) [0x7f274910d390] > 3: (gsignal()+0x38) [0x7f274809d428] > 4: (abort()+0x16a) [0x7f274809f02a] > 5: (__gnu_cxx::__verbose_terminate_handler()+0x16d) [0x7f27489e084d] > 6: (()+0x8d6b6) [0x7f27489de6b6] > 7: (()+0x8d701) [0x7f27489de701] > 8: (()+0xb8d38) [0x7f2748a09d38] > 9: (()+0x76ba) [0x7f27491036ba] > 10: (clone()+0x6d) [0x7f274816f3dd] > NOTE: a copy of the executable, or `objdump -rdS <executable>` is > needed to interpret this. > > > -9877> 2017-10-21 00:24:41.764088 7f2745bdf700 1 dump: > 00000000 f1 03 00 00 00 01 00 00 00 00 00 00 26 01 00 00 |............&...| > 00000010 ee a3 0e 00 00 05 00 00 00 00 00 00 77 e6 37 00 |............w.7.| > 00000020 00 02 00 00 00 00 00 00 46 50 13 00 00 04 00 00 |........FP......| > 00000030 00 00 00 00 24 57 a8 00 00 02 00 00 00 00 00 00 |....$W..........| > 00000040 ef aa 20 00 00 03 00 00 00 00 00 00 30 65 16 00 |.. .........0e..| > 00000050 00 04 00 00 00 00 00 00 de e3 93 00 00 02 00 00 |................| > 00000060 00 00 00 00 38 6f a5 00 00 02 00 00 00 00 00 00 |....8o..........| > 00000070 e9 71 16 00 00 03 00 00 00 00 00 00 59 e8 16 00 |.q..........Y...| > 00000080 00 04 00 00 00 00 00 00 95 1a 96 00 00 02 00 00 |................| > 00000090 00 00 00 00 40 52 6a 00 00 02 00 00 00 00 00 00 |....@Rj.........| > 000000a0 98 c7 a6 00 00 02 00 00 00 00 00 00 4b 97 80 00 |............K...| > 000000b0 00 02 00 00 00 00 00 00 b4 7e 1e 00 00 03 00 00 |.........~......| > 000000c0 00 00 00 00 3b 58 a3 00 00 02 00 00 00 00 00 00 |....;X..........| > 000000d0 da b7 15 00 00 04 00 00 00 00 00 00 5e 61 17 00 |............^a..| > 000000e0 00 04 00 00 00 00 00 00 0e 12 98 00 00 01 00 00 |................| > 000000f0 00 00 00 00 0e 03 1e 00 00 03 00 00 00 00 00 00 |................| > 00000100 93 d6 14 00 00 04 00 00 00 00 00 00 74 a4 97 00 |............t...| > 00000110 00 01 00 00 00 00 00 00 65 6a 13 00 00 04 00 00 |........ej......| > 00000120 00 00 00 00 cc 76 1f 00 00 03 00 00 00 00 00 00 |.....v..........| > 00000130 9b da 1f 00 00 03 00 00 00 00 00 00 5d 41 0d 00 |............]A..| > 00000140 00 04 00 00 00 00 00 00 e1 58 18 00 00 04 00 00 |.........X......| > 00000150 00 00 00 00 0a b8 1e 00 00 03 00 00 00 00 00 00 |................| > 00000160 97 f6 13 00 00 04 00 00 00 00 00 00 1b 50 20 00 |.............P .| > 00000170 00 03 00 00 00 00 00 00 10 e8 93 00 00 02 00 00 |................| > 00000180 00 00 00 00 77 71 6a 00 00 02 00 00 00 00 00 00 |....wqj.........| > 00000190 fc d2 b7 00 00 02 00 00 00 00 00 00 bf 6e 95 00 |.............n..| > 000001a0 00 02 00 00 00 00 00 00 fd 0d 93 00 00 02 00 00 |................| > 000001b0 00 00 00 00 fe e5 37 00 00 02 00 00 00 00 00 00 |......7.........| > 000001c0 3f e6 37 00 00 02 00 00 00 00 00 00 55 b5 16 00 |?.7.........U...| > 000001d0 00 03 00 00 00 00 00 00 1a 41 12 00 00 04 00 00 |.........A......| > 000001e0 00 00 00 00 96 e5 37 00 00 02 00 00 00 00 00 00 |......7.........| > 000001f0 a2 dc 10 00 00 04 00 00 00 00 00 00 8c a1 a4 00 |................| > 00000200 00 02 00 00 00 00 00 00 a0 3f 0d 00 00 04 00 00 |.........?......| > 00000210 00 00 00 00 2c 4b 16 00 00 03 00 00 00 00 00 00 |....,K..........| > 00000220 d7 8e 14 00 00 04 00 00 00 00 00 00 45 4d 0d 00 |............EM..| > 00000230 00 04 00 00 00 00 00 00 1a c9 a4 00 00 02 00 00 |................| > 00000240 00 00 00 00 75 ab 16 00 00 03 00 00 00 00 00 00 |....u...........| > 00000250 6f 53 37 00 00 02 00 00 00 00 00 00 c5 8a 37 00 |oS7...........7.| > 00000260 00 02 00 00 00 00 00 00 5c c9 ba 00 00 02 00 00 |........\.......| > 00000270 00 00 00 00 9a 44 ab 00 00 02 00 00 00 00 00 00 |.....D..........| > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
