On Tue, Jan 03, 2012 at 09:20:44PM +0100, Jens Axboe wrote: > On 2012-01-03 21:09, Tejun Heo wrote: > > On Tue, Jan 03, 2012 at 09:59:22AM -0800, Tejun Heo wrote: > >> That should have been service tree. I couldn't find more missing > >> removals other than the one Shaohua's patch already fixed. Close > >> cooperator selection in cfq_select_queue() seems suspicious tho. I > >> can't see what prevents it from returning an empty coopeator cfqq. > >> I'm trying to verify whether that's the case. Will update when I know > >> more. > > > > While testing, found another bug. > > > > Redzone: 0x9f911029d74e35b/0x9f911029d74e35b. > > Last user: [<ffffffff813a82ee>](cfq_put_queue+0x7e/0xd0) > > 070: e8 32 ab 1d 00 88 ff ff e8 32 ab 1d 00 88 ff ff .2.......2...... > > Prev obj: start=ffff88001dab3178, len=232 > > Redzone: 0x9f911029d74e35b/0x9f911029d74e35b. > > Last user: [<ffffffff813a82ee>](cfq_put_queue+0x7e/0xd0) > > 000: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk > > 010: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk > > Next obj: start=ffff88001dab3378, len=232 > > Redzone: 0xd84156c5635688c0/0xd84156c5635688c0. > > Last user: [<ffffffff813a8e53>](cfq_get_queue+0x153/0x670) > > 000: 02 00 00 00 21 01 00 00 e0 c9 b1 1d 00 88 ff ff ....!........... > > 010: 89 96 ae 18 00 88 ff ff 00 00 00 00 00 00 00 00 ................ > > > > The field at 0x70 which is being updated after being freed is > > cfqq->fifo. Interestingly, it didn't lead to any visible failure. > > That's pretty odd. Given Hughs report as well, it sure does sound like > we now have some life time issues with cfqq's. Hmmm... I disabled cfqq merge logic (commented out cfq_close_cooperator() and the following cfq_setup_merge() calls) in cfq_select_queue() and neither is triggering for quite a while now. Maybe cfqq refcnt is getting borked over cfqq merging / splitting? It would also explain the low frequency of the issue too. I'll try to further isolate it but It would be awesome if someone more familiar with the logic can go over that part. Thanks. -- tejun -- To unsubscribe from this list: send the line "unsubscribe linux-next" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
