On Wed, Dec 11, 2019 at 11:07:45AM -0500, Theodore Y. Ts'o wrote: > On Wed, Dec 11, 2019 at 12:00:58PM +0800, Ming Lei wrote: > > I didn't reproduce the issue in my test environment, and follows > > Andrea's test commands[1]: > > > > mount UUID=$uuid /mnt/pendrive 2>&1 |tee -a $logfile > > SECONDS=0 > > cp $testfile /mnt/pendrive 2>&1 |tee -a $logfile > > umount /mnt/pendrive 2>&1 |tee -a $logfile > > > > The 'cp' command supposes to open/close the file just once, however > > ext4_release_file() & write pages is observed to run for 4358 times > > when executing the above 'cp' test. > > Why are we sure the ext4_release_file() / _fput() is coming from the > cp command, as opposed to something else that might be running on the > system under test? _fput() is called by the kernel when the last Please see the log: https://lore.kernel.org/linux-scsi/3af3666920e7d46f8f0c6d88612f143ffabc743c.camel@xxxxxxxx/2-log_ming.zip Which is collected by: #!/bin/sh MAJ=$1 MIN=$2 MAJ=$(( $MAJ << 20 )) DEV=$(( $MAJ | $MIN )) /usr/share/bcc/tools/trace -t -C \ 't:block:block_rq_issue (args->dev == '$DEV') "%s %d %d", args->rwbs, args->sector, args->nr_sector' \ 't:block:block_rq_insert (args->dev == '$DEV') "%s %d %d", args->rwbs, args->sector, args->nr_sector' $MAJ:$MIN points to the USB storage disk. >From the above IO trace, there are two write paths, one is from cp, another is from writeback wq. The stackcount trace[1] is consistent with the IO trace log since it only shows two IO paths, that is why I concluded that the write done via ext4_release_file() is from 'cp'. [1] https://lore.kernel.org/linux-scsi/320b315b9c87543d4fb919ecbdf841596c8fbcea.camel@xxxxxxxx/2-log_ming_20191129_150609.zip > reference to a struct file is released. (Specifically, if you have a > fd which is dup'ed, it's only when the last fd corresponding to the > struct file is closed, and the struct file is about to be released, > does the file system's f_ops->release function get called.) > > So the first question I'd ask is whether there is anything else going > on the system, and whether the writes are happening to the USB thumb > drive, or to some other storage device. And if there is something > else which is writing to the pendrive, maybe that's why no one else > has been able to reproduce the OP's complaint.... OK, we can ask Andrea to confirm that via the following trace, which will add pid/comm info in the stack trace: /usr/share/bcc/tools/stackcount blk_mq_sched_request_inserted Andrew, could you collect the above log again when running new/bad kernel for confirming if the write done by ext4_release_file() is from the 'cp' process? Thanks, Ming
