On Thu, Feb 04, 2016 at 09:49:22AM +0100, Oliver Neukum wrote:
> On Wed, 2016-02-03 at 22:20 -0800, Marc MERLIN wrote:
>
> > But the main point remains that default mount options and default cp
> > work at horrible speed. I'm happy to try stuff to get around that
> > (nosync might be one, but having my stick write after I unmounted it,
> > makes me very nervous), so I'd love a solution where cp succeeds when
>
> It should. Have you tried to time a manual sync after writing?
Mmmh, this time it worked as expected:
legolas:~# mount -o async /dev/sdd1 /mnt/mnt
legolas:~# time cp /var/local/space/vid/282MB /mnt/mnt; time umount /mnt/mnt; time sync
real 0m0.598s
user 0m0.004s
sys 0m0.572s
real 0m11.454s
user 0m0.000s
sys 0m0.256s
real 0m0.117s
user 0m0.004s
sys 0m0.028s
This goes back to 23MB/s (size/time), which is proper speed.
I also verified that this time the LED wasn't flashing after unmount.
On Thu, Feb 04, 2016 at 10:58:04AM -0500, Alan Stern wrote:
> > And by faster, it's actually instant as long as the file is smaller than
> > my amount of free RAM :)
>
> The sync option really slows things down a lot. That's why you were
> originally seeing such slow transfers under Linux.
Correct. I just don't see this slowdown if I'm using a USB hard drive
(i.e. I use sync and it's 10x+ faster than writing to flash with sync),
and on windows the file is committed to flash when the copy is finished
whereas with async on linux, cp returns before bits have gone anywhere.
> Incidentally, did you do this test within VirtualBox (to match the
> Windows test) or running on the bare hardware?
I did not try running linux within virtualbox, I don't actually have a
linux image for virtualbox (could get one, just haven't needed so far).
If it's a useful test, I could do that.
> Still, it's a more appropriate value, indicating that the actual
> transfer speed is reasonably close to what it should be.
Correct. There seems to be a problem between sync and the underlying
usb-storage driver?
Maybe the windows driver takes many packets at a time, and returns sync
for a bunch of them while linux syncs them one by one, which ends up
being 50 times slower.
> > If I had to guess, sync causes cp(1) to be very conservative and wait
> > for a sync after each 4KB block while windows probably blasts all the
> > data with fewer sync calls in between?
> > (wild guess at this point)
>
> Probably something like that. It doesn't actually change the behavior
> of the cp program; it changes the behavior of the kernel's filesystem
> layer. The end effect is the same, though.
Right. Since running in async mode, especially with usb automounting, is
not safe, it would be good if usb-storage or whichever underlying driver
could do the same batching of blocks and sync them 100 or 1000 at a
time?
> > But the main point remains that default mount options and default cp
> > work at horrible speed.
>
> That's a new main point. Your original main point was that write
> transfers under Linux were horribly slow. You now know this was due to
> the sync option, not any fault in the USB driver stack.
> The default mount options are a question for another mailing list; they
> have nothing at all to do with USB.
You're correct.
Let's forget about async for now since it's not very safe for the
average user and windows gets async like speed in close enough to sync
mode.
Could the usb system aggregate requests and ack them by block?
(yes, I guess that means lying to cp and acking packets that haven't
been written yet, but I think everyone expects that if they pull a USB
stick in the middle of a copy, they'll lose data, so if we ack some data
that really hasn't made it to flash and won't for another second or so,
this likely would not cause problems to most (hopefully all?) users.
Either way, sync mode is so excruciatingly slow that few people must be
using it for any reasonable data.
> cp generally does not wait for data to be written (as far as I know).
> umount, on the other hand, _is_ supposed to wait. If it doesn't then
> it's a bug in umount, not a bug in the USB stack. Are you certain that
> umount doesn't wait for all the data to be transferred?
So, cp(1) seems to wait in sync mode.
And umount is waiting today. Let's assume that I did a bad test
yesterday.
Thanks for your answer.
Marc
--
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/ | PGP 1024R/763BE901
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
