2013/6/10 Anatol Pomozov <anatol.pomozov@xxxxxxxxx>: > > "sync" is not a workaround, it is a right solution. > > Under the hood copying in linux works following way. Every time you read > something from disk the file information will stay cached in memory region > called "buffer cache". Next time you read the same information kernel it > will be served from RAM, not from disk. This speedups the read operation a > lot - reading from RAM ~100000 faster than reading from disk [1]. > > "buffer cache" is used for write operations as well. When you write to disk > it is actually writes to to memory and operation reported as "finished". > Moments later special process called "writeback" sends this data to disk. > This trick also allows to speedup the process. Of course it supposes that > underlying disk will not suddenly disappear (like in your case with USB > pen). > > If you want to make sure that data is really written then you should do one > of the following things: > > 1) Unmount device correctly. Instead of just pulling the USB pen you should > do "umount YOUR_DEVICE_NAME". "umount" flushes all dirty blocks to the > device. I always unmount my devices before unplugging, but I found out that after writing a lot of data to a slow device (SD card in my case), umount or udiskie-umount would block for a long time, then exit with some weird error message and the data would have not been correctly written. > > 2) Call "sync" (that flushes dirty buffers) and then plug/umount USB pen. > Yeah, I do that, sync then umount, but it bothers me that "rsync --progress" shows hundreds of MB/s or so for the first few... GBs? of files, then slows down to the actual speed of the device when the write cache fills up. And then I hit sync and it blocks for a few more minutes until everything is copied. > 3) Call "dd" operation with "conv=fsync" flag, this tells that dd should > not return until all data is written to the device. > Is there a similar flag for rsync? There is no "fsync" string in the manpage, and countless "sync", for obvious reasons. Or a way to make the write cache smaller, or disable it entirely, for removable devices? I recall seeing such an option years ago in Windows. -- Pedro Emílio Machado de Brito Engenharia de Computação 2012 - Unicamp Coordenador Tecnológico - Centro Acadêmico da Computação (CACo) (34) 9673-0963 - Facebook/Skype: pedroembrito - Github: pemb 18hxPDGcNDpsYYKJ2aghSh3mm9n3c4C7pf "Repair what you can — but when you must fail, fail noisily and as soon as possible." · Eric S. Raymond
