On 11/5/21 10:59 AM, Nikola Ciprich wrote:
here's the output if it is of some use:
https://storage.linuxbox.cz/index.php/s/AyZGW5Xdfxg47f6
Just to be clear - I think Dave and I interpreted your original email slightly
differently.
Are these large files on the 1.5PB filesystem filesystem images themselves,
or some other type of file?
And - the repair you were running was against the 1.5PB filesystem?
(see also Dave's reply in this thread)
Hello Eric,
I was running fsck on the 1.5PB fs (I interrupted it, as it doesn't seem
to be the main problem now). Large files are archives of videofiles from camera
streaming software, I don't know much about them, I was told at the beginning
that all writes will be sequential, which apparently are not, so for new
files, we'll be preallocating them.
ok, thanks for the clarification.
Though I've never heard of streaming video writes that weren't sequential ...
have you actually observed that via strace or whatnot?
What might be happening is that if you are streaming multiple files into a single
directory at the same time, it competes for the allocator, and they will interleave.
XFS has an allocator mode called "filestreams" which was designed just for this
(video ingest).
If you set the "S" attribute on the target directory, IIRC it should enable this
mode. You can do that with the xfs_io "chattr" command.
Might be worth a test, or wait for dchinner to chime in on whether this is a
reasonable suggestion...
-Eric
btw blocked read from file I sent backtrace seems to have started finally (after
maybe an hour) and runs 8-20MB/s
