Re: btrfs: What to do with large growing files?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, Jan 11, 2021 at 12:57 PM Richard Shaw <hobbes1069@xxxxxxxxx> wrote:
>
> So from what I understand once a file is >0 that you can't set nodatacow.

Correct.

> I'm playing around with cryptocurrency and the current one downloads the whole blockchain and it's at 65GB and growing. I think the growing part is the problem. That and it's being stored on /home which is a spinning disk not an SSD like the system disk.
>
> I'm seeing a lot of IO wait in glances and the system is sluggish even though CPU and memory usage is relatively low.
>
> Ideas?

It'd be useful to know if this is some kind of database, if it's one
big file or many, if it's doing insertions (typical with database) or
append. It might just be busy and will settle itself out once it's
done downloading. And even maybe know what the program is. And I can
ask folks on #btrfs.

Btrfs does quite a lot of delayed allocation so you might see higher
IO wait than you're used to, but as a side effect it turns random
writes into sequential writes which write and read faster and have
less fragmentation. So it's overall better. But the system being
sluggish makes me wonder if something else is competing for IO with
this task and what that process's write pattern is too. It could be
lock contention, the easy way to alleviate that is give one of the
tasks its own subvolume.

I'm assuming this problem is all over by now but if you can reproduce
it the first thing to look at is sysrq+t
https://fedoraproject.org/wiki/QA/Sysrq

that likely fills the dmesg buffer, but 'journalctl -k --no-hostname
-o short-monotonic > dmesg.txt` will do it and then you can upload
that file somewhere and I'll bug someone about it :P if no one on here
knows how to read sysrq+t (i'm not great at it unless it's really
obvious).

Ideally have the sysrq+t ready to just hit enter. When the
sluggishness happens, hit return.

If that's all normal we might have to get into bcc-tools or something.

--
Chris Murphy
_______________________________________________
users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx



[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [EPEL Devel]     [Fedora Magazine]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Desktop]     [Fedora Fonts]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Fedora Sparc]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux