Re: btrfs: What to do with large growing files?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, Jan 11, 2021 at 11:49 PM Chris Murphy <lists@xxxxxxxxxxxxxxxxx> wrote:
On Mon, Jan 11, 2021 at 12:57 PM Richard Shaw <hobbes1069@xxxxxxxxx> wrote:

> I'm playing around with cryptocurrency and the current one downloads the whole blockchain and it's at 65GB and growing. I think the growing part is the problem. That and it's being stored on /home which is a spinning disk not an SSD like the system disk.
>
> I'm seeing a lot of IO wait in glances and the system is sluggish even though CPU and memory usage is relatively low.

It'd be useful to know if this is some kind of database, if it's one
big file or many, if it's doing insertions (typical with database) or
append. It might just be busy and will settle itself out once it's
done downloading. And even maybe know what the program is. And I can
ask folks on #btrfs.

I would think a blockchain is roughly equivalent to a database but I have no idea if the blocks were downloaded in order (appended) or randomly (inserted). The single file (data.mdb) ended up being 96GB in total. It seems to have settled down as I expected but it took about 2 days to download so it was a bit frustrating using my computer at that time. Anything that tried to access /home (my spinning btrfs drive) would basically stall including youtube videos that ran out of cache, browser tabs, etc. 

The program/cryptocurrency is Monero and was downloaded via Flatpack (I think, could be an Appimage). 


Btrfs does quite a lot of delayed allocation so you might see higher
IO wait than you're used to, but as a side effect it turns random
writes into sequential writes which write and read faster and have
less fragmentation. So it's overall better. But the system being
sluggish makes me wonder if something else is competing for IO with
this task and what that process's write pattern is too. It could be
lock contention, the easy way to alleviate that is give one of the
tasks its own subvolume.

I'm assuming this problem is all over by now but if you can reproduce
it the first thing to look at is sysrq+t
https://fedoraproject.org/wiki/QA/Sysrq

Yeah, it's done, otherwise I would do it :)

Thanks,
Richard
_______________________________________________
users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [EPEL Devel]     [Fedora Magazine]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Desktop]     [Fedora Fonts]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Fedora Sparc]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux