Re: btrfs: What to do with large growing files?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Tue, Jan 12, 2021 at 8:08 AM Roberto Ragusa <mail@xxxxxxxxxxxxxxxx> wrote:
On 1/12/21 2:21 PM, Richard Shaw wrote:

> I would think a blockchain is roughly equivalent to a database but I have no idea if the blocks were downloaded in order (appended) or randomly (inserted). The single file (data.mdb) ended up being 96GB in total. It seems to have settled down as I expected but it took about 2 days to download so it was a bit frustrating using my computer at that time. Anything that tried to access /home (my spinning btrfs drive) would basically stall including youtube videos that ran out of cache, browser tabs, etc.
98GB in 48 hours is an average of 500kB/s.
 
A spinning disk can seek 100 time per second, was the filesystem doing a seek every about 5kB of data?
Looks unreasonable, especially since I would not expect the access to be random at all in a blockchain.
Would be interesting to do the same operation in different conditions (SSD <-> HDD, ext4 <-> btrfs).

Yes, I almost stopped and copied the file to somewhere in /var since my system SSD (970 EVO m.2) is still EXT4. To be fair, it seemed to save up the download for some time and then dump to the disk all at once. My system wasn't unuseable all the time, but during the writes to disk it was very frustrating. 

I knew it wasn't an overall system problem since glances showed low CPU and memory usage. It's a Ryzen 5 3600 w/ 32GB of memory :)

Thanks,
Richard 
_______________________________________________
users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [EPEL Devel]     [Fedora Magazine]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Desktop]     [Fedora Fonts]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Fedora Sparc]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux