Instead of coming up with new aggressive pruning schemes, Copr really needs to come up with a reasonable amount of storage to satisfy user demands. HDDs in the multi-TB-range are available for fairly low budgets (extremely low by the standards of a company like IBM), and it just takes 2 of them to build a RAID 1 that is safe against data loss. Of course, that means that Copr needs to stop locking itself into third-party cloud providers that charge ridiculously high prices for storage.
1) Fedora and Copr are using AWS. We are not locked there as Fedora infrastructure has Ansible playbooks for everything and we can migrate somewhere else in few days. The cost of AWS is very competitive - it costs Fedora $0. Zero dollars! Amazon is sponsoring Fedora this way. And I am really grateful for that.
2) While we still try to optimize the "cost", the maintainability is equally important now.
We already use RAID. This is RAID configuration of copr-backend:
# cat
/proc/mdstat
Personalities : [raid1] [raid10]
md126 : active raid10 nvme5n1p1[1] nvme0n1p1[2] nvme4n1p1[0]
nvme2n1p1[3]
25769536512 blocks super 1.2 512K chunks 2 near-copies
[4/4] [UUUU]
bitmap: 22/192 pages [88KB], 65536KB chunk
md127 : active raid1 nvme1n1p1[1] nvme6n1p1[0]
16777081856 blocks super 1.2 [2/2] [UU]
bitmap: 11/125 pages [44KB], 65536KB chunk
Raid check takes more than week. Backup takes days. We did the training exercise of restore from backup and it took several weeks. We identified places for improvements and get it down to a week. Traversing all repositories to delete old build takes almost a day (and we several times had to work on speed improvements there). Some improvements found the way back to basic tools like createrepo_c.
We can easily create 100TB
volume using a RAID. Or even 1PB. But the maintenance would be a
hell. And several parts of Copr would be painfully slow. It is
like a things in a house. From certain size, moving to bigger
house does not improve a situation and makes certain things
harder: finding things, cleaning house...
If anyone think that the
maintenance of Copr can be done better (I am sure, it can!) -
then I want you to invite to join our group.
-- Miroslav Suchy, RHCA Red Hat, Manager, Packit and CPT, #brno, #fedora-buildsys
-- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
