Re: Proposal to mirror Docker images

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



It would also help to have the following information.  The mirrors will
need to have this information in order to make informed decisions.  (I
will also have to make changes to quick-fedora-mirror to accommodate.)

1) How much content will the mirrors need to store?  How will this
   amount change over time?

2) Do you have a plan for placing an upper bound on the total amount of
   data?  (In Fedora things are moved to archive, though that has its
   own problems and of course doesn't really place an upper bound on
   anything.)

3) How much change do you expect per day?  Churn is really important,
   and even now we can come close to the point where the master mirrors
   simply can't feed new content to the tier 1 mirrors fast enough for
   them to keep ahead of the changes we're making.

4) How will this be organized on the master mirrors?  It really should
   be in a separate rsync module, and the archive (if that happens)
   should also be in a separate rsync module.

For some background, I have three mirrors, each identical (1U, 4x4TB
disks, RAID0, cached on SSD using bcache).  I mirror all content from
the master mirrors.  Right now I have about 12TB used, 3.5TB free.  I
could upgrade to larger disks for more space, but of course that costs
money.

Basically we're past the point where we have to carefully consider how
we ask the mirror network for more storage and bandwidth, and any plan
for adding stuff should at least include some projections of disk and
bandwidth usage.

 - J<
 
_______________________________________________
infrastructure mailing list
infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
https://lists.fedoraproject.org/admin/lists/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx




[Index of Archives]     [Fedora Development]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [KDE Users]

  Powered by Linux