F40 Change Proposal:Passim Peer-to-Peer Metadata (Self Contained)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Wiki https://fedoraproject.org/wiki/Changes/Passim_P2P_Metadata

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.


== Summary ==

Passim is a local caching server that broadcasts specific shared
metadata to other clients on your local network to reduce the amount
of duplicate data downloaded from the internet.

== Owner ==
* Name: [[User:rhughes| Richard Hughes]]
* Email: richard@xxxxxxxxxxx



== Detailed Description ==
Much of the software running on your computer that connects to other
systems over the Internet needs to periodically download metadata or
information needed to perform other requests.

Everybody downloads the same file from a CDN, and although a CDN is
not super-expensive, it's certainly not free. Everybody on your
current network (perhaps thousands of users) has to download the same
1MB blob of metadata from a CDN over a perhaps-expensive internet
link.

What if we could download the file from the CDN on one machine, and
the next machine on the local network that needs it instead downloads
it from the first machine? We could put a limit on the number of times
it can be shared, and the maximum age so that we don't store
yesterday's metadata forever, and so that we don't turn a ThinkPad
X220 into a machine distributing 1Gb/s to every other machine in the
office. We could cut the CDN traffic by at least one order of
magnitude, but possibly much more. This is better for the person
paying the cloud bill, the person paying for the internet connection,
and the planet as a whole.

== Feedback ==
IPFS is an existing project that's existed for many years and allows
sharing with other users not on your local network. It's not packaged
in any distributions and not trivial to install correctly. Its main
drawback is that it requires an internet to IPFS "gateway" which costs
a large amount of money for the LVFS, and that it's not EAR/ITAR
compliant.

I've asked for feedback already on fedora-devel and have already
started making changes and suggestions from that discussion -- for
instance splitting out a -libs subpackage. See
https://www.spinics.net/lists/fedora-devel/msg315078.html for the
discussion.

== Benefit to Fedora ==
Fedora will consume less bandwidth when checking for firmware updates.
Per user there is only a 2MB/day saving, but for millions of Fedora
users this adds up to a huge amount of saved data (and money)
globally.

== Scope ==

The code is already written, tested and ready to go. The passim
package is already included in Fedora (although not installed by
default) and fwupd needs an upstream release which includes this
functionality -- which is scheduled for 2 weeks time.

== Upgrade/compatibility impact ==
Old versions of fwupd will be updated and start sharing metadata with
other local users with no changes required.

== How To Test ==
Install two physical machines or VMs with Fedora 40 that are both on
the local network. Run `fwupdmgr refresh` on the first, and observe
that `passim dump` or `https://localhost:27500/` lists the published
metadata file . Then run `fwupdmgr refresh --verbose` on the second
machine and see that the file is downloaded from
https://localhost:27500 rather than `cdn.fwupd.org`. Avahi needs to be
enabled and running, as does `passimd` although both are autostarted
as required.

== User Experience ==
Each user will use 2MB less bandwidth per day when there are other
users on the local network with the same metadata file. There is no
user-visible difference to any operation.

== Dependencies ==
None; fwupd will recommend passim to be installed by default and
autolaunch it as required.

== Contingency Plan ==
Change `Recommends: passim` to `Suggests: passim` in the `fwupd.spec`
file so that it's not autoinstalled by default. In this case fwupd
will fall back to downloading from the CDN every day.

== Documentation ==
 * https://github.com/hughsie/passim/blob/main/README.md

== Release Notes ==
Fedora now uses a peer-to-peer service called Passim to reduce the
amount of bandwidth used when downloading metadata.





-- 
Aoife Moloney

Product Owner

Community Platform Engineering Team

Red Hat EMEA

Communications House

Cork Road

Waterford
_______________________________________________
devel-announce mailing list -- devel-announce@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-announce-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux