Hi all, For the Reef release, Dashboard team wants to deliver Python packages embedded in the ceph-dashboard distro package. [1] This will make it unnecessary to depend on distro Python packages and allow us to: - *Use new/er Python packages:* that are not available in all distros (e.g.: fastapi), or which are available but in very disparate versions (and hence with different feature sets, resulting in spaghetti code and complicating test matrices), - *Simplify upgrades to newer Python versions*: ceph-mgr package currently depends on another 42 python packages, so everytime we move to a newer minor version of Python we have to ensure that a total of almost 300 new packages are available in the 7 supported distro-releases. Sometimes the distro maintainers provide those, but it's not infrequent that Ceph devels have to maintain themselves these missing packages, - *Improve security and quality*: contrary to common belief, the distro package approach does not intrinsically ensure a more secure, maintained environment, just a stable one. Many non-standard Python packages come from non-base repos (like Universe in Ubuntu, or EPEL in CentOS), which have pretty lax/no maintenance commitments at all. By controlling the versions of the Python deps, we can run tools like Dependabot to monitor for CVEs and security upgrades, and also perform our tests on a single-version stack. This is a paradoxical situation in which more duties translate into less (more effective) work. - *Summing up: spend more time exposing Ceph features in the Dashboard*, and less time implementing functionality that can be provided by these external libraries or dealing with cross-distro/environment issues. We also find that this approach is consistent with widely-adopted best practices for reproducible runtime environments (Python virtualenvs, containers, etc.). There're a couple of open points and clarifications: - A discussion about whether to give precedence, in the ceph-mgr sys.path, to the distro/site packages or to the embedded ones. The former approach would allow admins to override the embedded versions and provide their own versions, but it's very likely to result in issues (e.g.: distro packages provide an older version of a transitive dependency), so we would prefer the latter, in which only the Python standard libraries are imported via distro packages. - The embedded deps will still be under the control of the distro package management system. The only downside (or upside) is that these deps can't be shared/used by other Python modules. However, this also happens with containers, and is actually perceived as a positive outcome if you favor isolation/reproducibility over disk space savings. [1] https://github.com/ceph/ceph/pull/47501 We'd like to hear your comments, concerns, and suggestions. Thank you! Kind Regards, Ernesto Puerta He / Him / His Principal Software Engineer, Ceph Red Hat <https://www.redhat.com/> <https://www.redhat.com/> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |