On Wed, 2017-03-22 at 13:59 -0400, Matthew Miller wrote: [..] > This is a really nice, easy-to-understand presentation putting forward > the goals of the initiative, the current state, and where we hope to > have it for the upcoming Fedora 26 release and Fedora 27 later this > year. If you package software in Fedora — and, particularly, something > like a language or application stack where Fedora users might benefit > from choosing between several supported versions — I really recommend > checking this out. OK. Will try to write longer comment :) Looks like more or less Modularity people are trying to solve problems already solved in Solaris IPS mediators which allows solve problems of delivery software in different versions/variants. Trying to solve such dilemmas by manage as well software which is not delivered as regular package probably will break completely as software in working state as non-packaged software will not have proper requires/provides description. IPS trying to solve such problems is using know from Linux alternatives management + dependency resolver. Highly likely solving multiple versions software delivery issues on top of raw rpm packages (as they are now) will end up big mess behind. In other words such support IMO must be anchored way deeper into PM like it was done in IPS. Another issue is that some level of flexibility manipulation of versions of some components may be working only in strictly controlled sets of versions of the software. Deliver such such baselines is solved in IPS by incorporations idea which is mechanism guarantee consistency on some exact areas where delivery of alternatives is possible. Single incorporation simple locks all possible to install packages on exact version and releases. At the moment similar locking is done in rpm based distros like Fedora is done by delivery whole distribution. However all internal dependencies inside exact distro version are based only on serial:version-release(arch) dependencies and there is no mechanism which will start delivery alarms or will not allow to install anything from other versions of the distribution. There is no in current rpm based approaches mechanisms allowing make upgrade from distribution version N to version N+1 with signalisation that some already installed packages are/will be breaking dependencies encircled on areas on which exact distribution packages have been tested together, Generally delivery packaged software in multiple versions on top of packages like rpm will be really hard if not impossible, as long each variant adds yet another dimensions the same stuff which needs to be delivered. This is why IPS completely moved away from packages delivered in form archives and switched to serving software in form of repositories. IPS mediators + facets ideas really solves Modularity problems and few other things as well. Surprisingly something like IPS facet idea in some very limited form is available on top of rpm. For example at the moment is possible to choose install everything with or without documentation (rpm exclude doc mechanism) which is basing on %doc tokens in %files sections. The same is with choosing languages/locale dependent files basing on %lang() tokens. In each of those two "dimensions" are used "dimension" specific %files tokens. In other words in rpm world is possible to choose within only those two defined "dimensions". The same possibility of customisation in case of IPS is delivered in more general way in form of facet like doc=[true|false], locale.<lang_name>=[true|false], The same file in package description can be marked using multiple facets as well. Something ca e documenttion in exact language. How this approach may be used on some other areas? One example: someone on top of typical system want to compile something because additionally software must be tsted "in situ". Choosing some exact set of -devel packages to install to start compiling some software? No .. just "pkg change-facet devel=true" and within few minutes ALL already installed pakages will be enreached by adding all files with devel=true facet. On Solaris there is no separated devel packages!!!! This beautifully as well interacts with AI (Automated installer) manifest where on specifying installation of the system with exact attributes is possible nicely described this by: <software type="IPS"> <destination> <image> <facet set="false">facet.doc.html</facet> <facet set="false">facet.doc.pdf</facet> <facet set="false">facet.locale.*</facet> <facet set="true">facet.devel</facet> <facet set="true">facet.locale.en</facet> <facet set="true">facet.locale.en_GB</facet> </image> </destination> Solving problems of moving around software in form of archives? No problem 'software type="ARCHIVE"' (Unified ARchive). For example initial version of OpenStack packages on Solaris where provided as UAR. Example from my first experiments with OpenStack trying to use it by install and setup this software over AI manifest and profile: <software type="ARCHIVE"> <source> <file uri="http://<hostname>/AI/sol-11_3-openstack-x86.uar"/>; </source> <destination> <image> <facet set="false">facet.devel</facet> <facet set="false">facet.doc.html</facet> <facet set="false">facet.doc.pdf</facet> <facet set="false">facet.locale.*</facet> <facet set="true">facet.locale.en</facet> <facet set="true">facet.locale.en_GB</facet> </image> </destination> Back to example with compiling something on top of regular system .. So .. software has been compiled and we have now binaries. OK. Another command "pkg change-facet devel=false" and all devel stuff is removed. Someone want to have an access to documentation during development process? .. easy to guess "pkg change-facet 'doc*=true'" (there are few doc* facets used in Solaris packages). It is possible to add dependencies on facet dependent files. Few days ago bugs in build-id infrastructure which is now integrated within each rpm packages kicked hardly. Good that it has been already solved (partially). Generally build-id tries to solve delivery of debuginfo packages/resources for exact and matching versions-releas(arch) for example core dumping binaries (https://bugzilla.redhat.com/show_bug.cgi?id=1433837 are some of my comments how build-id could be still simplified if it would be relaying more on packages database). However some bugs are still around (https://bugzilla.redhat.com/show_bug .cgi?id=1434235) So how such problem cold be solved using IPS approach? Simple by debuginfo=true facet on exact set of packages. PM software will find exact files which needs to be installed on the system. All without hardcoding as it is now within additional ELF section build-id hashes. In repository exact packages will exist with debuginfo=true files however on exact system image those files may or may not be installed depends on facet debuginfo=[true|false] state. By addng dependencies between debuinfo resources in the same way as between regular rpm packages is possible to 100% reuse dependencies resolver. As I've mentioned rpm allows use only two facets-like types of tagging. Solaris uses more than two hundreds. Typical dilemma of cutting system image to absolute minimum is solvable by facets customisation without changing set of installed packages. No doubts that something like extreme cutting off used disk space will fail because will not allow to new states of facets with breaking some dependencies. ---- *** ---- What I'm trying to tell by above is that technologies like rpm have been designed on top of quite precise assumptions and used approaches on solving some scenarios have been architected within scope of those assumptions. When now original design design needs to be transformed to handle few new assumptions, without breaking scaffold of original design everything likely will end up in form of growing and breaking apart ball of yeast than something solid. Providing software in form of only repositories (and effectively breaking paradigm of package as file/archive) solves perfectly on source side providing software for multiple distro versions of within *single repository*. What it means? No longer updating repositories addresses on major upgrade -> one less point o fail on whole upgrade procedure. The same repository is used as well to provide software for multiple architectures and packages on repo side are sharing files with the same checksums. As long as new version of the package A delivers only one changed file in new version on new version package everything else on repo side will be shared between multiple versions of the packages. Packages can be way bigger and as long new versions of the same packages will be changeing only some subset of files owned by package over network automatically will be transferred only what has been changed. Solutions like drpm (delta rpm) are completely not needed because delta resources are automatically formed. If someone is interested some more details about IPS please try to have look on source code repo https://java.net/projects/ips/sources/ pkg-gate/show IPS code is probably something like +20 times smaller than rpm, dnf and all additional python modules code combined. Aditionally it provides all repo side services with caching and providing multi layered repos infrastructure services software. IPS is fully written in python and in many places still it solves a lot of more problems/scenarios which are still ahead of Fedora to solve. For example on may systems happens something like this that someone installs some additional package as JFDI solution. As it was done during weekend Monday the same person forgets that temporary solution need to be solved in some clean and tested way. After this someone else starts using this newly installed software adding to the system software some script. This scrip have been even added to install profile used in full OS reinstall/DR recovery procedure. However after next cycle of reinstallation such script starts failing and no one remember why and what is needed. How to avoid such scenarios? Easy: by locking whole set of packages after initial installation by executing from cron every day uninstall every package which is not within originally locked set of packages. By this our example script will fail next day after installation recalling automatically to finish solving JFDI properly much earlier. Part of the IPS internal simplicity lies as well on top of other OS provided technologies like using snapshots. Even single new package installation starts from creating on affecter volumes snapshots. If package installation fails and it is usually hard to say how to roll back all changes on PM layer. So .. no problem. Just roll back everything to checkpointed state in matter of fraction of second. The same approach is possible to use on Linux. However to solve this the same way all ext, xfs and few other FSes needs to be excluded from new approach and only btrfs ATM could be used as only fully supported platform. Radical approach .. but 100% it will be working without breaking internal simplicity. What is more important is that on top of IPS have been already proven that this new approach is working. In other words IPS it is stash of tested in combat ideas (I'm not suggesting to switch from rpm to IPS because ~99.99% Linux community ATM is not mentally ready to start thinking about more radical approaches to PM :) ) IMO it would be really good if people involved in Modularity will have closer look on IPS to avoid reinventing the wheel. Best would be to switch to IPS but probably this time again it will be not possible to avoid NIH syndrome :) Whole and so huge IPS simplification on code layer was possible only because about decade ago few people come to conclusion that it is no longer possible to solve new problems using old paradigms of SySV packages. rpm still sits very hardly on basic SySV packages ideas which as the set they've been invented *~30 years ago* (a lot of people here was born around the same time when those fundaments have been lied :) ). As I'm looking one more time now on Modularity I think that those problems which this project is trying to solve should be handled exactly in the same way because if not .. whole project IMO has high chance to fail. IMO it is only matter of time when rpm will be abandoned because no longer would be possible to stretch this software onto new needs without breaking internal consistency. rpm still is very strong but already with each day is slightly weaker and weaker. kloczek -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
