On Sun, Apr 30, 2017 at 10:30 PM, Nick Coghlan <ncoghlan@xxxxxxxxx> wrote: > On 1 May 2017 at 09:59, Nico Kadel-Garcia <nkadel@xxxxxxxxx> wrote: >> Wouldn't it be more effective, more safe, and more efficient to >> improve the existing support for doing "the right thing", rather than >> trying to outsmart a build and deployment system that is essentially >> outside your control? To encourage the use of "pyvenv" for local such >> modules, in a local userspace, rather than as a default system >> version? > > This is already happening, and is entirely independent of the > discussion at hand (e.g. see [1], a presentation I did with Graham > Dumpleton at Red Hat Summit/DevNation last year advocating for Red > Hat's customers to adopt exactly this model). On Fedora's Developer > Portal, virtual environments share the front page of the Python > section with running Python itself: > https://developer.fedoraproject.org/tech/languages/python/python-installation.html Good that the conversation is happening. > Responding to "We want to make running 'sudo pip install' less harmful > and easier to recover from when people do it by mistake" with "We > should discourage the use of 'sudo pip install'" remains an > unproductive non sequitur - we *know* it's inherently fragile, and > requires deep knowledge of both Python packaging and RPM packaging to > do reliably. However, Red Hat can't (and won't) answer a commercial > support call from a distressed customer with "Ooh, you shouldn't have > done that - sucks to be you" (although these supportability problems > *are* one of the reasons we only ship "pip" commercially through > Software Collections, and not as part of the system Python runtime). This seems wise. Good guidelines for safer practices would be invaluable. > Fortunately, we have the ability to influence *all* of the moving > parts (upstream pip, upstream CPython, downstream distros) here, and > help to incrementally adjust their respective default behaviours so > that the key risks involved in running "sudo pip install" are > comparable to those involved in running "curl $REMOTE_SCRIPT | sudo > sh". If only. The difficulty is the dependencies. The behavior is very difficult to predict, and dangerously inconsistent, if the dependencies happen to be already installed via RPM or already installed via even other "sudo pip install" operations. It's a fundamental risk of all the "build me on the fly" installers, historically including CPAN, ant, maven, gradle, and is currently playing out with "pip install". > If the intended benefit of this change remains unclear, it may help to > focus on a specific concrete case, which would be that the following > operations should be completely indistinguishable at the system level > from not having done anything at all (except in the sudo logs): > > $ sudo pip3 install contextlib2 > $ sudo pip3 uninstall contextlib2 And this successfully ignores *all* the dependencies which contextlib2 may add at installation time. An archetype for this right now is the "awscli" module, whose dependencies themselves require dependencies that require incompatible versions of "Sphinx" to generate their documentation. (Been there, done that, recently.) It also ignores this timeline: * sudo pip3 install contextlib2 # installs contextlib2 from 2017-05-01 * sudo pip3 install [module with more recent contextlib2 requirement from 2017-05-02] * sudo pip3 uninstall contextlib2 Which contextlib2 are you going to uninstall? It will typically uninstall the more recent one from 2017-05-02, and the module with the dependency. But the module from 2017-05-01 remains untouched.. And yes, I've seen this happen with complex package dependencies. > At the moment, that's *not* the case if you had previously done "sudo > dnf install python3-contextlib2", as both of the above commands will > mess with the system contextlib2 packages. With the proposed changes, > contextlib2 may behave unexpectedly while the non-system version is > installed, but removing the pip installed version will be sufficient > to restore the affected machine to its previous state. (I chose > contextlib2 as the example here, rather than something more complex > like ansible, as pip installing components with dependencies makes the > cleanup process more complicated. However, the /usr/local/ split helps Yup. This is one of the big problems of the whole scheme. What might be safe with one of the cleaner, more independent modules becomes quite hazardous for something larger and more complex, such as "pip" itself. > there as well, by making the installed-via-pip versions easier to > identify even without checking the PEP 376 installation metadata) At a first glance, in theory, yes. But by installing it in a default enabled location, I'm trying to point out that it puts other system modules at risk of incompatible upgrades, particularly from dependencies of dependencies that "sudo pip install" will activate automatically. And worse: some modules actually actually provide binaries with the same name, such as the "aws" and "awscli" modules, which both install /usr/bin/aws. The result is "flapping". You get whichever module was installed or upgraded last. "pip install" doesn't provide enough control to prevent this: And it doesn't even report, in advance, which modules are going to be installed or upgraded and provide the option to continue. What Red Hat did with the "software collections library" has, in my hands-on experience, proven much safer as a working non-default-enabled location for add-on modules. Encourage a working RPM structure, and keep these local, manually installed components out of /usr/lib, *and* out of /usr/local/lib default access, has been much safer in my own work. Put it over in /opt/python2 or "/opt/pythn33" or a similar, segregated location to be activated on an "as-needed" basis, or encourage the use of pyvenv to keep it segregated. > Cheers, > Nick. > > [1] https://www.slideshare.net/ncoghlan_dev/developing-in-python-on-red-hat-platforms-devnation-2016 > > -- > Nick Coghlan | ncoghlan@xxxxxxxxx | Brisbane, Australia > _______________________________________________ > devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
