Re: scrapy on F39

"George N. White III" <gnwiii@xxxxxxxxx> · Thu, 8 Feb 2024 08:38:13 -0400

On Thu, Feb 8, 2024 at 12:48 AM Frederic Muller <fred@xxxxxxxx> wrote:

    On 07/02/2024 22:09, George N. White
      III wrote:

        On Wed, Feb 7, 2024 at 2:11 AM Frederic Muller
          <fred@xxxxxxxx> wrote:

          Hi!

            Installed scrapy and just trying to start the tutorial gives
            me that:

            scrapy startproject tutorial
          [...] 

          /home/fred/.local/lib/python3.12/site-packages/parsel/csstranslator.py",

            line 8, in <module>

                 from cssselect.parser import Element,
            FunctionalPseudoElement, 

            PseudoElement

            ImportError: cannot import name 'PseudoElement' from
            'cssselect.parser' 

            (/usr/lib/python3.12/site-packages/cssselect/parser.py)

            Traceback (most recent call last):

              Any idea why it's not working?

        Looks like https://stackoverflow.com/questions/77476433/importerror-cannot-import-name-pseudoelement-from-cssselect-parser
        Fedora 39 has:
          python-cssselect-1.1.0-6  Have a look at the https://github.com/scrapy/scrapy/issues

        You should give enough details so
          others can reproduce the issue: how did you install scrapy,
          which version of Fedora, scrapy, and python-cssselect do you
          have?

        There are problems with the Fedora
          package: 

        Available Packages

          Name         : python3-scrapy

          Version      : 2.10.1

          Release      : 1.fc39

          Architecture : noarch

          Size         : 659 k

          Source       : python-scrapy-2.10.1-1.fc39.src.rpm

          Repository   : fedora

          Summary      : A high-level Python Screen Scraping framework

          URL          : https://scrapy.org

          License      : BSD

          Description  : Scrapy is a fast high-level screen scraping and
          web crawling

                       : framework, used to crawl websites and extract
          structured data

                       : from their pages. It can be used for a wide
          range of purposes,

                       : from data mining to monitoring and automated
          testing.

        Here, on Fedora 39:

        % doas dnf install python3-scrapy 

          doas (gnw3@imacf39) password: 

          Last metadata expiration check: 0:04:29 ago on Wed Feb  7
          10:39:54 2024.

          Error: 

           Problem: conflicting requests

            - nothing provides (python3.12dist(twisted) < 23.8~~ with
          python3.12dist(twisted) >= 18.9) needed by
          python3-scrapy-2.10.1-1.fc39.noarch from fedora

          (try to add '--skip-broken' to skip uninstallable packages)

        twisted has https://nvd.nist.gov/vuln/detail/CVE-2023-46137
          so 

        Many large Python systems have not been
          ported to python-3.12.  I've had success using 

        fedora's python-3.11 in a conda
          environment.  Anaconda python is also a way to run stuff that
        still needs python 3.10 or 3.11.

    So yes... just tried something totally different (opening an IFC file with FreeCAD) and the importer stops saying:
    " File "/app/lib/python3.11/site-packages/ifcopenshell/__init__.py", line 62, in <module>
    raise ImportError("IfcOpenShell not built for '%s'" % python_distribution)
<class 'ImportError'>: IfcOpenShell not built for 'linux/64bit/python3.11' "
You don't mention how you installed python-3.11 and FreeCAD.  Anaconda provides many python packages that are built in a 
way that is compatible with their python binaries.  Conda manages dependencies.   Fedora has /usr/bin/python3.11
from python3.11-3.11.7-2.fc39.x86_64. 

    Back to scrapy I am using F39 updated to the latest everything, I installed using the given instructions on their website which is pip install scrapy
scrapy is version 2.11.0
cssselect is version 1.1.

    So should I file a bug somewhere or patiently wait (it's been like for over a month at least)?
Python 3.12 requires major changes to some packages, and many developers aren't paid for
their contributions, so as long as people are able to continue using affected packages with
Python 3.10 or 3.11 there is no urgency to make things work with Python 3.12.

For scrapy there is the added complication of the recent CVE for twisted: do we get 
patches for older versions of twisted or does scrapy have to be changed to use the
current version?

I think the scrapy github Issues already has similar reports.   You may want to add 
your configuration to the existing issue.   Post on mail lists rarely get into search 
results, so github Issues are more likely to help others who have a problem..

-- 
George N. White III

--
_______________________________________________
users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/users@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue