[PATCH v5 00/23] scripts: convert most perl scripts to python

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



This series is an effort to reduce the number of different
languages we use by eliminating most use of perl in favour
of python.

This aligns with fact that the likely future build system
we'll use (meson) is written in python, and that python
is much more commonly used/understood by developers these
days than perl.

With this applied we use perl in a handful of places only:

 - src/rpc/gendispatch.pl - this is a horrendously large
   script and very hard to understand/follow. A straight
   syntax conversion to Python would still leave a hgue
   and hard to understand/follow script. It really needs
   a clean room rewrite from scratch, with better structure.

 - src/rpc/genprotocol.pl - fairly easy to convert, but
   might be obsolete depending on approach for rewriting
   gendispatch.pl, so ignored for now

 - tests/oomtrace.pl - will be purge by the patches that
   drop OOM handling anyway

 - tools/wireshark/util/genxdrstub.pl - a very large
   script, which I haven't got the courage to tackle
   yet.

 - cfg.mk/maint.mk - many syntax rules involve regexes
   which are fed to perl. Decision on what to do
   with syntax-check rules punted to another time.

 - build-aux/gitlog-to-changelog
 - build-aux/useless-if-before-free - Both pulled in
   from gnulib. Could be rewritten quite easily if
   desired, but given that we aren't maintainers of
   them right now, they're ignored as they don't
   really impact our developers.

Note that the check-spacing.py script is significantly
slower in Python than in Perl. After researching this
it appears there is nothing that can be done. The Perl
regex engine is simply much better optimized than the
Python one.  As previously discussed we need to loook
at uncrustify or clang-format or some other tool to
validate whitespace formatting. This is ongoing. We
can either accept the slow down in the short term or
keep the Perl version in the short term.

In v5:

 - Rebased to cope with changes to require VPATH build
 - Merged the already-ACKd scripts

In v4:

 - Moved all scripts into the scripts/ directory instead
   of having them scattered around source tree
 - Use re.search instead of re.match
 - Don't bother re-compiling regexes

In v3:

 - All scripts comply with all flake8 style rules with
   exception of

    E129 visually indented line with same indent as next logical line

In v2:

 - Pulled in patch to hacking file
 - Converted many more scripts
 - Forced UTF-8 character set to avoid ascii codec
   on py3 < 3.7

Daniel P. Berrangé (23):
  build-aux: rewrite duplicate header checker in Python
  build-aux: rewrite whitespace checker in Python
  build-aux: rewrite mock inline checker in Python
  build-aux: rewrite header ifdef checker in Python
  src: rewrite ACL permissions checker in Python
  src: rewrite symfile sorting checker in Python
  src: rewrite symfile library checker in Python
  src: rewrite systemtap probe generator in Python
  src: rewrite systemtap function generator in Python
  src: rewrite driver name checker in Python
  src: rewrite driver impl checker in Python
  src: rewrite ACL rule checker in Python
  src: rewrite polkit ACL generator in Python
  src: rewrite remote protocol checker in Python
  tests: rewrite test argv line wrapper in Python
  tests: rewrite qemu capability grouper in Python
  tests: rewrite file access checker in Python
  docs: rewrite hvsupport.html page generator in python
  docs: rewrite polkit docs generator in Python
  docs: move apibuild.py to the scripts/ directory
  docs: move reformat-news.py to the scripts/ directory
  docs: move esx_vi_generator.py to the scripts/ directory
  docs: move hyperv_wmi_generator.py to the scripts/ directory

 Makefile.am                                   |  33 +-
 build-aux/check-spacing.pl                    | 198 -------
 build-aux/header-ifdef.pl                     | 182 -------
 build-aux/mock-noinline.pl                    |  75 ---
 build-aux/prohibit-duplicate-header.pl        |  26 -
 build-aux/syntax-check.mk                     |  32 +-
 docs/Makefile.am                              |  16 +-
 docs/genaclperms.pl                           | 125 -----
 docs/hvsupport.pl                             | 459 ----------------
 {docs => scripts}/apibuild.py                 |   0
 scripts/check-aclperms.py                     |  75 +++
 scripts/check-aclrules.py                     | 263 +++++++++
 scripts/check-driverimpls.py                  | 102 ++++
 scripts/check-drivername.py                   | 114 ++++
 scripts/check-file-access.py                  | 123 +++++
 scripts/check-remote-protocol.py              | 136 +++++
 scripts/check-spacing.py                      | 229 ++++++++
 scripts/check-symfile.py                      |  80 +++
 scripts/check-symsorting.py                   | 117 ++++
 scripts/dtrace2systemtap.py                   | 143 +++++
 {src/esx => scripts}/esx_vi_generator.py      |   0
 scripts/genaclperms.py                        | 123 +++++
 scripts/genpolkit.py                          | 122 +++++
 scripts/gensystemtap.py                       | 184 +++++++
 scripts/group-qemu-caps.py                    | 123 +++++
 scripts/header-ifdef.py                       | 231 ++++++++
 scripts/hvsupport.py                          | 504 ++++++++++++++++++
 .../hyperv_wmi_generator.py                   |   0
 scripts/mock-noinline.py                      |  85 +++
 scripts/prohibit-duplicate-header.py          |  56 ++
 {docs => scripts}/reformat-news.py            |   0
 scripts/test-wrap-argv.py                     | 170 ++++++
 src/Makefile.am                               | 146 ++---
 src/access/Makefile.inc.am                    |   6 +-
 src/access/genpolkit.pl                       | 119 -----
 src/admin/Makefile.inc.am                     |   6 +-
 src/check-aclperms.pl                         |  73 ---
 src/check-aclrules.pl                         | 252 ---------
 src/check-driverimpls.pl                      |  80 ---
 src/check-drivername.pl                       |  83 ---
 src/check-symfile.pl                          |  70 ---
 src/check-symsorting.pl                       | 106 ----
 src/dtrace2systemtap.pl                       | 130 -----
 src/esx/Makefile.inc.am                       |   6 +-
 src/hyperv/Makefile.inc.am                    |   5 +-
 src/rpc/Makefile.inc.am                       |   1 -
 src/rpc/gensystemtap.pl                       | 193 -------
 tests/Makefile.am                             |   3 +-
 tests/check-file-access.pl                    | 126 -----
 tests/file_access_whitelist.txt               |   2 +-
 tests/group-qemu-caps.pl                      | 124 -----
 tests/test-wrap-argv.pl                       | 174 ------
 tests/testutils.c                             |  16 +-
 53 files changed, 3092 insertions(+), 2755 deletions(-)
 delete mode 100755 build-aux/check-spacing.pl
 delete mode 100644 build-aux/header-ifdef.pl
 delete mode 100644 build-aux/mock-noinline.pl
 delete mode 100644 build-aux/prohibit-duplicate-header.pl
 delete mode 100755 docs/genaclperms.pl
 delete mode 100755 docs/hvsupport.pl
 rename {docs => scripts}/apibuild.py (100%)
 create mode 100755 scripts/check-aclperms.py
 create mode 100755 scripts/check-aclrules.py
 create mode 100755 scripts/check-driverimpls.py
 create mode 100644 scripts/check-drivername.py
 create mode 100755 scripts/check-file-access.py
 create mode 100644 scripts/check-remote-protocol.py
 create mode 100755 scripts/check-spacing.py
 create mode 100755 scripts/check-symfile.py
 create mode 100755 scripts/check-symsorting.py
 create mode 100755 scripts/dtrace2systemtap.py
 rename {src/esx => scripts}/esx_vi_generator.py (100%)
 create mode 100755 scripts/genaclperms.py
 create mode 100755 scripts/genpolkit.py
 create mode 100755 scripts/gensystemtap.py
 create mode 100755 scripts/group-qemu-caps.py
 create mode 100644 scripts/header-ifdef.py
 create mode 100755 scripts/hvsupport.py
 rename {src/hyperv => scripts}/hyperv_wmi_generator.py (100%)
 create mode 100644 scripts/mock-noinline.py
 create mode 100644 scripts/prohibit-duplicate-header.py
 rename {docs => scripts}/reformat-news.py (100%)
 create mode 100755 scripts/test-wrap-argv.py
 delete mode 100755 src/access/genpolkit.pl
 delete mode 100755 src/check-aclperms.pl
 delete mode 100755 src/check-aclrules.pl
 delete mode 100755 src/check-driverimpls.pl
 delete mode 100755 src/check-drivername.pl
 delete mode 100755 src/check-symfile.pl
 delete mode 100755 src/check-symsorting.pl
 delete mode 100755 src/dtrace2systemtap.pl
 delete mode 100755 src/rpc/gensystemtap.pl
 delete mode 100755 tests/check-file-access.pl
 delete mode 100755 tests/group-qemu-caps.pl
 delete mode 100755 tests/test-wrap-argv.pl

-- 
2.21.0

--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list




[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]

  Powered by Linux