On Fri, Nov 15, 2019 at 03:30:15PM -0500, Cole Robinson wrote: > On 11/11/19 9:38 AM, Daniel P. Berrangé wrote: > > As part of an goal to eliminate Perl from libvirt build tools, > > rewrite the check-spacing.pl tool in Python. > > > > This was a straight conversion, manually going line-by-line to > > change the syntax from Perl to Python. Thus the overall structure > > of the file and approach is the same. > > > > I checked the speed difference mentioned in the cover letter. > > Old: > time ./build-aux/check-spacing.pl `cat ~/src/libvirt/cfiles` > real 0m1.423s > > New: > time ./scripts/check-spacing.py `cat cfiles` > real 0m5.607s > > About around there. The script can easily be parallelized though: > > diff --git a/scripts/check-spacing.py b/scripts/check-spacing.py > index 6b9f3ec1ba..f2ce376e80 100755 > --- a/scripts/check-spacing.py > +++ b/scripts/check-spacing.py > @@ -222,8 +222,12 @@ def check_whitespace(filename): > > > ret = 0 > -for filename in sys.argv[1:]: > - if check_whitespace(filename): > - ret = 1 > +filenames = sys.argv[1:] > +import multiprocessing > +pool = multiprocessing.Pool() > +results = pool.map(check_whitespace, filenames) > +pool.close() > +pool.join() > +ret = int(any(results)) > > sys.exit(ret) > > > After that: > > [:~/src/libvirt] (python3 *) $ time ./scripts/check-spacing.py `cat cfiles` > > real 0m1.674s > user 0m10.506s > sys 0m0.041s > > > Which is pretty close to the perl version, but at the cost of more > cycles. Not sure how that will play with 'make' job control. I believe > the Pool defaults to number of logical host CPUs, which is 8 on my > T480s. Going above 4 doesn't seem to make much of a difference in my > testing for this case FWIW, I always run with "make -j 8 syntax-check" so while this will speed up the job when run individually, it won't help a full syntax check as then it'll be competing with the other syntx checks that are running in parallel. I'd already said I'd drop this check previously but mistakenly re-posted it in this v5. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
