On Mon, Nov 21 2022, Eric Sunshine wrote: > On Mon, Nov 21, 2022 at 8:32 AM Ævar Arnfjörð Bjarmason > <avarab@xxxxxxxxx> wrote: >> On Sun, Nov 20 2022, Eric Sunshine wrote: >> > Somehow Windows manages to be unbelievably slow no matter what. I >> > mentioned elsewhere (after you sent this) that I tested on a five or >> > six year old 8-core dual-boot machine. Booted to Linux, running a >> > single chainlint.pl invocation using all 8 cores to check all scripts >> > in the project took under 1 second walltime. The same machine booted >> > to Windows using all 8 cores took just under two minutes(!) walltime >> > for the single Perl invocation to check all scripts in the project. >> >> I'd be really interested in seeing e.g. the NYTProf output for that run, >> compared with that on *nix (if you could upload the HTML versions of >> both somewhere, even better). > > Unfortunately, I no longer have access to that machine, or usable > Windows in general. Of course, someone else with access to a dual-boot > machine could generate such a report, but whether anyone will offer to > do so is a different matter. :( >> Maybe "chainlint.pl" is doing something odd, but this goes against the >> usual wisdom about what is and isn't slow in Perl on windows, as I >> understand it. >> >> I.e. process star-up etc. is slow there, and I/O's a bit slower, but >> once you're started up and e.g. slurping up all of those files & parsing >> them you're just running "perl-native" code. >> >> Which shouldn't be much slower at all. A perl compiled with ithreads is >> (last I checked) around 10-20% slower, and the Windows version is always >> compiled with that (it's needed for "fork" emulation). >> >> But most *nix versions are compiled with that too, and certainly the one >> you're using with "threads", so that's not the difference. >> >> So I suspect something odd's going on... > > This is all my understanding, as well, which is why I was so surprised > by the difference in speed. Aside from suspecting Windows I/O as the > culprit, another obvious possible culprit would be whatever > mechanism/primitives "ithreads" is using on Windows for > locking/synchronizing and passing messages between threads. I wouldn't > be surprised to learn that those mechanisms/primitives have very high > overhead on that platform. Yeah, that could be, but then... >> > Overall, I think Ævar's plan to parallelize linting via "make" is >> > probably the way to go. >> >> Yeah, but that seems to me to be orthagonal to why it's this slow on >> Windows, and if it is that wouldn't help much, except for incremental >> re-runs. > > Oh, I didn't at all mean that `make` parallelism would be helpful on > Windows; I can't imagine that it ever would be (though I could once > again be wrong). What I meant was that `make` parallelism would be a > nice improvement and simplification (of sorts), in general, > considering that I've given up hope of ever seeing linting be speedy > on Windows. ...that parallelism probably wouldn't be helpful, as it'll run into another thing that's slow. But just ditching the "ithreads" commit from chainlint.pl should make it much faster, as sequentially parsing all the files isn't that slow, and as that won't use threads should be much faster then.
