Re: [PATCH] ci: avoid unnecessary builds

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Tue, Nov 08, 2022 at 10:16:15AM +0100, Johannes Schindelin wrote:

> > As an aside, I wish there was a way to interpret per-repo environment
> > variables in the actual action config.
> 
> There kind of is. "Kind of" because it is not _really_ a per-repo variable
> (those do not exist on GitHub), but there are topics you can set. These
> are relatively free-form labels you can attach to _your_ fork, and these
> labels show up below the "About" section and the link to the home page (if
> any) on the right side of your respository. AFAICT these topics are not
> inherited automatically when forking a repository, which is precisely what
> we want. See
> https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/classifying-your-repository-with-topics
> for more details on that.

Ah, that's very clever, thank you!

For the original problem that motivated me adding ci-config in the first
place, branch selection, I think we could do this:

  if: |
    !contains(join(github.event.repository.topics), 'ci-only-') ||
    contains(github.event.repository.topics, format('ci-only-{0}', github.ref_name))

and then by default we'd continue to build for all pushes, but if you
add ci-only-foo as a repo topic, then we'd build only refs/heads/foo.

I may see if I can work this into our workflow file. Even if we can't
get rid of ci-config for the skip-successful-build feature, this would
still save CPU by dropping even ci-config when the branch should be
skipped entirely.

> > The current ci-config stuff works, but it's pretty horrible because (if
> > I understand correctly) it spins up a VM just to evaluate a glob and say
> > "nope, no CI needed on this branch". So:
> >
> >   1. It's wasteful of resources, compared to a system where the Actions
> >      parser can evaluate a variable.
> 
> Indeed. It might look like the job only takes a few seconds (at least that
> was the argument that got the `ci-config` patch accepted), but that misses
> the queue time, which turns this more into several dozens of seconds, and
> the recorded total duration is much longer than that. In
> https://github.com/gitgitgadget/git/actions/runs/3412982102 for example,
> the `ci-config` job only took 6 seconds, according to the page, but the
> total duration of the build was 6 minutes and 56 seconds.

Yes, but I don't think that's wasting 6 minutes of resources if we're
just sitting in a queue. It may increase the end-to-end latency of
getting the CI result, of course.

>     4. The way `ci-config` is configured is sufficiently "magic" that it
>        only benefits very, very few users, at the price of adding to
>        everybody's build time. To see what I mean, look at
>        https://github.com/gitgitgadget/git/actions/runs/3416084640/jobs/5685867765#step:1:1
>        and at
>        https://github.com/gitgitgadget/git/actions/runs/3416084640/jobs/5685879914#step:1:1
>        turning on the timestamps (i.e. click on the sprocket wheel on the
>        upper right side of the log and select "Show timestamps"). You will
>        see that the `ci-config` job started at 3:22:05 UTC and the next
>        job, `win-build`, started only at 4:16:03 UTC.

Ouch. Though I wonder in practice how fast that would have gone without
ci-config. It is just asking for a generic ubuntu vm, which many of
other jobs would. Was there a shortage of those vms at 3:22, and by the
time it finally ran that shortage was over? Or is there constant
contention, and it is increasing the end-to-end latency by asking for a
queue slot, running, and then asking for more queue slots?

>     5. There is official support for the desired behavior that comes
>        without any magic branch with special content that users somehow
>        need to learn about: If you push a branch with commit messages that
>        contain `[skip ci]`, the build will be skipped, and no time is
>        wasted on running any job. For full details, see
>        https://github.blog/changelog/2021-02-08-github-actions-skip-pull-request-and-push-workflows-with-skip-ci/

This existed back when I added ci-config originally, and I tried it, but
the results are quite painful to use. Because "skip ci" is really a
property of a branch that a commit is pushed to, and not a branch. So
you have to sprinkle these "skip ci" markers all over the branch tips,
but then remember to remove them when you actually want to do useful
things with the branches, like merge them or send them out.

-Peff



[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux