Re: [PATCH v4 0/9] Makefile: optimize noop runs, add shared.mak

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 02/03/2022 12:49, Ævar Arnfjörð Bjarmason wrote:
As the v1 notes (among other things):
https://lore.kernel.org/git/cover-0.8-00000000000-20211217T012902Z-avarab@xxxxxxxxx/

     This speeds up noop runs of "make" by a lot. After a "make" running a
     "make -j1" with this is ~1.5 faster than on "master"[2], and around 3x
     as fast with "make -j1 NO_TCLTK=Y" (the TCL part takes a lot of time,
     but that's another matter).

This v4 re-roll (for v3, see
https://lore.kernel.org/git/cover-v3-0.9-00000000000-20220225T090127Z-avarab@xxxxxxxxx/):

  * The "all" boilerplate goes first now, before "include"
  * Elaborated on DELETE_ON_ERROR ina commit message.
  * No longer change a ":=" to "=" while moving code.
  * Rephrased other commit messages (one of which referred to a
    function from another future series)
  * Typo fix in commit message.

Ævar Arnfjörð Bjarmason (9):
   scalar Makefile: use "The default target of..." pattern
   Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it
   Makefile: disable GNU make built-in wildcard rules
   Makefile: define $(LIB_H) in terms of $(FIND_SOURCE_FILES)
   Makefile: move ".SUFFIXES" rule to shared.mak
   Makefile: move $(comma), $(empty) and $(space) to shared.mak
   Makefile: add "$(QUIET)" boilerplate to shared.mak
   Makefile: use $(wspfx) for $(QUIET...) in shared.mak
   Makefiles: add and use wildcard "mkdir -p" template

  Documentation/Makefile    |  63 ++------------------
  Makefile                  | 118 ++++++++++++--------------------------
  config.mak.uname          |   1 -
  contrib/scalar/Makefile   |  20 ++-----
  contrib/scalar/t/Makefile |   3 +
  shared.mak                | 109 +++++++++++++++++++++++++++++++++++
  t/Makefile                |   3 +
  t/interop/Makefile        |   3 +
  t/perf/Makefile           |   3 +
  templates/Makefile        |   8 +--
  10 files changed, 171 insertions(+), 160 deletions(-)
  create mode 100644 shared.mak

Range-diff against v3:
  1:  2404c4d8b96 <  -:  ----------- scalar Makefile: set the default target after the includes
  -:  ----------- >  1:  26c6bb897cf scalar Makefile: use "The default target of..." pattern
  2:  96a490bec54 !  2:  74692458b70 Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it
     @@ Commit message
          .DELETE_ON_ERROR, 2021-05-21) for the addition and use of the
          ".DELETE_ON_ERROR" flag.
- This does have the potential downside that if e.g. templates/Makefile
     -    would like to include this "shared.mak" in the future the semantics of
     -    such a Makefile will change, but as noted in the above commits (and
     -    GNU make's own documentation) any such change would be for the better,
     -    so it's safe to do this.
     +    I.e. this changes the behavior of existing rules in the altered
     +    Makefiles (except "Makefile" & "Documentation/Makefile"). I'm
     +    confident that this is safe having read the relevant rules in those
     +    Makfiles, and as the GNU make manual notes that it isn't the default
     +    behavior is out of an abundance of backwards compatibility
     +    caution. From edition 0.75 of its manual, covering GNU make 4.3:
- This also doesn't introduce a bug by e.g. having this
     +        [Enabling '.DELETE_ON_ERROR' is] almost always what you want
     +        'make' to do, but it is not historical practice; so for
     +        compatibility, you must explicitly request it.
     +
     +    This doesn't introduce a bug by e.g. having this
          ".DELETE_ON_ERROR" flag only apply to this new shared.mak, Makefiles
          have no such scoping semantics.

I think this is much clearer now,

Best Wishes

Phillip

     +    It does increase the danger that any Makefile without an explicit "The
     +    default target of this Makefile is..." snippet to define the default
     +    target as "all" could have its default rule changed if our new
     +    shared.mak ever defines a "real" rule. In subsequent commits we'll be
     +    careful not to do that, and such breakage would be obvious e.g. in the
     +    case of "make -C t".
     +
     +    We might want to make that less fragile still (e.g. by using
     +    ".DEFAULT_GOAL" as noted in the preceding commit), but for now let's
     +    simply include "shared.mak" without adding that boilerplate to all the
     +    Makefiles that don't have it already. Most of those are already
     +    exposed to that potential caveat e.g. due to including "config.mak*".
     +
          Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@xxxxxxxxx>
## Documentation/Makefile ##
     @@ Documentation/Makefile: doc-l10n install-l10n::
## Makefile ##
      @@
     -+# Import tree-wide shared Makefile behavior and libraries
     -+include shared.mak
     -+
       # The default target of this Makefile is...
       all::
++# Import tree-wide shared Makefile behavior and libraries
     ++include shared.mak
     ++
     + # Define V=1 to have a more verbose compile.
     + #
     + # Define SHELL_PATH to a POSIX shell if your /bin/sh is broken.
      @@ Makefile: shell_compatibility_test: please_set_SHELL_PATH_to_a_more_modern_shell
       strip: $(PROGRAMS) git$X
       	$(STRIP) $(STRIP_OPTS) $^
     @@ Makefile: shell_compatibility_test: please_set_SHELL_PATH_to_a_more_modern_shell
## contrib/scalar/Makefile ##
      @@
     + # The default target of this Makefile is...
     + all::
     +
      +# Import tree-wide shared Makefile behavior and libraries
      +include ../../shared.mak
      +
     - QUIET_SUBDIR0  = +$(MAKE) -C # space to separate -C and subdir
     - QUIET_SUBDIR1  =
     -
     + include ../../config.mak.uname
     + -include ../../config.mak.autogen
     + -include ../../config.mak
## contrib/scalar/t/Makefile ##
      @@
  3:  9392e3c3e97 !  3:  0fbdeeffc7b Makefile: disable GNU make built-in wildcard rules
     @@ Commit message
          contents from RCS or SCCS. See [1] for an old mailing list discussion
          about how to disable these.
- The speed-up may wary. I've seen 1-10% depending on the speed of the
     +    The speed-up may vary. I've seen 1-10% depending on the speed of the
          local disk, caches, -jN etc. Running:
strace -f -c -S calls make -j1 NO_TCLTK=Y
  4:  07cf9daa9d6 =  4:  ea6b835308a Makefile: define $(LIB_H) in terms of $(FIND_SOURCE_FILES)
  5:  16f2e3ff35b =  5:  c2339694cf7 Makefile: move ".SUFFIXES" rule to shared.mak
  6:  1b6ecb27f02 !  6:  741fdfd48e2 Makefile: move $(comma), $(empty) and $(space) to shared.mak
     @@ Metadata
       ## Commit message ##
          Makefile: move $(comma), $(empty) and $(space) to shared.mak
- Move these variables over to the shared.max, we'll make use of them in
     -    a subsequent commit. There was no reason for these to be "simply
     -    expanded variables", so let's use the normal lazy "=" assignment here.
     +    Move these variables over to the shared.mak, we'll make use of them in
     +    a subsequent commit.
     +
     +    Note that there's reason for these to be "simply expanded variables",
     +    i.e. to use ":=" assignments instead of lazily expanded "="
     +    assignments. We could use "=", but let's leave this as-is for now for
     +    ease of review.
See 425ca6710b2 (Makefile: allow combining UBSan with other
          sanitizers, 2017-07-15) for the commit that introduced these.
     @@ shared.mak
      +
      +## comma, empty, space: handy variables as these tokens are either
      +## special or can be hard to spot among other Makefile syntax.
     -+comma = ,
     -+empty =
     -+space = $(empty) $(empty)
     ++comma := ,
     ++empty :=
     ++space := $(empty) $(empty)
  7:  471067deefc !  7:  a723cbce270 Makefile: add "$(QUIET)" boilerplate to shared.mak
     @@ config.mak.uname: vcxproj:
## contrib/scalar/Makefile ##
     -@@
     - # Import tree-wide shared Makefile behavior and libraries
     - include ../../shared.mak
     +@@ contrib/scalar/Makefile: include ../../config.mak.uname
     + -include ../../config.mak.autogen
     + -include ../../config.mak
-QUIET_SUBDIR0 = +$(MAKE) -C # space to separate -C and subdir
      -QUIET_SUBDIR1  =
     @@ contrib/scalar/Makefile
      -endif
      -endif
      -
     - include ../../config.mak.uname
     - -include ../../config.mak.autogen
     - -include ../../config.mak
     + TARGETS = scalar$(X) scalar.o
     + GITLIBS = ../../common-main.o ../../libgit.a ../../xdiff/lib.a
     +
## shared.mak ##
      @@
     - comma = ,
     - empty =
     - space = $(empty) $(empty)
     + comma := ,
     + empty :=
     + space := $(empty) $(empty)
      +
      +### Quieting
      +## common
  8:  510306d2219 !  8:  3733b0c8df1 Makefile: use $(wspfx) for $(QUIET...) in shared.mak
     @@ Commit message
          Makefile: use $(wspfx) for $(QUIET...) in shared.mak
Change the mostly move-only change in the preceding commit to use the
     -    $(wspfx) variable for defining the QUIET padding, to guarantee that
     -    it's consistent with the "TRACK_template" template.
     +    $(wspfx) variable for defining the QUIET padding. This refactoring
     +    will make it easier to emit that exact amount of padding in functions
     +    that we might add to shared.mak in the future.
- $ make CFLAGS=-I$RANDOM grep.o wspfx='$(space)->'
     -         -> GIT-CFLAGS PARAMETERS (changed)
     -         -> CC grep.o
     +    Such a function is not part of this patch series, but a
     +    "TRACK_template" that I'd like to add as a follow-up to it makes use
     +    of this. Let's make this change now while modifying these QUIET rules
     +    is fresh in our minds.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@xxxxxxxxx> ## shared.mak ##
     -@@ shared.mak: comma = ,
     - empty =
     - space = $(empty) $(empty)
     +@@ shared.mak: comma := ,
     + empty :=
     + space := $(empty) $(empty)
+## wspfx: the whitespace prefix padding for $(QUIET...) and similarly
      +## aligned output.
  9:  85bb74aa32f =  9:  4cc4aeabb20 Makefiles: add and use wildcard "mkdir -p" template




[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux