[RFC/PATCHv7 00/22] git notes

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

(Feel free to put this on hold until v1.6.5 is released. In any case,
I'm going to Berlin for the weekend, and don't expect to read much
email...)

Here is the 7th iteration of the git-notes series. Changes in this
iteration are as follows:

- Rebased onto current 'next'

- Patch 1: Include minor leak fix

- Patch 10: Rename free_commit_notes() to free_notes() (Notes are
  no longer bound to commits only, see patch 15 for details)

- Patch 12: Remove tests that are invalidated by concatenation code
  in patch 13.

Overall, I consider the 12 first patches fairly stable at this point.
There's also a slew of new patches, that has more of an RFC status:

- Patches 13-14: Concatenation of multiple notes annotating the same
  commit/object. This was originally suggested by mugwump many months
  ago, and the suggestion was re-iterated by Dscho. This change has a
  minor perfomance impact (see [1]), but I still think it's worth it.

- Patch 15: Allow notes to be attached to any object (not just commits).
  Rename get_commit_notes() to format_note() to reflect this change.

- Patch 16-19: Expand notes API in preparation for querying and
  manipulating notes from elsewhere in Git (see patch 22 for examples).

- Patch 20: Add a new notes_tree struct, and use it as the first
  parameter to all functions in the notes API. This allows API users to
  maintain their own (multiple, concurrent) notes trees (see patch 22
  for an example). We still have a default notes tree in notes.c as a
  fallback (when NULL is passed as to an API function).

- Patch 21: The default behaviour when there are multiple notes for a
  given object is to concatenate them. However, some callers (see patch
  22) want to tweak this behaviour. This patch defines a new function
  type: combine_notes_fn, for combining two notes that reference the
  same object. The notes API is then expanded to allow the caller to
  specify a suitable combine_notes_fn. For convenience, three simple
  combine_notes functions are available in the notes API:
  - combine_notes_concatenate(): Concatenates the contents of the two
    notes. (This is the default behaviour)
  - combine_notes_overwrite(): Overwrite the existing note with the
    new note.
  - combine_notes_ignore(): Keep the existing note, and ignore the new
    note.

- Patch 22: This teaches fast-import to use the new notes API when
  adding note objects to a commit. Since adding a note to a notes tree
  might cause restructuring of that notes tree, the note objects must
  be handled differently from regular blobs.
  There are some testcases for the new behaviour in this patch, but not
  enough. These will be added later.
  This patch is still very much in RFC mode...


Although this iteration brings the jh/notes topic towards feature-
completion, there are still some things left to do before I consider
the git notes feature fully complete:

- Builtin-ify git-notes shell script to take advantage of notes API

- Garbage-collect notes whose referenced objects are unreachable

- Handle note objects that are not blobs, but trees (e.g.
  refs/notes/<topic>:<commit>/<subtopic>)

- Add a simple notation for referring to an object's note (e.g.
  "<object>^{note}")

- Probably more that I haven't thought of yet...

However, It might be a good idea to consider merging the early/stable
parts of jh/notes, instead of waiting for everything to complete.


Have fun! :)

...Johan


[1] Performance impact of the concatenation rewrite.

In order to concatenate notes correctly, the tree traversal code must be
changed to more proactively unpack subtree entries (so that we can safely
determine whether there are multiple notes for a given key).

As before, the test case is as follows:
Linux kernel repo with 157101 commits, 1 note per commit, organized into
various fanout schemes. Hardware is Intel Core 2 Quad with 4GB RAM.


Algorithm / Notes tree   git log -n10 (x100)   git log --all

next / no-notes                 4.78s             63.90s

before / no-notes               4.77s             63.61s
before / no-fanout             56.59s             65.19s

16tree / no-notes               4.73s             63.80s
16tree / no-fanout             30.21s             65.11s
16tree / 2_38                   5.53s             65.24s
16tree / 2_2_36                 5.15s             65.12s

concat / no-notes               4.80s             64.21s
concat / no-fanout             30.66s             65.35s
concat / 2_38                   5.64s             65.87s
concat / 2_2_36                 5.23s             66.44s

Conclusion: There is a small, but measurable impact (about .1s or so in
the 100 x 'git log -n10' case), but I think this is small enough to be
acceptable.


Johan Herland (17):
  Teach "-m <msg>" and "-F <file>" to "git notes edit"
  fast-import: Add support for importing commit notes
  t3302-notes-index-expensive: Speed up create_repo()
  Add flags to get_commit_notes() to control the format of the note string
  Teach notes code to free its internal data structures on request
  Teach the notes lookup code to parse notes trees with various fanout schemes
  Add selftests verifying that we can parse notes trees with various fanouts
  Refactor notes code to concatenate multiple notes annotating the same object
  Add selftests verifying concatenation of multiple notes for the same commit
  Notes API: get_commit_notes() -> format_note() + remove the commit restriction
  Notes API: init_notes(): Initialize the notes tree from the given notes ref
  Notes API: add_note(): Add note objects to the internal notes tree structure
  Notes API: get_note(): Return the note annotating the given object
  Notes API: for_each_note(): Traverse the entire notes tree with a callback
  Notes API: Allow multiple concurrent notes trees with new struct notes_tree
  Refactor notes concatenation into a flexible interface for combining notes
  fast-import: Proper notes tree manipulation using the notes API

Johannes Schindelin (5):
  Introduce commit notes
  Add a script to edit/inspect notes
  Speed up git notes lookup
  Add an expensive test for git-notes
  Add '%N'-format for pretty-printing commit notes

 .gitignore                        |    1 +
 Documentation/config.txt          |   13 +
 Documentation/git-fast-import.txt |   45 +++-
 Documentation/git-notes.txt       |   60 ++++
 Documentation/pretty-formats.txt  |    1 +
 Makefile                          |    3 +
 cache.h                           |    4 +
 command-list.txt                  |    1 +
 commit.c                          |    1 +
 config.c                          |    5 +
 environment.c                     |    1 +
 fast-import.c                     |  176 +++++++++++-
 git-notes.sh                      |  121 ++++++++
 notes.c                           |  579 +++++++++++++++++++++++++++++++++++++
 notes.h                           |  113 +++++++
 pretty.c                          |   10 +
 t/t3301-notes.sh                  |  150 ++++++++++
 t/t3302-notes-index-expensive.sh  |  118 ++++++++
 t/t3303-notes-subtrees.sh         |  188 ++++++++++++
 t/t9300-fast-import.sh            |  296 +++++++++++++++++++
 20 files changed, 1875 insertions(+), 11 deletions(-)
 create mode 100644 Documentation/git-notes.txt
 create mode 100755 git-notes.sh
 create mode 100644 notes.c
 create mode 100644 notes.h
 create mode 100755 t/t3301-notes.sh
 create mode 100755 t/t3302-notes-index-expensive.sh
 create mode 100755 t/t3303-notes-subtrees.sh

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]