[PATCH 0/5] multithread traverse_commit_list (aka rev-list)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



rev-list is quite cpu extensive. While there's better approach to
tackle this (e.g. Shawn's bitmap cache idea), I was curious if
making it parellel could give us anything. So this is a naive (and
failed) attempt at that. I post it anyway, so people, especially gsoc
students if the related proposal is chosen, might learn from it, or
learn to avoid it, or just laugh at my attempt to relieve stress.

While this series does not break rev-list (but breaks all external
commands, I don't care anyway), and it proves lock-free pack access
works, it does not improve rev-list performance. I suspect extensive
locking around "struct object *" may be the culprit.  Or maybe it's
just my dumb partition method. A better partition method that avoids
locking might improve it. We may split the object hash table into 4,
one for each object type and assign one thread for tree, one for blob.
Or something else..

Anyway I'll stop here to finish my ongoing topics for 1.7.11 and maybe
implement Shawn's idea if I still have more time. Effectively generating
the cache from index-pack could be challenging.

Nguyễn Thái Ngọc Duy (5):
  Remove global pointer "packed_git" in favor or set/get function pair
  sha1_file: stuff various pack reading variables into a struct
  Make lookup_*() functions thread-safe
  Teach traverse_commit_list callsites about new parameter, nr_threads
  Support multithread in traverse_commit_list and rev-list

 blob.c                   |   11 ++-
 builtin/count-objects.c  |    4 +-
 builtin/fsck.c           |    4 +-
 builtin/gc.c             |    2 +-
 builtin/pack-objects.c   |   12 +-
 builtin/pack-redundant.c |    4 +-
 builtin/rev-list.c       |    5 +-
 cache.h                  |    8 +-
 commit.c                 |   12 ++-
 git.c                    |    1 +
 list-objects.c           |  157 +++++++++++++++++++++++++-
 list-objects.h           |    2 +-
 object.c                 |    9 ++-
 object.h                 |    6 +
 pack-revindex.c          |    4 +-
 revision.c               |   16 +++
 revision.h               |    2 +
 server-info.c            |    4 +-
 sha1_file.c              |  276 ++++++++++++++++++++++++++++------------------
 sha1_name.c              |    2 +-
 tag.c                    |    9 +-
 tree.c                   |   18 +++-
 upload-pack.c            |    2 +-
 23 files changed, 419 insertions(+), 151 deletions(-)

-- 
1.7.8.36.g69ee2

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]