This 17+ patches long patch series is intended as a replacement for large 'gitweb: File based caching layer (from git.kernel.org)' mega-patch by John 'Warthog9' Hawley aka J.H., by starting small and adding features piece by piece. This is fourthfifth version (5th release) of this series, and is available on http://repo.or.cz/w/git/jnareb-git.git as 'gitweb/cache-kernel-v5' branch. Earlier versions are available there as branches 'gitweb/cache-kernel', 'gitweb/cache-kernel-v2', 'gitweb/cache-kernel-v3' and 'gitweb/cache-kernel-v4'. Previous version of this series was sent to git mailing list as: * [RFC PATCHv4 00/17] gitweb: Simple file based output caching Message-Id: <1276531710-22945-1-git-send-email-jnareb@xxxxxxxxx> http://thread.gmane.org/gmane.comp.version-control.git/149101 This version is based on top of aad24ad (gitweb: Move call to evaluate_git_version after evaluate_gitweb_config, 2010-10-05) in 'gitweb/web' branch of http://repo.or.cz/w/git/jnareb-git.git repository. The 'gitweb/web' branch consist of the following commits on top of 1e63341 (Merge branch 'maint', 2010-09-30) in 'master' branch of main git repository: gitweb/Makefile: Include gitweb/config.mak t/gitweb-lib.sh: Add support for GITWEB_TEST_INSTALLED gitweb/Makefile: Add 'test' and 'test-installed' targets gitweb: Move call to evaluate_git_version after evaluate_gitweb_config The first three patches were send as 3-part series: * [PATCHv2 0/3] Testing installed gitweb Message-Id: <1285506146-8009-1-git-send-email-jnareb@xxxxxxxxx> http://thread.gmane.org/gmane.comp.version-control.git/157222 Those are required to be able to test installed version of gitweb and modules, to check if there are any problems with process of installing gitweb. The last patch in 'gitweb/web' branch was send as: * [PATCH] gitweb: Move call to evaluate_git_version after evaluate_gitweb_config Message-ID: <20100926113431.28918.53550.stgit@xxxxxxxxxxxxxxxxxxxxx> http://article.gmane.org/gmane.comp.version-control.git/157220 This was needed during development of this series, to avoid spurious output of gitweb tests when using `--debug' option. The MAIN DIFFERENCE from previous version is that it is based on never version of gitweb; in particular putting all code in subroutines: gitweb calls run() subroutine on startup, and run_request() subroutine on each request (per connection) from the 'jn/gitweb-fastcgi' branch. The patches gitweb: Return or exit after done serving request gitweb: Fix typo in hash key name in %opts in git_header_html present at beginning of previous (fourth) version of this series are already merged in. The patches gitweb/lib - Benchmarking GitwebCache::SimpleFileCache (in t/9603/) gitweb/lib - Alternate ways of capturing output which were interleaved with other patches in previous version of this series got moved to the end of series, and marked proof of concept ("PoC"). The following changes are available in the git repository at: git://repo.or.cz/git/jnareb-git.git gitweb/cache-kernel-v5 All comments welcome! J.H., could you comment on how this series relates to the gitweb code *currently* running on git.kernel.org wrt. code? If possible, could you try to compare performance of those two implementations, the one presented here in this series, and the one used by git.kernel.org. Pasky or other repo.or.cz admins, could you tell us if and what kind of caching gitweb used by repo.or.cz uses, and how it compares (wrt code and if possible also performance) to solution implemented in this series? TODO list and areas of possible improvements would be send in separate email. Jakub Narebski (21): t/test-lib.sh: Export also GIT_BUILD_DIR in test_external gitweb: Prepare for splitting gitweb gitweb/lib - Very simple file based cache gitweb/lib - Stat-based cache expiration gitweb/lib - Regenerate entry if the cache file has size of 0 gitweb/lib - Simple select(FH) based output capture gitweb/lib - Cache captured output (using get/set) gitweb: Add optional output caching (from gitweb/cache.pm) gitweb/lib - Adaptive cache expiration time gitweb/lib - Use CHI compatibile (compute method) caching interface gitweb/lib - Use locking to avoid 'cache miss stampede' problem gitweb/lib - No need for File::Temp when locking gitweb/lib - Serve stale data when waiting for filling cache gitweb/lib - Regenerate (refresh) cache in background gitweb: Introduce %actions_info, gathering information about actions gitweb: Show appropriate "Generating..." page when regenerating cache gitweb: Add startup delay to activity indicator for cache gitweb/lib - Add clear() and size() methods to caching interface gitweb: Add beginnings of cache administration page gitweb/lib - Benchmarking GitwebCache::SimpleFileCache (in t/9603/) gitweb/lib - Alternate ways of capturing output gitweb/Makefile | 24 ++- gitweb/README | 61 +++ gitweb/gitweb.perl | 459 ++++++++++++++++++++++-- gitweb/lib/GitwebCache/CacheOutput.pm | 95 +++++ gitweb/lib/GitwebCache/Capture.pm | 66 ++++ gitweb/lib/GitwebCache/Capture/PerlIO.pm | 79 ++++ gitweb/lib/GitwebCache/Capture/SelectFH.pm | 82 ++++ gitweb/lib/GitwebCache/Capture/TiedCapture.pm | 76 ++++ gitweb/lib/GitwebCache/FileCacheWithLocking.pm | 280 ++++++++++++++ gitweb/lib/GitwebCache/SimpleFileCache.pm | 472 ++++++++++++++++++++++++ gitweb/lib/Tie/Restore.pm | 24 ++ gitweb/lib/TiedCapture/PerlIO.pm | 56 +++ gitweb/lib/TiedCapture/String.pm | 53 +++ t/t9500-gitweb-standalone-no-errors.sh | 20 + t/t9503-gitweb-caching-interface.sh | 34 ++ t/t9503/benchmark_caching_interface.pl | 209 +++++++++++ t/t9503/test_cache_interface.pl | 407 ++++++++++++++++++++ t/t9504-gitweb-capture-interface.sh | 34 ++ t/t9504/benchmark_capture_implementations.pl | 226 +++++++++++ t/t9504/test_capture_implementations.pl | 85 +++++ t/t9504/test_capture_interface.pl | 76 ++++ t/t9505-gitweb-cache.sh | 34 ++ t/t9505/test_cache_output.pl | 70 ++++ t/test-lib.sh | 4 +- 24 files changed, 2997 insertions(+), 29 deletions(-) create mode 100644 gitweb/lib/GitwebCache/CacheOutput.pm create mode 100644 gitweb/lib/GitwebCache/Capture.pm create mode 100644 gitweb/lib/GitwebCache/Capture/PerlIO.pm create mode 100644 gitweb/lib/GitwebCache/Capture/SelectFH.pm create mode 100644 gitweb/lib/GitwebCache/Capture/TiedCapture.pm create mode 100644 gitweb/lib/GitwebCache/FileCacheWithLocking.pm create mode 100644 gitweb/lib/GitwebCache/SimpleFileCache.pm create mode 100644 gitweb/lib/Tie/Restore.pm create mode 100644 gitweb/lib/TiedCapture/PerlIO.pm create mode 100644 gitweb/lib/TiedCapture/String.pm create mode 100755 t/t9503-gitweb-caching-interface.sh create mode 100755 t/t9503/benchmark_caching_interface.pl create mode 100755 t/t9503/test_cache_interface.pl create mode 100755 t/t9504-gitweb-capture-interface.sh create mode 100755 t/t9504/benchmark_capture_implementations.pl create mode 100755 t/t9504/test_capture_implementations.pl create mode 100755 t/t9504/test_capture_interface.pl create mode 100755 t/t9505-gitweb-cache.sh create mode 100755 t/t9505/test_cache_output.pl -- 1.7.3 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html