Re: [PATCH v6 3/7] eoie: add End of Index Entry (EOIE) extension

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 





On 9/27/2018 8:19 PM, SZEDER Gábor wrote:
On Wed, Sep 26, 2018 at 03:54:38PM -0400, Ben Peart wrote:
The End of Index Entry (EOIE) is used to locate the end of the variable

Nit: perhaps start with:

   The End of Index Entry (EOIE) optional extension can be used to ...

to make it clearer for those who don't immediately realize the
significance of the upper case 'E' in the extension's signature.

length index entries and the beginning of the extensions. Code can take
advantage of this to quickly locate the index extensions without having
to parse through all of the index entries.

Because it must be able to be loaded before the variable length cache
entries and other index extensions, this extension must be written last.
The signature for this extension is { 'E', 'O', 'I', 'E' }.

The extension consists of:

- 32-bit offset to the end of the index entries

- 160-bit SHA-1 over the extension types and their sizes (but not
their contents).  E.g. if we have "TREE" extension that is N-bytes
long, "REUC" extension that is M-bytes long, followed by "EOIE",
then the hash would be:

SHA-1("TREE" + <binary representation of N> +
	"REUC" + <binary representation of M>)

Signed-off-by: Ben Peart <Ben.Peart@xxxxxxxxxxxxx>
---
  Documentation/technical/index-format.txt |  23 ++++
  read-cache.c                             | 151 +++++++++++++++++++++--
  t/README                                 |   5 +
  t/t1700-split-index.sh                   |   1 +
  4 files changed, 172 insertions(+), 8 deletions(-)


diff --git a/t/README b/t/README
index 3ea6c85460..aa33ac4f26 100644
--- a/t/README
+++ b/t/README
@@ -327,6 +327,11 @@ GIT_TEST_COMMIT_GRAPH=<boolean>, when true, forces the commit-graph to
  be written after every 'git commit' command, and overrides the
  'core.commitGraph' setting to true.
+GIT_TEST_DISABLE_EOIE=<boolean> disables writing the EOIE extension.
+This is used to allow tests 1, 4-9 in t1700-split-index.sh to succeed
+as they currently hard code SHA values for the index which are no longer
+valid due to the addition of the EOIE extension.

Is this extension enabled by default?  The commit message doesn't
explicitly say so, but I don't see any way to turn it on or off, while
there is this new GIT_TEST environment variable to disable it for one
particular test, so it seems so.  If that's indeed the case, then
wouldn't it be better to update those hard-coded SHA1 values in t1700
instead?


Yes, it is enabled by default and the only way to disable it is the GIT_TEST_DISABLE_EOIE environment variable.

The tests in t1700-split-index.sh assume that there are no extensions in the index file so anything that adds an extension, will break one or more of the tests.

First in 'enable split index', they hard code SHA values assuming there are no extensions. If some option adds an extension, these hard coded values no longer match and the test fails.

Later in 'disable split index' they save off the SHA of the index with split-index turned off and then in later tests, compare it to the SHA of the shared index. Because extensions are stripped when the shared index is written out this only works if there were not extensions in the original index.

I'll document this behavior and reasoning in the test directly.

This did cause me to reexamine how EOIE and IEOT behave when split index is turned on. These two extensions help most with a large index. When split index is turned on, the large index is actually the shared index as the index is now the smaller set of deltas.

Currently, the extensions are stripped out of the shared index which means they are not available when they are needed to quickly load the shared index. I'll see if I can update the patch so that these extensions are still written out and available in the shared index to speed up when it is loaded.

Thanks!

  Naming Tests
  ------------
diff --git a/t/t1700-split-index.sh b/t/t1700-split-index.sh
index be22398a85..1f168378c8 100755
--- a/t/t1700-split-index.sh
+++ b/t/t1700-split-index.sh
@@ -7,6 +7,7 @@ test_description='split index mode tests'
  # We need total control of index splitting here
  sane_unset GIT_TEST_SPLIT_INDEX
  sane_unset GIT_FSMONITOR_TEST
+GIT_TEST_DISABLE_EOIE=true; export GIT_TEST_DISABLE_EOIE
test_expect_success 'enable split index' '
  	git config splitIndex.maxPercentChange 100 &&
--
2.18.0.windows.1




[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux