This adds the reftable library, and hooks it up as a ref backend.
Feedback wanted:
* spots marked with XXX in the Git source code.
* what is a good test strategy? Could we have a CI flavor where we flip the
default to reftable, and see how it fares?
v8
* propagate errors to git.
* discard empty tables in the stack.
* one very basic test (t0031.sh)
v9
* Added spec as technical doc.
* Use 4-byte hash ID as API. Storage format for SHA256 is still pending
discussion.
v10
* Serialize 4 byte hash ID to the version-2 format
v11
* use reftable_ namespace prefix.
* transaction API to provide locking.
* reorganized record.{h,c}. More doc in reftable.h
* support log record deletion.
* pluggable malloc/free
Han-Wen Nienhuys (8):
refs.h: clarify reflog iteration order
create .git/refs in files-backend.c
refs: document how ref_iterator_advance_fn should handle symrefs
Add .gitattributes for the reftable/ directory
reftable: define version 2 of the spec to accomodate SHA256
reftable: clarify how empty tables should be written
Add reftable library
Reftable support for git-core
Jonathan Nieder (1):
reftable: file format documentation
Documentation/Makefile | 1 +
Documentation/technical/reftable.txt | 1080 ++++++++++++++++
.../technical/repository-version.txt | 7 +
Makefile | 24 +-
builtin/clone.c | 5 +-
builtin/init-db.c | 51 +-
cache.h | 4 +-
refs.c | 20 +-
refs.h | 7 +-
refs/files-backend.c | 6 +
refs/refs-internal.h | 6 +
refs/reftable-backend.c | 1002 +++++++++++++++
reftable/.gitattributes | 1 +
reftable/LICENSE | 31 +
reftable/README.md | 11 +
reftable/VERSION | 5 +
reftable/basics.c | 194 +++
reftable/basics.h | 35 +
reftable/block.c | 433 +++++++
reftable/block.h | 76 ++
reftable/blocksource.h | 22 +
reftable/bytes.c | 0
reftable/config.h | 1 +
reftable/constants.h | 21 +
reftable/dump.c | 97 ++
reftable/file.c | 98 ++
reftable/iter.c | 234 ++++
reftable/iter.h | 56 +
reftable/merged.c | 299 +++++
reftable/merged.h | 34 +
reftable/pq.c | 115 ++
reftable/pq.h | 34 +
reftable/reader.c | 756 +++++++++++
reftable/reader.h | 60 +
reftable/record.c | 1098 ++++++++++++++++
reftable/record.h | 106 ++
reftable/reftable.h | 520 ++++++++
reftable/slice.c | 219 ++++
reftable/slice.h | 42 +
reftable/stack.c | 1132 +++++++++++++++++
reftable/stack.h | 42 +
reftable/system.h | 53 +
reftable/tree.c | 67 +
reftable/tree.h | 24 +
reftable/update.sh | 14 +
reftable/writer.c | 653 ++++++++++
reftable/writer.h | 45 +
reftable/zlib-compat.c | 92 ++
repository.c | 2 +
repository.h | 3 +
setup.c | 12 +-
t/t0031-reftable.sh | 35 +
52 files changed, 8954 insertions(+), 31 deletions(-)
create mode 100644 Documentation/technical/reftable.txt
create mode 100644 refs/reftable-backend.c
create mode 100644 reftable/.gitattributes
create mode 100644 reftable/LICENSE
create mode 100644 reftable/README.md
create mode 100644 reftable/VERSION
create mode 100644 reftable/basics.c
create mode 100644 reftable/basics.h
create mode 100644 reftable/block.c
create mode 100644 reftable/block.h
create mode 100644 reftable/blocksource.h
create mode 100644 reftable/bytes.c
create mode 100644 reftable/config.h
create mode 100644 reftable/constants.h
create mode 100644 reftable/dump.c
create mode 100644 reftable/file.c
create mode 100644 reftable/iter.c
create mode 100644 reftable/iter.h
create mode 100644 reftable/merged.c
create mode 100644 reftable/merged.h
create mode 100644 reftable/pq.c
create mode 100644 reftable/pq.h
create mode 100644 reftable/reader.c
create mode 100644 reftable/reader.h
create mode 100644 reftable/record.c
create mode 100644 reftable/record.h
create mode 100644 reftable/reftable.h
create mode 100644 reftable/slice.c
create mode 100644 reftable/slice.h
create mode 100644 reftable/stack.c
create mode 100644 reftable/stack.h
create mode 100644 reftable/system.h
create mode 100644 reftable/tree.c
create mode 100644 reftable/tree.h
create mode 100644 reftable/update.sh
create mode 100644 reftable/writer.c
create mode 100644 reftable/writer.h
create mode 100644 reftable/zlib-compat.c
create mode 100755 t/t0031-reftable.sh
base-commit: 9fadedd637b312089337d73c3ed8447e9f0aa775
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-539%2Fhanwen%2Freftable-v8
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-539/hanwen/reftable-v8
Pull-Request: https://github.com/gitgitgadget/git/pull/539
Range-diff vs v7:
1: b1e44bc431e = 1: 05634d26dd5 refs.h: clarify reflog iteration order
2: b68488a095e = 2: 8d34e2c5c8b create .git/refs in files-backend.c
3: da538ef7421 = 3: d08f823844d refs: document how ref_iterator_advance_fn should handle symrefs
-: ----------- > 4: ca95b3a371e Add .gitattributes for the reftable/ directory
4: aae26814983 = 5: dfc8b131294 reftable: file format documentation
-: ----------- > 6: 45ab5361750 reftable: define version 2 of the spec to accomodate SHA256
-: ----------- > 7: 67ee5962e85 reftable: clarify how empty tables should be written
5: 30ed43a4fdb ! 8: 4f9bdd7312b Add reftable library
@@ -5,21 +5,21 @@
Reftable is a format for storing the ref database. Its rationale and
specification is in the preceding commit.
- Further context and motivation can be found in background reading:
+ This imports the upstream library as one big commit. For understanding
+ the code, it is suggested to read the code in the following order:
- * Original discussion on JGit-dev: https://www.eclipse.org/lists/jgit-dev/msg03389.html
+ * The specification under Documentation/technical/reftable.txt
- * First design discussion on git@vger: https://public-inbox.org/git/CAJo=hJtTp2eA3z9wW9cHo-nA7kK40vVThqh6inXpbCcqfdMP9g@xxxxxxxxxxxxxx/
+ * reftable.h - the public API
- * Last design discussion on git@vger: https://public-inbox.org/git/CAJo=hJsZcAM9sipdVr7TMD-FD2V2W6_pvMQ791EGCDsDkQ033w@xxxxxxxxxxxxxx/
+ * block.{c,h} - reading and writing blocks.
- * First attempt at implementation: https://public-inbox.org/git/CAP8UFD0PPZSjBnxCA7ez91vBuatcHKQ+JUWvTD1iHcXzPBjPBg@xxxxxxxxxxxxxx/
+ * writer.{c,h} - writing a complete reftable file.
- * libgit2 support issue: https://github.com/libgit2/libgit2/issues
+ * merged.{c,h} and pq.{c,h} - reading a stack of reftables
- * GitLab support issue: https://gitlab.com/gitlab-org/git/issues/6
-
- * go-git support issue: https://github.com/src-d/go-git/issues/1059
+ * stack.{c,h} - writing and compacting stacks of reftable on the
+ filesystem.
Signed-off-by: Han-Wen Nienhuys <hanwen@xxxxxxxxxx>
@@ -82,11 +82,11 @@
--- /dev/null
+++ b/reftable/VERSION
@@
-+commit ccce3b3eb763e79b23a3b4677d65ecceb4155ba3
++commit 920f7413f7a25f308d9a203f584873f6753192ec
+Author: Han-Wen Nienhuys <hanwen@xxxxxxxxxx>
-+Date: Tue Feb 25 20:42:29 2020 +0100
++Date: Wed Apr 1 12:41:41 2020 +0200
+
-+ C: rephrase the hash ID API in terms of a uint32_t
++ C: remove stray debug printfs
diff --git a/reftable/basics.c b/reftable/basics.c
new file mode 100644
@@ -165,10 +165,10 @@
+ return;
+ }
+ while (*p) {
-+ free(*p);
++ reftable_free(*p);
+ p++;
+ }
-+ free(a);
++ reftable_free(a);
+}
+
+int names_length(char **names)
@@ -201,8 +201,8 @@
+ if (p < next) {
+ if (names_len == names_cap) {
+ names_cap = 2 * names_cap + 1;
-+ names = realloc(names,
-+ names_cap * sizeof(char *));
++ names = reftable_realloc(
++ names, names_cap * sizeof(char *));
+ }
+ names[names_len++] = xstrdup(p);
+ }
@@ -211,7 +211,7 @@
+
+ if (names_len == names_cap) {
+ names_cap = 2 * names_cap + 1;
-+ names = realloc(names, names_cap * sizeof(char *));
++ names = reftable_realloc(names, names_cap * sizeof(char *));
+ }
+
+ names[names_len] = NULL;
@@ -232,7 +232,7 @@
+ return *a == *b;
+}
+
-+const char *error_str(int err)
++const char *reftable_error_str(int err)
+{
+ switch (err) {
+ case IO_ERROR:
@@ -252,6 +252,40 @@
+ default:
+ return "unknown error code";
+ }
++}
++
++void *(*reftable_malloc_ptr)(size_t sz) = &malloc;
++void *(*reftable_realloc_ptr)(void *, size_t) = &realloc;
++void (*reftable_free_ptr)(void *) = &free;
++
++void *reftable_malloc(size_t sz)
++{
++ return (*reftable_malloc_ptr)(sz);
++}
++
++void *reftable_realloc(void *p, size_t sz)
++{
++ return (*reftable_realloc_ptr)(p, sz);
++}
++
++void reftable_free(void *p)
++{
++ reftable_free_ptr(p);
++}
++
++void *reftable_calloc(size_t sz)
++{
++ void *p = reftable_malloc(sz);
++ memset(p, 0, sz);
++ return p;
++}
++
++void reftable_set_alloc(void *(*malloc)(size_t),
++ void *(*realloc)(void *, size_t), void (*free)(void *))
++{
++ reftable_malloc_ptr = malloc;
++ reftable_realloc_ptr = realloc;
++ reftable_free_ptr = free;
+}
diff --git a/reftable/basics.h b/reftable/basics.h
@@ -288,6 +322,11 @@
+int names_equal(char **a, char **b);
+int names_length(char **names);
+
++void *reftable_malloc(size_t sz);
++void *reftable_realloc(void *p, size_t sz);
++void reftable_free(void *p);
++void *reftable_calloc(size_t sz);
++
+#endif
diff --git a/reftable/block.c b/reftable/block.c
@@ -313,10 +352,32 @@
+#include "reftable.h"
+#include "zlib.h"
+
-+int block_writer_register_restart(struct block_writer *w, int n, bool restart,
-+ struct slice key);
++int header_size(int version)
++{
++ switch (version) {
++ case 1:
++ return 24;
++ case 2:
++ return 28;
++ }
++ abort();
++}
++
++int footer_size(int version)
++{
++ switch (version) {
++ case 1:
++ return 68;
++ case 2:
++ return 72;
++ }
++ abort();
++}
++
++int block_writer_register_restart(struct reftable_block_writer *w, int n,
++ bool restart, struct slice key);
+
-+void block_writer_init(struct block_writer *bw, byte typ, byte *buf,
++void block_writer_init(struct reftable_block_writer *bw, byte typ, byte *buf,
+ uint32_t block_size, uint32_t header_off, int hash_size)
+{
+ bw->buf = buf;
@@ -329,14 +390,14 @@
+ bw->entries = 0;
+}
+
-+byte block_writer_type(struct block_writer *bw)
++byte block_writer_type(struct reftable_block_writer *bw)
+{
+ return bw->buf[bw->header_off];
+}
+
+/* adds the record to the block. Returns -1 if it does not fit, 0 on
+ success */
-+int block_writer_add(struct block_writer *w, struct record rec)
++int block_writer_add(struct reftable_block_writer *w, struct record rec)
+{
+ struct slice empty = { 0 };
+ struct slice last = w->entries % w->restart_interval == 0 ? empty :
@@ -357,32 +418,29 @@
+ if (n < 0) {
+ goto err;
+ }
-+ out.buf += n;
-+ out.len -= n;
++ slice_consume(&out, n);
+
+ n = record_encode(rec, out, w->hash_size);
+ if (n < 0) {
+ goto err;
+ }
-+
-+ out.buf += n;
-+ out.len -= n;
++ slice_consume(&out, n);
+
+ if (block_writer_register_restart(w, start.len - out.len, restart,
+ key) < 0) {
+ goto err;
+ }
+
-+ free(slice_yield(&key));
++ reftable_free(slice_yield(&key));
+ return 0;
+
+err:
-+ free(slice_yield(&key));
++ reftable_free(slice_yield(&key));
+ return -1;
+}
+
-+int block_writer_register_restart(struct block_writer *w, int n, bool restart,
-+ struct slice key)
++int block_writer_register_restart(struct reftable_block_writer *w, int n,
++ bool restart, struct slice key)
+{
+ int rlen = w->restart_len;
+ if (rlen >= MAX_RESTARTS) {
@@ -398,7 +456,7 @@
+ if (restart) {
+ if (w->restart_len == w->restart_cap) {
+ w->restart_cap = w->restart_cap * 2 + 1;
-+ w->restarts = realloc(
++ w->restarts = reftable_realloc(
+ w->restarts, sizeof(uint32_t) * w->restart_cap);
+ }
+
@@ -411,7 +469,7 @@
+ return 0;
+}
+
-+int block_writer_finish(struct block_writer *w)
++int block_writer_finish(struct reftable_block_writer *w)
+{
+ int i = 0;
+ for (i = 0; i < w->restart_len; i++) {
@@ -442,7 +500,7 @@
+ }
+
+ if (Z_OK != zresult) {
-+ free(slice_yield(&compressed));
++ reftable_free(slice_yield(&compressed));
+ return ZLIB_ERROR;
+ }
+
@@ -455,14 +513,14 @@
+ return w->next;
+}
+
-+byte block_reader_type(struct block_reader *r)
++byte block_reader_type(struct reftable_block_reader *r)
+{
+ return r->block.data[r->header_off];
+}
+
-+int block_reader_init(struct block_reader *br, struct block *block,
-+ uint32_t header_off, uint32_t table_block_size,
-+ int hash_size)
++int block_reader_init(struct reftable_block_reader *br,
++ struct reftable_block *block, uint32_t header_off,
++ uint32_t table_block_size, int hash_size)
+{
+ uint32_t full_block_size = table_block_size;
+ byte typ = block->data[header_off];
@@ -485,7 +543,7 @@
+ uncompressed.buf + block_header_skip,
+ &dst_len, block->data + block_header_skip,
+ &src_len)) {
-+ free(slice_yield(&uncompressed));
++ reftable_free(slice_yield(&uncompressed));
+ return ZLIB_ERROR;
+ }
+
@@ -526,12 +584,14 @@
+ return 0;
+}
+
-+static uint32_t block_reader_restart_offset(struct block_reader *br, int i)
++static uint32_t block_reader_restart_offset(struct reftable_block_reader *br,
++ int i)
+{
+ return get_be24(br->restart_bytes + 3 * i);
+}
+
-+void block_reader_start(struct block_reader *br, struct block_iter *it)
++void block_reader_start(struct reftable_block_reader *br,
++ struct reftable_block_iter *it)
+{
+ it->br = br;
+ slice_resize(&it->last_key, 0);
@@ -541,7 +601,7 @@
+struct restart_find_args {
+ int error;
+ struct slice key;
-+ struct block_reader *r;
++ struct reftable_block_reader *r;
+};
+
+static int restart_key_less(int idx, void *args)
@@ -566,12 +626,13 @@
+
+ {
+ int result = slice_compare(a->key, rkey);
-+ free(slice_yield(&rkey));
++ reftable_free(slice_yield(&rkey));
+ return result;
+ }
+}
+
-+void block_iter_copy_from(struct block_iter *dest, struct block_iter *src)
++void block_iter_copy_from(struct reftable_block_iter *dest,
++ struct reftable_block_iter *src)
+{
+ dest->br = src->br;
+ dest->next_off = src->next_off;
@@ -579,7 +640,7 @@
+}
+
+/* return < 0 for error, 0 for OK, > 0 for EOF. */
-+int block_iter_next(struct block_iter *it, struct record rec)
++int block_iter_next(struct reftable_block_iter *it, struct record rec)
+{
+ if (it->next_off >= it->br->block_len) {
+ return 1;
@@ -598,23 +659,21 @@
+ return -1;
+ }
+
-+ in.buf += n;
-+ in.len -= n;
++ slice_consume(&in, n);
+ n = record_decode(rec, key, extra, in, it->br->hash_size);
+ if (n < 0) {
+ return -1;
+ }
-+ in.buf += n;
-+ in.len -= n;
++ slice_consume(&in, n);
+
+ slice_copy(&it->last_key, key);
+ it->next_off += start.len - in.len;
-+ free(slice_yield(&key));
++ reftable_free(slice_yield(&key));
+ return 0;
+ }
+}
+
-+int block_reader_first_key(struct block_reader *br, struct slice *key)
++int block_reader_first_key(struct reftable_block_reader *br, struct slice *key)
+{
+ struct slice empty = { 0 };
+ int off = br->header_off + 4;
@@ -631,18 +690,18 @@
+ return 0;
+}
+
-+int block_iter_seek(struct block_iter *it, struct slice want)
++int block_iter_seek(struct reftable_block_iter *it, struct slice want)
+{
+ return block_reader_seek(it->br, it, want);
+}
+
-+void block_iter_close(struct block_iter *it)
++void block_iter_close(struct reftable_block_iter *it)
+{
-+ free(slice_yield(&it->last_key));
++ reftable_free(slice_yield(&it->last_key));
+}
+
-+int block_reader_seek(struct block_reader *br, struct block_iter *it,
-+ struct slice want)
++int block_reader_seek(struct reftable_block_reader *br,
++ struct reftable_block_iter *it, struct slice want)
+{
+ struct restart_find_args args = {
+ .key = want,
@@ -667,7 +726,7 @@
+ struct slice key = { 0 };
+ int result = 0;
+ int err = 0;
-+ struct block_iter next = { 0 };
++ struct reftable_block_iter next = { 0 };
+ while (true) {
+ block_iter_copy_from(&next, it);
+
@@ -687,25 +746,25 @@
+ }
+
+ exit:
-+ free(slice_yield(&key));
-+ free(slice_yield(&next.last_key));
++ reftable_free(slice_yield(&key));
++ reftable_free(slice_yield(&next.last_key));
+ record_clear(rec);
-+ free(record_yield(&rec));
++ reftable_free(record_yield(&rec));
+
+ return result;
+ }
+}
+
-+void block_writer_reset(struct block_writer *bw)
++void block_writer_reset(struct reftable_block_writer *bw)
+{
+ bw->restart_len = 0;
+ bw->last_key.len = 0;
+}
+
-+void block_writer_clear(struct block_writer *bw)
++void block_writer_clear(struct reftable_block_writer *bw)
+{
+ FREE_AND_NULL(bw->restarts);
-+ free(slice_yield(&bw->last_key));
++ reftable_free(slice_yield(&bw->last_key));
+ /* the block is not owned. */
+}
@@ -729,7 +788,7 @@
+#include "record.h"
+#include "reftable.h"
+
-+struct block_writer {
++struct reftable_block_writer {
+ byte *buf;
+ uint32_t block_size;
+ uint32_t header_off;
@@ -744,17 +803,17 @@
+ int entries;
+};
+
-+void block_writer_init(struct block_writer *bw, byte typ, byte *buf,
++void block_writer_init(struct reftable_block_writer *bw, byte typ, byte *buf,
+ uint32_t block_size, uint32_t header_off, int hash_size);
-+byte block_writer_type(struct block_writer *bw);
-+int block_writer_add(struct block_writer *w, struct record rec);
-+int block_writer_finish(struct block_writer *w);
-+void block_writer_reset(struct block_writer *bw);
-+void block_writer_clear(struct block_writer *bw);
++byte block_writer_type(struct reftable_block_writer *bw);
++int block_writer_add(struct reftable_block_writer *w, struct record rec);
++int block_writer_finish(struct reftable_block_writer *w);
++void block_writer_reset(struct reftable_block_writer *bw);
++void block_writer_clear(struct reftable_block_writer *bw);
+
-+struct block_reader {
++struct reftable_block_reader {
+ uint32_t header_off;
-+ struct block block;
++ struct reftable_block block;
+ int hash_size;
+
+ /* size of the data, excluding restart data. */
@@ -764,25 +823,30 @@
+ uint16_t restart_count;
+};
+
-+struct block_iter {
++struct reftable_block_iter {
+ uint32_t next_off;
-+ struct block_reader *br;
++ struct reftable_block_reader *br;
+ struct slice last_key;
+};
+
-+int block_reader_init(struct block_reader *br, struct block *bl,
-+ uint32_t header_off, uint32_t table_block_size,
-+ int hash_size);
-+void block_reader_start(struct block_reader *br, struct block_iter *it);
-+int block_reader_seek(struct block_reader *br, struct block_iter *it,
-+ struct slice want);
-+byte block_reader_type(struct block_reader *r);
-+int block_reader_first_key(struct block_reader *br, struct slice *key);
-+
-+void block_iter_copy_from(struct block_iter *dest, struct block_iter *src);
-+int block_iter_next(struct block_iter *it, struct record rec);
-+int block_iter_seek(struct block_iter *it, struct slice want);
-+void block_iter_close(struct block_iter *it);
++int block_reader_init(struct reftable_block_reader *br,
++ struct reftable_block *bl, uint32_t header_off,
++ uint32_t table_block_size, int hash_size);
++void block_reader_start(struct reftable_block_reader *br,
++ struct reftable_block_iter *it);
++int block_reader_seek(struct reftable_block_reader *br,
++ struct reftable_block_iter *it, struct slice want);
++byte block_reader_type(struct reftable_block_reader *r);
++int block_reader_first_key(struct reftable_block_reader *br, struct slice *key);
++
++void block_iter_copy_from(struct reftable_block_iter *dest,
++ struct reftable_block_iter *src);
++int block_iter_next(struct reftable_block_iter *it, struct record rec);
++int block_iter_seek(struct reftable_block_iter *it, struct slice want);
++void block_iter_close(struct reftable_block_iter *it);
++
++int header_size(int version);
++int footer_size(int version);
+
+#endif
@@ -804,11 +868,13 @@
+
+#include "reftable.h"
+
-+uint64_t block_source_size(struct block_source source);
-+int block_source_read_block(struct block_source source, struct block *dest,
-+ uint64_t off, uint32_t size);
-+void block_source_return_block(struct block_source source, struct block *ret);
-+void block_source_close(struct block_source source);
++uint64_t block_source_size(struct reftable_block_source source);
++int block_source_read_block(struct reftable_block_source source,
++ struct reftable_block *dest, uint64_t off,
++ uint32_t size);
++void block_source_return_block(struct reftable_block_source source,
++ struct reftable_block *ret);
++void block_source_close(struct reftable_block_source source);
+
+#endif
@@ -838,10 +904,6 @@
+#ifndef CONSTANTS_H
+#define CONSTANTS_H
+
-+#define VERSION 1
-+#define HEADER_SIZE 24
-+#define FOOTER_SIZE 68
-+
+#define BLOCK_TYPE_LOG 'g'
+#define BLOCK_TYPE_INDEX 'i'
+#define BLOCK_TYPE_REF 'r'
@@ -987,10 +1049,10 @@
+ return ((struct file_block_source *)b)->size;
+}
+
-+static void file_return_block(void *b, struct block *dest)
++static void file_return_block(void *b, struct reftable_block *dest)
+{
+ memset(dest->data, 0xff, dest->len);
-+ free(dest->data);
++ reftable_free(dest->data);
+}
+
+static void file_close(void *b)
@@ -1001,15 +1063,15 @@
+ ((struct file_block_source *)b)->fd = 0;
+ }
+
-+ free(b);
++ reftable_free(b);
+}
+
-+static int file_read_block(void *v, struct block *dest, uint64_t off,
++static int file_read_block(void *v, struct reftable_block *dest, uint64_t off,
+ uint32_t size)
+{
+ struct file_block_source *b = (struct file_block_source *)v;
+ assert(off + size <= b->size);
-+ dest->data = malloc(size);
++ dest->data = reftable_malloc(size);
+ if (pread(b->fd, dest->data, size, off) != size) {
+ return -1;
+ }
@@ -1017,14 +1079,15 @@
+ return size;
+}
+
-+struct block_source_vtable file_vtable = {
++struct reftable_block_source_vtable file_vtable = {
+ .size = &file_size,
+ .read_block = &file_read_block,
+ .return_block = &file_return_block,
+ .close = &file_close,
+};
+
-+int block_source_from_file(struct block_source *bs, const char *name)
++int reftable_block_source_from_file(struct reftable_block_source *bs,
++ const char *name)
+{
+ struct stat st = { 0 };
+ int err = 0;
@@ -1043,7 +1106,7 @@
+
+ {
+ struct file_block_source *p =
-+ calloc(sizeof(struct file_block_source), 1);
++ reftable_calloc(sizeof(struct file_block_source));
+ p->size = st.st_size;
+ p->fd = fd;
+
@@ -1053,7 +1116,7 @@
+ return 0;
+}
+
-+int fd_writer(void *arg, byte *data, int sz)
++int reftable_fd_write(void *arg, byte *data, int sz)
+{
+ int *fdp = (int *)arg;
+ return write(*fdp, data, sz);
@@ -1081,7 +1144,7 @@
+#include "reader.h"
+#include "reftable.h"
+
-+bool iterator_is_null(struct iterator it)
++bool iterator_is_null(struct reftable_iterator it)
+{
+ return it.ops == NULL;
+}
@@ -1095,23 +1158,23 @@
+{
+}
+
-+struct iterator_vtable empty_vtable = {
++struct reftable_iterator_vtable empty_vtable = {
+ .next = &empty_iterator_next,
+ .close = &empty_iterator_close,
+};
+
-+void iterator_set_empty(struct iterator *it)
++void iterator_set_empty(struct reftable_iterator *it)
+{
+ it->iter_arg = NULL;
+ it->ops = &empty_vtable;
+}
+
-+int iterator_next(struct iterator it, struct record rec)
++int iterator_next(struct reftable_iterator it, struct record rec)
+{
+ return it.ops->next(it.iter_arg, rec);
+}
+
-+void iterator_destroy(struct iterator *it)
++void reftable_iterator_destroy(struct reftable_iterator *it)
+{
+ if (it->ops == NULL) {
+ return;
@@ -1121,14 +1184,16 @@
+ FREE_AND_NULL(it->iter_arg);
+}
+
-+int iterator_next_ref(struct iterator it, struct ref_record *ref)
++int reftable_iterator_next_ref(struct reftable_iterator it,
++ struct reftable_ref_record *ref)
+{
+ struct record rec = { 0 };
+ record_from_ref(&rec, ref);
+ return iterator_next(it, rec);
+}
+
-+int iterator_next_log(struct iterator it, struct log_record *log)
++int reftable_iterator_next_log(struct reftable_iterator it,
++ struct reftable_log_record *log)
+{
+ struct record rec = { 0 };
+ record_from_log(&rec, log);
@@ -1139,31 +1204,33 @@
+{
+ struct filtering_ref_iterator *fri =
+ (struct filtering_ref_iterator *)iter_arg;
-+ free(slice_yield(&fri->oid));
-+ iterator_destroy(&fri->it);
++ reftable_free(slice_yield(&fri->oid));
++ reftable_iterator_destroy(&fri->it);
+}
+
+static int filtering_ref_iterator_next(void *iter_arg, struct record rec)
+{
+ struct filtering_ref_iterator *fri =
+ (struct filtering_ref_iterator *)iter_arg;
-+ struct ref_record *ref = (struct ref_record *)rec.data;
++ struct reftable_ref_record *ref =
++ (struct reftable_ref_record *)rec.data;
+
+ while (true) {
-+ int err = iterator_next_ref(fri->it, ref);
++ int err = reftable_iterator_next_ref(fri->it, ref);
+ if (err != 0) {
+ return err;
+ }
+
+ if (fri->double_check) {
-+ struct iterator it = { 0 };
++ struct reftable_iterator it = { 0 };
+
-+ int err = reader_seek_ref(fri->r, &it, ref->ref_name);
++ int err = reftable_reader_seek_ref(fri->r, &it,
++ ref->ref_name);
+ if (err == 0) {
-+ err = iterator_next_ref(it, ref);
++ err = reftable_iterator_next_ref(it, ref);
+ }
+
-+ iterator_destroy(&it);
++ reftable_iterator_destroy(&it);
+
+ if (err < 0) {
+ return err;
@@ -1183,12 +1250,12 @@
+ }
+}
+
-+struct iterator_vtable filtering_ref_iterator_vtable = {
++struct reftable_iterator_vtable filtering_ref_iterator_vtable = {
+ .next = &filtering_ref_iterator_next,
+ .close = &filtering_ref_iterator_close,
+};
+
-+void iterator_from_filtering_ref_iterator(struct iterator *it,
++void iterator_from_filtering_ref_iterator(struct reftable_iterator *it,
+ struct filtering_ref_iterator *fri)
+{
+ it->iter_arg = fri;
@@ -1200,7 +1267,7 @@
+ struct indexed_table_ref_iter *it = (struct indexed_table_ref_iter *)p;
+ block_iter_close(&it->cur);
+ reader_return_block(it->r, &it->block_reader.block);
-+ free(slice_yield(&it->oid));
++ reftable_free(slice_yield(&it->oid));
+}
+
+static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)
@@ -1231,7 +1298,8 @@
+static int indexed_table_ref_iter_next(void *p, struct record rec)
+{
+ struct indexed_table_ref_iter *it = (struct indexed_table_ref_iter *)p;
-+ struct ref_record *ref = (struct ref_record *)rec.data;
++ struct reftable_ref_record *ref =
++ (struct reftable_ref_record *)rec.data;
+
+ while (true) {
+ int err = block_iter_next(&it->cur, rec);
@@ -1259,11 +1327,11 @@
+}
+
+int new_indexed_table_ref_iter(struct indexed_table_ref_iter **dest,
-+ struct reader *r, byte *oid, int oid_len,
-+ uint64_t *offsets, int offset_len)
++ struct reftable_reader *r, byte *oid,
++ int oid_len, uint64_t *offsets, int offset_len)
+{
+ struct indexed_table_ref_iter *itr =
-+ calloc(sizeof(struct indexed_table_ref_iter), 1);
++ reftable_calloc(sizeof(struct indexed_table_ref_iter));
+ int err = 0;
+
+ itr->r = r;
@@ -1275,19 +1343,19 @@
+
+ err = indexed_table_ref_iter_next_block(itr);
+ if (err < 0) {
-+ free(itr);
++ reftable_free(itr);
+ } else {
+ *dest = itr;
+ }
+ return err;
+}
+
-+struct iterator_vtable indexed_table_ref_iter_vtable = {
++struct reftable_iterator_vtable indexed_table_ref_iter_vtable = {
+ .next = &indexed_table_ref_iter_next,
+ .close = &indexed_table_ref_iter_close,
+};
+
-+void iterator_from_indexed_table_ref_iter(struct iterator *it,
++void iterator_from_indexed_table_ref_iter(struct reftable_iterator *it,
+ struct indexed_table_ref_iter *itr)
+{
+ it->iter_arg = itr;
@@ -1314,27 +1382,27 @@
+#include "record.h"
+#include "slice.h"
+
-+struct iterator_vtable {
++struct reftable_iterator_vtable {
+ int (*next)(void *iter_arg, struct record rec);
+ void (*close)(void *iter_arg);
+};
+
-+void iterator_set_empty(struct iterator *it);
-+int iterator_next(struct iterator it, struct record rec);
-+bool iterator_is_null(struct iterator it);
++void iterator_set_empty(struct reftable_iterator *it);
++int iterator_next(struct reftable_iterator it, struct record rec);
++bool iterator_is_null(struct reftable_iterator it);
+
+struct filtering_ref_iterator {
+ bool double_check;
-+ struct reader *r;
++ struct reftable_reader *r;
+ struct slice oid;
-+ struct iterator it;
++ struct reftable_iterator it;
+};
+
-+void iterator_from_filtering_ref_iterator(struct iterator *,
++void iterator_from_filtering_ref_iterator(struct reftable_iterator *,
+ struct filtering_ref_iterator *);
+
+struct indexed_table_ref_iter {
-+ struct reader *r;
++ struct reftable_reader *r;
+ struct slice oid;
+
+ /* mutable */
@@ -1343,16 +1411,16 @@
+ /* Points to the next offset to read. */
+ int offset_idx;
+ int offset_len;
-+ struct block_reader block_reader;
-+ struct block_iter cur;
++ struct reftable_block_reader block_reader;
++ struct reftable_block_iter cur;
+ bool finished;
+};
+
-+void iterator_from_indexed_table_ref_iter(struct iterator *it,
++void iterator_from_indexed_table_ref_iter(struct reftable_iterator *it,
+ struct indexed_table_ref_iter *itr);
+int new_indexed_table_ref_iter(struct indexed_table_ref_iter **dest,
-+ struct reader *r, byte *oid, int oid_len,
-+ uint64_t *offsets, int offset_len);
++ struct reftable_reader *r, byte *oid,
++ int oid_len, uint64_t *offsets, int offset_len);
+
+#endif
@@ -1389,9 +1457,9 @@
+ }
+
+ if (err > 0) {
-+ iterator_destroy(&mi->stack[i]);
++ reftable_iterator_destroy(&mi->stack[i]);
+ record_clear(rec);
-+ free(record_yield(&rec));
++ reftable_free(record_yield(&rec));
+ } else {
+ struct pq_entry e = {
+ .rec = rec,
@@ -1410,9 +1478,9 @@
+ int i = 0;
+ merged_iter_pqueue_clear(&mi->pq);
+ for (i = 0; i < mi->stack_len; i++) {
-+ iterator_destroy(&mi->stack[i]);
++ reftable_iterator_destroy(&mi->stack[i]);
+ }
-+ free(mi->stack);
++ reftable_free(mi->stack);
+}
+
+static int merged_iter_advance_subiter(struct merged_iter *mi, int idx)
@@ -1433,9 +1501,9 @@
+ }
+
+ if (err > 0) {
-+ iterator_destroy(&mi->stack[idx]);
++ reftable_iterator_destroy(&mi->stack[idx]);
+ record_clear(rec);
-+ free(record_yield(&rec));
++ reftable_free(record_yield(&rec));
+ return 0;
+ }
+
@@ -1462,7 +1530,7 @@
+ record_key(top.rec, &k);
+
+ cmp = slice_compare(k, entry_key);
-+ free(slice_yield(&k));
++ reftable_free(slice_yield(&k));
+
+ if (cmp > 0) {
+ break;
@@ -1474,13 +1542,13 @@
+ return err;
+ }
+ record_clear(top.rec);
-+ free(record_yield(&top.rec));
++ reftable_free(record_yield(&top.rec));
+ }
+
+ record_copy_from(rec, entry.rec, hash_size(mi->hash_id));
+ record_clear(entry.rec);
-+ free(record_yield(&entry.rec));
-+ free(slice_yield(&entry_key));
++ reftable_free(record_yield(&entry.rec));
++ reftable_free(slice_yield(&entry_key));
+ return 0;
+}
+
@@ -1494,41 +1562,42 @@
+ return merged_iter_next(mi, rec);
+}
+
-+struct iterator_vtable merged_iter_vtable = {
++struct reftable_iterator_vtable merged_iter_vtable = {
+ .next = &merged_iter_next_void,
+ .close = &merged_iter_close,
+};
+
-+static void iterator_from_merged_iter(struct iterator *it,
++static void iterator_from_merged_iter(struct reftable_iterator *it,
+ struct merged_iter *mi)
+{
+ it->iter_arg = mi;
+ it->ops = &merged_iter_vtable;
+}
+
-+int new_merged_table(struct merged_table **dest, struct reader **stack, int n,
-+ uint32_t hash_id)
++int reftable_new_merged_table(struct reftable_merged_table **dest,
++ struct reftable_reader **stack, int n,
++ uint32_t hash_id)
+{
+ uint64_t last_max = 0;
+ uint64_t first_min = 0;
+ int i = 0;
+ for (i = 0; i < n; i++) {
-+ struct reader *r = stack[i];
++ struct reftable_reader *r = stack[i];
+ if (r->hash_id != hash_id) {
+ return FORMAT_ERROR;
+ }
-+ if (i > 0 && last_max >= reader_min_update_index(r)) {
++ if (i > 0 && last_max >= reftable_reader_min_update_index(r)) {
+ return FORMAT_ERROR;
+ }
+ if (i == 0) {
-+ first_min = reader_min_update_index(r);
++ first_min = reftable_reader_min_update_index(r);
+ }
+
-+ last_max = reader_max_update_index(r);
++ last_max = reftable_reader_max_update_index(r);
+ }
+
+ {
-+ struct merged_table m = {
++ struct reftable_merged_table m = {
+ .stack = stack,
+ .stack_len = n,
+ .min = first_min,
@@ -1536,52 +1605,56 @@
+ .hash_id = hash_id,
+ };
+
-+ *dest = calloc(sizeof(struct merged_table), 1);
++ *dest = reftable_calloc(sizeof(struct reftable_merged_table));
+ **dest = m;
+ }
+ return 0;
+}
+
-+void merged_table_close(struct merged_table *mt)
++void reftable_merged_table_close(struct reftable_merged_table *mt)
+{
+ int i = 0;
+ for (i = 0; i < mt->stack_len; i++) {
-+ reader_free(mt->stack[i]);
++ reftable_reader_free(mt->stack[i]);
+ }
+ FREE_AND_NULL(mt->stack);
+ mt->stack_len = 0;
+}
+
+/* clears the list of subtable, without affecting the readers themselves. */
-+void merged_table_clear(struct merged_table *mt)
++void merged_table_clear(struct reftable_merged_table *mt)
+{
+ FREE_AND_NULL(mt->stack);
+ mt->stack_len = 0;
+}
+
-+void merged_table_free(struct merged_table *mt)
++void reftable_merged_table_free(struct reftable_merged_table *mt)
+{
+ if (mt == NULL) {
+ return;
+ }
+ merged_table_clear(mt);
-+ free(mt);
++ reftable_free(mt);
+}
+
-+uint64_t merged_max_update_index(struct merged_table *mt)
++uint64_t
++reftable_merged_table_max_update_index(struct reftable_merged_table *mt)
+{
+ return mt->max;
+}
+
-+uint64_t merged_min_update_index(struct merged_table *mt)
++uint64_t
++reftable_merged_table_min_update_index(struct reftable_merged_table *mt)
+{
+ return mt->min;
+}
+
-+static int merged_table_seek_record(struct merged_table *mt,
-+ struct iterator *it, struct record rec)
++static int merged_table_seek_record(struct reftable_merged_table *mt,
++ struct reftable_iterator *it,
++ struct record rec)
+{
-+ struct iterator *iters = calloc(sizeof(struct iterator), mt->stack_len);
++ struct reftable_iterator *iters = reftable_calloc(
++ sizeof(struct reftable_iterator) * mt->stack_len);
+ struct merged_iter merged = {
+ .stack = iters,
+ .typ = record_type(rec),
@@ -1602,9 +1675,9 @@
+ if (err < 0) {
+ int i = 0;
+ for (i = 0; i < n; i++) {
-+ iterator_destroy(&iters[i]);
++ reftable_iterator_destroy(&iters[i]);
+ }
-+ free(iters);
++ reftable_free(iters);
+ return err;
+ }
+
@@ -1615,17 +1688,19 @@
+ }
+
+ {
-+ struct merged_iter *p = malloc(sizeof(struct merged_iter));
++ struct merged_iter *p =
++ reftable_malloc(sizeof(struct merged_iter));
+ *p = merged;
+ iterator_from_merged_iter(it, p);
+ }
+ return 0;
+}
+
-+int merged_table_seek_ref(struct merged_table *mt, struct iterator *it,
-+ const char *name)
++int reftable_merged_table_seek_ref(struct reftable_merged_table *mt,
++ struct reftable_iterator *it,
++ const char *name)
+{
-+ struct ref_record ref = {
++ struct reftable_ref_record ref = {
+ .ref_name = (char *)name,
+ };
+ struct record rec = { 0 };
@@ -1633,10 +1708,11 @@
+ return merged_table_seek_record(mt, it, rec);
+}
+
-+int merged_table_seek_log_at(struct merged_table *mt, struct iterator *it,
-+ const char *name, uint64_t update_index)
++int reftable_merged_table_seek_log_at(struct reftable_merged_table *mt,
++ struct reftable_iterator *it,
++ const char *name, uint64_t update_index)
+{
-+ struct log_record log = {
++ struct reftable_log_record log = {
+ .ref_name = (char *)name,
+ .update_index = update_index,
+ };
@@ -1645,11 +1721,12 @@
+ return merged_table_seek_record(mt, it, rec);
+}
+
-+int merged_table_seek_log(struct merged_table *mt, struct iterator *it,
-+ const char *name)
++int reftable_merged_table_seek_log(struct reftable_merged_table *mt,
++ struct reftable_iterator *it,
++ const char *name)
+{
+ uint64_t max = ~((uint64_t)0);
-+ return merged_table_seek_log_at(mt, it, name, max);
++ return reftable_merged_table_seek_log_at(mt, it, name, max);
+}
diff --git a/reftable/merged.h b/reftable/merged.h
@@ -1671,8 +1748,8 @@
+#include "pq.h"
+#include "reftable.h"
+
-+struct merged_table {
-+ struct reader **stack;
++struct reftable_merged_table {
++ struct reftable_reader **stack;
+ int stack_len;
+ uint32_t hash_id;
+
@@ -1681,14 +1758,14 @@
+};
+
+struct merged_iter {
-+ struct iterator *stack;
++ struct reftable_iterator *stack;
+ uint32_t hash_id;
+ int stack_len;
+ byte typ;
+ struct merged_iter_pqueue pq;
-+} merged_iter;
++};
+
-+void merged_table_clear(struct merged_table *mt);
++void merged_table_clear(struct reftable_merged_table *mt);
+
+#endif
@@ -1719,8 +1796,8 @@
+
+ cmp = slice_compare(ak, bk);
+
-+ free(slice_yield(&ak));
-+ free(slice_yield(&bk));
++ reftable_free(slice_yield(&ak));
++ reftable_free(slice_yield(&bk));
+
+ if (cmp == 0) {
+ return a.index > b.index;
@@ -1784,7 +1861,8 @@
+ int i = 0;
+ if (pq->len == pq->cap) {
+ pq->cap = 2 * pq->cap + 1;
-+ pq->heap = realloc(pq->heap, pq->cap * sizeof(struct pq_entry));
++ pq->heap = reftable_realloc(pq->heap,
++ pq->cap * sizeof(struct pq_entry));
+ }
+
+ pq->heap[pq->len++] = e;
@@ -1806,7 +1884,7 @@
+ int i = 0;
+ for (i = 0; i < pq->len; i++) {
+ record_clear(pq->heap[i].rec);
-+ free(record_yield(&pq->heap[i].rec));
++ reftable_free(record_yield(&pq->heap[i].rec));
+ }
+ FREE_AND_NULL(pq->heap);
+ pq->len = pq->cap = 0;
@@ -1876,20 +1954,22 @@
+#include "reftable.h"
+#include "tree.h"
+
-+uint64_t block_source_size(struct block_source source)
++uint64_t block_source_size(struct reftable_block_source source)
+{
+ return source.ops->size(source.arg);
+}
+
-+int block_source_read_block(struct block_source source, struct block *dest,
-+ uint64_t off, uint32_t size)
++int block_source_read_block(struct reftable_block_source source,
++ struct reftable_block *dest, uint64_t off,
++ uint32_t size)
+{
+ int result = source.ops->read_block(source.arg, dest, off, size);
+ dest->source = source;
+ return result;
+}
+
-+void block_source_return_block(struct block_source source, struct block *blockp)
++void block_source_return_block(struct reftable_block_source source,
++ struct reftable_block *blockp)
+{
+ source.ops->return_block(source.arg, blockp);
+ blockp->data = NULL;
@@ -1898,7 +1978,7 @@
+ blockp->source.arg = NULL;
+}
+
-+void block_source_close(struct block_source *source)
++void block_source_close(struct reftable_block_source *source)
+{
+ if (source->ops == NULL) {
+ return;
@@ -1908,7 +1988,8 @@
+ source->ops = NULL;
+}
+
-+static struct reader_offsets *reader_offsets_for(struct reader *r, byte typ)
++static struct reftable_reader_offsets *
++reader_offsets_for(struct reftable_reader *r, byte typ)
+{
+ switch (typ) {
+ case BLOCK_TYPE_REF:
@@ -1921,7 +2002,8 @@
+ abort();
+}
+
-+static int reader_get_block(struct reader *r, struct block *dest, uint64_t off,
++static int reader_get_block(struct reftable_reader *r,
++ struct reftable_block *dest, uint64_t off,
+ uint32_t sz)
+{
+ if (off >= r->size) {
@@ -1935,22 +2017,22 @@
+ return block_source_read_block(r->source, dest, off, sz);
+}
+
-+void reader_return_block(struct reader *r, struct block *p)
++void reader_return_block(struct reftable_reader *r, struct reftable_block *p)
+{
+ block_source_return_block(r->source, p);
+}
+
-+uint32_t reader_hash_id(struct reader *r)
++uint32_t reftable_reader_hash_id(struct reftable_reader *r)
+{
+ return r->hash_id;
+}
+
-+const char *reader_name(struct reader *r)
++const char *reader_name(struct reftable_reader *r)
+{
+ return r->name;
+}
+
-+static int parse_footer(struct reader *r, byte *footer, byte *header)
++static int parse_footer(struct reftable_reader *r, byte *footer, byte *header)
+{
+ byte *f = footer;
+ int err = 0;
@@ -1960,25 +2042,12 @@
+ }
+ f += 4;
+
-+ if (memcmp(footer, header, HEADER_SIZE)) {
++ if (memcmp(footer, header, header_size(r->version))) {
+ err = FORMAT_ERROR;
+ goto exit;
+ }
+
-+ r->hash_id = SHA1_ID;
-+ {
-+ byte version = *f++;
-+ if (version == 2) {
-+ /* DO NOT SUBMIT. Not yet in the standard. */
-+ r->hash_id = SHA256_ID;
-+ version = 1;
-+ }
-+ if (version != 1) {
-+ err = FORMAT_ERROR;
-+ goto exit;
-+ }
-+ }
-+
++ f++;
+ r->block_size = get_be24(f);
+
+ f += 3;
@@ -1987,6 +2056,22 @@
+ r->max_update_index = get_be64(f);
+ f += 8;
+
++ if (r->version == 1) {
++ r->hash_id = SHA1_ID;
++ } else {
++ r->hash_id = get_be32(f);
++ switch (r->hash_id) {
++ case SHA1_ID:
++ break;
++ case SHA256_ID:
++ break;
++ default:
++ err = FORMAT_ERROR;
++ goto exit;
++ }
++ f += 4;
++ }
++
+ r->ref_offsets.index_offset = get_be64(f);
+ f += 8;
+
@@ -2014,7 +2099,7 @@
+ }
+
+ {
-+ byte first_block_typ = header[HEADER_SIZE];
++ byte first_block_typ = header[header_size(r->version)];
+ r->ref_offsets.present = (first_block_typ == BLOCK_TYPE_REF);
+ r->ref_offsets.offset = 0;
+ r->log_offsets.present = (first_block_typ == BLOCK_TYPE_LOG ||
@@ -2026,27 +2111,40 @@
+ return err;
+}
+
-+int init_reader(struct reader *r, struct block_source source, const char *name)
++int init_reader(struct reftable_reader *r, struct reftable_block_source source,
++ const char *name)
+{
-+ struct block footer = { 0 };
-+ struct block header = { 0 };
++ struct reftable_block footer = { 0 };
++ struct reftable_block header = { 0 };
+ int err = 0;
+
-+ memset(r, 0, sizeof(struct reader));
-+ r->size = block_source_size(source) - FOOTER_SIZE;
-+ r->source = source;
-+ r->name = xstrdup(name);
-+ r->hash_id = 0;
++ memset(r, 0, sizeof(struct reftable_reader));
+
-+ err = block_source_read_block(source, &footer, r->size, FOOTER_SIZE);
-+ if (err != FOOTER_SIZE) {
++ /* Need +1 to read type of first block. */
++ err = block_source_read_block(source, &header, 0, header_size(2) + 1);
++ if (err != header_size(2) + 1) {
+ err = IO_ERROR;
+ goto exit;
+ }
+
-+ /* Need +1 to read type of first block. */
-+ err = reader_get_block(r, &header, 0, HEADER_SIZE + 1);
-+ if (err != HEADER_SIZE + 1) {
++ if (memcmp(header.data, "REFT", 4)) {
++ err = FORMAT_ERROR;
++ goto exit;
++ }
++ r->version = header.data[4];
++ if (r->version != 1 && r->version != 2) {
++ err = FORMAT_ERROR;
++ goto exit;
++ }
++
++ r->size = block_source_size(source) - footer_size(r->version);
++ r->source = source;
++ r->name = xstrdup(name);
++ r->hash_id = 0;
++
++ err = block_source_read_block(source, &footer, r->size,
++ footer_size(r->version));
++ if (err != footer_size(r->version)) {
+ err = IO_ERROR;
+ goto exit;
+ }
@@ -2059,10 +2157,10 @@
+}
+
+struct table_iter {
-+ struct reader *r;
++ struct reftable_reader *r;
+ byte typ;
+ uint64_t block_off;
-+ struct block_iter bi;
++ struct reftable_block_iter bi;
+ bool finished;
+};
+
@@ -2080,7 +2178,7 @@
+{
+ int res = block_iter_next(&ti->bi, rec);
+ if (res == 0 && record_type(rec) == BLOCK_TYPE_REF) {
-+ ((struct ref_record *)rec.data)->update_index +=
++ ((struct reftable_ref_record *)rec.data)->update_index +=
+ ti->r->min_update_index;
+ }
+
@@ -2099,12 +2197,13 @@
+ ti->bi.next_off = 0;
+}
+
-+static int32_t extract_block_size(byte *data, byte *typ, uint64_t off)
++static int32_t extract_block_size(byte *data, byte *typ, uint64_t off,
++ int version)
+{
+ int32_t result = 0;
+
+ if (off == 0) {
-+ data += 24;
++ data += header_size(version);
+ }
+
+ *typ = data[0];
@@ -2114,15 +2213,16 @@
+ return result;
+}
+
-+int reader_init_block_reader(struct reader *r, struct block_reader *br,
++int reader_init_block_reader(struct reftable_reader *r,
++ struct reftable_block_reader *br,
+ uint64_t next_off, byte want_typ)
+{
+ int32_t guess_block_size = r->block_size ? r->block_size :
+ DEFAULT_BLOCK_SIZE;
-+ struct block block = { 0 };
++ struct reftable_block block = { 0 };
+ byte block_typ = 0;
+ int err = 0;
-+ uint32_t header_off = next_off ? 0 : HEADER_SIZE;
++ uint32_t header_off = next_off ? 0 : header_size(r->version);
+ int32_t block_size = 0;
+
+ if (next_off >= r->size) {
@@ -2134,7 +2234,8 @@
+ return err;
+ }
+
-+ block_size = extract_block_size(block.data, &block_typ, next_off);
++ block_size = extract_block_size(block.data, &block_typ, next_off,
++ r->version);
+ if (block_size < 0) {
+ return block_size;
+ }
@@ -2160,7 +2261,7 @@
+ struct table_iter *src)
+{
+ uint64_t next_block_off = src->block_off + src->bi.br->full_block_size;
-+ struct block_reader br = { 0 };
++ struct reftable_block_reader br = { 0 };
+ int err = 0;
+
+ dest->r = src->r;
@@ -2177,7 +2278,8 @@
+ }
+
+ {
-+ struct block_reader *brp = malloc(sizeof(struct block_reader));
++ struct reftable_block_reader *brp =
++ reftable_malloc(sizeof(struct reftable_block_reader));
+ *brp = br;
+
+ dest->finished = false;
@@ -2229,29 +2331,30 @@
+ block_iter_close(&ti->bi);
+}
+
-+struct iterator_vtable table_iter_vtable = {
++struct reftable_iterator_vtable table_iter_vtable = {
+ .next = &table_iter_next_void,
+ .close = &table_iter_close,
+};
+
-+static void iterator_from_table_iter(struct iterator *it, struct table_iter *ti)
++static void iterator_from_table_iter(struct reftable_iterator *it,
++ struct table_iter *ti)
+{
+ it->iter_arg = ti;
+ it->ops = &table_iter_vtable;
+}
+
-+static int reader_table_iter_at(struct reader *r, struct table_iter *ti,
-+ uint64_t off, byte typ)
++static int reader_table_iter_at(struct reftable_reader *r,
++ struct table_iter *ti, uint64_t off, byte typ)
+{
-+ struct block_reader br = { 0 };
-+ struct block_reader *brp = NULL;
++ struct reftable_block_reader br = { 0 };
++ struct reftable_block_reader *brp = NULL;
+
+ int err = reader_init_block_reader(r, &br, off, typ);
+ if (err != 0) {
+ return err;
+ }
+
-+ brp = malloc(sizeof(struct block_reader));
++ brp = reftable_malloc(sizeof(struct reftable_block_reader));
+ *brp = br;
+ ti->r = r;
+ ti->typ = block_reader_type(brp);
@@ -2260,10 +2363,10 @@
+ return 0;
+}
+
-+static int reader_start(struct reader *r, struct table_iter *ti, byte typ,
-+ bool index)
++static int reader_start(struct reftable_reader *r, struct table_iter *ti,
++ byte typ, bool index)
+{
-+ struct reader_offsets *offs = reader_offsets_for(r, typ);
++ struct reftable_reader_offsets *offs = reader_offsets_for(r, typ);
+ uint64_t off = offs->offset;
+ if (index) {
+ off = offs->index_offset;
@@ -2276,7 +2379,7 @@
+ return reader_table_iter_at(r, ti, off, typ);
+}
+
-+static int reader_seek_linear(struct reader *r, struct table_iter *ti,
++static int reader_seek_linear(struct reftable_reader *r, struct table_iter *ti,
+ struct record want)
+{
+ struct record rec = new_record(record_type(want));
@@ -2321,14 +2424,14 @@
+exit:
+ block_iter_close(&next.bi);
+ record_clear(rec);
-+ free(record_yield(&rec));
-+ free(slice_yield(&want_key));
-+ free(slice_yield(&got_key));
++ reftable_free(record_yield(&rec));
++ reftable_free(slice_yield(&want_key));
++ reftable_free(slice_yield(&got_key));
+ return err;
+}
+
-+static int reader_seek_indexed(struct reader *r, struct iterator *it,
-+ struct record rec)
++static int reader_seek_indexed(struct reftable_reader *r,
++ struct reftable_iterator *it, struct record rec)
+{
+ struct index_record want_index = { 0 };
+ struct record want_index_rec = { 0 };
@@ -2380,7 +2483,7 @@
+
+ if (err == 0) {
+ struct table_iter *malloced =
-+ calloc(sizeof(struct table_iter), 1);
++ reftable_calloc(sizeof(struct table_iter));
+ table_iter_copy_from(malloced, &next);
+ iterator_from_table_iter(it, malloced);
+ }
@@ -2392,10 +2495,11 @@
+ return err;
+}
+
-+static int reader_seek_internal(struct reader *r, struct iterator *it,
-+ struct record rec)
++static int reader_seek_internal(struct reftable_reader *r,
++ struct reftable_iterator *it, struct record rec)
+{
-+ struct reader_offsets *offs = reader_offsets_for(r, record_type(rec));
++ struct reftable_reader_offsets *offs =
++ reader_offsets_for(r, record_type(rec));
+ uint64_t idx = offs->index_offset;
+ struct table_iter ti = { 0 };
+ int err = 0;
@@ -2413,7 +2517,8 @@
+ }
+
+ {
-+ struct table_iter *p = malloc(sizeof(struct table_iter));
++ struct table_iter *p =
++ reftable_malloc(sizeof(struct table_iter));
+ *p = ti;
+ iterator_from_table_iter(it, p);
+ }
@@ -2421,11 +2526,12 @@
+ return 0;
+}
+
-+int reader_seek(struct reader *r, struct iterator *it, struct record rec)
++int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,
++ struct record rec)
+{
+ byte typ = record_type(rec);
+
-+ struct reader_offsets *offs = reader_offsets_for(r, typ);
++ struct reftable_reader_offsets *offs = reader_offsets_for(r, typ);
+ if (!offs->present) {
+ iterator_set_empty(it);
+ return 0;
@@ -2434,9 +2540,10 @@
+ return reader_seek_internal(r, it, rec);
+}
+
-+int reader_seek_ref(struct reader *r, struct iterator *it, const char *name)
++int reftable_reader_seek_ref(struct reftable_reader *r,
++ struct reftable_iterator *it, const char *name)
+{
-+ struct ref_record ref = {
++ struct reftable_ref_record ref = {
+ .ref_name = (char *)name,
+ };
+ struct record rec = { 0 };
@@ -2444,10 +2551,11 @@
+ return reader_seek(r, it, rec);
+}
+
-+int reader_seek_log_at(struct reader *r, struct iterator *it, const char *name,
-+ uint64_t update_index)
++int reftable_reader_seek_log_at(struct reftable_reader *r,
++ struct reftable_iterator *it, const char *name,
++ uint64_t update_index)
+{
-+ struct log_record log = {
++ struct reftable_log_record log = {
+ .ref_name = (char *)name,
+ .update_index = update_index,
+ };
@@ -2456,45 +2564,49 @@
+ return reader_seek(r, it, rec);
+}
+
-+int reader_seek_log(struct reader *r, struct iterator *it, const char *name)
++int reftable_reader_seek_log(struct reftable_reader *r,
++ struct reftable_iterator *it, const char *name)
+{
+ uint64_t max = ~((uint64_t)0);
-+ return reader_seek_log_at(r, it, name, max);
++ return reftable_reader_seek_log_at(r, it, name, max);
+}
+
-+void reader_close(struct reader *r)
++void reader_close(struct reftable_reader *r)
+{
+ block_source_close(&r->source);
+ FREE_AND_NULL(r->name);
+}
+
-+int new_reader(struct reader **p, struct block_source src, char const *name)
++int reftable_new_reader(struct reftable_reader **p,
++ struct reftable_block_source src, char const *name)
+{
-+ struct reader *rd = calloc(sizeof(struct reader), 1);
++ struct reftable_reader *rd =
++ reftable_calloc(sizeof(struct reftable_reader));
+ int err = init_reader(rd, src, name);
+ if (err == 0) {
+ *p = rd;
+ } else {
-+ free(rd);
++ reftable_free(rd);
+ }
+ return err;
+}
+
-+void reader_free(struct reader *r)
++void reftable_reader_free(struct reftable_reader *r)
+{
+ reader_close(r);
-+ free(r);
++ reftable_free(r);
+}
+
-+static int reader_refs_for_indexed(struct reader *r, struct iterator *it,
-+ byte *oid)
++static int reftable_reader_refs_for_indexed(struct reftable_reader *r,
++ struct reftable_iterator *it,
++ byte *oid)
+{
+ struct obj_record want = {
+ .hash_prefix = oid,
+ .hash_prefix_len = r->object_id_len,
+ };
+ struct record want_rec = { 0 };
-+ struct iterator oit = { 0 };
++ struct reftable_iterator oit = { 0 };
+ struct obj_record got = { 0 };
+ struct record got_rec = { 0 };
+ int err = 0;
@@ -2508,7 +2620,7 @@
+
+ record_from_obj(&got_rec, &got);
+ err = iterator_next(oit, got_rec);
-+ iterator_destroy(&oit);
++ reftable_iterator_destroy(&oit);
+ if (err < 0) {
+ return err;
+ }
@@ -2537,18 +2649,19 @@
+ return 0;
+}
+
-+static int reader_refs_for_unindexed(struct reader *r, struct iterator *it,
-+ byte *oid, int oid_len)
++static int reftable_reader_refs_for_unindexed(struct reftable_reader *r,
++ struct reftable_iterator *it,
++ byte *oid, int oid_len)
+{
-+ struct table_iter *ti = calloc(sizeof(struct table_iter), 1);
++ struct table_iter *ti = reftable_calloc(sizeof(struct table_iter));
+ struct filtering_ref_iterator *filter = NULL;
+ int err = reader_start(r, ti, BLOCK_TYPE_REF, false);
+ if (err < 0) {
-+ free(ti);
++ reftable_free(ti);
+ return err;
+ }
+
-+ filter = calloc(sizeof(struct filtering_ref_iterator), 1);
++ filter = reftable_calloc(sizeof(struct filtering_ref_iterator));
+ slice_resize(&filter->oid, oid_len);
+ memcpy(filter->oid.buf, oid, oid_len);
+ filter->r = r;
@@ -2559,21 +2672,22 @@
+ return 0;
+}
+
-+int reader_refs_for(struct reader *r, struct iterator *it, byte *oid,
-+ int oid_len)
++int reftable_reader_refs_for(struct reftable_reader *r,
++ struct reftable_iterator *it, byte *oid,
++ int oid_len)
+{
+ if (r->obj_offsets.present) {
-+ return reader_refs_for_indexed(r, it, oid);
++ return reftable_reader_refs_for_indexed(r, it, oid);
+ }
-+ return reader_refs_for_unindexed(r, it, oid, oid_len);
++ return reftable_reader_refs_for_unindexed(r, it, oid, oid_len);
+}
+
-+uint64_t reader_max_update_index(struct reader *r)
++uint64_t reftable_reader_max_update_index(struct reftable_reader *r)
+{
+ return r->max_update_index;
+}
+
-+uint64_t reader_min_update_index(struct reader *r)
++uint64_t reftable_reader_min_update_index(struct reftable_reader *r)
+{
+ return r->min_update_index;
+}
@@ -2598,40 +2712,48 @@
+#include "record.h"
+#include "reftable.h"
+
-+uint64_t block_source_size(struct block_source source);
++uint64_t block_source_size(struct reftable_block_source source);
+
-+int block_source_read_block(struct block_source source, struct block *dest,
-+ uint64_t off, uint32_t size);
-+void block_source_return_block(struct block_source source, struct block *ret);
-+void block_source_close(struct block_source *source);
++int block_source_read_block(struct reftable_block_source source,
++ struct reftable_block *dest, uint64_t off,
++ uint32_t size);
++void block_source_return_block(struct reftable_block_source source,
++ struct reftable_block *ret);
++void block_source_close(struct reftable_block_source *source);
+
-+struct reader_offsets {
++struct reftable_reader_offsets {
+ bool present;
+ uint64_t offset;
+ uint64_t index_offset;
+};
+
-+struct reader {
++struct reftable_reader {
+ char *name;
-+ struct block_source source;
++ struct reftable_block_source source;
+ uint32_t hash_id;
++
++ // Size of the file, excluding the footer.
+ uint64_t size;
+ uint32_t block_size;
+ uint64_t min_update_index;
+ uint64_t max_update_index;
+ int object_id_len;
++ int version;
+
-+ struct reader_offsets ref_offsets;
-+ struct reader_offsets obj_offsets;
-+ struct reader_offsets log_offsets;
++ struct reftable_reader_offsets ref_offsets;
++ struct reftable_reader_offsets obj_offsets;
++ struct reftable_reader_offsets log_offsets;
+};
+
-+int init_reader(struct reader *r, struct block_source source, const char *name);
-+int reader_seek(struct reader *r, struct iterator *it, struct record rec);
-+void reader_close(struct reader *r);
-+const char *reader_name(struct reader *r);
-+void reader_return_block(struct reader *r, struct block *p);
-+int reader_init_block_reader(struct reader *r, struct block_reader *br,
++int init_reader(struct reftable_reader *r, struct reftable_block_source source,
++ const char *name);
++int reader_seek(struct reftable_reader *r, struct reftable_iterator *it,
++ struct record rec);
++void reader_close(struct reftable_reader *r);
++const char *reader_name(struct reftable_reader *r);
++void reader_return_block(struct reftable_reader *r, struct reftable_block *p);
++int reader_init_block_reader(struct reftable_reader *r,
++ struct reftable_block_reader *br,
+ uint64_t next_off, byte want_typ);
+
+#endif
@@ -2649,6 +2771,8 @@
+https://developers.google.com/open-source/licenses/bsd
+*/
+
++/* record.c - methods for different types of records. */
++
+#include "record.h"
+
+#include "system.h"
@@ -2656,18 +2780,6 @@
+#include "constants.h"
+#include "reftable.h"
+
-+int is_block_type(byte typ)
-+{
-+ switch (typ) {
-+ case BLOCK_TYPE_REF:
-+ case BLOCK_TYPE_LOG:
-+ case BLOCK_TYPE_OBJ:
-+ case BLOCK_TYPE_INDEX:
-+ return true;
-+ }
-+ return false;
-+}
-+
+int get_var_int(uint64_t *dest, struct slice in)
+{
+ int ptr = 0;
@@ -2716,17 +2828,16 @@
+ }
+}
+
-+int common_prefix_size(struct slice a, struct slice b)
++int is_block_type(byte typ)
+{
-+ int p = 0;
-+ while (p < a.len && p < b.len) {
-+ if (a.buf[p] != b.buf[p]) {
-+ break;
-+ }
-+ p++;
++ switch (typ) {
++ case BLOCK_TYPE_REF:
++ case BLOCK_TYPE_LOG:
++ case BLOCK_TYPE_OBJ:
++ case BLOCK_TYPE_INDEX:
++ return true;
+ }
-+
-+ return p;
++ return false;
+}
+
+static int decode_string(struct slice *dest, struct slice in)
@@ -2737,8 +2848,7 @@
+ if (n <= 0) {
+ return -1;
+ }
-+ in.buf += n;
-+ in.len -= n;
++ slice_consume(&in, n);
+ if (in.len < tsize) {
+ return -1;
+ }
@@ -2746,12 +2856,29 @@
+ slice_resize(dest, tsize + 1);
+ dest->buf[tsize] = 0;
+ memcpy(dest->buf, in.buf, tsize);
-+ in.buf += tsize;
-+ in.len -= tsize;
++ slice_consume(&in, tsize);
+
+ return start_len - in.len;
+}
+
++static int encode_string(char *str, struct slice s)
++{
++ struct slice start = s;
++ int l = strlen(str);
++ int n = put_var_int(s, l);
++ if (n < 0) {
++ return -1;
++ }
++ slice_consume(&s, n);
++ if (s.len < l) {
++ return -1;
++ }
++ memcpy(s.buf, str, l);
++ slice_consume(&s, l);
++
++ return start.len - s.len;
++}
++
+int encode_key(bool *restart, struct slice dest, struct slice prev_key,
+ struct slice key, byte extra)
+{
@@ -2762,8 +2889,7 @@
+ if (n < 0) {
+ return -1;
+ }
-+ dest.buf += n;
-+ dest.len -= n;
++ slice_consume(&dest, n);
+
+ *restart = (prefix_len == 0);
+
@@ -2771,39 +2897,77 @@
+ if (n < 0) {
+ return -1;
+ }
-+ dest.buf += n;
-+ dest.len -= n;
++ slice_consume(&dest, n);
+
+ if (dest.len < suffix_len) {
+ return -1;
+ }
+ memcpy(dest.buf, key.buf + prefix_len, suffix_len);
-+ dest.buf += suffix_len;
-+ dest.len -= suffix_len;
++ slice_consume(&dest, suffix_len);
+
+ return start.len - dest.len;
+}
+
-+static byte ref_record_type(void)
++int decode_key(struct slice *key, byte *extra, struct slice last_key,
++ struct slice in)
++{
++ int start_len = in.len;
++ uint64_t prefix_len = 0;
++ uint64_t suffix_len = 0;
++ int n = get_var_int(&prefix_len, in);
++ if (n < 0) {
++ return -1;
++ }
++ slice_consume(&in, n);
++
++ if (prefix_len > last_key.len) {
++ return -1;
++ }
++
++ n = get_var_int(&suffix_len, in);
++ if (n <= 0) {
++ return -1;
++ }
++ slice_consume(&in, n);
++
++ *extra = (byte)(suffix_len & 0x7);
++ suffix_len >>= 3;
++
++ if (in.len < suffix_len) {
++ return -1;
++ }
++
++ slice_resize(key, suffix_len + prefix_len);
++ memcpy(key->buf, last_key.buf, prefix_len);
++
++ memcpy(key->buf + prefix_len, in.buf, suffix_len);
++ slice_consume(&in, suffix_len);
++
++ return start_len - in.len;
++}
++
++static byte reftable_ref_record_type(void)
+{
+ return BLOCK_TYPE_REF;
+}
+
-+static void ref_record_key(const void *r, struct slice *dest)
++static void reftable_ref_record_key(const void *r, struct slice *dest)
+{
-+ const struct ref_record *rec = (const struct ref_record *)r;
++ const struct reftable_ref_record *rec =
++ (const struct reftable_ref_record *)r;
+ slice_set_string(dest, rec->ref_name);
+}
+
-+static void ref_record_copy_from(void *rec, const void *src_rec, int hash_size)
++static void reftable_ref_record_copy_from(void *rec, const void *src_rec,
++ int hash_size)
+{
-+ struct ref_record *ref = (struct ref_record *)rec;
-+ struct ref_record *src = (struct ref_record *)src_rec;
++ struct reftable_ref_record *ref = (struct reftable_ref_record *)rec;
++ struct reftable_ref_record *src = (struct reftable_ref_record *)src_rec;
+ assert(hash_size > 0);
+
+ /* This is simple and correct, but we could probably reuse the hash
+ fields. */
-+ ref_record_clear(ref);
++ reftable_ref_record_clear(ref);
+ if (src->ref_name != NULL) {
+ ref->ref_name = xstrdup(src->ref_name);
+ }
@@ -2813,12 +2977,12 @@
+ }
+
+ if (src->target_value != NULL) {
-+ ref->target_value = malloc(hash_size);
++ ref->target_value = reftable_malloc(hash_size);
+ memcpy(ref->target_value, src->target_value, hash_size);
+ }
+
+ if (src->value != NULL) {
-+ ref->value = malloc(hash_size);
++ ref->value = reftable_malloc(hash_size);
+ memcpy(ref->value, src->value, hash_size);
+ }
+ ref->update_index = src->update_index;
@@ -2845,7 +3009,7 @@
+ }
+}
+
-+void ref_record_print(struct ref_record *ref, int hash_size)
++void reftable_ref_record_print(struct reftable_ref_record *ref, int hash_size)
+{
+ char hex[SHA256_SIZE + 1] = { 0 };
+
@@ -2864,23 +3028,24 @@
+ printf("}\n");
+}
+
-+static void ref_record_clear_void(void *rec)
++static void reftable_ref_record_clear_void(void *rec)
+{
-+ ref_record_clear((struct ref_record *)rec);
++ reftable_ref_record_clear((struct reftable_ref_record *)rec);
+}
+
-+void ref_record_clear(struct ref_record *ref)
++void reftable_ref_record_clear(struct reftable_ref_record *ref)
+{
-+ free(ref->ref_name);
-+ free(ref->target);
-+ free(ref->target_value);
-+ free(ref->value);
-+ memset(ref, 0, sizeof(struct ref_record));
++ reftable_free(ref->ref_name);
++ reftable_free(ref->target);
++ reftable_free(ref->target_value);
++ reftable_free(ref->value);
++ memset(ref, 0, sizeof(struct reftable_ref_record));
+}
+
-+static byte ref_record_val_type(const void *rec)
++static byte reftable_ref_record_val_type(const void *rec)
+{
-+ const struct ref_record *r = (const struct ref_record *)rec;
++ const struct reftable_ref_record *r =
++ (const struct reftable_ref_record *)rec;
+ if (r->value != NULL) {
+ if (r->target_value != NULL) {
+ return 2;
@@ -2893,45 +3058,25 @@
+ return 0;
+}
+
-+static int encode_string(char *str, struct slice s)
-+{
-+ struct slice start = s;
-+ int l = strlen(str);
-+ int n = put_var_int(s, l);
-+ if (n < 0) {
-+ return -1;
-+ }
-+ s.buf += n;
-+ s.len -= n;
-+ if (s.len < l) {
-+ return -1;
-+ }
-+ memcpy(s.buf, str, l);
-+ s.buf += l;
-+ s.len -= l;
-+
-+ return start.len - s.len;
-+}
-+
-+static int ref_record_encode(const void *rec, struct slice s, int hash_size)
++static int reftable_ref_record_encode(const void *rec, struct slice s,
++ int hash_size)
+{
-+ const struct ref_record *r = (const struct ref_record *)rec;
++ const struct reftable_ref_record *r =
++ (const struct reftable_ref_record *)rec;
+ struct slice start = s;
+ int n = put_var_int(s, r->update_index);
+ assert(hash_size > 0);
+ if (n < 0) {
+ return -1;
+ }
-+ s.buf += n;
-+ s.len -= n;
++ slice_consume(&s, n);
+
+ if (r->value != NULL) {
+ if (s.len < hash_size) {
+ return -1;
+ }
+ memcpy(s.buf, r->value, hash_size);
-+ s.buf += hash_size;
-+ s.len -= hash_size;
++ slice_consume(&s, hash_size);
+ }
+
+ if (r->target_value != NULL) {
@@ -2939,8 +3084,7 @@
+ return -1;
+ }
+ memcpy(s.buf, r->target_value, hash_size);
-+ s.buf += hash_size;
-+ s.len -= hash_size;
++ slice_consume(&s, hash_size);
+ }
+
+ if (r->target != NULL) {
@@ -2948,17 +3092,17 @@
+ if (n < 0) {
+ return -1;
+ }
-+ s.buf += n;
-+ s.len -= n;
++ slice_consume(&s, n);
+ }
+
+ return start.len - s.len;
+}
+
-+static int ref_record_decode(void *rec, struct slice key, byte val_type,
-+ struct slice in, int hash_size)
++static int reftable_ref_record_decode(void *rec, struct slice key,
++ byte val_type, struct slice in,
++ int hash_size)
+{
-+ struct ref_record *r = (struct ref_record *)rec;
++ struct reftable_ref_record *r = (struct reftable_ref_record *)rec;
+ struct slice start = in;
+ bool seen_value = false;
+ bool seen_target_value = false;
@@ -2970,10 +3114,9 @@
+ }
+ assert(hash_size > 0);
+
-+ in.buf += n;
-+ in.len -= n;
++ slice_consume(&in, n);
+
-+ r->ref_name = realloc(r->ref_name, key.len + 1);
++ r->ref_name = reftable_realloc(r->ref_name, key.len + 1);
+ memcpy(r->ref_name, key.buf, key.len);
+ r->ref_name[key.len] = 0;
+
@@ -2985,22 +3128,20 @@
+ }
+
+ if (r->value == NULL) {
-+ r->value = malloc(hash_size);
++ r->value = reftable_malloc(hash_size);
+ }
+ seen_value = true;
+ memcpy(r->value, in.buf, hash_size);
-+ in.buf += hash_size;
-+ in.len -= hash_size;
++ slice_consume(&in, hash_size);
+ if (val_type == 1) {
+ break;
+ }
+ if (r->target_value == NULL) {
-+ r->target_value = malloc(hash_size);
++ r->target_value = reftable_malloc(hash_size);
+ }
+ seen_target_value = true;
+ memcpy(r->target_value, in.buf, hash_size);
-+ in.buf += hash_size;
-+ in.len -= hash_size;
++ slice_consume(&in, hash_size);
+ break;
+ case 3: {
+ struct slice dest = { 0 };
@@ -3008,8 +3149,7 @@
+ if (n < 0) {
+ return -1;
+ }
-+ in.buf += n;
-+ in.len -= n;
++ slice_consume(&in, n);
+ seen_target = true;
+ r->target = (char *)slice_as_string(&dest);
+ } break;
@@ -3034,55 +3174,14 @@
+ return start.len - in.len;
+}
+
-+int decode_key(struct slice *key, byte *extra, struct slice last_key,
-+ struct slice in)
-+{
-+ int start_len = in.len;
-+ uint64_t prefix_len = 0;
-+ uint64_t suffix_len = 0;
-+ int n = get_var_int(&prefix_len, in);
-+ if (n < 0) {
-+ return -1;
-+ }
-+ in.buf += n;
-+ in.len -= n;
-+
-+ if (prefix_len > last_key.len) {
-+ return -1;
-+ }
-+
-+ n = get_var_int(&suffix_len, in);
-+ if (n <= 0) {
-+ return -1;
-+ }
-+ in.buf += n;
-+ in.len -= n;
-+
-+ *extra = (byte)(suffix_len & 0x7);
-+ suffix_len >>= 3;
-+
-+ if (in.len < suffix_len) {
-+ return -1;
-+ }
-+
-+ slice_resize(key, suffix_len + prefix_len);
-+ memcpy(key->buf, last_key.buf, prefix_len);
-+
-+ memcpy(key->buf + prefix_len, in.buf, suffix_len);
-+ in.buf += suffix_len;
-+ in.len -= suffix_len;
-+
-+ return start_len - in.len;
-+}
-+
-+struct record_vtable ref_record_vtable = {
-+ .key = &ref_record_key,
-+ .type = &ref_record_type,
-+ .copy_from = &ref_record_copy_from,
-+ .val_type = &ref_record_val_type,
-+ .encode = &ref_record_encode,
-+ .decode = &ref_record_decode,
-+ .clear = &ref_record_clear_void,
++struct record_vtable reftable_ref_record_vtable = {
++ .key = &reftable_ref_record_key,
++ .type = &reftable_ref_record_type,
++ .copy_from = &reftable_ref_record_copy_from,
++ .val_type = &reftable_ref_record_val_type,
++ .encode = &reftable_ref_record_encode,
++ .decode = &reftable_ref_record_decode,
++ .clear = &reftable_ref_record_clear_void,
+};
+
+static byte obj_record_type(void)
@@ -3103,12 +3202,12 @@
+ const struct obj_record *src = (const struct obj_record *)src_rec;
+
+ *ref = *src;
-+ ref->hash_prefix = malloc(ref->hash_prefix_len);
++ ref->hash_prefix = reftable_malloc(ref->hash_prefix_len);
+ memcpy(ref->hash_prefix, src->hash_prefix, ref->hash_prefix_len);
+
+ {
+ int olen = ref->offset_len * sizeof(uint64_t);
-+ ref->offsets = malloc(olen);
++ ref->offsets = reftable_malloc(olen);
+ memcpy(ref->offsets, src->offsets, olen);
+ }
+}
@@ -3140,8 +3239,7 @@
+ if (n < 0) {
+ return -1;
+ }
-+ s.buf += n;
-+ s.len -= n;
++ slice_consume(&s, n);
+ }
+ if (r->offset_len == 0) {
+ return start.len - s.len;
@@ -3150,8 +3248,7 @@
+ if (n < 0) {
+ return -1;
+ }
-+ s.buf += n;
-+ s.len -= n;
++ slice_consume(&s, n);
+
+ {
+ uint64_t last = r->offsets[0];
@@ -3161,8 +3258,7 @@
+ if (n < 0) {
+ return -1;
+ }
-+ s.buf += n;
-+ s.len -= n;
++ slice_consume(&s, n);
+ last = r->offsets[i];
+ }
+ }
@@ -3176,7 +3272,7 @@
+ struct obj_record *r = (struct obj_record *)rec;
+ uint64_t count = val_type;
+ int n = 0;
-+ r->hash_prefix = malloc(key.len);
++ r->hash_prefix = reftable_malloc(key.len);
+ memcpy(r->hash_prefix, key.buf, key.len);
+ r->hash_prefix_len = key.len;
+
@@ -3186,8 +3282,7 @@
+ return n;
+ }
+
-+ in.buf += n;
-+ in.len -= n;
++ slice_consume(&in, n);
+ }
+
+ r->offsets = NULL;
@@ -3196,16 +3291,14 @@
+ return start.len - in.len;
+ }
+
-+ r->offsets = malloc(count * sizeof(uint64_t));
++ r->offsets = reftable_malloc(count * sizeof(uint64_t));
+ r->offset_len = count;
+
+ n = get_var_int(&r->offsets[0], in);
+ if (n < 0) {
+ return n;
+ }
-+
-+ in.buf += n;
-+ in.len -= n;
++ slice_consume(&in, n);
+
+ {
+ uint64_t last = r->offsets[0];
@@ -3216,9 +3309,7 @@
+ if (n < 0) {
+ return n;
+ }
-+
-+ in.buf += n;
-+ in.len -= n;
++ slice_consume(&in, n);
+
+ last = r->offsets[j] = (delta + last);
+ j++;
@@ -3237,7 +3328,7 @@
+ .clear = &obj_record_clear,
+};
+
-+void log_record_print(struct log_record *log, int hash_size)
++void reftable_log_record_print(struct reftable_log_record *log, int hash_size)
+{
+ char hex[SHA256_SIZE + 1] = { 0 };
+
@@ -3250,14 +3341,15 @@
+ printf("%s\n\n%s\n}\n", hex, log->message);
+}
+
-+static byte log_record_type(void)
++static byte reftable_log_record_type(void)
+{
+ return BLOCK_TYPE_LOG;
+}
+
-+static void log_record_key(const void *r, struct slice *dest)
++static void reftable_log_record_key(const void *r, struct slice *dest)
+{
-+ const struct log_record *rec = (const struct log_record *)r;
++ const struct reftable_log_record *rec =
++ (const struct reftable_log_record *)r;
+ int len = strlen(rec->ref_name);
+ uint64_t ts = 0;
+ slice_resize(dest, len + 9);
@@ -3266,10 +3358,12 @@
+ put_be64(dest->buf + 1 + len, ts);
+}
+
-+static void log_record_copy_from(void *rec, const void *src_rec, int hash_size)
++static void reftable_log_record_copy_from(void *rec, const void *src_rec,
++ int hash_size)
+{
-+ struct log_record *dst = (struct log_record *)rec;
-+ const struct log_record *src = (const struct log_record *)src_rec;
++ struct reftable_log_record *dst = (struct reftable_log_record *)rec;
++ const struct reftable_log_record *src =
++ (const struct reftable_log_record *)src_rec;
+
+ *dst = *src;
+ dst->ref_name = xstrdup(dst->ref_name);
@@ -3277,46 +3371,54 @@
+ dst->name = xstrdup(dst->name);
+ dst->message = xstrdup(dst->message);
+ if (dst->new_hash != NULL) {
-+ dst->new_hash = malloc(hash_size);
++ dst->new_hash = reftable_malloc(hash_size);
+ memcpy(dst->new_hash, src->new_hash, hash_size);
+ }
+ if (dst->old_hash != NULL) {
-+ dst->old_hash = malloc(hash_size);
++ dst->old_hash = reftable_malloc(hash_size);
+ memcpy(dst->old_hash, src->old_hash, hash_size);
+ }
+}
+
-+static void log_record_clear_void(void *rec)
++static void reftable_log_record_clear_void(void *rec)
+{
-+ struct log_record *r = (struct log_record *)rec;
-+ log_record_clear(r);
++ struct reftable_log_record *r = (struct reftable_log_record *)rec;
++ reftable_log_record_clear(r);
+}
+
-+void log_record_clear(struct log_record *r)
++void reftable_log_record_clear(struct reftable_log_record *r)
+{
-+ free(r->ref_name);
-+ free(r->new_hash);
-+ free(r->old_hash);
-+ free(r->name);
-+ free(r->email);
-+ free(r->message);
-+ memset(r, 0, sizeof(struct log_record));
++ reftable_free(r->ref_name);
++ reftable_free(r->new_hash);
++ reftable_free(r->old_hash);
++ reftable_free(r->name);
++ reftable_free(r->email);
++ reftable_free(r->message);
++ memset(r, 0, sizeof(struct reftable_log_record));
+}
+
-+static byte log_record_val_type(const void *rec)
++static byte reftable_log_record_val_type(const void *rec)
+{
-+ return 1;
++ const struct reftable_log_record *log =
++ (const struct reftable_log_record *)rec;
++
++ return reftable_log_record_is_deletion(log) ? 0 : 1;
+}
+
+static byte zero[SHA256_SIZE] = { 0 };
+
-+static int log_record_encode(const void *rec, struct slice s, int hash_size)
++static int reftable_log_record_encode(const void *rec, struct slice s,
++ int hash_size)
+{
-+ struct log_record *r = (struct log_record *)rec;
++ struct reftable_log_record *r = (struct reftable_log_record *)rec;
+ struct slice start = s;
+ int n = 0;
+ byte *oldh = r->old_hash;
+ byte *newh = r->new_hash;
++ if (reftable_log_record_is_deletion(r)) {
++ return 0;
++ }
++
+ if (oldh == NULL) {
+ oldh = zero;
+ }
@@ -3330,53 +3432,48 @@
+
+ memcpy(s.buf, oldh, hash_size);
+ memcpy(s.buf + hash_size, newh, hash_size);
-+ s.buf += 2 * hash_size;
-+ s.len -= 2 * hash_size;
++ slice_consume(&s, 2 * hash_size);
+
+ n = encode_string(r->name ? r->name : "", s);
+ if (n < 0) {
+ return -1;
+ }
-+ s.len -= n;
-+ s.buf += n;
++ slice_consume(&s, n);
+
+ n = encode_string(r->email ? r->email : "", s);
+ if (n < 0) {
+ return -1;
+ }
-+ s.len -= n;
-+ s.buf += n;
++ slice_consume(&s, n);
+
+ n = put_var_int(s, r->time);
+ if (n < 0) {
+ return -1;
+ }
-+ s.buf += n;
-+ s.len -= n;
++ slice_consume(&s, n);
+
+ if (s.len < 2) {
+ return -1;
+ }
+
+ put_be16(s.buf, r->tz_offset);
-+ s.buf += 2;
-+ s.len -= 2;
++ slice_consume(&s, 2);
+
+ n = encode_string(r->message ? r->message : "", s);
+ if (n < 0) {
+ return -1;
+ }
-+ s.len -= n;
-+ s.buf += n;
++ slice_consume(&s, n);
+
+ return start.len - s.len;
+}
+
-+static int log_record_decode(void *rec, struct slice key, byte val_type,
-+ struct slice in, int hash_size)
++static int reftable_log_record_decode(void *rec, struct slice key,
++ byte val_type, struct slice in,
++ int hash_size)
+{
+ struct slice start = in;
-+ struct log_record *r = (struct log_record *)rec;
++ struct reftable_log_record *r = (struct reftable_log_record *)rec;
+ uint64_t max = 0;
+ uint64_t ts = 0;
+ struct slice dest = { 0 };
@@ -3386,33 +3483,35 @@
+ return FORMAT_ERROR;
+ }
+
-+ r->ref_name = realloc(r->ref_name, key.len - 8);
++ r->ref_name = reftable_realloc(r->ref_name, key.len - 8);
+ memcpy(r->ref_name, key.buf, key.len - 8);
+ ts = get_be64(key.buf + key.len - 8);
+
+ r->update_index = (~max) - ts;
+
++ if (val_type == 0) {
++ return 0;
++ }
++
+ if (in.len < 2 * hash_size) {
+ return FORMAT_ERROR;
+ }
+
-+ r->old_hash = realloc(r->old_hash, hash_size);
-+ r->new_hash = realloc(r->new_hash, hash_size);
++ r->old_hash = reftable_realloc(r->old_hash, hash_size);
++ r->new_hash = reftable_realloc(r->new_hash, hash_size);
+
+ memcpy(r->old_hash, in.buf, hash_size);
+ memcpy(r->new_hash, in.buf + hash_size, hash_size);
+
-+ in.buf += 2 * hash_size;
-+ in.len -= 2 * hash_size;
++ slice_consume(&in, 2 * hash_size);
+
+ n = decode_string(&dest, in);
+ if (n < 0) {
+ goto error;
+ }
-+ in.len -= n;
-+ in.buf += n;
++ slice_consume(&in, n);
+
-+ r->name = realloc(r->name, dest.len + 1);
++ r->name = reftable_realloc(r->name, dest.len + 1);
+ memcpy(r->name, dest.buf, dest.len);
+ r->name[dest.len] = 0;
+
@@ -3421,10 +3520,9 @@
+ if (n < 0) {
+ goto error;
+ }
-+ in.len -= n;
-+ in.buf += n;
++ slice_consume(&in, n);
+
-+ r->email = realloc(r->email, dest.len + 1);
++ r->email = reftable_realloc(r->email, dest.len + 1);
+ memcpy(r->email, dest.buf, dest.len);
+ r->email[dest.len] = 0;
+
@@ -3433,33 +3531,30 @@
+ if (n < 0) {
+ goto error;
+ }
-+ in.len -= n;
-+ in.buf += n;
++ slice_consume(&in, n);
+ r->time = ts;
+ if (in.len < 2) {
+ goto error;
+ }
+
+ r->tz_offset = get_be16(in.buf);
-+ in.buf += 2;
-+ in.len -= 2;
++ slice_consume(&in, 2);
+
+ slice_resize(&dest, 0);
+ n = decode_string(&dest, in);
+ if (n < 0) {
+ goto error;
+ }
-+ in.len -= n;
-+ in.buf += n;
++ slice_consume(&in, n);
+
-+ r->message = realloc(r->message, dest.len + 1);
++ r->message = reftable_realloc(r->message, dest.len + 1);
+ memcpy(r->message, dest.buf, dest.len);
+ r->message[dest.len] = 0;
+
+ return start.len - in.len;
+
+error:
-+ free(slice_yield(&dest));
++ reftable_free(slice_yield(&dest));
+ return FORMAT_ERROR;
+}
+
@@ -3486,7 +3581,8 @@
+ return !memcmp(a, b, sz);
+}
+
-+bool log_record_equal(struct log_record *a, struct log_record *b, int hash_size)
++bool reftable_log_record_equal(struct reftable_log_record *a,
++ struct reftable_log_record *b, int hash_size)
+{
+ return null_streq(a->name, b->name) && null_streq(a->email, b->email) &&
+ null_streq(a->message, b->message) &&
@@ -3496,14 +3592,14 @@
+ a->update_index == b->update_index;
+}
+
-+struct record_vtable log_record_vtable = {
-+ .key = &log_record_key,
-+ .type = &log_record_type,
-+ .copy_from = &log_record_copy_from,
-+ .val_type = &log_record_val_type,
-+ .encode = &log_record_encode,
-+ .decode = &log_record_decode,
-+ .clear = &log_record_clear_void,
++struct record_vtable reftable_log_record_vtable = {
++ .key = &reftable_log_record_key,
++ .type = &reftable_log_record_type,
++ .copy_from = &reftable_log_record_copy_from,
++ .val_type = &reftable_log_record_val_type,
++ .encode = &reftable_log_record_encode,
++ .decode = &reftable_log_record_decode,
++ .clear = &reftable_log_record_clear_void,
+};
+
+struct record new_record(byte typ)
@@ -3511,23 +3607,27 @@
+ struct record rec;
+ switch (typ) {
+ case BLOCK_TYPE_REF: {
-+ struct ref_record *r = calloc(1, sizeof(struct ref_record));
++ struct reftable_ref_record *r =
++ reftable_calloc(sizeof(struct reftable_ref_record));
+ record_from_ref(&rec, r);
+ return rec;
+ }
+
+ case BLOCK_TYPE_OBJ: {
-+ struct obj_record *r = calloc(1, sizeof(struct obj_record));
++ struct obj_record *r =
++ reftable_calloc(sizeof(struct obj_record));
+ record_from_obj(&rec, r);
+ return rec;
+ }
+ case BLOCK_TYPE_LOG: {
-+ struct log_record *r = calloc(1, sizeof(struct log_record));
++ struct reftable_log_record *r =
++ reftable_calloc(sizeof(struct reftable_log_record));
+ record_from_log(&rec, r);
+ return rec;
+ }
+ case BLOCK_TYPE_INDEX: {
-+ struct index_record *r = calloc(1, sizeof(struct index_record));
++ struct index_record *r =
++ reftable_calloc(sizeof(struct index_record));
+ record_from_index(&rec, r);
+ return rec;
+ }
@@ -3560,7 +3660,7 @@
+static void index_record_clear(void *rec)
+{
+ struct index_record *idx = (struct index_record *)rec;
-+ free(slice_yield(&idx->last_key));
++ reftable_free(slice_yield(&idx->last_key));
+}
+
+static byte index_record_val_type(const void *rec)
@@ -3578,8 +3678,7 @@
+ return n;
+ }
+
-+ out.buf += n;
-+ out.len -= n;
++ slice_consume(&out, n);
+
+ return start.len - out.len;
+}
@@ -3598,8 +3697,7 @@
+ return n;
+ }
+
-+ in.buf += n;
-+ in.len -= n;
++ slice_consume(&in, n);
+ return start.len - in.len;
+}
+
@@ -3651,10 +3749,10 @@
+ return rec.ops->clear(rec.data);
+}
+
-+void record_from_ref(struct record *rec, struct ref_record *ref_rec)
++void record_from_ref(struct record *rec, struct reftable_ref_record *ref_rec)
+{
+ rec->data = ref_rec;
-+ rec->ops = &ref_record_vtable;
++ rec->ops = &reftable_ref_record_vtable;
+}
+
+void record_from_obj(struct record *rec, struct obj_record *obj_rec)
@@ -3669,10 +3767,10 @@
+ rec->ops = &index_record_vtable;
+}
+
-+void record_from_log(struct record *rec, struct log_record *log_rec)
++void record_from_log(struct record *rec, struct reftable_log_record *log_rec)
+{
+ rec->data = log_rec;
-+ rec->ops = &log_record_vtable;
++ rec->ops = &reftable_log_record_vtable;
+}
+
+void *record_yield(struct record *rec)
@@ -3682,10 +3780,10 @@
+ return p;
+}
+
-+struct ref_record *record_as_ref(struct record rec)
++struct reftable_ref_record *record_as_ref(struct record rec)
+{
+ assert(record_type(rec) == BLOCK_TYPE_REF);
-+ return (struct ref_record *)rec.data;
++ return (struct reftable_ref_record *)rec.data;
+}
+
+static bool hash_equal(byte *a, byte *b, int hash_size)
@@ -3706,7 +3804,8 @@
+ return a == b;
+}
+
-+bool ref_record_equal(struct ref_record *a, struct ref_record *b, int hash_size)
++bool reftable_ref_record_equal(struct reftable_ref_record *a,
++ struct reftable_ref_record *b, int hash_size)
+{
+ assert(hash_size > 0);
+ return 0 == strcmp(a->ref_name, b->ref_name) &&
@@ -3716,22 +3815,22 @@
+ str_equal(a->target, b->target);
+}
+
-+int ref_record_compare_name(const void *a, const void *b)
++int reftable_ref_record_compare_name(const void *a, const void *b)
+{
-+ return strcmp(((struct ref_record *)a)->ref_name,
-+ ((struct ref_record *)b)->ref_name);
++ return strcmp(((struct reftable_ref_record *)a)->ref_name,
++ ((struct reftable_ref_record *)b)->ref_name);
+}
+
-+bool ref_record_is_deletion(const struct ref_record *ref)
++bool reftable_ref_record_is_deletion(const struct reftable_ref_record *ref)
+{
+ return ref->value == NULL && ref->target == NULL &&
+ ref->target_value == NULL;
+}
+
-+int log_record_compare_key(const void *a, const void *b)
++int reftable_log_record_compare_key(const void *a, const void *b)
+{
-+ struct log_record *la = (struct log_record *)a;
-+ struct log_record *lb = (struct log_record *)b;
++ struct reftable_log_record *la = (struct reftable_log_record *)a;
++ struct reftable_log_record *lb = (struct reftable_log_record *)b;
+
+ int cmp = strcmp(la->ref_name, lb->ref_name);
+ if (cmp) {
@@ -3743,10 +3842,12 @@
+ return (la->update_index < lb->update_index) ? 1 : 0;
+}
+
-+bool log_record_is_deletion(const struct log_record *log)
++bool reftable_log_record_is_deletion(const struct reftable_log_record *log)
+{
-+ /* XXX */
-+ return false;
++ return (log->new_hash == NULL && log->old_hash == NULL &&
++ log->name == NULL && log->email == NULL &&
++ log->message == NULL && log->time == 0 && log->tz_offset == 0 &&
++ log->message == NULL);
+}
+
+int hash_size(uint32_t id)
@@ -3780,14 +3881,33 @@
+#include "reftable.h"
+#include "slice.h"
+
++/* utilities for de/encoding varints */
++
++int get_var_int(uint64_t *dest, struct slice in);
++int put_var_int(struct slice dest, uint64_t val);
++
++/* Methods for records. */
+struct record_vtable {
++ /* encode the key of to a byte slice. */
+ void (*key)(const void *rec, struct slice *dest);
++
++ /* The record type of ('r' for ref). */
+ byte (*type)(void);
-+ void (*copy_from)(void *rec, const void *src, int hash_size);
++
++ void (*copy_from)(void *dest, const void *src, int hash_size);
++
++ /* a value of [0..7], indicating record subvariants (eg. ref vs. symref
++ * vs ref deletion) */
+ byte (*val_type)(const void *rec);
++
++ /* encodes rec into dest, returning how much space was used. */
+ int (*encode)(const void *rec, struct slice dest, int hash_size);
++
++ /* decode data from `src` into the record. */
+ int (*decode)(void *rec, struct slice key, byte extra, struct slice src,
+ int hash_size);
++
++ /* deallocate and null the record. */
+ void (*clear)(void *rec);
+};
+
@@ -3797,32 +3917,34 @@
+ struct record_vtable *ops;
+};
+
-+int get_var_int(uint64_t *dest, struct slice in);
-+int put_var_int(struct slice dest, uint64_t val);
-+int common_prefix_size(struct slice a, struct slice b);
-+
+int is_block_type(byte typ);
++
+struct record new_record(byte typ);
+
-+extern struct record_vtable ref_record_vtable;
++extern struct record_vtable reftable_ref_record_vtable;
+
+int encode_key(bool *restart, struct slice dest, struct slice prev_key,
+ struct slice key, byte extra);
+int decode_key(struct slice *key, byte *extra, struct slice last_key,
+ struct slice in);
+
++/* index_record are used internally to speed up lookups. */
+struct index_record {
-+ uint64_t offset;
-+ struct slice last_key;
++ uint64_t offset; /* Offset of block */
++ struct slice last_key; /* Last key of the block. */
+};
+
++/* obj_record stores an object ID => ref mapping. */
+struct obj_record {
-+ byte *hash_prefix;
-+ int hash_prefix_len;
-+ uint64_t *offsets;
++ byte *hash_prefix; /* leading bytes of the object ID */
++ int hash_prefix_len; /* number of leading bytes. Constant
++ * across a single table. */
++ uint64_t *offsets; /* a vector of file offsets. */
+ int offset_len;
+};
+
++/* see struct record_vtable */
++
+void record_key(struct record rec, struct slice *dest);
+byte record_type(struct record rec);
+void record_copy_from(struct record rec, struct record src, int hash_size);
@@ -3831,18 +3953,24 @@
+int record_decode(struct record rec, struct slice key, byte extra,
+ struct slice src, int hash_size);
+void record_clear(struct record rec);
++
++/* clear out the record, yielding the record data that was encapsulated. */
+void *record_yield(struct record *rec);
++
++/* initialize generic records from concrete records. The generic record should
++ * be zeroed out. */
++
+void record_from_obj(struct record *rec, struct obj_record *objrec);
+void record_from_index(struct record *rec, struct index_record *idxrec);
-+void record_from_ref(struct record *rec, struct ref_record *refrec);
-+void record_from_log(struct record *rec, struct log_record *logrec);
-+struct ref_record *record_as_ref(struct record ref);
++void record_from_ref(struct record *rec, struct reftable_ref_record *refrec);
++void record_from_log(struct record *rec, struct reftable_log_record *logrec);
++struct reftable_ref_record *record_as_ref(struct record ref);
+
+/* for qsort. */
-+int ref_record_compare_name(const void *a, const void *b);
++int reftable_ref_record_compare_name(const void *a, const void *b);
+
+/* for qsort. */
-+int log_record_compare_key(const void *a, const void *b);
++int reftable_log_record_compare_key(const void *a, const void *b);
+
+#endif
@@ -3863,65 +3991,20 @@
+#define REFTABLE_H
+
+#include <stdint.h>
++#include <stddef.h>
+
-+/* block_source is a generic wrapper for a seekable readable file.
-+ It is generally passed around by value.
-+ */
-+struct block_source {
-+ struct block_source_vtable *ops;
-+ void *arg;
-+};
++void reftable_set_alloc(void *(*malloc)(size_t),
++ void *(*realloc)(void *, size_t), void (*free)(void *));
+
-+/* a contiguous segment of bytes. It keeps track of its generating block_source
-+ so it can return itself into the pool.
-+*/
-+struct block {
-+ uint8_t *data;
-+ int len;
-+ struct block_source source;
-+};
++/****************************************************************
++ Basic data types
+
-+/* block_source_vtable are the operations that make up block_source */
-+struct block_source_vtable {
-+ /* returns the size of a block source */
-+ uint64_t (*size)(void *source);
++ Reftables store the state of each ref in struct reftable_ref_record, and they
++ store a sequence of reflog updates in struct reftable_log_record.
++ ****************************************************************/
+
-+ /* reads a segment from the block source. It is an error to read
-+ beyond the end of the block */
-+ int (*read_block)(void *source, struct block *dest, uint64_t off,
-+ uint32_t size);
-+ /* mark the block as read; may return the data back to malloc */
-+ void (*return_block)(void *source, struct block *blockp);
-+
-+ /* release all resources associated with the block source */
-+ void (*close)(void *source);
-+};
-+
-+/* opens a file on the file system as a block_source */
-+int block_source_from_file(struct block_source *block_src, const char *name);
-+
-+/* write_options sets options for writing a single reftable. */
-+struct write_options {
-+ /* boolean: do not pad out blocks to block size. */
-+ int unpadded;
-+
-+ /* the blocksize. Should be less than 2^24. */
-+ uint32_t block_size;
-+
-+ /* boolean: do not generate a SHA1 => ref index. */
-+ int skip_index_objects;
-+
-+ /* how often to write complete keys in each block. */
-+ int restart_interval;
-+
-+ /* 4-byte identifier ("sha1", "s256") of the hash.
-+ * Defaults to SHA1 if unset
-+ */
-+ uint32_t hash_id;
-+};
-+
-+/* ref_record holds a ref database entry target_value */
-+struct ref_record {
++/* reftable_ref_record holds a ref database entry target_value */
++struct reftable_ref_record {
+ char *ref_name; /* Name of the ref, malloced. */
+ uint64_t update_index; /* Logical timestamp at which this value is
+ written */
@@ -3931,21 +4014,23 @@
+};
+
+/* returns whether 'ref' represents a deletion */
-+int ref_record_is_deletion(const struct ref_record *ref);
++int reftable_ref_record_is_deletion(const struct reftable_ref_record *ref);
+
-+/* prints a ref_record onto stdout */
-+void ref_record_print(struct ref_record *ref, int hash_size);
++/* prints a reftable_ref_record onto stdout */
++void reftable_ref_record_print(struct reftable_ref_record *ref, int hash_size);
+
+/* frees and nulls all pointer values. */
-+void ref_record_clear(struct ref_record *ref);
++void reftable_ref_record_clear(struct reftable_ref_record *ref);
+
-+/* returns whether two ref_records are the same */
-+int ref_record_equal(struct ref_record *a, struct ref_record *b, int hash_size);
++/* returns whether two reftable_ref_records are the same */
++int reftable_ref_record_equal(struct reftable_ref_record *a,
++ struct reftable_ref_record *b, int hash_size);
+
-+/* log_record holds a reflog entry */
-+struct log_record {
++/* reftable_log_record holds a reflog entry */
++struct reftable_log_record {
+ char *ref_name;
-+ uint64_t update_index;
++ uint64_t update_index; /* logical timestamp of a transactional update.
++ */
+ uint8_t *new_hash;
+ uint8_t *old_hash;
+ char *name;
@@ -3956,39 +4041,88 @@
+};
+
+/* returns whether 'ref' represents the deletion of a log record. */
-+int log_record_is_deletion(const struct log_record *log);
++int reftable_log_record_is_deletion(const struct reftable_log_record *log);
+
+/* frees and nulls all pointer values. */
-+void log_record_clear(struct log_record *log);
++void reftable_log_record_clear(struct reftable_log_record *log);
+
+/* returns whether two records are equal. */
-+int log_record_equal(struct log_record *a, struct log_record *b, int hash_size);
++int reftable_log_record_equal(struct reftable_log_record *a,
++ struct reftable_log_record *b, int hash_size);
+
-+void log_record_print(struct log_record *log, int hash_size);
++/* dumps a reftable_log_record on stdout, for debugging/testing. */
++void reftable_log_record_print(struct reftable_log_record *log, int hash_size);
+
-+/* iterator is the generic interface for walking over data stored in a
-+ reftable. It is generally passed around by value.
-+*/
-+struct iterator {
-+ struct iterator_vtable *ops;
-+ void *iter_arg;
++/****************************************************************
++ Error handling
++
++ Error are signaled with negative integer return values. 0 means success.
++ ****************************************************************/
++
++/* different types of errors */
++enum reftable_error {
++ /* Unexpected file system behavior */
++ IO_ERROR = -2,
++
++ /* Format inconsistency on reading data
++ */
++ FORMAT_ERROR = -3,
++
++ /* File does not exist. Returned from block_source_from_file(), because
++ it needs special handling in stack.
++ */
++ NOT_EXIST_ERROR = -4,
++
++ /* Trying to write out-of-date data. */
++ LOCK_ERROR = -5,
++
++ /* Misuse of the API:
++ - on writing a record with NULL ref_name.
++ - on writing a reftable_ref_record outside the table limits
++ - on writing a ref or log record before the stack's next_update_index
++ - on reading a reftable_ref_record from log iterator, or vice versa.
++ */
++ API_ERROR = -6,
++
++ /* Decompression error */
++ ZLIB_ERROR = -7,
++
++ /* Wrote a table without blocks. */
++ EMPTY_TABLE_ERROR = -8,
+};
+
-+/* reads the next ref_record. Returns < 0 for error, 0 for OK and > 0:
-+ end of iteration.
-+*/
-+int iterator_next_ref(struct iterator it, struct ref_record *ref);
++/* convert the numeric error code to a string. The string should not be
++ * deallocated. */
++const char *reftable_error_str(int err);
+
-+/* reads the next log_record. Returns < 0 for error, 0 for OK and > 0:
-+ end of iteration.
-+*/
-+int iterator_next_log(struct iterator it, struct log_record *log);
++/****************************************************************
++ Writing
+
-+/* releases resources associated with an iterator. */
-+void iterator_destroy(struct iterator *it);
++ Writing single reftables
++ ****************************************************************/
++
++/* reftable_write_options sets options for writing a single reftable. */
++struct reftable_write_options {
++ /* boolean: do not pad out blocks to block size. */
++ int unpadded;
++
++ /* the blocksize. Should be less than 2^24. */
++ uint32_t block_size;
+
-+/* block_stats holds statistics for a single block type */
-+struct block_stats {
++ /* boolean: do not generate a SHA1 => ref index. */
++ int skip_index_objects;
++
++ /* how often to write complete keys in each block. */
++ int restart_interval;
++
++ /* 4-byte identifier ("sha1", "s256") of the hash.
++ * Defaults to SHA1 if unset
++ */
++ uint32_t hash_id;
++};
++
++/* reftable_block_stats holds statistics for a single block type */
++struct reftable_block_stats {
+ /* total number of entries written */
+ int entries;
+ /* total number of key restarts */
@@ -4008,119 +4142,175 @@
+};
+
+/* stats holds overall statistics for a single reftable */
-+struct stats {
++struct reftable_stats {
+ /* total number of blocks written. */
+ int blocks;
+ /* stats for ref data */
-+ struct block_stats ref_stats;
++ struct reftable_block_stats ref_stats;
+ /* stats for the SHA1 to ref map. */
-+ struct block_stats obj_stats;
++ struct reftable_block_stats obj_stats;
+ /* stats for index blocks */
-+ struct block_stats idx_stats;
++ struct reftable_block_stats idx_stats;
+ /* stats for log blocks */
-+ struct block_stats log_stats;
++ struct reftable_block_stats log_stats;
+
+ /* disambiguation length of shortened object IDs. */
+ int object_id_len;
+};
+
-+/* different types of errors */
-+
-+/* Unexpected file system behavior */
-+#define IO_ERROR -2
-+
-+/* Format inconsistency on reading data
-+ */
-+#define FORMAT_ERROR -3
-+
-+/* File does not exist. Returned from block_source_from_file(), because it
-+ needs special handling in stack.
-+*/
-+#define NOT_EXIST_ERROR -4
-+
-+/* Trying to write out-of-date data. */
-+#define LOCK_ERROR -5
-+
-+/* Misuse of the API:
-+ - on writing a record with NULL ref_name.
-+ - on writing a ref_record outside the table limits
-+ - on writing a ref or log record before the stack's next_update_index
-+ - on reading a ref_record from log iterator, or vice versa.
-+ */
-+#define API_ERROR -6
-+
-+/* Decompression error */
-+#define ZLIB_ERROR -7
-+
-+/* Wrote a table without blocks. */
-+#define EMPTY_TABLE_ERROR -8
-+
-+const char *error_str(int err);
-+
-+/* new_writer creates a new writer */
-+struct writer *new_writer(int (*writer_func)(void *, uint8_t *, int),
-+ void *writer_arg, struct write_options *opts);
++/* reftable_new_writer creates a new writer */
++struct reftable_writer *
++reftable_new_writer(int (*writer_func)(void *, uint8_t *, int),
++ void *writer_arg, struct reftable_write_options *opts);
+
+/* write to a file descriptor. fdp should be an int* pointing to the fd. */
-+int fd_writer(void *fdp, uint8_t *data, int size);
++int reftable_fd_write(void *fdp, uint8_t *data, int size);
+
+/* Set the range of update indices for the records we will add. When
+ writing a table into a stack, the min should be at least
-+ stack_next_update_index(), or API_ERROR is returned.
++ reftable_stack_next_update_index(), or API_ERROR is returned.
++
++ For transactional updates, typically min==max. When converting an existing
++ ref database into a single reftable, this would be a range of update-index
++ timestamps.
+ */
-+void writer_set_limits(struct writer *w, uint64_t min, uint64_t max);
++void reftable_writer_set_limits(struct reftable_writer *w, uint64_t min,
++ uint64_t max);
+
-+/* adds a ref_record. Must be called in ascending
++/* adds a reftable_ref_record. Must be called in ascending
+ order. The update_index must be within the limits set by
-+ writer_set_limits(), or API_ERROR is returned.
++ reftable_writer_set_limits(), or API_ERROR is returned.
++
++ It is an error to write a ref record after a log record.
+ */
-+int writer_add_ref(struct writer *w, struct ref_record *ref);
++int reftable_writer_add_ref(struct reftable_writer *w,
++ struct reftable_ref_record *ref);
+
+/* Convenience function to add multiple refs. Will sort the refs by
+ name before adding. */
-+int writer_add_refs(struct writer *w, struct ref_record *refs, int n);
++int reftable_writer_add_refs(struct reftable_writer *w,
++ struct reftable_ref_record *refs, int n);
+
-+/* adds a log_record. Must be called in ascending order (with more
++/* adds a reftable_log_record. Must be called in ascending order (with more
+ recent log entries first.)
+ */
-+int writer_add_log(struct writer *w, struct log_record *log);
++int reftable_writer_add_log(struct reftable_writer *w,
++ struct reftable_log_record *log);
+
+/* Convenience function to add multiple logs. Will sort the records by
+ key before adding. */
-+int writer_add_logs(struct writer *w, struct log_record *logs, int n);
++int reftable_writer_add_logs(struct reftable_writer *w,
++ struct reftable_log_record *logs, int n);
++
++/* reftable_writer_close finalizes the reftable. The writer is retained so
++ * statistics can be inspected. */
++int reftable_writer_close(struct reftable_writer *w);
++
++/* writer_stats returns the statistics on the reftable being written.
++
++ This struct becomes invalid when the writer is freed.
++ */
++const struct reftable_stats *writer_stats(struct reftable_writer *w);
++
++/* reftable_writer_free deallocates memory for the writer */
++void reftable_writer_free(struct reftable_writer *w);
++
++/****************************************************************
++ * ITERATING
++ ****************************************************************/
+
-+/* writer_close finalizes the reftable. The writer is retained so statistics can
-+ * be inspected. */
-+int writer_close(struct writer *w);
++/* iterator is the generic interface for walking over data stored in a
++ reftable. It is generally passed around by value.
++*/
++struct reftable_iterator {
++ struct reftable_iterator_vtable *ops;
++ void *iter_arg;
++};
+
-+/* writer_stats returns the statistics on the reftable being written. */
-+struct stats *writer_stats(struct writer *w);
++/* reads the next reftable_ref_record. Returns < 0 for error, 0 for OK and > 0:
++ end of iteration.
++*/
++int reftable_iterator_next_ref(struct reftable_iterator it,
++ struct reftable_ref_record *ref);
+
-+/* writer_free deallocates memory for the writer */
-+void writer_free(struct writer *w);
++/* reads the next reftable_log_record. Returns < 0 for error, 0 for OK and > 0:
++ end of iteration.
++*/
++int reftable_iterator_next_log(struct reftable_iterator it,
++ struct reftable_log_record *log);
+
-+struct reader;
++/* releases resources associated with an iterator. */
++void reftable_iterator_destroy(struct reftable_iterator *it);
++
++/****************************************************************
++ Reading single tables
+
-+/* new_reader opens a reftable for reading. If successful, returns 0
++ The follow routines are for reading single files. For an application-level
++ interface, skip ahead to struct reftable_merged_table and struct
++ reftable_stack.
++ ****************************************************************/
++
++/* block_source is a generic wrapper for a seekable readable file.
++ It is generally passed around by value.
++ */
++struct reftable_block_source {
++ struct reftable_block_source_vtable *ops;
++ void *arg;
++};
++
++/* a contiguous segment of bytes. It keeps track of its generating block_source
++ so it can return itself into the pool.
++*/
++struct reftable_block {
++ uint8_t *data;
++ int len;
++ struct reftable_block_source source;
++};
++
++/* block_source_vtable are the operations that make up block_source */
++struct reftable_block_source_vtable {
++ /* returns the size of a block source */
++ uint64_t (*size)(void *source);
++
++ /* reads a segment from the block source. It is an error to read
++ beyond the end of the block */
++ int (*read_block)(void *source, struct reftable_block *dest,
++ uint64_t off, uint32_t size);
++ /* mark the block as read; may return the data back to malloc */
++ void (*return_block)(void *source, struct reftable_block *blockp);
++
++ /* release all resources associated with the block source */
++ void (*close)(void *source);
++};
++
++/* opens a file on the file system as a block_source */
++int reftable_block_source_from_file(struct reftable_block_source *block_src,
++ const char *name);
++
++/* The reader struct is a handle to an open reftable file. */
++struct reftable_reader;
++
++/* reftable_new_reader opens a reftable for reading. If successful, returns 0
+ * code and sets pp. The name is used for creating a
+ * stack. Typically, it is the basename of the file.
+ */
-+int new_reader(struct reader **pp, struct block_source, const char *name);
++int reftable_new_reader(struct reftable_reader **pp,
++ struct reftable_block_source, const char *name);
+
-+/* reader_seek_ref returns an iterator where 'name' would be inserted in the
-+ table.
++/* reftable_reader_seek_ref returns an iterator where 'name' would be inserted
++ in the table. To seek to the start of the table, use name = "".
+
+ example:
+
-+ struct reader *r = NULL;
-+ int err = new_reader(&r, src, "filename");
++ struct reftable_reader *r = NULL;
++ int err = reftable_new_reader(&r, src, "filename");
+ if (err < 0) { ... }
-+ struct iterator it = {0};
-+ err = reader_seek_ref(r, &it, "refs/heads/master");
++ struct reftable_iterator it = {0};
++ err = reftable_reader_seek_ref(r, &it, "refs/heads/master");
+ if (err < 0) { ... }
-+ struct ref_record ref = {0};
++ struct reftable_ref_record ref = {0};
+ while (1) {
-+ err = iterator_next_ref(it, &ref);
++ err = reftable_iterator_next_ref(it, &ref);
+ if (err > 0) {
+ break;
+ }
@@ -4129,103 +4319,149 @@
+ }
+ ..found..
+ }
-+ iterator_destroy(&it);
-+ ref_record_clear(&ref);
++ reftable_iterator_destroy(&it);
++ reftable_ref_record_clear(&ref);
+ */
-+int reader_seek_ref(struct reader *r, struct iterator *it, const char *name);
++int reftable_reader_seek_ref(struct reftable_reader *r,
++ struct reftable_iterator *it, const char *name);
+
+/* returns the hash ID used in this table. */
-+uint32_t reader_hash_id(struct reader *r);
++uint32_t reftable_reader_hash_id(struct reftable_reader *r);
+
-+/* seek to logs for the given name, older than update_index. */
-+int reader_seek_log_at(struct reader *r, struct iterator *it, const char *name,
-+ uint64_t update_index);
++/* seek to logs for the given name, older than update_index. To seek to the
++ start of the table, use name = "".
++ */
++int reftable_reader_seek_log_at(struct reftable_reader *r,
++ struct reftable_iterator *it, const char *name,
++ uint64_t update_index);
+
+/* seek to newest log entry for given name. */
-+int reader_seek_log(struct reader *r, struct iterator *it, const char *name);
++int reftable_reader_seek_log(struct reftable_reader *r,
++ struct reftable_iterator *it, const char *name);
+
+/* closes and deallocates a reader. */
-+void reader_free(struct reader *);
++void reftable_reader_free(struct reftable_reader *);
+
+/* return an iterator for the refs pointing to oid */
-+int reader_refs_for(struct reader *r, struct iterator *it, uint8_t *oid,
-+ int oid_len);
++int reftable_reader_refs_for(struct reftable_reader *r,
++ struct reftable_iterator *it, uint8_t *oid,
++ int oid_len);
+
+/* return the max_update_index for a table */
-+uint64_t reader_max_update_index(struct reader *r);
++uint64_t reftable_reader_max_update_index(struct reftable_reader *r);
+
+/* return the min_update_index for a table */
-+uint64_t reader_min_update_index(struct reader *r);
++uint64_t reftable_reader_min_update_index(struct reftable_reader *r);
++
++/****************************************************************
++ Merged tables
+
-+/* a merged table is implements seeking/iterating over a stack of tables. */
-+struct merged_table;
++ A ref database kept in a sequence of table files. The merged_table presents a
++ unified view to reading (seeking, iterating) a sequence of immutable tables.
++ ****************************************************************/
+
-+/* new_merged_table creates a new merged table. It takes ownership of the stack
-+ array.
++/* A merged table is implements seeking/iterating over a stack of tables. */
++struct reftable_merged_table;
++
++/* reftable_new_merged_table creates a new merged table. It takes ownership of
++ the stack array.
+*/
-+int new_merged_table(struct merged_table **dest, struct reader **stack, int n,
-+ uint32_t hash_id);
++int reftable_new_merged_table(struct reftable_merged_table **dest,
++ struct reftable_reader **stack, int n,
++ uint32_t hash_id);
+
+/* returns the hash id used in this merged table. */
-+uint32_t merged_hash_id(struct merged_table *mt);
++uint32_t reftable_merged_table_hash_id(struct reftable_merged_table *mt);
+
+/* returns an iterator positioned just before 'name' */
-+int merged_table_seek_ref(struct merged_table *mt, struct iterator *it,
-+ const char *name);
++int reftable_merged_table_seek_ref(struct reftable_merged_table *mt,
++ struct reftable_iterator *it,
++ const char *name);
+
+/* returns an iterator for log entry, at given update_index */
-+int merged_table_seek_log_at(struct merged_table *mt, struct iterator *it,
-+ const char *name, uint64_t update_index);
++int reftable_merged_table_seek_log_at(struct reftable_merged_table *mt,
++ struct reftable_iterator *it,
++ const char *name, uint64_t update_index);
+
-+/* like merged_table_seek_log_at but look for the newest entry. */
-+int merged_table_seek_log(struct merged_table *mt, struct iterator *it,
-+ const char *name);
++/* like reftable_merged_table_seek_log_at but look for the newest entry. */
++int reftable_merged_table_seek_log(struct reftable_merged_table *mt,
++ struct reftable_iterator *it,
++ const char *name);
+
+/* returns the max update_index covered by this merged table. */
-+uint64_t merged_max_update_index(struct merged_table *mt);
++uint64_t
++reftable_merged_table_max_update_index(struct reftable_merged_table *mt);
+
+/* returns the min update_index covered by this merged table. */
-+uint64_t merged_min_update_index(struct merged_table *mt);
++uint64_t
++reftable_merged_table_min_update_index(struct reftable_merged_table *mt);
+
+/* closes readers for the merged tables */
-+void merged_table_close(struct merged_table *mt);
++void reftable_merged_table_close(struct reftable_merged_table *mt);
+
+/* releases memory for the merged_table */
-+void merged_table_free(struct merged_table *m);
++void reftable_merged_table_free(struct reftable_merged_table *m);
++
++/****************************************************************
++ Mutable ref database
++
++ The stack presents an interface to a mutable sequence of reftables.
++ ****************************************************************/
+
+/* a stack is a stack of reftables, which can be mutated by pushing a table to
+ * the top of the stack */
-+struct stack;
++struct reftable_stack;
+
-+/* open a new reftable stack. The tables will be stored in 'dir', while the list
-+ of tables is in 'list_file'. Typically, this should be .git/reftables and
-+ .git/refs respectively.
++/* open a new reftable stack. The tables along with the table list will be
++ stored in 'dir'. Typically, this should be .git/reftables.
+*/
-+int new_stack(struct stack **dest, const char *dir, const char *list_file,
-+ struct write_options config);
++int reftable_new_stack(struct reftable_stack **dest, const char *dir,
++ struct reftable_write_options config);
+
+/* returns the update_index at which a next table should be written. */
-+uint64_t stack_next_update_index(struct stack *st);
++uint64_t reftable_stack_next_update_index(struct reftable_stack *st);
++
++/* holds a transaction to add tables at the top of a stack. */
++struct reftable_addition;
++
++/*
++ returns a new transaction to add reftables to the given stack. As a side
++ effect, the ref database is locked.
++*/
++int reftable_stack_new_addition(struct reftable_addition **dest, struct reftable_stack *st);
++
++/* Adds a reftable to transaction. */
++int reftable_addition_add(struct reftable_addition *add,
++ int (*write_table)(struct reftable_writer *wr, void *arg),
++ void *arg);
++
++/* Commits the transaction, releasing the lock. */
++int reftable_addition_commit(struct reftable_addition *add);
++
++/* Release all non-committed data from the transaction; releases the lock if held. */
++void reftable_addition_close(struct reftable_addition *add);
+
+/* add a new table to the stack. The write_table function must call
-+ writer_set_limits, add refs and return an error value. */
-+int stack_add(struct stack *st,
-+ int (*write_table)(struct writer *wr, void *write_arg),
-+ void *write_arg);
++ reftable_writer_set_limits, add refs and return an error value. */
++int reftable_stack_add(struct reftable_stack *st,
++ int (*write_table)(struct reftable_writer *wr,
++ void *write_arg),
++ void *write_arg);
+
+/* returns the merged_table for seeking. This table is valid until the
+ next write or reload, and should not be closed or deleted.
+*/
-+struct merged_table *stack_merged_table(struct stack *st);
++struct reftable_merged_table *
++reftable_stack_merged_table(struct reftable_stack *st);
+
+/* frees all resources associated with the stack. */
-+void stack_destroy(struct stack *st);
++void reftable_stack_destroy(struct reftable_stack *st);
+
+/* reloads the stack if necessary. */
-+int stack_reload(struct stack *st);
++int reftable_stack_reload(struct reftable_stack *st);
+
+/* Policy for expiring reflog entries. */
-+struct log_expiry_config {
++struct reftable_log_expiry_config {
+ /* Drop entries older than this timestamp */
+ uint64_t time;
+
@@ -4235,29 +4471,32 @@
+
+/* compacts all reftables into a giant table. Expire reflog entries if config is
+ * non-NULL */
-+int stack_compact_all(struct stack *st, struct log_expiry_config *config);
++int reftable_stack_compact_all(struct reftable_stack *st,
++ struct reftable_log_expiry_config *config);
+
+/* heuristically compact unbalanced table stack. */
-+int stack_auto_compact(struct stack *st);
++int reftable_stack_auto_compact(struct reftable_stack *st);
+
+/* convenience function to read a single ref. Returns < 0 for error, 0
+ for success, and 1 if ref not found. */
-+int stack_read_ref(struct stack *st, const char *refname,
-+ struct ref_record *ref);
++int reftable_stack_read_ref(struct reftable_stack *st, const char *refname,
++ struct reftable_ref_record *ref);
+
+/* convenience function to read a single log. Returns < 0 for error, 0
+ for success, and 1 if ref not found. */
-+int stack_read_log(struct stack *st, const char *refname,
-+ struct log_record *log);
++int reftable_stack_read_log(struct reftable_stack *st, const char *refname,
++ struct reftable_log_record *log);
+
+/* statistics on past compactions. */
-+struct compaction_stats {
-+ uint64_t bytes;
-+ int attempts;
-+ int failures;
++struct reftable_compaction_stats {
++ uint64_t bytes; /* total number of bytes written */
++ int attempts; /* how often we tried to compact */
++ int failures; /* failures happen on concurrent updates */
+};
+
-+struct compaction_stats *stack_compaction_stats(struct stack *st);
++/* return statistics for compaction up till now. */
++struct reftable_compaction_stats *
++reftable_stack_compaction_stats(struct reftable_stack *st);
+
+#endif
@@ -4304,7 +4543,7 @@
+ c = l;
+ }
+ s->cap = c;
-+ s->buf = realloc(s->buf, s->cap);
++ s->buf = reftable_realloc(s->buf, s->cap);
+ }
+ s->len = l;
+}
@@ -4325,6 +4564,12 @@
+ memcpy(s->buf + end, a.buf, a.len);
+}
+
++void slice_consume(struct slice *s, int n)
++{
++ s->buf += n;
++ s->len -= n;
++}
++
+byte *slice_yield(struct slice *s)
+{
+ byte *p = s->buf;
@@ -4394,7 +4639,7 @@
+ if (newcap < b->len + sz) {
+ newcap = (b->len + sz);
+ }
-+ b->buf = realloc(b->buf, newcap);
++ b->buf = reftable_realloc(b->buf, newcap);
+ b->cap = newcap;
+ }
+
@@ -4413,57 +4658,71 @@
+ return ((struct slice *)b)->len;
+}
+
-+static void slice_return_block(void *b, struct block *dest)
++static void slice_return_block(void *b, struct reftable_block *dest)
+{
+ memset(dest->data, 0xff, dest->len);
-+ free(dest->data);
++ reftable_free(dest->data);
+}
+
+static void slice_close(void *b)
+{
+}
+
-+static int slice_read_block(void *v, struct block *dest, uint64_t off,
++static int slice_read_block(void *v, struct reftable_block *dest, uint64_t off,
+ uint32_t size)
+{
+ struct slice *b = (struct slice *)v;
+ assert(off + size <= b->len);
-+ dest->data = calloc(size, 1);
++ dest->data = reftable_calloc(size);
+ memcpy(dest->data, b->buf + off, size);
+ dest->len = size;
+ return size;
+}
+
-+struct block_source_vtable slice_vtable = {
++struct reftable_block_source_vtable slice_vtable = {
+ .size = &slice_size,
+ .read_block = &slice_read_block,
+ .return_block = &slice_return_block,
+ .close = &slice_close,
+};
+
-+void block_source_from_slice(struct block_source *bs, struct slice *buf)
++void block_source_from_slice(struct reftable_block_source *bs,
++ struct slice *buf)
+{
+ bs->ops = &slice_vtable;
+ bs->arg = buf;
+}
+
-+static void malloc_return_block(void *b, struct block *dest)
++static void malloc_return_block(void *b, struct reftable_block *dest)
+{
+ memset(dest->data, 0xff, dest->len);
-+ free(dest->data);
++ reftable_free(dest->data);
+}
+
-+struct block_source_vtable malloc_vtable = {
++struct reftable_block_source_vtable malloc_vtable = {
+ .return_block = &malloc_return_block,
+};
+
-+struct block_source malloc_block_source_instance = {
++struct reftable_block_source malloc_block_source_instance = {
+ .ops = &malloc_vtable,
+};
+
-+struct block_source malloc_block_source(void)
++struct reftable_block_source malloc_block_source(void)
+{
+ return malloc_block_source_instance;
++}
++
++int common_prefix_size(struct slice a, struct slice b)
++{
++ int p = 0;
++ while (p < a.len && p < b.len) {
++ if (a.buf[p] != b.buf[p]) {
++ break;
++ }
++ p++;
++ }
++
++ return p;
+}
diff --git a/reftable/slice.h b/reftable/slice.h
@@ -4498,16 +4757,19 @@
+bool slice_equal(struct slice a, struct slice b);
+byte *slice_yield(struct slice *s);
+void slice_copy(struct slice *dest, struct slice src);
++void slice_consume(struct slice *s, int n);
+void slice_resize(struct slice *s, int l);
+int slice_compare(struct slice a, struct slice b);
+int slice_write(struct slice *b, byte *data, int sz);
+int slice_write_void(void *b, byte *data, int sz);
+void slice_append(struct slice *dest, struct slice add);
++int common_prefix_size(struct slice a, struct slice b);
+
-+struct block_source;
-+void block_source_from_slice(struct block_source *bs, struct slice *buf);
++struct reftable_block_source;
++void block_source_from_slice(struct reftable_block_source *bs,
++ struct slice *buf);
+
-+struct block_source malloc_block_source(void);
++struct reftable_block_source malloc_block_source(void);
+
+#endif
@@ -4532,19 +4794,26 @@
+#include "reftable.h"
+#include "writer.h"
+
-+int new_stack(struct stack **dest, const char *dir, const char *list_file,
-+ struct write_options config)
++int reftable_new_stack(struct reftable_stack **dest, const char *dir,
++ struct reftable_write_options config)
+{
-+ struct stack *p = calloc(sizeof(struct stack), 1);
++ struct reftable_stack *p =
++ reftable_calloc(sizeof(struct reftable_stack));
++ struct slice list_file_name = {};
+ int err = 0;
+ *dest = NULL;
-+ p->list_file = xstrdup(list_file);
++
++ slice_set_string(&list_file_name, dir);
++ slice_append_string(&list_file_name, "/reftables.list");
++
++ p->list_file = slice_to_string(list_file_name);
++ reftable_free(slice_yield(&list_file_name));
+ p->reftable_dir = xstrdup(dir);
+ p->config = config;
+
-+ err = stack_reload(p);
++ err = reftable_stack_reload(p);
+ if (err < 0) {
-+ stack_destroy(p);
++ reftable_stack_destroy(p);
+ } else {
+ *dest = p;
+ }
@@ -4571,7 +4840,7 @@
+ goto exit;
+ }
+
-+ buf = malloc(size + 1);
++ buf = reftable_malloc(size + 1);
+ if (fread(buf, 1, size, f) != size) {
+ err = IO_ERROR;
+ goto exit;
@@ -4580,7 +4849,7 @@
+
+ parse_names(buf, size, namesp);
+exit:
-+ free(buf);
++ reftable_free(buf);
+ return err;
+}
+
@@ -4590,7 +4859,7 @@
+ int err = 0;
+ if (f == NULL) {
+ if (errno == ENOENT) {
-+ *namesp = calloc(sizeof(char *), 1);
++ *namesp = reftable_calloc(sizeof(char *));
+ return 0;
+ }
+
@@ -4601,30 +4870,33 @@
+ return err;
+}
+
-+struct merged_table *stack_merged_table(struct stack *st)
++struct reftable_merged_table *
++reftable_stack_merged_table(struct reftable_stack *st)
+{
+ return st->merged;
+}
+
+/* Close and free the stack */
-+void stack_destroy(struct stack *st)
++void reftable_stack_destroy(struct reftable_stack *st)
+{
+ if (st->merged == NULL) {
+ return;
+ }
+
-+ merged_table_close(st->merged);
-+ merged_table_free(st->merged);
++ reftable_merged_table_close(st->merged);
++ reftable_merged_table_free(st->merged);
+ st->merged = NULL;
+
+ FREE_AND_NULL(st->list_file);
+ FREE_AND_NULL(st->reftable_dir);
-+ free(st);
++ reftable_free(st);
+}
+
-+static struct reader **stack_copy_readers(struct stack *st, int cur_len)
++static struct reftable_reader **stack_copy_readers(struct reftable_stack *st,
++ int cur_len)
+{
-+ struct reader **cur = calloc(sizeof(struct reader *), cur_len);
++ struct reftable_reader **cur =
++ reftable_calloc(sizeof(struct reftable_reader *) * cur_len);
+ int i = 0;
+ for (i = 0; i < cur_len; i++) {
+ cur[i] = st->merged->stack[i];
@@ -4632,21 +4904,22 @@
+ return cur;
+}
+
-+static int stack_reload_once(struct stack *st, char **names, bool reuse_open)
++static int reftable_stack_reload_once(struct reftable_stack *st, char **names,
++ bool reuse_open)
+{
+ int cur_len = st->merged == NULL ? 0 : st->merged->stack_len;
-+ struct reader **cur = stack_copy_readers(st, cur_len);
++ struct reftable_reader **cur = stack_copy_readers(st, cur_len);
+ int err = 0;
+ int names_len = names_length(names);
-+ struct reader **new_tables =
-+ malloc(sizeof(struct reader *) * names_len);
++ struct reftable_reader **new_tables =
++ reftable_malloc(sizeof(struct reftable_reader *) * names_len);
+ int new_tables_len = 0;
-+ struct merged_table *new_merged = NULL;
++ struct reftable_merged_table *new_merged = NULL;
+
+ struct slice table_path = { 0 };
+
+ while (*names) {
-+ struct reader *rd = NULL;
++ struct reftable_reader *rd = NULL;
+ char *name = *names++;
+
+ /* this is linear; we assume compaction keeps the number of
@@ -4661,18 +4934,18 @@
+ }
+
+ if (rd == NULL) {
-+ struct block_source src = { 0 };
++ struct reftable_block_source src = { 0 };
+ slice_set_string(&table_path, st->reftable_dir);
+ slice_append_string(&table_path, "/");
+ slice_append_string(&table_path, name);
+
-+ err = block_source_from_file(
++ err = reftable_block_source_from_file(
+ &src, slice_as_string(&table_path));
+ if (err < 0) {
+ goto exit;
+ }
+
-+ err = new_reader(&rd, src, name);
++ err = reftable_new_reader(&rd, src, name);
+ if (err < 0) {
+ goto exit;
+ }
@@ -4682,8 +4955,8 @@
+ }
+
+ /* success! */
-+ err = new_merged_table(&new_merged, new_tables, new_tables_len,
-+ st->config.hash_id);
++ err = reftable_new_merged_table(&new_merged, new_tables, new_tables_len,
++ st->config.hash_id);
+ if (err < 0) {
+ goto exit;
+ }
@@ -4692,7 +4965,7 @@
+ new_tables_len = 0;
+ if (st->merged != NULL) {
+ merged_table_clear(st->merged);
-+ merged_table_free(st->merged);
++ reftable_merged_table_free(st->merged);
+ }
+ st->merged = new_merged;
+
@@ -4701,20 +4974,20 @@
+ for (i = 0; i < cur_len; i++) {
+ if (cur[i] != NULL) {
+ reader_close(cur[i]);
-+ reader_free(cur[i]);
++ reftable_reader_free(cur[i]);
+ }
+ }
+ }
+exit:
-+ free(slice_yield(&table_path));
++ reftable_free(slice_yield(&table_path));
+ {
+ int i = 0;
+ for (i = 0; i < new_tables_len; i++) {
+ reader_close(new_tables[i]);
+ }
+ }
-+ free(new_tables);
-+ free(cur);
++ reftable_free(new_tables);
++ reftable_free(cur);
+ return err;
+}
+
@@ -4731,7 +5004,8 @@
+ return udiff;
+}
+
-+static int stack_reload_maybe_reuse(struct stack *st, bool reuse_open)
++static int reftable_stack_reload_maybe_reuse(struct reftable_stack *st,
++ bool reuse_open)
+{
+ struct timeval deadline = { 0 };
+ int err = gettimeofday(&deadline, NULL);
@@ -4763,7 +5037,7 @@
+ free_names(names);
+ return err;
+ }
-+ err = stack_reload_once(st, names, reuse_open);
++ err = reftable_stack_reload_once(st, names, reuse_open);
+ if (err == 0) {
+ free_names(names);
+ break;
@@ -4794,15 +5068,15 @@
+ return 0;
+}
+
-+int stack_reload(struct stack *st)
++int reftable_stack_reload(struct reftable_stack *st)
+{
-+ return stack_reload_maybe_reuse(st, true);
++ return reftable_stack_reload_maybe_reuse(st, true);
+}
+
+/* -1 = error
+ 0 = up to date
+ 1 = changed. */
-+static int stack_uptodate(struct stack *st)
++static int stack_uptodate(struct reftable_stack *st)
+{
+ char **names = NULL;
+ int err = read_lines(st->list_file, &names);
@@ -4833,18 +5107,19 @@
+ return err;
+}
+
-+int stack_add(struct stack *st, int (*write)(struct writer *wr, void *arg),
-+ void *arg)
++int reftable_stack_add(struct reftable_stack *st,
++ int (*write)(struct reftable_writer *wr, void *arg),
++ void *arg)
+{
+ int err = stack_try_add(st, write, arg);
+ if (err < 0) {
+ if (err == LOCK_ERROR) {
-+ err = stack_reload(st);
++ err = reftable_stack_reload(st);
+ }
+ return err;
+ }
+
-+ return stack_auto_compact(st);
++ return reftable_stack_auto_compact(st);
+}
+
+static void format_name(struct slice *dest, uint64_t min, uint64_t max)
@@ -4854,34 +5129,35 @@
+ slice_set_string(dest, buf);
+}
+
-+int stack_try_add(struct stack *st,
-+ int (*write_table)(struct writer *wr, void *arg), void *arg)
++struct reftable_addition {
++ int lock_file_fd;
++ struct slice lock_file_name;
++ struct reftable_stack *stack;
++ char **names;
++ char **new_tables;
++ int new_tables_len;
++ uint64_t next_update_index;
++};
++
++static int reftable_stack_init_addition(struct reftable_addition *add,
++ struct reftable_stack *st)
+{
-+ struct slice lock_name = { 0 };
-+ struct slice temp_tab_name = { 0 };
-+ struct slice tab_name = { 0 };
-+ struct slice next_name = { 0 };
-+ struct slice table_list = { 0 };
-+ struct writer *wr = NULL;
+ int err = 0;
-+ int tab_fd = 0;
-+ int lock_fd = 0;
-+ uint64_t next_update_index = 0;
++ add->stack = st;
+
-+ slice_set_string(&lock_name, st->list_file);
-+ slice_append_string(&lock_name, ".lock");
++ slice_set_string(&add->lock_file_name, st->list_file);
++ slice_append_string(&add->lock_file_name, ".lock");
+
-+ lock_fd = open(slice_as_string(&lock_name), O_EXCL | O_CREAT | O_WRONLY,
-+ 0644);
-+ if (lock_fd < 0) {
++ add->lock_file_fd = open(slice_as_string(&add->lock_file_name),
++ O_EXCL | O_CREAT | O_WRONLY, 0644);
++ if (add->lock_file_fd < 0) {
+ if (errno == EEXIST) {
+ err = LOCK_ERROR;
-+ goto exit;
++ } else {
++ err = IO_ERROR;
+ }
-+ err = IO_ERROR;
+ goto exit;
+ }
-+
+ err = stack_uptodate(st);
+ if (err < 0) {
+ goto exit;
@@ -4892,29 +5168,160 @@
+ goto exit;
+ }
+
-+ next_update_index = stack_next_update_index(st);
++ add->next_update_index = reftable_stack_next_update_index(st);
++exit:
++ if (err) {
++ reftable_addition_close(add);
++ }
++ return err;
++}
++
++void reftable_addition_close(struct reftable_addition *add)
++{
++ int i = 0;
++ struct slice nm = {};
++ for (i = 0; i < add->new_tables_len; i++) {
++ slice_set_string(&nm, add->stack->list_file);
++ slice_append_string(&nm, "/");
++ slice_append_string(&nm, add->new_tables[i]);
++ unlink(slice_as_string(&nm));
++
++ reftable_free(add->new_tables[i]);
++ add->new_tables[i] = NULL;
++ }
++ reftable_free(add->new_tables);
++ add->new_tables = NULL;
++ add->new_tables_len = 0;
++
++ if (add->lock_file_fd > 0) {
++ close(add->lock_file_fd);
++ add->lock_file_fd = 0;
++ }
++ if (add->lock_file_name.len > 0) {
++ unlink(slice_as_string(&add->lock_file_name));
++ reftable_free(slice_yield(&add->lock_file_name));
++ }
++
++ free_names(add->names);
++ add->names = NULL;
++}
++
++int reftable_addition_commit(struct reftable_addition *add)
++{
++ struct slice table_list = { 0 };
++ int i = 0;
++ int err = 0;
++ if (add->new_tables_len == 0) {
++ goto exit;
++ }
++
++ for (i = 0; i < add->stack->merged->stack_len; i++) {
++ slice_append_string(&table_list,
++ add->stack->merged->stack[i]->name);
++ slice_append_string(&table_list, "\n");
++ }
++ for (i = 0; i < add->new_tables_len; i++) {
++ slice_append_string(&table_list, add->new_tables[i]);
++ slice_append_string(&table_list, "\n");
++ }
++
++ err = write(add->lock_file_fd, table_list.buf, table_list.len);
++ free(slice_yield(&table_list));
++ if (err < 0) {
++ err = IO_ERROR;
++ goto exit;
++ }
++
++ err = close(add->lock_file_fd);
++ add->lock_file_fd = 0;
++ if (err < 0) {
++ err = IO_ERROR;
++ goto exit;
++ }
++
++ err = rename(slice_as_string(&add->lock_file_name),
++ add->stack->list_file);
++ if (err < 0) {
++ err = IO_ERROR;
++ goto exit;
++ }
++
++ err = reftable_stack_reload(add->stack);
++
++exit:
++ reftable_addition_close(add);
++ return err;
++}
++
++int reftable_stack_new_addition(struct reftable_addition **dest,
++ struct reftable_stack *st)
++{
++ int err = 0;
++ *dest = reftable_malloc(sizeof(**dest));
++ err = reftable_stack_init_addition(*dest, st);
++ if (err) {
++ reftable_free(*dest);
++ *dest = NULL;
++ }
++ return err;
++}
++
++int stack_try_add(struct reftable_stack *st,
++ int (*write_table)(struct reftable_writer *wr, void *arg),
++ void *arg)
++{
++ struct reftable_addition add = { 0 };
++ int err = reftable_stack_init_addition(&add, st);
++ if (err < 0) {
++ goto exit;
++ }
++
++ err = reftable_addition_add(&add, write_table, arg);
++ if (err < 0) {
++ goto exit;
++ }
++
++ err = reftable_addition_commit(&add);
++exit:
++ reftable_addition_close(&add);
++ return err;
++}
++
++int reftable_addition_add(struct reftable_addition *add,
++ int (*write_table)(struct reftable_writer *wr,
++ void *arg),
++ void *arg)
++{
++ struct slice temp_tab_file_name = { 0 };
++ struct slice tab_file_name = { 0 };
++ struct slice next_name = { 0 };
++ struct reftable_writer *wr = NULL;
++ int err = 0;
++ int tab_fd = 0;
++ uint64_t next_update_index = 0;
+
+ slice_resize(&next_name, 0);
+ format_name(&next_name, next_update_index, next_update_index);
+
-+ slice_set_string(&temp_tab_name, st->reftable_dir);
-+ slice_append_string(&temp_tab_name, "/");
-+ slice_append(&temp_tab_name, next_name);
-+ slice_append_string(&temp_tab_name, ".temp.XXXXXX");
++ slice_set_string(&temp_tab_file_name, add->stack->reftable_dir);
++ slice_append_string(&temp_tab_file_name, "/");
++ slice_append(&temp_tab_file_name, next_name);
++ slice_append_string(&temp_tab_file_name, ".temp.XXXXXX");
+
-+ tab_fd = mkstemp((char *)slice_as_string(&temp_tab_name));
++ tab_fd = mkstemp((char *)slice_as_string(&temp_tab_file_name));
+ if (tab_fd < 0) {
+ err = IO_ERROR;
+ goto exit;
+ }
+
-+ wr = new_writer(fd_writer, &tab_fd, &st->config);
++ wr = reftable_new_writer(reftable_fd_write, &tab_fd,
++ &add->stack->config);
+ err = write_table(wr, arg);
+ if (err < 0) {
+ goto exit;
+ }
+
-+ err = writer_close(wr);
++ err = reftable_writer_close(wr);
+ if (err == EMPTY_TABLE_ERROR) {
+ err = 0;
+ goto exit;
@@ -4935,98 +5342,64 @@
+ goto exit;
+ }
+
-+ {
-+ int i = 0;
-+ for (i = 0; i < st->merged->stack_len; i++) {
-+ slice_append_string(&table_list,
-+ st->merged->stack[i]->name);
-+ slice_append_string(&table_list, "\n");
-+ }
-+ }
-+
+ format_name(&next_name, wr->min_update_index, wr->max_update_index);
+ slice_append_string(&next_name, ".ref");
-+ slice_append(&table_list, next_name);
-+ slice_append_string(&table_list, "\n");
+
-+ slice_set_string(&tab_name, st->reftable_dir);
-+ slice_append_string(&tab_name, "/");
-+ slice_append(&tab_name, next_name);
++ slice_set_string(&tab_file_name, add->stack->reftable_dir);
++ slice_append_string(&tab_file_name, "/");
++ slice_append(&tab_file_name, next_name);
+
-+ err = rename(slice_as_string(&temp_tab_name),
-+ slice_as_string(&tab_name));
++ err = rename(slice_as_string(&temp_tab_file_name),
++ slice_as_string(&tab_file_name));
+ if (err < 0) {
+ err = IO_ERROR;
+ goto exit;
+ }
-+ free(slice_yield(&temp_tab_name));
+
-+ err = write(lock_fd, table_list.buf, table_list.len);
-+ if (err < 0) {
-+ err = IO_ERROR;
-+ goto exit;
-+ }
-+ err = close(lock_fd);
-+ lock_fd = 0;
-+ if (err < 0) {
-+ unlink(slice_as_string(&tab_name));
-+ err = IO_ERROR;
-+ goto exit;
-+ }
-+
-+ err = rename(slice_as_string(&lock_name), st->list_file);
-+ if (err < 0) {
-+ unlink(slice_as_string(&tab_name));
-+ err = IO_ERROR;
-+ goto exit;
-+ }
-+
-+ err = stack_reload(st);
++ add->new_tables = reftable_realloc(add->new_tables,
++ sizeof(*add->new_tables) *
++ (add->new_tables_len + 1));
++ add->new_tables[add->new_tables_len] = slice_to_string(next_name);
++ add->new_tables_len++;
+exit:
+ if (tab_fd > 0) {
+ close(tab_fd);
+ tab_fd = 0;
+ }
-+ if (temp_tab_name.len > 0) {
-+ unlink(slice_as_string(&temp_tab_name));
-+ }
-+ unlink(slice_as_string(&lock_name));
-+
-+ if (lock_fd > 0) {
-+ close(lock_fd);
-+ lock_fd = 0;
++ if (temp_tab_file_name.len > 0) {
++ unlink(slice_as_string(&temp_tab_file_name));
+ }
+
-+ free(slice_yield(&lock_name));
-+ free(slice_yield(&temp_tab_name));
-+ free(slice_yield(&tab_name));
-+ free(slice_yield(&next_name));
-+ free(slice_yield(&table_list));
-+ writer_free(wr);
++ reftable_free(slice_yield(&temp_tab_file_name));
++ reftable_free(slice_yield(&tab_file_name));
++ reftable_free(slice_yield(&next_name));
++ reftable_writer_free(wr);
+ return err;
+}
+
-+uint64_t stack_next_update_index(struct stack *st)
++uint64_t reftable_stack_next_update_index(struct reftable_stack *st)
+{
+ int sz = st->merged->stack_len;
+ if (sz > 0) {
-+ return reader_max_update_index(st->merged->stack[sz - 1]) + 1;
++ return reftable_reader_max_update_index(
++ st->merged->stack[sz - 1]) +
++ 1;
+ }
+ return 1;
+}
+
-+static int stack_compact_locked(struct stack *st, int first, int last,
++static int stack_compact_locked(struct reftable_stack *st, int first, int last,
+ struct slice *temp_tab,
-+ struct log_expiry_config *config)
++ struct reftable_log_expiry_config *config)
+{
+ struct slice next_name = { 0 };
+ int tab_fd = -1;
-+ struct writer *wr = NULL;
++ struct reftable_writer *wr = NULL;
+ int err = 0;
+
+ format_name(&next_name,
-+ reader_min_update_index(st->merged->stack[first]),
-+ reader_max_update_index(st->merged->stack[first]));
++ reftable_reader_min_update_index(st->merged->stack[first]),
++ reftable_reader_max_update_index(st->merged->stack[first]));
+
+ slice_set_string(temp_tab, st->reftable_dir);
+ slice_append_string(temp_tab, "/");
@@ -5034,17 +5407,17 @@
+ slice_append_string(temp_tab, ".temp.XXXXXX");
+
+ tab_fd = mkstemp((char *)slice_as_string(temp_tab));
-+ wr = new_writer(fd_writer, &tab_fd, &st->config);
++ wr = reftable_new_writer(reftable_fd_write, &tab_fd, &st->config);
+
+ err = stack_write_compact(st, wr, first, last, config);
+ if (err < 0) {
+ goto exit;
+ }
-+ err = writer_close(wr);
++ err = reftable_writer_close(wr);
+ if (err < 0) {
+ goto exit;
+ }
-+ writer_free(wr);
++ reftable_writer_free(wr);
+
+ err = close(tab_fd);
+ tab_fd = 0;
@@ -5056,46 +5429,49 @@
+ }
+ if (err != 0 && temp_tab->len > 0) {
+ unlink(slice_as_string(temp_tab));
-+ free(slice_yield(temp_tab));
++ reftable_free(slice_yield(temp_tab));
+ }
-+ free(slice_yield(&next_name));
++ reftable_free(slice_yield(&next_name));
+ return err;
+}
+
-+int stack_write_compact(struct stack *st, struct writer *wr, int first,
-+ int last, struct log_expiry_config *config)
++int stack_write_compact(struct reftable_stack *st, struct reftable_writer *wr,
++ int first, int last,
++ struct reftable_log_expiry_config *config)
+{
+ int subtabs_len = last - first + 1;
-+ struct reader **subtabs =
-+ calloc(sizeof(struct reader *), last - first + 1);
-+ struct merged_table *mt = NULL;
++ struct reftable_reader **subtabs = reftable_calloc(
++ sizeof(struct reftable_reader *) * (last - first + 1));
++ struct reftable_merged_table *mt = NULL;
+ int err = 0;
-+ struct iterator it = { 0 };
-+ struct ref_record ref = { 0 };
-+ struct log_record log = { 0 };
++ struct reftable_iterator it = { 0 };
++ struct reftable_ref_record ref = { 0 };
++ struct reftable_log_record log = { 0 };
+
+ int i = 0, j = 0;
+ for (i = first, j = 0; i <= last; i++) {
-+ struct reader *t = st->merged->stack[i];
++ struct reftable_reader *t = st->merged->stack[i];
+ subtabs[j++] = t;
+ st->stats.bytes += t->size;
+ }
-+ writer_set_limits(wr, st->merged->stack[first]->min_update_index,
-+ st->merged->stack[last]->max_update_index);
++ reftable_writer_set_limits(wr,
++ st->merged->stack[first]->min_update_index,
++ st->merged->stack[last]->max_update_index);
+
-+ err = new_merged_table(&mt, subtabs, subtabs_len, st->config.hash_id);
++ err = reftable_new_merged_table(&mt, subtabs, subtabs_len,
++ st->config.hash_id);
+ if (err < 0) {
-+ free(subtabs);
++ reftable_free(subtabs);
+ goto exit;
+ }
+
-+ err = merged_table_seek_ref(mt, &it, "");
++ err = reftable_merged_table_seek_ref(mt, &it, "");
+ if (err < 0) {
+ goto exit;
+ }
+
+ while (true) {
-+ err = iterator_next_ref(it, &ref);
++ err = reftable_iterator_next_ref(it, &ref);
+ if (err > 0) {
+ err = 0;
+ break;
@@ -5103,23 +5479,23 @@
+ if (err < 0) {
+ break;
+ }
-+ if (first == 0 && ref_record_is_deletion(&ref)) {
++ if (first == 0 && reftable_ref_record_is_deletion(&ref)) {
+ continue;
+ }
+
-+ err = writer_add_ref(wr, &ref);
++ err = reftable_writer_add_ref(wr, &ref);
+ if (err < 0) {
+ break;
+ }
+ }
+
-+ err = merged_table_seek_log(mt, &it, "");
++ err = reftable_merged_table_seek_log(mt, &it, "");
+ if (err < 0) {
+ goto exit;
+ }
+
+ while (true) {
-+ err = iterator_next_log(it, &log);
++ err = reftable_iterator_next_log(it, &log);
+ if (err > 0) {
+ err = 0;
+ break;
@@ -5127,7 +5503,7 @@
+ if (err < 0) {
+ break;
+ }
-+ if (first == 0 && log_record_is_deletion(&log)) {
++ if (first == 0 && reftable_log_record_is_deletion(&log)) {
+ continue;
+ }
+
@@ -5143,28 +5519,28 @@
+ continue;
+ }
+
-+ err = writer_add_log(wr, &log);
++ err = reftable_writer_add_log(wr, &log);
+ if (err < 0) {
+ break;
+ }
+ }
+
+exit:
-+ iterator_destroy(&it);
++ reftable_iterator_destroy(&it);
+ if (mt != NULL) {
+ merged_table_clear(mt);
-+ merged_table_free(mt);
++ reftable_merged_table_free(mt);
+ }
-+ ref_record_clear(&ref);
++ reftable_ref_record_clear(&ref);
+
+ return err;
+}
+
+/* < 0: error. 0 == OK, > 0 attempt failed; could retry. */
-+static int stack_compact_range(struct stack *st, int first, int last,
-+ struct log_expiry_config *expiry)
++static int stack_compact_range(struct reftable_stack *st, int first, int last,
++ struct reftable_log_expiry_config *expiry)
+{
-+ struct slice temp_tab_name = { 0 };
++ struct slice temp_tab_file_name = { 0 };
+ struct slice new_table_name = { 0 };
+ struct slice lock_file_name = { 0 };
+ struct slice ref_list_contents = { 0 };
@@ -5173,8 +5549,10 @@
+ bool have_lock = false;
+ int lock_file_fd = 0;
+ int compact_count = last - first + 1;
-+ char **delete_on_success = calloc(sizeof(char *), compact_count + 1);
-+ char **subtable_locks = calloc(sizeof(char *), compact_count + 1);
++ char **delete_on_success =
++ reftable_calloc(sizeof(char *) * (compact_count + 1));
++ char **subtable_locks =
++ reftable_calloc(sizeof(char *) * (compact_count + 1));
+ int i = 0;
+ int j = 0;
+ bool is_empty_table = false;
@@ -5206,14 +5584,14 @@
+ }
+
+ for (i = first, j = 0; i <= last; i++) {
-+ struct slice subtab_name = { 0 };
++ struct slice subtab_file_name = { 0 };
+ struct slice subtab_lock = { 0 };
-+ slice_set_string(&subtab_name, st->reftable_dir);
-+ slice_append_string(&subtab_name, "/");
-+ slice_append_string(&subtab_name,
++ slice_set_string(&subtab_file_name, st->reftable_dir);
++ slice_append_string(&subtab_file_name, "/");
++ slice_append_string(&subtab_file_name,
+ reader_name(st->merged->stack[i]));
+
-+ slice_copy(&subtab_lock, subtab_name);
++ slice_copy(&subtab_lock, subtab_file_name);
+ slice_append_string(&subtab_lock, ".lock");
+
+ {
@@ -5231,7 +5609,8 @@
+ }
+
+ subtable_locks[j] = (char *)slice_as_string(&subtab_lock);
-+ delete_on_success[j] = (char *)slice_as_string(&subtab_name);
++ delete_on_success[j] =
++ (char *)slice_as_string(&subtab_file_name);
+ j++;
+
+ if (err != 0) {
@@ -5245,7 +5624,8 @@
+ }
+ have_lock = false;
+
-+ err = stack_compact_locked(st, first, last, &temp_tab_name, expiry);
++ err = stack_compact_locked(st, first, last, &temp_tab_file_name,
++ expiry);
+ /* Compaction + tombstones can create an empty table out of non-empty
+ * tables. */
+ is_empty_table = (err == EMPTY_TABLE_ERROR);
@@ -5278,7 +5658,7 @@
+ slice_append(&new_table_path, new_table_name);
+
+ if (!is_empty_table) {
-+ err = rename(slice_as_string(&temp_tab_name),
++ err = rename(slice_as_string(&temp_tab_file_name),
+ slice_as_string(&new_table_path));
+ if (err < 0) {
+ goto exit;
@@ -5329,7 +5709,7 @@
+ }
+ }
+
-+ err = stack_reload_maybe_reuse(st, first < last);
++ err = reftable_stack_reload_maybe_reuse(st, first < last);
+exit:
+ free_names(delete_on_success);
+ {
@@ -5347,21 +5727,23 @@
+ if (have_lock) {
+ unlink(slice_as_string(&lock_file_name));
+ }
-+ free(slice_yield(&new_table_name));
-+ free(slice_yield(&new_table_path));
-+ free(slice_yield(&ref_list_contents));
-+ free(slice_yield(&temp_tab_name));
-+ free(slice_yield(&lock_file_name));
++ reftable_free(slice_yield(&new_table_name));
++ reftable_free(slice_yield(&new_table_path));
++ reftable_free(slice_yield(&ref_list_contents));
++ reftable_free(slice_yield(&temp_tab_file_name));
++ reftable_free(slice_yield(&lock_file_name));
+ return err;
+}
+
-+int stack_compact_all(struct stack *st, struct log_expiry_config *config)
++int reftable_stack_compact_all(struct reftable_stack *st,
++ struct reftable_log_expiry_config *config)
+{
+ return stack_compact_range(st, 0, st->merged->stack_len - 1, config);
+}
+
-+static int stack_compact_range_stats(struct stack *st, int first, int last,
-+ struct log_expiry_config *config)
++static int stack_compact_range_stats(struct reftable_stack *st, int first,
++ int last,
++ struct reftable_log_expiry_config *config)
+{
+ int err = stack_compact_range(st, first, last, config);
+ if (err > 0) {
@@ -5387,7 +5769,7 @@
+
+struct segment *sizes_to_segments(int *seglen, uint64_t *sizes, int n)
+{
-+ struct segment *segs = calloc(sizeof(struct segment), n);
++ struct segment *segs = reftable_calloc(sizeof(struct segment) * n);
+ int next = 0;
+ struct segment cur = { 0 };
+ int i = 0;
@@ -5406,8 +5788,9 @@
+ cur.end = i + 1;
+ cur.bytes += sizes[i];
+ }
-+
-+ segs[next++] = cur;
++ if (next > 0) {
++ segs[next++] = cur;
++ }
+ *seglen = next;
+ return segs;
+}
@@ -5440,13 +5823,14 @@
+ min_seg.bytes += sizes[prev];
+ }
+
-+ free(segs);
++ reftable_free(segs);
+ return min_seg;
+}
+
-+static uint64_t *stack_table_sizes_for_compaction(struct stack *st)
++static uint64_t *stack_table_sizes_for_compaction(struct reftable_stack *st)
+{
-+ uint64_t *sizes = calloc(sizeof(uint64_t), st->merged->stack_len);
++ uint64_t *sizes =
++ reftable_calloc(sizeof(uint64_t) * st->merged->stack_len);
+ int i = 0;
+ for (i = 0; i < st->merged->stack_len; i++) {
+ /* overhead is 24 + 68 = 92. */
@@ -5455,12 +5839,12 @@
+ return sizes;
+}
+
-+int stack_auto_compact(struct stack *st)
++int reftable_stack_auto_compact(struct reftable_stack *st)
+{
+ uint64_t *sizes = stack_table_sizes_for_compaction(st);
+ struct segment seg =
+ suggest_compaction_segment(sizes, st->merged->stack_len);
-+ free(sizes);
++ reftable_free(sizes);
+ if (segment_size(&seg) > 0) {
+ return stack_compact_range_stats(st, seg.start, seg.end - 1,
+ NULL);
@@ -5469,58 +5853,61 @@
+ return 0;
+}
+
-+struct compaction_stats *stack_compaction_stats(struct stack *st)
++struct reftable_compaction_stats *
++reftable_stack_compaction_stats(struct reftable_stack *st)
+{
+ return &st->stats;
+}
+
-+int stack_read_ref(struct stack *st, const char *refname,
-+ struct ref_record *ref)
++int reftable_stack_read_ref(struct reftable_stack *st, const char *refname,
++ struct reftable_ref_record *ref)
+{
-+ struct iterator it = { 0 };
-+ struct merged_table *mt = stack_merged_table(st);
-+ int err = merged_table_seek_ref(mt, &it, refname);
++ struct reftable_iterator it = { 0 };
++ struct reftable_merged_table *mt = reftable_stack_merged_table(st);
++ int err = reftable_merged_table_seek_ref(mt, &it, refname);
+ if (err) {
+ goto exit;
+ }
+
-+ err = iterator_next_ref(it, ref);
++ err = reftable_iterator_next_ref(it, ref);
+ if (err) {
+ goto exit;
+ }
+
-+ if (strcmp(ref->ref_name, refname) || ref_record_is_deletion(ref)) {
++ if (strcmp(ref->ref_name, refname) ||
++ reftable_ref_record_is_deletion(ref)) {
+ err = 1;
+ goto exit;
+ }
+
+exit:
-+ iterator_destroy(&it);
++ reftable_iterator_destroy(&it);
+ return err;
+}
+
-+int stack_read_log(struct stack *st, const char *refname,
-+ struct log_record *log)
++int reftable_stack_read_log(struct reftable_stack *st, const char *refname,
++ struct reftable_log_record *log)
+{
-+ struct iterator it = { 0 };
-+ struct merged_table *mt = stack_merged_table(st);
-+ int err = merged_table_seek_log(mt, &it, refname);
++ struct reftable_iterator it = { 0 };
++ struct reftable_merged_table *mt = reftable_stack_merged_table(st);
++ int err = reftable_merged_table_seek_log(mt, &it, refname);
+ if (err) {
+ goto exit;
+ }
+
-+ err = iterator_next_log(it, log);
++ err = reftable_iterator_next_log(it, log);
+ if (err) {
+ goto exit;
+ }
+
-+ if (strcmp(log->ref_name, refname) || log_record_is_deletion(log)) {
++ if (strcmp(log->ref_name, refname) ||
++ reftable_log_record_is_deletion(log)) {
+ err = 1;
+ goto exit;
+ }
+
+exit:
-+ iterator_destroy(&it);
++ reftable_iterator_destroy(&it);
+ return err;
+}
@@ -5542,21 +5929,23 @@
+
+#include "reftable.h"
+
-+struct stack {
++struct reftable_stack {
+ char *list_file;
+ char *reftable_dir;
+
-+ struct write_options config;
++ struct reftable_write_options config;
+
-+ struct merged_table *merged;
-+ struct compaction_stats stats;
++ struct reftable_merged_table *merged;
++ struct reftable_compaction_stats stats;
+};
+
+int read_lines(const char *filename, char ***lines);
-+int stack_try_add(struct stack *st,
-+ int (*write_table)(struct writer *wr, void *arg), void *arg);
-+int stack_write_compact(struct stack *st, struct writer *wr, int first,
-+ int last, struct log_expiry_config *config);
++int stack_try_add(struct reftable_stack *st,
++ int (*write_table)(struct reftable_writer *wr, void *arg),
++ void *arg);
++int stack_write_compact(struct reftable_stack *st, struct reftable_writer *wr,
++ int first, int last,
++ struct reftable_log_expiry_config *config);
+int fastlog2(uint64_t sz);
+
+struct segment {
@@ -5644,6 +6033,7 @@
+
+#include "tree.h"
+
++#include "basics.h"
+#include "system.h"
+
+struct tree_node *tree_search(void *key, struct tree_node **rootp,
@@ -5655,7 +6045,7 @@
+ return NULL;
+ } else {
+ struct tree_node *n =
-+ calloc(sizeof(struct tree_node), 1);
++ reftable_calloc(sizeof(struct tree_node));
+ n->key = key;
+ *rootp = n;
+ return *rootp;
@@ -5698,7 +6088,7 @@
+ if (t->right != NULL) {
+ tree_free(t->right);
+ }
-+ free(t);
++ reftable_free(t);
+}
diff --git a/reftable/tree.h b/reftable/tree.h
@@ -5743,6 +6133,7 @@
+((cd reftable-repo && git fetch origin && git checkout origin/master ) ||
+git clone https://github.com/google/reftable reftable-repo) && \
+cp reftable-repo/c/*.[ch] reftable/ && \
++cp reftable-repo/c/include/*.[ch] reftable/ && \
+cp reftable-repo/LICENSE reftable/ &&
+git --git-dir reftable-repo/.git show --no-patch origin/master \
+> reftable/VERSION && \
@@ -5773,7 +6164,8 @@
+#include "reftable.h"
+#include "tree.h"
+
-+static struct block_stats *writer_block_stats(struct writer *w, byte typ)
++static struct reftable_block_stats *
++writer_reftable_block_stats(struct reftable_writer *w, byte typ)
+{
+ switch (typ) {
+ case 'r':
@@ -5791,18 +6183,19 @@
+
+/* write data, queuing the padding for the next write. Returns negative for
+ * error. */
-+static int padded_write(struct writer *w, byte *data, size_t len, int padding)
++static int padded_write(struct reftable_writer *w, byte *data, size_t len,
++ int padding)
+{
+ int n = 0;
+ if (w->pending_padding > 0) {
-+ byte *zeroed = calloc(w->pending_padding, 1);
++ byte *zeroed = reftable_calloc(w->pending_padding);
+ int n = w->write(w->write_arg, zeroed, w->pending_padding);
+ if (n < 0) {
+ return n;
+ }
+
+ w->pending_padding = 0;
-+ free(zeroed);
++ reftable_free(zeroed);
+ }
+
+ w->pending_padding = padding;
@@ -5814,7 +6207,7 @@
+ return 0;
+}
+
-+static void options_set_defaults(struct write_options *opts)
++static void options_set_defaults(struct reftable_write_options *opts)
+{
+ if (opts->restart_interval == 0) {
+ opts->restart_interval = 16;
@@ -5828,25 +6221,31 @@
+ }
+}
+
-+static int writer_write_header(struct writer *w, byte *dest)
++static int writer_version(struct reftable_writer *w)
++{
++ return (w->opts.hash_id == 0 || w->opts.hash_id == SHA1_ID) ? 1 : 2;
++}
++
++static int writer_write_header(struct reftable_writer *w, byte *dest)
+{
+ memcpy((char *)dest, "REFT", 4);
+
-+ /* DO NOT SUBMIT. This has not been encoded in the standard yet. */
-+ dest[4] = (hash_size(w->opts.hash_id) == SHA1_SIZE) ? 1 : 2; /* version
-+ */
++ dest[4] = writer_version(w);
+
+ put_be24(dest + 5, w->opts.block_size);
+ put_be64(dest + 8, w->min_update_index);
+ put_be64(dest + 16, w->max_update_index);
-+ return 24;
++ if (writer_version(w) == 2) {
++ put_be32(dest + 24, w->opts.hash_id);
++ }
++ return header_size(writer_version(w));
+}
+
-+static void writer_reinit_block_writer(struct writer *w, byte typ)
++static void writer_reinit_block_writer(struct reftable_writer *w, byte typ)
+{
+ int block_start = 0;
+ if (w->next == 0) {
-+ block_start = HEADER_SIZE;
++ block_start = header_size(writer_version(w));
+ }
+
+ block_writer_init(&w->block_writer_data, typ, w->block,
@@ -5856,16 +6255,18 @@
+ w->block_writer->restart_interval = w->opts.restart_interval;
+}
+
-+struct writer *new_writer(int (*writer_func)(void *, byte *, int),
-+ void *writer_arg, struct write_options *opts)
++struct reftable_writer *
++reftable_new_writer(int (*writer_func)(void *, byte *, int), void *writer_arg,
++ struct reftable_write_options *opts)
+{
-+ struct writer *wp = calloc(sizeof(struct writer), 1);
++ struct reftable_writer *wp =
++ reftable_calloc(sizeof(struct reftable_writer));
+ options_set_defaults(opts);
+ if (opts->block_size >= (1 << 24)) {
+ /* TODO - error return? */
+ abort();
+ }
-+ wp->block = calloc(opts->block_size, 1);
++ wp->block = reftable_calloc(opts->block_size);
+ wp->write = writer_func;
+ wp->write_arg = writer_arg;
+ wp->opts = *opts;
@@ -5874,16 +6275,17 @@
+ return wp;
+}
+
-+void writer_set_limits(struct writer *w, uint64_t min, uint64_t max)
++void reftable_writer_set_limits(struct reftable_writer *w, uint64_t min,
++ uint64_t max)
+{
+ w->min_update_index = min;
+ w->max_update_index = max;
+}
+
-+void writer_free(struct writer *w)
++void reftable_writer_free(struct reftable_writer *w)
+{
-+ free(w->block);
-+ free(w);
++ reftable_free(w->block);
++ reftable_free(w);
+}
+
+struct obj_index_tree_node {
@@ -5899,7 +6301,7 @@
+ ((const struct obj_index_tree_node *)b)->hash);
+}
+
-+static void writer_index_hash(struct writer *w, struct slice hash)
++static void writer_index_hash(struct reftable_writer *w, struct slice hash)
+{
+ uint64_t off = w->next;
+
@@ -5909,7 +6311,7 @@
+ &obj_index_tree_node_compare, 0);
+ struct obj_index_tree_node *key = NULL;
+ if (node == NULL) {
-+ key = calloc(sizeof(struct obj_index_tree_node), 1);
++ key = reftable_calloc(sizeof(struct obj_index_tree_node));
+ slice_copy(&key->hash, hash);
+ tree_search((void *)key, &w->obj_index_tree,
+ &obj_index_tree_node_compare, 1);
@@ -5923,14 +6325,14 @@
+
+ if (key->offset_len == key->offset_cap) {
+ key->offset_cap = 2 * key->offset_cap + 1;
-+ key->offsets = realloc(key->offsets,
-+ sizeof(uint64_t) * key->offset_cap);
++ key->offsets = reftable_realloc(
++ key->offsets, sizeof(uint64_t) * key->offset_cap);
+ }
+
+ key->offsets[key->offset_len++] = off;
+}
+
-+static int writer_add_record(struct writer *w, struct record rec)
++static int writer_add_record(struct reftable_writer *w, struct record rec)
+{
+ int result = -1;
+ struct slice key = { 0 };
@@ -5967,14 +6369,15 @@
+
+ result = 0;
+exit:
-+ free(slice_yield(&key));
++ reftable_free(slice_yield(&key));
+ return result;
+}
+
-+int writer_add_ref(struct writer *w, struct ref_record *ref)
++int reftable_writer_add_ref(struct reftable_writer *w,
++ struct reftable_ref_record *ref)
+{
+ struct record rec = { 0 };
-+ struct ref_record copy = *ref;
++ struct reftable_ref_record copy = *ref;
+ int err = 0;
+
+ if (ref->ref_name == NULL) {
@@ -6010,18 +6413,20 @@
+ return 0;
+}
+
-+int writer_add_refs(struct writer *w, struct ref_record *refs, int n)
++int reftable_writer_add_refs(struct reftable_writer *w,
++ struct reftable_ref_record *refs, int n)
+{
+ int err = 0;
+ int i = 0;
-+ QSORT(refs, n, ref_record_compare_name);
++ QSORT(refs, n, reftable_ref_record_compare_name);
+ for (i = 0; err == 0 && i < n; i++) {
-+ err = writer_add_ref(w, &refs[i]);
++ err = reftable_writer_add_ref(w, &refs[i]);
+ }
+ return err;
+}
+
-+int writer_add_log(struct writer *w, struct log_record *log)
++int reftable_writer_add_log(struct reftable_writer *w,
++ struct reftable_log_record *log)
+{
+ if (log->ref_name == NULL) {
+ return API_ERROR;
@@ -6047,18 +6452,19 @@
+ }
+}
+
-+int writer_add_logs(struct writer *w, struct log_record *logs, int n)
++int reftable_writer_add_logs(struct reftable_writer *w,
++ struct reftable_log_record *logs, int n)
+{
+ int err = 0;
+ int i = 0;
-+ QSORT(logs, n, log_record_compare_key);
++ QSORT(logs, n, reftable_log_record_compare_key);
+ for (i = 0; err == 0 && i < n; i++) {
-+ err = writer_add_log(w, &logs[i]);
++ err = reftable_writer_add_log(w, &logs[i]);
+ }
+ return err;
+}
+
-+static int writer_finish_section(struct writer *w)
++static int writer_finish_section(struct reftable_writer *w)
+{
+ byte typ = block_writer_type(w->block_writer);
+ uint64_t index_start = 0;
@@ -6105,9 +6511,9 @@
+ assert(err == 0);
+ }
+ for (i = 0; i < idx_len; i++) {
-+ free(slice_yield(&idx[i].last_key));
++ reftable_free(slice_yield(&idx[i].last_key));
+ }
-+ free(idx);
++ reftable_free(idx);
+ }
+
+ writer_clear_index(w);
@@ -6118,7 +6524,8 @@
+ }
+
+ {
-+ struct block_stats *bstats = writer_block_stats(w, typ);
++ struct reftable_block_stats *bstats =
++ writer_reftable_block_stats(w, typ);
+ bstats->index_blocks =
+ w->stats.idx_stats.blocks - before_blocks;
+ bstats->index_offset = index_start;
@@ -6150,7 +6557,7 @@
+}
+
+struct write_record_arg {
-+ struct writer *w;
++ struct reftable_writer *w;
+ int err;
+};
+
@@ -6199,11 +6606,11 @@
+ struct obj_index_tree_node *entry = (struct obj_index_tree_node *)key;
+
+ FREE_AND_NULL(entry->offsets);
-+ free(slice_yield(&entry->hash));
-+ free(entry);
++ reftable_free(slice_yield(&entry->hash));
++ reftable_free(entry);
+}
+
-+static int writer_dump_object_index(struct writer *w)
++static int writer_dump_object_index(struct reftable_writer *w)
+{
+ struct write_record_arg closure = { .w = w };
+ struct common_prefix_arg common = { 0 };
@@ -6224,7 +6631,7 @@
+ return writer_finish_section(w);
+}
+
-+int writer_finish_public_section(struct writer *w)
++int writer_finish_public_section(struct reftable_writer *w)
+{
+ byte typ = 0;
+ int err = 0;
@@ -6256,9 +6663,9 @@
+ return 0;
+}
+
-+int writer_close(struct writer *w)
++int reftable_writer_close(struct reftable_writer *w)
+{
-+ byte footer[68];
++ byte footer[72];
+ byte *p = footer;
+
+ int err = writer_finish_public_section(w);
@@ -6266,8 +6673,7 @@
+ goto exit;
+ }
+
-+ writer_write_header(w, footer);
-+ p += 24;
++ p += writer_write_header(w, footer);
+ put_be64(p, w->stats.ref_stats.index_offset);
+ p += 8;
+ put_be64(p, (w->stats.obj_stats.offset) << 5 | w->stats.object_id_len);
@@ -6284,7 +6690,7 @@
+ p += 4;
+ w->pending_padding = 0;
+
-+ err = padded_write(w, footer, sizeof(footer), 0);
++ err = padded_write(w, footer, footer_size(writer_version(w)), 0);
+ if (err < 0) {
+ goto exit;
+ }
@@ -6298,15 +6704,15 @@
+ /* free up memory. */
+ block_writer_clear(&w->block_writer_data);
+ writer_clear_index(w);
-+ free(slice_yield(&w->last_key));
++ reftable_free(slice_yield(&w->last_key));
+ return err;
+}
+
-+void writer_clear_index(struct writer *w)
++void writer_clear_index(struct reftable_writer *w)
+{
+ int i = 0;
+ for (i = 0; i < w->index_len; i++) {
-+ free(slice_yield(&w->index[i].last_key));
++ reftable_free(slice_yield(&w->index[i].last_key));
+ }
+
+ FREE_AND_NULL(w->index);
@@ -6316,10 +6722,11 @@
+
+const int debug = 0;
+
-+static int writer_flush_nonempty_block(struct writer *w)
++static int writer_flush_nonempty_block(struct reftable_writer *w)
+{
+ byte typ = block_writer_type(w->block_writer);
-+ struct block_stats *bstats = writer_block_stats(w, typ);
++ struct reftable_block_stats *bstats =
++ writer_reftable_block_stats(w, typ);
+ uint64_t block_typ_off = (bstats->blocks == 0) ? w->next : 0;
+ int raw_bytes = block_writer_finish(w->block_writer);
+ int padding = 0;
@@ -6358,8 +6765,8 @@
+
+ if (w->index_cap == w->index_len) {
+ w->index_cap = 2 * w->index_cap + 1;
-+ w->index = realloc(w->index,
-+ sizeof(struct index_record) * w->index_cap);
++ w->index = reftable_realloc(
++ w->index, sizeof(struct index_record) * w->index_cap);
+ }
+
+ {
@@ -6377,7 +6784,7 @@
+ return 0;
+}
+
-+int writer_flush_block(struct writer *w)
++int writer_flush_block(struct reftable_writer *w)
+{
+ if (w->block_writer == NULL) {
+ return 0;
@@ -6388,7 +6795,7 @@
+ return writer_flush_nonempty_block(w);
+}
+
-+struct stats *writer_stats(struct writer *w)
++const struct reftable_stats *writer_stats(struct reftable_writer *w)
+{
+ return &w->stats;
+}
@@ -6415,7 +6822,7 @@
+#include "slice.h"
+#include "tree.h"
+
-+struct writer {
++struct reftable_writer {
+ int (*write)(void *, byte *, int);
+ void *write_arg;
+ int pending_padding;
@@ -6423,11 +6830,11 @@
+
+ uint64_t next;
+ uint64_t min_update_index, max_update_index;
-+ struct write_options opts;
++ struct reftable_write_options opts;
+
+ byte *block;
-+ struct block_writer *block_writer;
-+ struct block_writer block_writer_data;
++ struct reftable_block_writer *block_writer;
++ struct reftable_block_writer block_writer_data;
+ struct index_record *index;
+ int index_len;
+ int index_cap;
@@ -6435,12 +6842,12 @@
+ /* tree for use with tsearch */
+ struct tree_node *obj_index_tree;
+
-+ struct stats stats;
++ struct reftable_stats stats;
+};
+
-+int writer_flush_block(struct writer *w);
-+void writer_clear_index(struct writer *w);
-+int writer_finish_public_section(struct writer *w);
++int writer_flush_block(struct reftable_writer *w);
++void writer_clear_index(struct reftable_writer *w);
++int writer_finish_public_section(struct reftable_writer *w);
+
+#endif
6: a622d8066c7 ! 9: b29c4ecc1c4 Reftable support for git-core
@@ -6,6 +6,7 @@
TODO:
+ * "git show-ref" shows "HEAD"
* Resolve spots marked with XXX
* Test strategy?
@@ -104,8 +105,9 @@ $^
}
}
-- init_db(git_dir, real_git_dir, option_template, INIT_DB_QUIET);
+- init_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN, INIT_DB_QUIET);
+ init_db(git_dir, real_git_dir, option_template,
++ GIT_HASH_UNKNOWN,
+ DEFAULT_REF_STORAGE, /* XXX */
+ INIT_DB_QUIET);
@@ -116,20 +118,28 @@ $^
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@
+ return 1;
}
- static int create_default_files(const char *template_path,
-- const char *original_git_dir)
-+ const char *original_git_dir,
-+ const char *ref_storage_format, int flags)
+-void initialize_repository_version(int hash_algo)
++void initialize_repository_version(int hash_algo, const char *ref_storage_format)
{
- struct stat st1;
- struct strbuf buf = STRBUF_INIT;
+ char repo_version_string[10];
+ int repo_version = GIT_REPO_VERSION;
+@@
+ die(_("The hash algorithm %s is not supported in this build."), hash_algos[hash_algo].name);
+ #endif
+
+- if (hash_algo != GIT_HASH_SHA1)
++ if (hash_algo != GIT_HASH_SHA1 || !strcmp(ref_storage_format, "reftable"))
+ repo_version = GIT_REPO_VERSION_READ;
+
+ /* This forces creation of new config file */
@@
is_bare_repository_cfg = init_is_bare_repository;
if (init_shared_repository != -1)
set_shared_repository(init_shared_repository);
-+ the_repository->ref_storage_format = xstrdup(ref_storage_format);
++ the_repository->ref_storage_format = xstrdup(fmt->ref_storage);
/*
* We would have created the above under user's umask -- under
@@ -166,35 +176,28 @@ $^
+ */
}
- /* This forces creation of new config file */
-- xsnprintf(repo_version_string, sizeof(repo_version_string),
-- "%d", GIT_REPO_VERSION);
-+ xsnprintf(repo_version_string, sizeof(repo_version_string), "%d",
-+ !strcmp(ref_storage_format, "reftable") ?
-+ GIT_REPO_VERSION_READ :
-+ GIT_REPO_VERSION);
- git_config_set("core.repositoryformatversion", repo_version_string);
+- initialize_repository_version(fmt->hash_algo);
++ initialize_repository_version(fmt->hash_algo, fmt->ref_storage);
/* Check filemode trustability */
+ path = git_path_buf(&buf, "config");
@@
}
int init_db(const char *git_dir, const char *real_git_dir,
-- const char *template_dir, unsigned int flags)
-+ const char *template_dir, const char *ref_storage_format,
-+ unsigned int flags)
+- const char *template_dir, int hash, unsigned int flags)
++ const char *template_dir, int hash, const char *ref_storage_format,
++ unsigned int flags)
{
int reinit;
int exist_ok = flags & INIT_DB_EXIST_OK;
@@
+ * is an attempt to reinitialize new repository with an old tool.
*/
- check_repository_format();
-
-- reinit = create_default_files(template_dir, original_git_dir);
-+ reinit = create_default_files(template_dir, original_git_dir,
-+ ref_storage_format, flags);
+ check_repository_format(&repo_fmt);
++ repo_fmt.ref_storage = xstrdup(ref_storage_format);
- create_object_directory();
+ validate_hash_algorithm(&repo_fmt, hash);
@@
git_config_set("receive.denyNonFastforwards", "true");
@@ -213,7 +216,9 @@ $^
const char *real_git_dir = NULL;
const char *work_tree;
const char *template_dir = NULL;
- unsigned int flags = 0;
+@@
+ const char *object_format = NULL;
+ int hash_algo = GIT_HASH_UNKNOWN;
const struct option init_db_options[] = {
- OPT_STRING(0, "template", &template_dir, N_("template-directory"),
- N_("directory from which templates will be used")),
@@ -235,7 +240,7 @@ $^
+ N_("the ref storage format to use")),
OPT_STRING(0, "separate-git-dir", &real_git_dir, N_("gitdir"),
N_("separate git dir from working tree")),
- OPT_END()
+ OPT_STRING(0, "object-format", &object_format, N_("hash"),
@@
}
@@ -245,21 +250,21 @@ $^
UNLEAK(work_tree);
flags |= INIT_DB_EXIST_OK;
-- return init_db(git_dir, real_git_dir, template_dir, flags);
-+ return init_db(git_dir, real_git_dir, template_dir, ref_storage_format,
-+ flags);
+- return init_db(git_dir, real_git_dir, template_dir, hash_algo, flags);
++ return init_db(git_dir, real_git_dir, template_dir, hash_algo, ref_storage_format, flags);
}
diff --git a/cache.h b/cache.h
--- a/cache.h
+++ b/cache.h
@@
- #define INIT_DB_EXIST_OK 0x0002
int init_db(const char *git_dir, const char *real_git_dir,
-- const char *template_dir, unsigned int flags);
-+ const char *template_dir, const char *ref_storage_format,
-+ unsigned int flags);
+ const char *template_dir, int hash_algo,
++ const char *ref_storage_format,
+ unsigned int flags);
+-void initialize_repository_version(int hash_algo);
++void initialize_repository_version(int hash_algo, const char *ref_storage_format);
void sanitize_stdfds(void);
int daemonize(void);
@@ -350,7 +355,6 @@ $^
struct string_list_item;
struct worktree;
-+/* XXX where should this be? */
+#define DEFAULT_REF_STORAGE "files"
+
/*
@@ -386,26 +390,25 @@ $^
+
+extern struct ref_storage_be refs_be_reftable;
+
-+struct reftable_ref_store {
++struct git_reftable_ref_store {
+ struct ref_store base;
+ unsigned int store_flags;
+
+ int err;
+ char *reftable_dir;
-+ char *table_list_file;
-+ struct stack *stack;
++ struct reftable_stack *stack;
+};
+
-+static void clear_log_record(struct log_record *log)
++static void clear_reftable_log_record(struct reftable_log_record *log)
+{
+ log->old_hash = NULL;
+ log->new_hash = NULL;
+ log->message = NULL;
+ log->ref_name = NULL;
-+ log_record_clear(log);
++ reftable_log_record_clear(log);
+}
+
-+static void fill_log_record(struct log_record *log)
++static void fill_reftable_log_record(struct reftable_log_record *log)
+{
+ const char *info = git_committer_info(0);
+ struct ident_split split = {};
@@ -413,7 +416,7 @@ $^
+ int sign = 1;
+ assert(0 == result);
+
-+ log_record_clear(log);
++ reftable_log_record_clear(log);
+ log->name =
+ xstrndup(split.name_begin, split.name_end - split.name_begin);
+ log->email =
@@ -431,12 +434,12 @@ $^
+ log->tz_offset = sign * atoi(split.tz_begin);
+}
+
-+static struct ref_store *reftable_ref_store_create(const char *path,
++static struct ref_store *git_reftable_ref_store_create(const char *path,
+ unsigned int store_flags)
+{
-+ struct reftable_ref_store *refs = xcalloc(1, sizeof(*refs));
++ struct git_reftable_ref_store *refs = xcalloc(1, sizeof(*refs));
+ struct ref_store *ref_store = (struct ref_store *)refs;
-+ struct write_options cfg = {
++ struct reftable_write_options cfg = {
+ .block_size = 4096,
+ .hash_id = the_hash_algo->format_id,
+ };
@@ -449,10 +452,6 @@ $^
+ refs->reftable_dir = xstrdup(sb.buf);
+ strbuf_reset(&sb);
+
-+ strbuf_addf(&sb, "%s/reftable/tables.list", path);
-+ refs->table_list_file = xstrdup(sb.buf);
-+ strbuf_reset(&sb);
-+
+ strbuf_addf(&sb, "%s/refs", path);
+ safe_create_dir(sb.buf, 1);
+ strbuf_reset(&sb);
@@ -464,31 +463,23 @@ $^
+ strbuf_addf(&sb, "%s/refs/heads", path);
+ write_file(sb.buf, "this repository uses the reftable format");
+
-+ refs->err = new_stack(&refs->stack, refs->reftable_dir,
-+ refs->table_list_file, cfg);
++ refs->err = reftable_new_stack(&refs->stack, refs->reftable_dir, cfg);
+ strbuf_release(&sb);
+ return ref_store;
+}
+
+static int reftable_init_db(struct ref_store *ref_store, struct strbuf *err)
+{
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
-+ FILE *file;
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
+ safe_create_dir(refs->reftable_dir, 1);
-+
-+ file = fopen(refs->table_list_file, "a");
-+ if (file == NULL) {
-+ return -1;
-+ }
-+ fclose(file);
+ return 0;
+}
+
-+struct reftable_iterator {
++struct git_reftable_iterator {
+ struct ref_iterator base;
-+ struct iterator iter;
-+ struct ref_record ref;
++ struct reftable_iterator iter;
++ struct reftable_ref_record ref;
+ struct object_id oid;
+ struct ref_store *ref_store;
+ unsigned int flags;
@@ -498,9 +489,9 @@ $^
+
+static int reftable_ref_iterator_advance(struct ref_iterator *ref_iterator)
+{
-+ struct reftable_iterator *ri = (struct reftable_iterator *)ref_iterator;
++ struct git_reftable_iterator *ri = (struct git_reftable_iterator *)ref_iterator;
+ while (ri->err == 0) {
-+ ri->err = iterator_next_ref(ri->iter, &ri->ref);
++ ri->err = reftable_iterator_next_ref(ri->iter, &ri->ref);
+ if (ri->err) {
+ break;
+ }
@@ -554,7 +545,7 @@ $^
+static int reftable_ref_iterator_peel(struct ref_iterator *ref_iterator,
+ struct object_id *peeled)
+{
-+ struct reftable_iterator *ri = (struct reftable_iterator *)ref_iterator;
++ struct git_reftable_iterator *ri = (struct git_reftable_iterator *)ref_iterator;
+ if (ri->ref.target_value != NULL) {
+ hashcpy(peeled->hash, ri->ref.target_value);
+ return 0;
@@ -565,9 +556,9 @@ $^
+
+static int reftable_ref_iterator_abort(struct ref_iterator *ref_iterator)
+{
-+ struct reftable_iterator *ri = (struct reftable_iterator *)ref_iterator;
-+ ref_record_clear(&ri->ref);
-+ iterator_destroy(&ri->iter);
++ struct git_reftable_iterator *ri = (struct git_reftable_iterator *)ref_iterator;
++ reftable_ref_record_clear(&ri->ref);
++ reftable_iterator_destroy(&ri->iter);
+ return 0;
+}
+
@@ -580,16 +571,16 @@ $^
+reftable_ref_iterator_begin(struct ref_store *ref_store, const char *prefix,
+ unsigned int flags)
+{
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
-+ struct reftable_iterator *ri = xcalloc(1, sizeof(*ri));
-+ struct merged_table *mt = NULL;
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
++ struct git_reftable_iterator *ri = xcalloc(1, sizeof(*ri));
++ struct reftable_merged_table *mt = NULL;
+
+ if (refs->err < 0) {
+ ri->err = refs->err;
+ } else {
-+ mt = stack_merged_table(refs->stack);
-+ ri->err = merged_table_seek_ref(mt, &ri->iter, prefix);
++ mt = reftable_stack_merged_table(refs->stack);
++ ri->err = reftable_merged_table_seek_ref(mt, &ri->iter, prefix);
+ }
+
+ base_ref_iterator_init(&ri->base, &reftable_ref_iterator_vtable, 1);
@@ -610,8 +601,8 @@ $^
+ struct ref_transaction *transaction,
+ struct strbuf *err)
+{
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
+ (void)refs;
+ return 0;
+}
@@ -640,19 +631,19 @@ $^
+ ((struct ref_update *)b)->refname);
+}
+
-+static int write_transaction_table(struct writer *writer, void *arg)
++static int write_transaction_table(struct reftable_writer *writer, void *arg)
+{
+ struct ref_transaction *transaction = (struct ref_transaction *)arg;
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)transaction->ref_store;
-+ uint64_t ts = stack_next_update_index(refs->stack);
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)transaction->ref_store;
++ uint64_t ts = reftable_stack_next_update_index(refs->stack);
+ int err = 0;
-+ struct log_record *logs = calloc(transaction->nr, sizeof(*logs));
++ struct reftable_log_record *logs = calloc(transaction->nr, sizeof(*logs));
+ struct ref_update **sorted =
+ malloc(transaction->nr * sizeof(struct ref_update *));
+ COPY_ARRAY(sorted, transaction->updates, transaction->nr);
+ QSORT(sorted, transaction->nr, ref_update_cmp);
-+ writer_set_limits(writer, ts, ts);
++ reftable_writer_set_limits(writer, ts, ts);
+
+ for (int i = 0; i < transaction->nr; i++) {
+ struct ref_update *u = sorted[i];
@@ -667,8 +658,8 @@ $^
+
+ for (int i = 0; i < transaction->nr; i++) {
+ struct ref_update *u = sorted[i];
-+ struct log_record *log = &logs[i];
-+ fill_log_record(log);
++ struct reftable_log_record *log = &logs[i];
++ fill_reftable_log_record(log);
+ log->ref_name = (char *)u->refname;
+ log->old_hash = u->old_oid.hash;
+ log->new_hash = u->new_oid.hash;
@@ -683,13 +674,13 @@ $^
+ const char *resolved = refs_resolve_ref_unsafe(
+ transaction->ref_store, u->refname, 0, &out_oid,
+ &out_flags);
-+ struct ref_record ref = {};
++ struct reftable_ref_record ref = {};
+ ref.ref_name =
+ (char *)(resolved ? resolved : u->refname);
+ log->ref_name = ref.ref_name;
+ ref.value = u->new_oid.hash;
+ ref.update_index = ts;
-+ err = writer_add_ref(writer, &ref);
++ err = reftable_writer_add_ref(writer, &ref);
+ if (err < 0) {
+ goto exit;
+ }
@@ -697,8 +688,8 @@ $^
+ }
+
+ for (int i = 0; i < transaction->nr; i++) {
-+ err = writer_add_log(writer, &logs[i]);
-+ clear_log_record(&logs[i]);
++ err = reftable_writer_add_log(writer, &logs[i]);
++ clear_reftable_log_record(&logs[i]);
+ if (err < 0) {
+ goto exit;
+ }
@@ -714,17 +705,17 @@ $^
+ struct ref_transaction *transaction,
+ struct strbuf *errmsg)
+{
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
+ int err = 0;
+ if (refs->err < 0) {
+ return refs->err;
+ }
+
-+ err = stack_add(refs->stack, &write_transaction_table, transaction);
++ err = reftable_stack_add(refs->stack, &write_transaction_table, transaction);
+ if (err < 0) {
+ strbuf_addf(errmsg, "reftable: transaction failure %s",
-+ error_str(err));
++ reftable_error_str(err));
+ return -1;
+ }
+
@@ -739,49 +730,49 @@ $^
+}
+
+struct write_delete_refs_arg {
-+ struct stack *stack;
++ struct reftable_stack *stack;
+ struct string_list *refnames;
+ const char *logmsg;
+ unsigned int flags;
+};
+
-+static int write_delete_refs_table(struct writer *writer, void *argv)
++static int write_delete_refs_table(struct reftable_writer *writer, void *argv)
+{
+ struct write_delete_refs_arg *arg =
+ (struct write_delete_refs_arg *)argv;
-+ uint64_t ts = stack_next_update_index(arg->stack);
++ uint64_t ts = reftable_stack_next_update_index(arg->stack);
+ int err = 0;
+
-+ writer_set_limits(writer, ts, ts);
++ reftable_writer_set_limits(writer, ts, ts);
+ for (int i = 0; i < arg->refnames->nr; i++) {
-+ struct ref_record ref = {
++ struct reftable_ref_record ref = {
+ .ref_name = (char *)arg->refnames->items[i].string,
+ .update_index = ts,
+ };
-+ err = writer_add_ref(writer, &ref);
++ err = reftable_writer_add_ref(writer, &ref);
+ if (err < 0) {
+ return err;
+ }
+ }
+
+ for (int i = 0; i < arg->refnames->nr; i++) {
-+ struct log_record log = {};
-+ struct ref_record current = {};
-+ fill_log_record(&log);
++ struct reftable_log_record log = {};
++ struct reftable_ref_record current = {};
++ fill_reftable_log_record(&log);
+ log.message = xstrdup(arg->logmsg);
+ log.new_hash = NULL;
+ log.old_hash = NULL;
+ log.update_index = ts;
+ log.ref_name = (char *)arg->refnames->items[i].string;
+
-+ if (stack_read_ref(arg->stack, log.ref_name, ¤t) == 0) {
++ if (reftable_stack_read_ref(arg->stack, log.ref_name, ¤t) == 0) {
+ log.old_hash = current.value;
+ }
-+ err = writer_add_log(writer, &log);
++ err = reftable_writer_add_log(writer, &log);
+ log.old_hash = NULL;
-+ ref_record_clear(¤t);
++ reftable_ref_record_clear(¤t);
+
-+ clear_log_record(&log);
++ clear_reftable_log_record(&log);
+ if (err < 0) {
+ return err;
+ }
@@ -793,8 +784,8 @@ $^
+ struct string_list *refnames,
+ unsigned int flags)
+{
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
+ struct write_delete_refs_arg arg = {
+ .stack = refs->stack,
+ .refnames = refnames,
@@ -805,52 +796,52 @@ $^
+ return refs->err;
+ }
+
-+ return stack_add(refs->stack, &write_delete_refs_table, &arg);
++ return reftable_stack_add(refs->stack, &write_delete_refs_table, &arg);
+}
+
+static int reftable_pack_refs(struct ref_store *ref_store, unsigned int flags)
+{
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
+ if (refs->err < 0) {
+ return refs->err;
+ }
-+ return stack_compact_all(refs->stack, NULL);
++ return reftable_stack_compact_all(refs->stack, NULL);
+}
+
+struct write_create_symref_arg {
-+ struct reftable_ref_store *refs;
++ struct git_reftable_ref_store *refs;
+ const char *refname;
+ const char *target;
+ const char *logmsg;
+};
+
-+static int write_create_symref_table(struct writer *writer, void *arg)
++static int write_create_symref_table(struct reftable_writer *writer, void *arg)
+{
+ struct write_create_symref_arg *create =
+ (struct write_create_symref_arg *)arg;
-+ uint64_t ts = stack_next_update_index(create->refs->stack);
++ uint64_t ts = reftable_stack_next_update_index(create->refs->stack);
+ int err = 0;
+
-+ struct ref_record ref = {
++ struct reftable_ref_record ref = {
+ .ref_name = (char *)create->refname,
+ .target = (char *)create->target,
+ .update_index = ts,
+ };
-+ writer_set_limits(writer, ts, ts);
-+ err = writer_add_ref(writer, &ref);
++ reftable_writer_set_limits(writer, ts, ts);
++ err = reftable_writer_add_ref(writer, &ref);
+ if (err < 0) {
+ return err;
+ }
+
+ {
-+ struct log_record log = {};
++ struct reftable_log_record log = {};
+ struct object_id new_oid = {};
+ struct object_id old_oid = {};
-+ struct ref_record current = {};
-+ stack_read_ref(create->refs->stack, create->refname, ¤t);
++ struct reftable_ref_record current = {};
++ reftable_stack_read_ref(create->refs->stack, create->refname, ¤t);
+
-+ fill_log_record(&log);
++ fill_reftable_log_record(&log);
+ log.ref_name = current.ref_name;
+ if (refs_resolve_ref_unsafe(
+ (struct ref_store *)create->refs, create->refname,
@@ -865,12 +856,12 @@ $^
+ }
+
+ if (log.old_hash != NULL || log.new_hash != NULL) {
-+ writer_add_log(writer, &log);
++ reftable_writer_add_log(writer, &log);
+ }
+ log.ref_name = NULL;
+ log.old_hash = NULL;
+ log.new_hash = NULL;
-+ clear_log_record(&log);
++ clear_reftable_log_record(&log);
+ }
+ return 0;
+}
@@ -879,8 +870,8 @@ $^
+ const char *refname, const char *target,
+ const char *logmsg)
+{
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
+ struct write_create_symref_arg arg = { .refs = refs,
+ .refname = refname,
+ .target = target,
@@ -888,55 +879,55 @@ $^
+ if (refs->err < 0) {
+ return refs->err;
+ }
-+ return stack_add(refs->stack, &write_create_symref_table, &arg);
++ return reftable_stack_add(refs->stack, &write_create_symref_table, &arg);
+}
+
+struct write_rename_arg {
-+ struct stack *stack;
++ struct reftable_stack *stack;
+ const char *oldname;
+ const char *newname;
+ const char *logmsg;
+};
+
-+static int write_rename_table(struct writer *writer, void *argv)
++static int write_rename_table(struct reftable_writer *writer, void *argv)
+{
+ struct write_rename_arg *arg = (struct write_rename_arg *)argv;
-+ uint64_t ts = stack_next_update_index(arg->stack);
-+ struct ref_record ref = {};
-+ int err = stack_read_ref(arg->stack, arg->oldname, &ref);
++ uint64_t ts = reftable_stack_next_update_index(arg->stack);
++ struct reftable_ref_record ref = {};
++ int err = reftable_stack_read_ref(arg->stack, arg->oldname, &ref);
+
+ if (err) {
+ goto exit;
+ }
+
+ /* XXX do ref renames overwrite the target? */
-+ if (stack_read_ref(arg->stack, arg->newname, &ref) == 0) {
++ if (reftable_stack_read_ref(arg->stack, arg->newname, &ref) == 0) {
+ goto exit;
+ }
+
+ free(ref.ref_name);
+ ref.ref_name = strdup(arg->newname);
-+ writer_set_limits(writer, ts, ts);
++ reftable_writer_set_limits(writer, ts, ts);
+ ref.update_index = ts;
+
+ {
-+ struct ref_record todo[2] = {};
++ struct reftable_ref_record todo[2] = {};
+ todo[0].ref_name = (char *)arg->oldname;
+ todo[0].update_index = ts;
+ /* leave todo[0] empty */
+ todo[1] = ref;
+ todo[1].update_index = ts;
+
-+ err = writer_add_refs(writer, todo, 2);
++ err = reftable_writer_add_refs(writer, todo, 2);
+ if (err < 0) {
+ goto exit;
+ }
+ }
+
+ if (ref.value != NULL) {
-+ struct log_record todo[2] = {};
-+ fill_log_record(&todo[0]);
-+ fill_log_record(&todo[1]);
++ struct reftable_log_record todo[2] = {};
++ fill_reftable_log_record(&todo[0]);
++ fill_reftable_log_record(&todo[1]);
+
+ todo[0].ref_name = (char *)arg->oldname;
+ todo[0].update_index = ts;
@@ -950,10 +941,10 @@ $^
+ todo[1].new_hash = ref.value;
+ todo[1].message = (char *)arg->logmsg;
+
-+ err = writer_add_logs(writer, todo, 2);
++ err = reftable_writer_add_logs(writer, todo, 2);
+
-+ clear_log_record(&todo[0]);
-+ clear_log_record(&todo[1]);
++ clear_reftable_log_record(&todo[0]);
++ clear_reftable_log_record(&todo[1]);
+
+ if (err < 0) {
+ goto exit;
@@ -964,7 +955,7 @@ $^
+ }
+
+exit:
-+ ref_record_clear(&ref);
++ reftable_ref_record_clear(&ref);
+ return err;
+}
+
@@ -972,8 +963,8 @@ $^
+ const char *oldrefname, const char *newrefname,
+ const char *logmsg)
+{
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
+ struct write_rename_arg arg = {
+ .stack = refs->stack,
+ .oldname = oldrefname,
@@ -984,7 +975,7 @@ $^
+ return refs->err;
+ }
+
-+ return stack_add(refs->stack, &write_rename_table, &arg);
++ return reftable_stack_add(refs->stack, &write_rename_table, &arg);
+}
+
+static int reftable_copy_ref(struct ref_store *ref_store,
@@ -996,8 +987,8 @@ $^
+
+struct reftable_reflog_ref_iterator {
+ struct ref_iterator base;
-+ struct iterator iter;
-+ struct log_record log;
++ struct reftable_iterator iter;
++ struct reftable_log_record log;
+ struct object_id oid;
+ char *last_name;
+};
@@ -1009,7 +1000,7 @@ $^
+ (struct reftable_reflog_ref_iterator *)ref_iterator;
+
+ while (1) {
-+ int err = iterator_next_log(ri->iter, &ri->log);
++ int err = reftable_iterator_next_log(ri->iter, &ri->log);
+ if (err > 0) {
+ return ITER_DONE;
+ }
@@ -1041,8 +1032,8 @@ $^
+{
+ struct reftable_reflog_ref_iterator *ri =
+ (struct reftable_reflog_ref_iterator *)ref_iterator;
-+ log_record_clear(&ri->log);
-+ iterator_destroy(&ri->iter);
++ reftable_log_record_clear(&ri->log);
++ reftable_iterator_destroy(&ri->iter);
+ return 0;
+}
+
@@ -1055,11 +1046,11 @@ $^
+reftable_reflog_iterator_begin(struct ref_store *ref_store)
+{
+ struct reftable_reflog_ref_iterator *ri = xcalloc(sizeof(*ri), 1);
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
+
-+ struct merged_table *mt = stack_merged_table(refs->stack);
-+ int err = merged_table_seek_log(mt, &ri->iter, "");
++ struct reftable_merged_table *mt = reftable_stack_merged_table(refs->stack);
++ int err = reftable_merged_table_seek_log(mt, &ri->iter, "");
+ if (err < 0) {
+ free(ri);
+ return NULL;
@@ -1077,21 +1068,21 @@ $^
+ const char *refname,
+ each_reflog_ent_fn fn, void *cb_data)
+{
-+ struct iterator it = {};
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
-+ struct merged_table *mt = NULL;
++ struct reftable_iterator it = {};
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
++ struct reftable_merged_table *mt = NULL;
+ int err = 0;
-+ struct log_record log = {};
++ struct reftable_log_record log = {};
+
+ if (refs->err < 0) {
+ return refs->err;
+ }
+
-+ mt = stack_merged_table(refs->stack);
-+ err = merged_table_seek_log(mt, &it, refname);
++ mt = reftable_stack_merged_table(refs->stack);
++ err = reftable_merged_table_seek_log(mt, &it, refname);
+ while (err == 0) {
-+ err = iterator_next_log(it, &log);
++ err = reftable_iterator_next_log(it, &log);
+ if (err != 0) {
+ break;
+ }
@@ -1120,8 +1111,8 @@ $^
+ }
+ }
+
-+ log_record_clear(&log);
-+ iterator_destroy(&it);
++ reftable_log_record_clear(&log);
++ reftable_iterator_destroy(&it);
+ if (err > 0) {
+ err = 0;
+ }
@@ -1133,11 +1124,11 @@ $^
+ const char *refname,
+ each_reflog_ent_fn fn, void *cb_data)
+{
-+ struct iterator it = {};
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
-+ struct merged_table *mt = NULL;
-+ struct log_record *logs = NULL;
++ struct reftable_iterator it = {};
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
++ struct reftable_merged_table *mt = NULL;
++ struct reftable_log_record *logs = NULL;
+ int cap = 0;
+ int len = 0;
+ int err = 0;
@@ -1145,12 +1136,12 @@ $^
+ if (refs->err < 0) {
+ return refs->err;
+ }
-+ mt = stack_merged_table(refs->stack);
-+ err = merged_table_seek_log(mt, &it, refname);
++ mt = reftable_stack_merged_table(refs->stack);
++ err = reftable_merged_table_seek_log(mt, &it, refname);
+
+ while (err == 0) {
-+ struct log_record log = {};
-+ err = iterator_next_log(it, &log);
++ struct reftable_log_record log = {};
++ err = reftable_iterator_next_log(it, &log);
+ if (err != 0) {
+ break;
+ }
@@ -1168,7 +1159,7 @@ $^
+ }
+
+ for (int i = len; i--;) {
-+ struct log_record *log = &logs[i];
++ struct reftable_log_record *log = &logs[i];
+ struct object_id old_oid = {};
+ struct object_id new_oid = {};
+ const char *full_committer = "";
@@ -1187,11 +1178,11 @@ $^
+ }
+
+ for (int i = 0; i < len; i++) {
-+ log_record_clear(&logs[i]);
++ reftable_log_record_clear(&logs[i]);
+ }
+ free(logs);
+
-+ iterator_destroy(&it);
++ reftable_iterator_destroy(&it);
+ if (err > 0) {
+ err = 0;
+ }
@@ -1219,8 +1210,8 @@ $^
+}
+
+struct reflog_expiry_arg {
-+ struct reftable_ref_store *refs;
-+ struct log_record *tombstones;
++ struct git_reftable_ref_store *refs;
++ struct reftable_log_record *tombstones;
+ int len;
+ int cap;
+};
@@ -1229,7 +1220,7 @@ $^
+{
+ int i = 0;
+ for (; i < arg->len; i++) {
-+ log_record_clear(&arg->tombstones[i]);
++ reftable_log_record_clear(&arg->tombstones[i]);
+ }
+
+ FREE_AND_NULL(arg->tombstones);
@@ -1238,7 +1229,7 @@ $^
+static void add_log_tombstone(struct reflog_expiry_arg *arg,
+ const char *refname, uint64_t ts)
+{
-+ struct log_record tombstone = {
++ struct reftable_log_record tombstone = {
+ .ref_name = xstrdup(refname),
+ .update_index = ts,
+ };
@@ -1250,14 +1241,14 @@ $^
+ arg->tombstones[arg->len++] = tombstone;
+}
+
-+static int write_reflog_expiry_table(struct writer *writer, void *argv)
++static int write_reflog_expiry_table(struct reftable_writer *writer, void *argv)
+{
+ struct reflog_expiry_arg *arg = (struct reflog_expiry_arg *)argv;
-+ uint64_t ts = stack_next_update_index(arg->refs->stack);
++ uint64_t ts = reftable_stack_next_update_index(arg->refs->stack);
+ int i = 0;
-+ writer_set_limits(writer, ts, ts);
++ reftable_writer_set_limits(writer, ts, ts);
+ for (i = 0; i < arg->len; i++) {
-+ int err = writer_add_log(writer, &arg->tombstones[i]);
++ int err = reftable_writer_add_log(writer, &arg->tombstones[i]);
+ if (err) {
+ return err;
+ }
@@ -1286,21 +1277,21 @@ $^
+ It would be better if the refs backend supported an API that sets a
+ criterion for all refs, passing the criterion to pack_refs().
+ */
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
-+ struct merged_table *mt = NULL;
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
++ struct reftable_merged_table *mt = NULL;
+ struct reflog_expiry_arg arg = {
+ .refs = refs,
+ };
-+ struct log_record log = {};
-+ struct iterator it = {};
++ struct reftable_log_record log = {};
++ struct reftable_iterator it = {};
+ int err = 0;
+ if (refs->err < 0) {
+ return refs->err;
+ }
+
-+ mt = stack_merged_table(refs->stack);
-+ err = merged_table_seek_log(mt, &it, refname);
++ mt = reftable_stack_merged_table(refs->stack);
++ err = reftable_merged_table_seek_log(mt, &it, refname);
+ if (err < 0) {
+ return err;
+ }
@@ -1309,7 +1300,7 @@ $^
+ struct object_id ooid = {};
+ struct object_id noid = {};
+
-+ int err = iterator_next_log(it, &log);
++ int err = reftable_iterator_next_log(it, &log);
+ if (err < 0) {
+ return err;
+ }
@@ -1326,9 +1317,9 @@ $^
+ add_log_tombstone(&arg, refname, log.update_index);
+ }
+ }
-+ log_record_clear(&log);
-+ iterator_destroy(&it);
-+ err = stack_add(refs->stack, &write_reflog_expiry_table, &arg);
++ reftable_log_record_clear(&log);
++ reftable_iterator_destroy(&it);
++ err = reftable_stack_add(refs->stack, &write_reflog_expiry_table, &arg);
+ clear_log_tombstones(&arg);
+ return err;
+}
@@ -1337,15 +1328,15 @@ $^
+ const char *refname, struct object_id *oid,
+ struct strbuf *referent, unsigned int *type)
+{
-+ struct reftable_ref_store *refs =
-+ (struct reftable_ref_store *)ref_store;
-+ struct ref_record ref = {};
++ struct git_reftable_ref_store *refs =
++ (struct git_reftable_ref_store *)ref_store;
++ struct reftable_ref_record ref = {};
+ int err = 0;
+ if (refs->err < 0) {
+ return refs->err;
+ }
+
-+ err = stack_read_ref(refs->stack, refname, &ref);
++ err = reftable_stack_read_ref(refs->stack, refname, &ref);
+ if (err) {
+ goto exit;
+ }
@@ -1358,14 +1349,14 @@ $^
+ hashcpy(oid->hash, ref.value);
+ }
+exit:
-+ ref_record_clear(&ref);
++ reftable_ref_record_clear(&ref);
+ return err;
+}
+
+struct ref_storage_be refs_be_reftable = {
+ &refs_be_files,
+ "reftable",
-+ reftable_ref_store_create,
++ git_reftable_ref_store_create,
+ reftable_init_db,
+ reftable_transaction_prepare,
+ reftable_transaction_finish,
@@ -1471,24 +1462,28 @@ $^
+. ./test-lib.sh
+
+test_expect_success 'basic operation of reftable storage' '
-+ git init --ref-storage=reftable repo &&
-+ cd repo &&
-+ echo "hello" > world.txt &&
-+ git add world.txt &&
-+ git commit -m "first post" &&
-+ printf "HEAD\nrefs/heads/master\n" > want &&
-+ git show-ref | cut -f2 -d" " > got &&
-+ test_cmp got want &&
-+ for count in $(test_seq 1 10); do
-+ echo "hello" >> world.txt
-+ git commit -m "number ${count}" world.txt
-+ done &&
-+ git gc &&
-+ nfiles=$(ls -1 .git/reftable | wc -l | tr -d "[ \t]" ) &&
-+ test "${nfiles}" = "2" &&
-+ git reflog refs/heads/master > output &&
-+ grep "commit (initial): first post" output &&
-+ grep "commit: number 10" output
++ git init --ref-storage=reftable repo && (
++ cd repo &&
++ echo "hello" >world.txt &&
++ git add world.txt &&
++ git commit -m "first post" &&
++ test_write_lines HEAD refs/heads/master >expect &&
++ git show-ref &&
++ git show-ref | cut -f2 -d" " > actual &&
++ test_cmp actual expect &&
++ for count in $(test_seq 1 10)
++ do
++ echo "hello" >>world.txt
++ git commit -m "number ${count}" world.txt ||
++ return 1
++ done &&
++ git gc &&
++ nfiles=$(ls -1 .git/reftable | wc -l ) &&
++ test ${nfiles} = "2" &&
++ git reflog refs/heads/master >output &&
++ test_line_count = 11 output &&
++ grep "commit (initial): first post" output &&
++ grep "commit: number 10" output )
+'
+
+test_done
--
gitgitgadget
