From: Neeraj Singh <neerajsi@xxxxxxxxxxxxx>
This commit prepares for adding batch-fsync to the bulk-checkin
infrastructure.
The bulk-checkin infrastructure is currently used to batch up addition
of large blobs to a packfile. When a blob is larger than
big_file_threshold, we unconditionally add it to a pack. If bulk
checkins are 'plugged', we allow multiple large blobs to be added to a
single pack until we reach the packfile size limit; otherwise, we simply
make a new packfile for each large blob. The 'unplug' call tells us when
the series of blob additions is done so that we can finish the packfiles
and make their objects available to subsequent operations.
Stated another way, bulk-checkin allows callers to define a transaction
that adds multiple objects to the object database, where the object
database can optimize its internal operations within the transaction
boundary.
Batched fsync will fit into bulk-checkin by taking advantage of the
plug/unplug functionality to determine the appropriate time to fsync
and make newly-added objects available in the primary object database.
* Rename 'state' variable to 'bulk_checkin_state', since we will later
be adding 'bulk_fsync_objdir'. This also makes the variable easier to
find in the debugger, since the name is more unique.
* Move the 'plugged' data member of 'bulk_checkin_state' into a separate
static variable. Doing this avoids resetting the variable in
finish_bulk_checkin when zeroing the 'bulk_checkin_state'. As-is, we
seem to unintentionally disable the plugging functionality the first
time a new packfile must be created due to packfile size limits. While
disabling the plugging state only results in suboptimal behavior for
the current code, it would be fatal for the bulk-fsync functionality
later in this patch series.
Signed-off-by: Neeraj Singh <neerajsi@xxxxxxxxxxxxx>
---
bulk-checkin.c | 22 ++++++++++++----------
1 file changed, 12 insertions(+), 10 deletions(-)
diff --git a/bulk-checkin.c b/bulk-checkin.c
index a16ae3c629d..ffe142841b2 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -10,9 +10,9 @@
#include "packfile.h"
#include "object-store.h"
-static struct bulk_checkin_state {
- unsigned plugged:1;
+static int bulk_checkin_plugged;
+static struct bulk_checkin_state {
char *pack_tmp_name;
struct hashfile *f;
off_t offset;
@@ -21,7 +21,7 @@ static struct bulk_checkin_state {
struct pack_idx_entry **written;
uint32_t alloc_written;
uint32_t nr_written;
-} state;
+} bulk_checkin_state;
static void finish_tmp_packfile(struct strbuf *basename,
const char *pack_tmp_name,
@@ -278,21 +278,23 @@ int index_bulk_checkin(struct object_id *oid,
int fd, size_t size, enum object_type type,
const char *path, unsigned flags)
{
- int status = deflate_to_pack(&state, oid, fd, size, type,
+ int status = deflate_to_pack(&bulk_checkin_state, oid, fd, size, type,
path, flags);
- if (!state.plugged)
- finish_bulk_checkin(&state);
+ if (!bulk_checkin_plugged)
+ finish_bulk_checkin(&bulk_checkin_state);
return status;
}
void begin_odb_transaction(void)
{
- state.plugged = 1;
+ assert(!bulk_checkin_plugged);
+ bulk_checkin_plugged = 1;
}
void end_odb_transaction(void)
{
- state.plugged = 0;
- if (state.f)
- finish_bulk_checkin(&state);
+ assert(bulk_checkin_plugged);
+ bulk_checkin_plugged = 0;
+ if (bulk_checkin_state.f)
+ finish_bulk_checkin(&bulk_checkin_state);
}
--
gitgitgadget
