The "Filtering content" progress in entry.c:finish_delayed_checkout()
is unusual because of how it calculates the progress count and because
it shows the progress of a nested loop. It works basically like this:
start_delayed_progress(p, nr_of_paths_to_filter)
for_each_filter {
display_progress(p, nr_of_paths_to_filter - nr_of_paths_still_left_to_filter)
for_each_path_handled_by_the_current_filter {
checkout_entry()
}
}
stop_progress(p)
There are two issues with this approach:
- The work done by the last filter (or the only filter if there is
only one) is never counted, so if the last filter still has some
paths to process, then the counter shown in the "done" progress
line will not match the expected total.
This is, in part, responsible for the failures of the tests
'missing file in delayed checkout' and 'invalid file in delayed
checkout' in 't0021-conversion.sh' when run with
GIT_TEST_CHECK_PROGRESS=1, because both use only one filter. (The
test 'delayed checkout in process filter' uses two filters but the
first one does all the work, so that test already happens to
succeed even with GIT_TEST_CHECK_PROGRESS=1.)
- The progress counter is updated only once per filter, not once per
processed path, so if a filter has a lot of paths to process, then
the counter might stay unchanged for a long while and then make a
big jump (though the user still gets a sense of progress, because
we call display_throughput() after each processed path to show the
amount of processed data).
Move the display_progress() call to the inner loop, right next to that
checkout_entry() call that does the hard work for each path, and use a
dedicated counter variable that is incremented upon processing each
path.
After this change the 'invalid file in delayed checkout' in
't0021-conversion.sh' will succeed with GIT_TEST_CHECK_PROGRESS=1, but
the 'missing file in delayed checkout' test will still fail, because
its purposefully buggy filter doesn't process any paths, so we won't
execute that inner loop at all (this will be fixed in the next patch).
Signed-off-by: SZEDER Gábor <szeder.dev@xxxxxxxxx>
---
entry.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/entry.c b/entry.c
index 711ee0693c..bc4b8fcc98 100644
--- a/entry.c
+++ b/entry.c
@@ -162,7 +162,7 @@ static int remove_available_paths(struct string_list_item *item, void *cb_data)
int finish_delayed_checkout(struct checkout *state, int *nr_checkouts)
{
int errs = 0;
- unsigned delayed_object_count;
+ unsigned processed_paths = 0;
off_t filtered_bytes = 0;
struct string_list_item *filter, *path;
struct progress *progress;
@@ -172,12 +172,10 @@ int finish_delayed_checkout(struct checkout *state, int *nr_checkouts)
return errs;
dco->state = CE_RETRY;
- delayed_object_count = dco->paths.nr;
- progress = start_delayed_progress(_("Filtering content"), delayed_object_count);
+ progress = start_delayed_progress(_("Filtering content"), dco->paths.nr);
while (dco->filters.nr > 0) {
for_each_string_list_item(filter, &dco->filters) {
struct string_list available_paths = STRING_LIST_INIT_NODUP;
- display_progress(progress, delayed_object_count - dco->paths.nr);
if (!async_query_available_blobs(filter->string, &available_paths)) {
/* Filter reported an error */
@@ -224,6 +222,7 @@ int finish_delayed_checkout(struct checkout *state, int *nr_checkouts)
ce = index_file_exists(state->istate, path->string,
strlen(path->string), 0);
if (ce) {
+ display_progress(progress, ++processed_paths);
errs |= checkout_entry(ce, state, NULL, nr_checkouts);
filtered_bytes += ce->ce_stat_data.sd_size;
display_throughput(progress, filtered_bytes);
--
2.32.0.289.g44fbea0957
