This patchset fixes a few directory traversal issues, where fill_directory() would traverse into directories that it shouldn't and not traverse into directories that it should (one of which was originally reported on this list at [1]). And it includes a few cleanups Changes since v4: * Tweak the trace2 statistics emitting a bit, as per suggestions from Jeff. * Introduce a new readdir_skip_dot_and_dotdot() helper at the end of the series, and use it everywhere we repeat the same code to skip '.' and '..' entries from readdir. Also use it in dir.c's read_cached_dir() so we can be consistent about skipping it, even for statistics, across platforms. If anyone has any ideas about a better place to put the "Some sidenotes" from the sixth commit message rather than keeping them in a random commit message, that might be helpful. [1] See https://lore.kernel.org/git/DM6PR00MB06829EC5B85E0C5AC595004E894E9@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/ or alternatively https://github.com/git-for-windows/git/issues/2732. Derrick Stolee (1): dir: update stale description of treat_directory() Elijah Newren (8): dir: convert trace calls to trace2 equivalents dir: report number of visited directories and paths with trace2 ls-files: error out on -i unless -o or -c are specified t7300: add testcase showing unnecessary traversal into ignored directory t3001, t7300: add testcase showcasing missed directory traversal dir: avoid unnecessary traversal into ignored directory dir: traverse into untracked directories if they may have ignored subfiles dir: introduce readdir_skip_dot_and_dotdot() helper builtin/clean.c | 4 +- builtin/ls-files.c | 3 + builtin/worktree.c | 4 +- diff-no-index.c | 5 +- dir.c | 146 +++++++++++++------- dir.h | 6 + entry.c | 5 +- notes-merge.c | 5 +- object-file.c | 4 +- packfile.c | 5 +- rerere.c | 4 +- t/t1306-xdg-files.sh | 2 +- t/t3001-ls-files-others-exclude.sh | 5 + t/t3003-ls-files-exclude.sh | 4 +- t/t7063-status-untracked-cache.sh | 206 +++++++++++++++++------------ t/t7300-clean.sh | 42 ++++++ t/t7519-status-fsmonitor.sh | 8 +- worktree.c | 12 +- 18 files changed, 298 insertions(+), 172 deletions(-) base-commit: 311531c9de557d25ac087c1637818bd2aad6eb3a Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1020%2Fnewren%2Fdirectory-traversal-fixes-v5 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1020/newren/directory-traversal-fixes-v5 Pull-Request: https://github.com/git/git/pull/1020 Range-diff vs v4: 1: 9204e36b7e90 ! 1: 6b1b4820dd20 dir: convert trace calls to trace2 equivalents @@ dir.c: static struct untracked_cache_dir *validate_untracked_cache(struct dir_st return root; } -+static void trace2_read_directory_statistics(struct dir_struct *dir, -+ struct repository *repo, -+ const char *path) ++static void emit_traversal_statistics(struct dir_struct *dir, ++ struct repository *repo, ++ const char *path, ++ int path_len) +{ ++ if (!trace2_is_enabled()) ++ return; ++ ++ if (!path_len) { ++ trace2_data_string("read_directory", repo, "path", ""); ++ } else { ++ struct strbuf tmp = STRBUF_INIT; ++ strbuf_add(&tmp, path, path_len); ++ trace2_data_string("read_directory", repo, "path", tmp.buf); ++ strbuf_release(&tmp); ++ } ++ + if (!dir->untracked) + return; -+ trace2_data_string("read_directory", repo, "path", path); + trace2_data_intmax("read_directory", repo, + "node-creation", dir->untracked->dir_created); + trace2_data_intmax("read_directory", repo, @@ dir.c: int read_directory(struct dir_struct *dir, struct index_state *istate, QSORT(dir->ignored, dir->ignored_nr, cmp_dir_entry); - trace_performance_leave("read directory %.*s", len, path); -+ if (trace2_is_enabled()) { -+ struct strbuf tmp = STRBUF_INIT; -+ strbuf_add(&tmp, path, len); -+ trace2_read_directory_statistics(dir, istate->repo, tmp.buf); -+ strbuf_release(&tmp); -+ } ++ emit_traversal_statistics(dir, istate->repo, path, len); + + trace2_region_leave("dir", "read_directory", istate->repo); if (dir->untracked) { 2: 6939253be825 ! 2: cfe2898b7a7e dir: report number of visited directories and paths with trace2 @@ dir.c: static enum path_treatment read_directory_recursive(struct dir_struct *di if (state > dir_state) dir_state = state; -@@ dir.c: static void trace2_read_directory_statistics(struct dir_struct *dir, - struct repository *repo, - const char *path) - { +@@ dir.c: static void emit_traversal_statistics(struct dir_struct *dir, + strbuf_release(&tmp); + } + + trace2_data_intmax("read_directory", repo, + "directories-visited", dir->visited_directories); + trace2_data_intmax("read_directory", repo, + "paths-visited", dir->visited_paths); ++ if (!dir->untracked) return; - trace2_data_string("read_directory", repo, "path", path); + trace2_data_intmax("read_directory", repo, @@ dir.c: int read_directory(struct dir_struct *dir, struct index_state *istate, struct untracked_cache_dir *untracked; 3: 8d0ca8104be6 = 3: 279ef30ffbc2 ls-files: error out on -i unless -o or -c are specified 4: 317abab3571e = 4: 5a8807a1992c t7300: add testcase showing unnecessary traversal into ignored directory 5: 5eb019327b57 = 5: b014ccbbaf3e t3001, t7300: add testcase showcasing missed directory traversal 6: 89cc01ef8598 = 6: ae1c9e37b21b dir: avoid unnecessary traversal into ignored directory 7: 4a561e1229e4 = 7: 6fa1e85edf2f dir: traverse into untracked directories if they may have ignored subfiles 8: 2945e749f5e3 = 8: 179f992edc92 dir: update stale description of treat_directory() -: ------------ > 9: b7c6176560bd dir: introduce readdir_skip_dot_and_dotdot() helper -- gitgitgadget
