Re: [PATCH 01/11] t9902-completion: add tests demonstrating issues with quoted pathnames

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Gábor,

On Tue, 17 Apr 2018, SZEDER Gábor wrote:

> Completion functions see all words on the command line verbatim,
> including any backslash-escapes, single and double quotes that might
> be there.  Furthermore, git commands quote pathnames if they contain
> certain special characters.  All these create various issues when
> doing git-aware path completion.
> 
> Add a couple of failing tests to demonstrate these issues.
> 
> Later patches in this series will discuss these issues in detail as
> they fix them.
> 
> Signed-off-by: SZEDER Gábor <szeder.dev@xxxxxxxxx>
> ---
> 
> Notes:
>     Do any more new tests need FUNNYNAMES* prereq?

Yes.

>  t/t9902-completion.sh | 91 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 91 insertions(+)
> 
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index b7f5b1e632..ff2e4a8f5f 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -1427,6 +1427,97 @@ test_expect_success 'complete files' '
>  	test_completion "git add mom" "momified"
>  '
>  
> +# The next tests only care about how the completion script deals with
> +# unusual characters in path names.  By defining a custom completion
> +# function to list untracked files they won't be influenced by future
> +# changes of the completion functions of real git commands, and we
> +# don't have to bother with adding files to the index in these tests.
> +_git_test_path_comp ()
> +{
> +	__git_complete_index_file --others
> +}
> +
> +test_expect_failure 'complete files - escaped characters on cmdline' '
> +	test_when_finished "rm -rf \"New|Dir\"" &&
> +	mkdir "New|Dir" &&
> +	>"New|Dir/New&File.c" &&
> +
> +	test_completion "git test-path-comp N" \
> +			"New|Dir" &&	# Bash will turn this into "New\|Dir/"
> +	test_completion "git test-path-comp New\\|D" \
> +			"New|Dir" &&
> +	test_completion "git test-path-comp New\\|Dir/N" \
> +			"New|Dir/New&File.c" &&	# Bash will turn this into
> +						# "New\|Dir/New\&File.c "
> +	test_completion "git test-path-comp New\\|Dir/New\\&F" \
> +			"New|Dir/New&File.c"
> +'

This fails with:

2018-04-18T11:12:55.0436371Z expecting success: 
2018-04-18T11:12:55.0436665Z 	test_when_finished "rm -rf \"New|Dir\"" &&
2018-04-18T11:12:55.0436799Z 	mkdir "New|Dir" &&
2018-04-18T11:12:55.0436904Z 	>"New|Dir/New&File.c" &&
2018-04-18T11:12:55.0436972Z 
2018-04-18T11:12:55.0437158Z 	test_completion "git test-path-comp N" \
2018-04-18T11:12:55.0437296Z 			"New|Dir" &&	# Bash will turn this into "New\|Dir/"
2018-04-18T11:12:55.0437413Z 	test_completion "git test-path-comp New\\|D" \
2018-04-18T11:12:55.0437522Z 			"New|Dir" &&
2018-04-18T11:12:55.0437629Z 	test_completion "git test-path-comp New\\|Dir/N" \
2018-04-18T11:12:55.0437767Z 			"New|Dir/New&File.c" &&	# Bash will turn this into
2018-04-18T11:12:55.0438040Z 						# "New\|Dir/New\&File.c "
2018-04-18T11:12:55.0438152Z 	test_completion "git test-path-comp New\\|Dir/New\\&F" \
2018-04-18T11:12:55.0438504Z 			"New|Dir/New&File.c"
2018-04-18T11:12:55.0438742Z 
2018-04-18T11:12:55.0590984Z ++ test_when_finished 'rm -rf "New|Dir"'
2018-04-18T11:12:55.0591722Z ++ test 0 = 0
2018-04-18T11:12:55.0592001Z ++ test_cleanup='{ rm -rf "New|Dir"
2018-04-18T11:12:55.0592290Z 		} && (exit "$eval_ret"); eval_ret=$?; :'
2018-04-18T11:12:55.0592472Z ++ mkdir 'New|Dir'
2018-04-18T11:12:55.0717255Z ++ test_completion 'git test-path-comp N' 'New|Dir'
2018-04-18T11:12:55.0717680Z ++ test 2 -gt 1
2018-04-18T11:12:55.0718062Z ++ printf '%s\n' 'New|Dir'
2018-04-18T11:12:55.0718275Z ++ run_completion 'git test-path-comp N'
2018-04-18T11:12:55.0718447Z ++ local -a COMPREPLY _words
2018-04-18T11:12:55.0718631Z ++ local _cword
2018-04-18T11:12:55.0718806Z ++ _words=($1)
2018-04-18T11:12:55.0718965Z ++ test N = ' '
2018-04-18T11:12:55.0719124Z ++ ((  _cword = 3 - 1  ))
2018-04-18T11:12:55.0719286Z ++ __git_wrap__git_main
2018-04-18T11:12:55.0719467Z ++ __git_func_wrap __git_main
2018-04-18T11:12:55.0719633Z ++ local cur words cword prev
2018-04-18T11:12:55.0719801Z ++ _get_comp_words_by_ref -n =: cur words cword prev
2018-04-18T11:12:55.0720074Z ++ '[' 6 -gt 0 ']'
2018-04-18T11:12:55.0720239Z ++ case "$1" in
2018-04-18T11:12:55.0720406Z ++ shift
2018-04-18T11:12:55.0720584Z ++ '[' 5 -gt 0 ']'
2018-04-18T11:12:55.0720742Z ++ case "$1" in
2018-04-18T11:12:55.0720899Z ++ shift
2018-04-18T11:12:55.0721054Z ++ '[' 4 -gt 0 ']'
2018-04-18T11:12:55.0721240Z ++ case "$1" in
2018-04-18T11:12:55.0721392Z ++ cur=N
2018-04-18T11:12:55.0721547Z ++ shift
2018-04-18T11:12:55.0721717Z ++ '[' 3 -gt 0 ']'
2018-04-18T11:12:55.0721879Z ++ case "$1" in
2018-04-18T11:12:55.0722040Z ++ words=("${_words[@]}")
2018-04-18T11:12:55.0722201Z ++ shift
2018-04-18T11:12:55.0722396Z ++ '[' 2 -gt 0 ']'
2018-04-18T11:12:55.0722931Z ++ case "$1" in
2018-04-18T11:12:55.0723070Z ++ cword=2
2018-04-18T11:12:55.0723221Z ++ shift
2018-04-18T11:12:55.0723357Z ++ '[' 1 -gt 0 ']'
2018-04-18T11:12:55.0723575Z ++ case "$1" in
2018-04-18T11:12:55.0723735Z ++ prev=test-path-comp
2018-04-18T11:12:55.0723874Z ++ shift
2018-04-18T11:12:55.0724009Z ++ '[' 0 -gt 0 ']'
2018-04-18T11:12:55.0724397Z ++ __git_main
2018-04-18T11:12:55.0724984Z ++ local i c=1 command __git_dir __git_repo_path
2018-04-18T11:12:55.0725183Z ++ local __git_C_args C_args_count=0
2018-04-18T11:12:55.0725353Z ++ '[' 1 -lt 2 ']'
2018-04-18T11:12:55.0725537Z ++ i=test-path-comp
2018-04-18T11:12:55.0725712Z ++ case "$i" in
2018-04-18T11:12:55.0725882Z ++ command=test-path-comp
2018-04-18T11:12:55.0726057Z ++ break
2018-04-18T11:12:55.0726270Z ++ '[' -z test-path-comp ']'
2018-04-18T11:12:55.0726446Z ++ __git_complete_command test-path-comp
2018-04-18T11:12:55.0726621Z ++ local command=test-path-comp
2018-04-18T11:12:55.0726816Z ++ local completion_func=_git_test_path_comp
2018-04-18T11:12:55.0726992Z ++ declare -f _git_test_path_comp
2018-04-18T11:12:55.0727353Z ++ declare -f _git_test_path_comp
2018-04-18T11:12:55.0727547Z ++ _git_test_path_comp
2018-04-18T11:12:55.0727716Z ++ __git_complete_index_file --others
2018-04-18T11:12:55.0727890Z ++ local dequoted_word pfx= cur_
2018-04-18T11:12:55.0728234Z ++ __git_dequote N
2018-04-18T11:12:55.0728418Z ++ local rest=N len ch
2018-04-18T11:12:55.0728869Z ++ dequoted_word=
2018-04-18T11:12:55.0729020Z ++ test -n N
2018-04-18T11:12:55.0729152Z ++ len=0
2018-04-18T11:12:55.0729309Z ++ dequoted_word=N
2018-04-18T11:12:55.0729440Z ++ rest=
2018-04-18T11:12:55.0729666Z ++ case "${rest:0:1}" in
2018-04-18T11:12:55.0729822Z ++ test -n ''
2018-04-18T11:12:55.0729993Z ++ case "$dequoted_word" in
2018-04-18T11:12:55.0730133Z ++ cur_=N
2018-04-18T11:12:55.0782504Z +++ __git_index_files --others '' N
2018-04-18T11:12:55.0782805Z +++ local root= match=N
2018-04-18T11:12:55.0845235Z +++ __git_ls_files_helper '' --others N
2018-04-18T11:12:55.0845440Z +++ '[' --others == --committable ']'
2018-04-18T11:12:55.0845567Z +++ __git -C '' -c core.quotePath=false ls-files --exclude-standard --others -- 'N*'
2018-04-18T11:12:55.0845706Z +++ git -C '' -c core.quotePath=false ls-files --exclude-standard --others -- 'N*'
2018-04-18T11:12:55.0907632Z +++ awk -F / -v pfx= '{
2018-04-18T11:12:55.0907806Z 		paths[$1] = 1
2018-04-18T11:12:55.0908985Z 	}
2018-04-18T11:12:55.0942839Z 	END {
2018-04-18T11:12:55.0943072Z 		for (p in paths) {
2018-04-18T11:12:55.0949175Z 			if (substr(p, 1, 1) != "\"") {
2018-04-18T11:12:55.0949458Z 				# No special characters, easy!
2018-04-18T11:12:55.0949659Z 				print pfx p
2018-04-18T11:12:55.0949823Z 				continue
2018-04-18T11:12:55.0949999Z 			}
2018-04-18T11:12:55.0950121Z 
2018-04-18T11:12:55.0950335Z 			# The path is quoted.
2018-04-18T11:12:55.0950829Z 			p = dequote(p)
2018-04-18T11:12:55.0951171Z 			if (p == "")
2018-04-18T11:12:55.0951555Z 				continue
2018-04-18T11:12:55.0951672Z 
2018-04-18T11:12:55.0951856Z 			# Even when a directory name itself does not contain
2018-04-18T11:12:55.0952038Z 			# any special characters, it will still be quoted if
2018-04-18T11:12:55.0952213Z 			# any of its (stripped) trailing path components do.
2018-04-18T11:12:55.0952407Z 			# Because of this we may have seen the same direcory
2018-04-18T11:12:55.0952583Z 			# both quoted and unquoted.
2018-04-18T11:12:55.0952762Z 			if (p in paths)
2018-04-18T11:12:55.0952948Z 				# We have seen the same directory unquoted,
2018-04-18T11:12:55.0953117Z 				# skip it.
2018-04-18T11:12:55.0953276Z 				continue
2018-04-18T11:12:55.0953441Z 			else
2018-04-18T11:12:55.0953613Z 				print pfx p
2018-04-18T11:12:55.0953766Z 		}
2018-04-18T11:12:55.0953914Z 	}
2018-04-18T11:12:55.0954461Z 	function dequote(p,    bs_idx, out, esc, esc_idx, dec) {
2018-04-18T11:12:55.0954650Z 		# Skip opening double quote.
2018-04-18T11:12:55.0954813Z 		p = substr(p, 2)
2018-04-18T11:12:55.0954935Z 
2018-04-18T11:12:55.0955237Z 		# Interpret backslash escape sequences.
2018-04-18T11:12:55.0955415Z 		while ((bs_idx = index(p, "\\")) != 0) {
2018-04-18T11:12:55.0955533Z 			out = out substr(p, 1, bs_idx - 1)
2018-04-18T11:12:55.0955638Z 			esc = substr(p, bs_idx + 1, 1)
2018-04-18T11:12:55.0955743Z 			p = substr(p, bs_idx + 2)
2018-04-18T11:12:55.0955830Z 
2018-04-18T11:12:55.0955939Z 			if ((esc_idx = index("abtvfr\"\\", esc)) != 0) {
2018-04-18T11:12:55.0956079Z 				# C-style one-character escape sequence.
2018-04-18T11:12:55.0956513Z 				out = out substr("\a\b\t\v\f\r\"\\",
2018-04-18T11:12:55.0956631Z esc_idx, 1)
2018-04-18T11:12:55.0956745Z 			} else if (esc == "n") {
2018-04-18T11:12:55.0956853Z 				# Uh-oh, a newline character.
2018-04-18T11:12:55.0956973Z 				# We cant reliably put a pathname
2018-04-18T11:12:55.0957086Z 				# containing a newline into COMPREPLY,
2018-04-18T11:12:55.0957193Z 				# and the newline would create a mess.
2018-04-18T11:12:55.0957300Z 				# Skip this path.
2018-04-18T11:12:55.0957413Z 				return ""
2018-04-18T11:12:55.0957510Z 			} else {
2018-04-18T11:12:55.0957808Z 				# Must be a \nnn octal value, then.
2018-04-18T11:12:55.0958070Z 				dec = esc * 64 + \
2018-04-18T11:12:55.0958184Z 				      substr(p, 1, 1) * 8  + \
2018-04-18T11:12:55.0958274Z 				      substr(p, 2, 1)
2018-04-18T11:12:55.0958369Z 				out = out sprintf("%c", dec)
2018-04-18T11:12:55.0958587Z 				p = substr(p, 3)
2018-04-18T11:12:55.0958692Z 			}
2018-04-18T11:12:55.0958769Z 		}
2018-04-18T11:12:55.0958862Z 		# Drop closing double quote, if there is one.
2018-04-18T11:12:55.0958969Z 		# (There isnt any if this is a directory, as it was
2018-04-18T11:12:55.0959153Z 		# already stripped with the trailing path components.)
2018-04-18T11:12:55.0959256Z 		if (substr(p, length(p), 1) == "\"")
2018-04-18T11:12:55.0959356Z 			out = out substr(p, 1, length(p) - 1)
2018-04-18T11:12:55.0959441Z 		else
2018-04-18T11:12:55.0959541Z 			out = out p
2018-04-18T11:12:55.0959598Z 
2018-04-18T11:12:55.0959682Z 		return out
2018-04-18T11:12:55.0959763Z 	}'
2018-04-18T11:12:55.1182135Z ++ __gitcomp_file_direct $'New∩\201╝Dir'
2018-04-18T11:12:55.1182355Z ++ local 'IFS=
2018-04-18T11:12:55.1182439Z '
2018-04-18T11:12:55.1182518Z ++ COMPREPLY=($1)
2018-04-18T11:12:55.1182622Z ++ compopt -o filenames +o nospace
2018-04-18T11:12:55.1182877Z ++ compgen -f /non-existing-dir/
2018-04-18T11:12:55.1182979Z ++ return 0
2018-04-18T11:12:55.1183055Z ++ return
2018-04-18T11:12:55.1183147Z ++ print_comp
2018-04-18T11:12:55.1183224Z ++ local 'IFS=
2018-04-18T11:12:55.1183300Z '
2018-04-18T11:12:55.1183398Z ++ echo $'New∩\201╝Dir'
2018-04-18T11:12:55.1183508Z ++ sort out
2018-04-18T11:12:55.1183605Z ++ /usr/bin/sort out
2018-04-18T11:12:55.1306331Z ++ test_cmp expected out_sorted
2018-04-18T11:12:55.1306825Z ++ mingw_test_cmp expected out_sorted
2018-04-18T11:12:55.1307024Z ++ local test_cmp_a= test_cmp_b=
2018-04-18T11:12:55.1307233Z ++ local stdin_for_diff=
2018-04-18T11:12:55.1307401Z ++ test -s expected
2018-04-18T11:12:55.1307568Z ++ test -s out_sorted
2018-04-18T11:12:55.1307742Z ++ mingw_read_file_strip_cr_ test_cmp_a
2018-04-18T11:12:55.1308083Z ++ local line
2018-04-18T11:12:55.1308424Z ++ :
2018-04-18T11:12:55.1308566Z ++ IFS=$'\r'
2018-04-18T11:12:55.1308717Z ++ read -r -d '
2018-04-18T11:12:55.1308852Z ' line
2018-04-18T11:12:55.1317521Z ++ line='New|Dir
2018-04-18T11:12:55.1317784Z '
2018-04-18T11:12:55.1318257Z ++ eval 'test_cmp_a=$test_cmp_a$line'
2018-04-18T11:12:55.1318424Z +++ test_cmp_a='New|Dir
2018-04-18T11:12:55.1318569Z '
2018-04-18T11:12:55.1318724Z ++ :
2018-04-18T11:12:55.1318871Z ++ IFS=$'\r'
2018-04-18T11:12:55.1319027Z ++ read -r -d '
2018-04-18T11:12:55.1319170Z ' line
2018-04-18T11:12:55.1319334Z ++ test -z ''
2018-04-18T11:12:55.1319476Z ++ break
2018-04-18T11:12:55.1319628Z ++ mingw_read_file_strip_cr_ test_cmp_b
2018-04-18T11:12:55.1319797Z ++ local line
2018-04-18T11:12:55.1319939Z ++ :
2018-04-18T11:12:55.1320081Z ++ IFS=$'\r'
2018-04-18T11:12:55.1320240Z ++ read -r -d '
2018-04-18T11:12:55.1320384Z ' line
2018-04-18T11:12:55.1320555Z ++ line='NewDir
2018-04-18T11:12:55.1320915Z '
2018-04-18T11:12:55.1321099Z ++ eval 'test_cmp_b=$test_cmp_b$line'
2018-04-18T11:12:55.1321266Z +++ test_cmp_b='NewDir
2018-04-18T11:12:55.1321422Z '
2018-04-18T11:12:55.1321570Z ++ :
2018-04-18T11:12:55.1321705Z ++ IFS=$'\r'
2018-04-18T11:12:55.1321859Z ++ read -r -d '
2018-04-18T11:12:55.1321994Z ' line
2018-04-18T11:12:55.1322219Z ++ test -z ''
2018-04-18T11:12:55.1322361Z ++ break
2018-04-18T11:12:55.1322497Z ++ test -n 'New|Dir
2018-04-18T11:12:55.1322649Z '
2018-04-18T11:12:55.1322828Z ++ test -n 'NewDir
2018-04-18T11:12:55.1322977Z '
2018-04-18T11:12:55.1323109Z ++ test 'New|Dir
2018-04-18T11:12:55.1323397Z ' = 'NewDir
2018-04-18T11:12:55.1323540Z '
2018-04-18T11:12:55.1323680Z ++ eval 'diff -u "$@" '
2018-04-18T11:12:55.1323840Z +++ diff -u expected out_sorted
2018-04-18T11:12:55.1454977Z --- expected	2018-04-18 11:12:55.065444100 +0000
2018-04-18T11:12:55.1455785Z error: last command exited with $?=1
2018-04-18T11:12:55.1456722Z +++ out_sorted	2018-04-18 11:12:55.127568400 +0000
2018-04-18T11:12:55.1457211Z @@ -1 +1 @@
2018-04-18T11:12:55.1457408Z -New|Dir
2018-04-18T11:12:55.1457752Z +NewDir
2018-04-18T11:12:55.1457975Z not ok 111 - complete files - escaped characters on cmdline
2018-04-18T11:12:55.1645995Z #	
2018-04-18T11:12:55.1646221Z #		test_when_finished "rm -rf \"New|Dir\"" &&
2018-04-18T11:12:55.1646380Z #		mkdir "New|Dir" &&
2018-04-18T11:12:55.1646487Z #		>"New|Dir/New&File.c" &&
2018-04-18T11:12:55.1646583Z #	
2018-04-18T11:12:55.1646865Z #		test_completion "git test-path-comp N" \
2018-04-18T11:12:55.1646986Z #				"New|Dir" &&	# Bash will turn this into "New\|Dir/"
2018-04-18T11:12:55.1647108Z #		test_completion "git test-path-comp New\\|D" \
2018-04-18T11:12:55.1647212Z #				"New|Dir" &&
2018-04-18T11:12:55.1647346Z #		test_completion "git test-path-comp New\\|Dir/N" \
2018-04-18T11:12:55.1647510Z # "New|Dir/New&File.c" &&	# Bash will turn this into
2018-04-18T11:12:55.1647636Z # # "New\|Dir/New\&File.c "
2018-04-18T11:12:55.1647775Z #		test_completion "git test-path-comp New\\|Dir/New\\&F" \
2018-04-18T11:12:55.1647886Z # "New|Dir/New&File.c"

I suspect that the culprit is once again Cygwin's trick where illegal
characters are mapped into a private Unicode page. Cygwin (and therefore
MSYS2 runtime, and therefore the Bash used to run the test script) can use
those filenames all right, but Git cannot.

So even testing whether you could write an illegal file name via shell
script is *not* enough to determine whether the file system supports funny
characters.

As far as I can tell from a *really* cursory glance, this is the only
affected test case. Apparently your prereq catches, somehow, on Windows:

2018-04-18T11:12:43.0459702Z     Your filesystem does not allow \ and " in filenames.
2018-04-18T11:12:43.0459823Z     skipped: complete files - C-style escapes in ls-files output (missing FUNNYNAMES_BS_DQ)

Ciao,
Dscho

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux