Re: expand: Do not quote backslashes in unquoted parameter expansion

[Jilles Tjoelker <jilles@xxxxxxxx>]

On Tue, Mar 27, 2018 at 08:19:19PM +0800, Herbert Xu wrote:
> On Tue, Mar 27, 2018 at 01:07:21PM +0200, Harald van Dijk wrote:

> > If POSIX specifies a result, and a shell applies an optimisation that causes
> > a different result to be produced, doesn't that inherently disallow that
> > optimisation? There may be some confusion and/or disagreement over what
> > exactly POSIX specifies and/or intends to specify, but I don't see how you
> > can argue that POSIX specifies result A, but it's okay to apply an
> > optimisation that produces result B.

> Wait, you're talking about something completely different.  The
> crux of the issue is whether POSIX allows the backslash to escape
> the character after it or does it have to be a literal backslash.

> In fact POSIX is perfectly clear on this:

> 	2.13.1 Patterns Matching a Single Character

> 	The following patterns matching a single character shall match a
> 	single character: ordinary characters, special pattern characters,
> 	and pattern bracket expressions. The pattern bracket expression
> 	also shall match a single collating element. A <backslash>
> 	character shall escape the following character. The escaping
> 	<backslash> shall be discarded. If a pattern ends with an
> 	unescaped <backslash>, it is unspecified whether the pattern
> 	does not match anything or the pattern is treated as invalid.

> Nowhere does it say that backslashes that come from the result of
> parameter expansion is always literal.

> So it's clear that any shell that treats the backslash as a literal
> backslash is already broken.

I don't think it is clear at all. Note the final paragraph of 2.13.1:

] When pattern matching is used where shell quote removal is not
] performed (such as in the argument to the find -name primary when find
] is being called using one of the exec functions as defined in the
] System Interfaces volume of POSIX.1-2008, or in the pattern argument
] to the fnmatch() function), special characters can be escaped to
] remove their special meaning by preceding them with a <backslash>
] character. This escaping <backslash> is discarded. The sequence "\\"
] represents one literal <backslash>. All of the requirements and
] effects of quoting on ordinary, shell special, and special pattern
] characters shall apply to escaping in this context.

This implies that special characters cannot be escaped using a backslash
in a context where shell quote removal is performed. Instead, special
characters can be escaped using shell quoting. As a result, the simplest
form of parameter expansion has either all or none of the generated
characters quoted (depending on whether the expansion is in
double-quotes or not).

There is also a sentence "The shell special characters always require
quoting." which seems untrue in practice if the characters come from an
expansion. Something like

  touch 'a&b'
  sh -c 'w=*\&*; printf "%s\n" $w'

works for many shells as sh. However, this could be explained away as
undefined behaviour.

Furthermore, POSIX generally intends to specify the behaviour of the
Bourne shell and ksh88. If the Bourne shell and ksh88 both agree, and do
not match an interpretation of POSIX, then it is likely that the
interpretation is wrong or the standard text has a bug.

> > Can you show me any shell other than bash that lets this
> > optimisation affect the results?

> The fact is that the other shells are not doing what they do because
> they're not doing this optimisation.  They do what they do because
> they have broken POSIX because they always treat backslashes that
> arise from parameter expansion as literal backslashes.

Let's use clear terminology: if it affects behaviour, it is by
definition not an optimization. It is a special case for words not
containing special pattern characters (for some value of "special
pattern characters").

-- 
Jilles Tjoelker
--
To unsubscribe from this list: send the line "unsubscribe dash" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html