On Sun, Apr 2, 2023 at 2:53 AM Nicolas Schier <nicolas@xxxxxxxxx> wrote:
>
> On Sat, Apr 01, 2023 at 11:55:44PM +0900 Masahiro Yamada wrote:
> > On Thu, Mar 30, 2023 at 5:02 AM Nicolas Schier <nicolas@xxxxxxxxx> wrote:
> > >
> > > On Sat 25 Mar 2023 23:19:09 GMT, Masahiro Yamada wrote:
> > > > 'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in
> > > > a dirty source tree. Handle symlinks properly, and also, keep the
> > > > executable permission.
> > > >
> > > > Fixes: 05e96e96a315 ("kbuild: use git-archive for source package creation")
> > > > Signed-off-by: Masahiro Yamada <masahiroy@xxxxxxxxxx>
> > > > ---
> > > >
> > > > scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++---------
> > > > 1 file changed, 27 insertions(+), 9 deletions(-)
> > > >
> > > > diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
> > > > index f842ab50a780..23551de92e1b 100755
> > > > --- a/scripts/package/gen-diff-patch
> > > > +++ b/scripts/package/gen-diff-patch
> > > > @@ -23,16 +23,34 @@ fi
> > > > git -C ${srctree} status --porcelain --untracked-files=all |
> > > > while read stat path
> > > > do
> > > > - if [ "${stat}" = '??' ]; then
> > > > -
> > > > - if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
> > > > - ! head -n1 .tmp_diff | grep -q "Binary files"; then
> > > > - {
> > > > - echo "--- /dev/null"
> > > > - echo "+++ linux/$path"
> > > > - cat .tmp_diff | tail -n +3
> > > > - } >> ${untracked_patch}
> > > > + if [ "${stat}" != '??' ]; then
> > > > + continue
> > > > + fi
> > > > +
> > > > + if [ -L "${path}" ]; then
> > > > + {
> > > > + echo "diff --git a/${path} b/${path}"
> > > > + echo "new file mode 120000"
> > > > + echo "--- /dev/null"
> > > > + echo "+++ b/$path"
> > > > + echo "@@ -0,0 +1 @@"
> > > > + printf "+"; readlink ${path}
> > >
> > > Better quote "${path}"?
> >
> >
> > Thanks for the suggestion.
> >
> > Quoting variables are correct in most cases.
> > But, that is not enough to generate a valid
> > patch when a file path contains spaces.
> >
> >
> >
> > 'git format-patch' produces a patch that
> > is accepted by GNU patch and also by dpkg-source.
> >
> > I learned a trick from GIT source code.
> >
> >
> > If you are interested, what GIT does [1].
> >
> >
> > [1] https://github.com/git/git/commit/1a9eb3b9d50367bee8fe85022684d812816fe531
>
> thanks for the pointer, that is really interesting!
>
> Kind regards,
> Nicolas
>
>
> > I will send v2 later, where I made some more efforts
> > to fix several corner cases even if that is not perfect.
I wrote a patch (attached to this email) but I am
still thinking if this is worth the complicated scripting...
Another approach would be to stop caring untracked files.
--
Best Regards
Masahiro Yamada
From 4b91ae3f482950209c356a98b79604394f16765c Mon Sep 17 00:00:00 2001
From: Masahiro Yamada <masahiroy@xxxxxxxxxx>
Date: Sat, 1 Apr 2023 03:57:31 +0900
Subject: [PATCH] kbuild: make package builds more robust
Running 'make deb-pkg' in a dirty repository (that is, 'git diff'
outputs something) has some problems.
[1] If a file path contains whitespaces, 'make deb-pkg' fails with
an error message like follows:
diff: ./scripts/package/../../"a b": No such file or directory
[snip]
dpkg-source: error: expected ^@@ at line 2 of diff ...
'git status --porcelain' double-quotes a file path that contains
whitespaces. Remove the surrounding double-quotes, and add TAB to the
'+++' line to make it a valid patch. This is the same as what GIT does.
See GIT commit 1a9eb3b9d503 ("git-diff/git-apply: make diff output a
bit friendlier to GNU patch (part 2)") for reference.
I gave up other special characters such as TAB, new line. It would be
possible to create a patch that GNU patch accepts by quoting the file
paths (and 'git diff' does this), but dpkg-source fails anyway.
[2] The executable bit is lost.
[3] Symbolic links are not handled.
Rewrite the code to make it work more correctly.
Signed-off-by: Masahiro Yamada <masahiroy@xxxxxxxxxx>
---
scripts/package/gen-diff-patch | 79 ++++++++++++++++++++++++++++------
1 file changed, 66 insertions(+), 13 deletions(-)
diff --git a/scripts/package/gen-diff-patch b/scripts/package/gen-diff-patch
index f842ab50a780..405b7d1925a5 100755
--- a/scripts/package/gen-diff-patch
+++ b/scripts/package/gen-diff-patch
@@ -21,24 +21,77 @@ if [ ! -s "${diff_patch}" ]; then
fi
git -C ${srctree} status --porcelain --untracked-files=all |
-while read stat path
+while read -r stat path
do
- if [ "${stat}" = '??' ]; then
-
- if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff &&
- ! head -n1 .tmp_diff | grep -q "Binary files"; then
- {
- echo "--- /dev/null"
- echo "+++ linux/$path"
- cat .tmp_diff | tail -n +3
- } >> ${untracked_patch}
- fi
+ if [ "${stat}" != '??' ]; then
+ continue
fi
-done
+
+ # path may be quoted if it contains whitespaces. Unquote it
+ path="${path#\"}"
+ path="${path%\"}"
+
+ case "${path}" in
+ *\\*)
+ # Give up escape sequences such as \n, \t.
+ # It is possible to create a patch by quoting file paths
+ # (and it is what GIT does) but dpkg-source fails with
+ # "... patches file with C-style encoded filename" error.
+ continue
+ ;;
+ *\ *)
+ # GNU patch expects TAB on ---/+++ lines for paths with spaces.
+ tab="$(printf '\t')"
+ ;;
+ *)
+ tab=
+ ;;
+ esac
+
+ if [ ! -f "${path}" ]; then
+ continue
+ fi
+
+ if [ -L "${path}" ]; then
+ echo "diff --git a/${path} b/${path}"
+ echo "new file mode 120000"
+ echo "--- /dev/null"
+ echo "+++ b/${path}${tab}"
+ echo "@@ -0,0 +1 @@"
+ printf "+"; readlink "${path}"
+ echo '\ No newline at end of file'
+ continue
+ fi
+
+ if [ -x "${path}" ]; then
+ mode=100755
+ else
+ mode=100644
+ fi
+
+ if [ ! -s "${path}" ]; then
+ echo "diff --git a/${path} b/${path}"
+ echo "new file mode ${mode}"
+ continue
+ fi
+
+ if diff -u /dev/null "${srctree}/${path}" > .tmp_diff || [ "$?" != 1 ]; then
+ continue
+ fi
+
+ if [ "$(head -n6 .tmp_diff)" = Binary ] ; then
+ continue
+ fi
+
+ echo "diff --git a/${path} b/${path}"
+ echo "new file mode ${mode}"
+ echo "--- /dev/null"
+ echo "+++ b/${path}${tab}"
+ cat .tmp_diff | tail -n +3
+done > "${untracked_patch}"
rm -f .tmp_diff
if [ ! -s "${diff_patch}" ]; then
rm -f "${diff_patch}"
- exit
fi
--
2.37.2
