In shell scripts it is sometimes useful to be able to read
exactly N bytes from a pipe. Doing this portably turns out
to be surprisingly difficult.
We want a solution that:
- is portable
- never reads more than N bytes due to buffering (which
would mean those bytes are not available to the next
program to read from the same pipe)
- handles partial reads by looping until N bytes or read
(or we see EOF)
- is resilient to stray signals giving us EINTR while
trying to read (even though we don't send them, things
like SIGWINCH could cause apparently-random failures)
Some possible solutions are:
- "head -c" is not portable, and implementations may
buffer (though GNU head does not)
- "read -N" is a bash-ism, and thus not portable
- "dd bs=$n count=1" does not handle partial reads. GNU dd
has iflags=fullblock, but that is not portable
- "dd bs=1 count=$n" fixes the partial read problem (all
reads are 1-byte, so there can be no partial response).
It does make a lot of write() calls, but for our tests
that's unlikely to matter. It's fairly portable. We
already use it in our tests, and it's unlikely that
implementations would screw up any of our criteria. The
most unknown one would be signal handling.
- perl can do a sysread() loop pretty easily. On my Linux
system, at least, it seems to restart the read() call
automatically. If that turns out not to be portable,
though, it would be easy for us to handle it.
That makes the perl solution the least bad (because we
conveniently omitted "length of code" as a criterion).
It's also what t9300 is currently using, so we can just pull
the implementation from there.
Signed-off-by: Jeff King <peff@xxxxxxxx>
---
t/t9300-fast-import.sh | 23 +++--------------------
t/test-lib-functions.sh | 14 ++++++++++++++
2 files changed, 17 insertions(+), 20 deletions(-)
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index 74d740d..2e0ba3e 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -7,23 +7,6 @@ test_description='test git fast-import utility'
. ./test-lib.sh
. "$TEST_DIRECTORY"/diff-lib.sh ;# test-lib chdir's into trash
-# Print $1 bytes from stdin to stdout.
-#
-# This could be written as "head -c $1", but IRIX "head" does not
-# support the -c option.
-head_c () {
- perl -e '
- my $len = $ARGV[1];
- while ($len > 0) {
- my $s;
- my $nread = sysread(STDIN, $s, $len);
- die "cannot read: $!" unless defined($nread);
- print $s;
- $len -= $nread;
- }
- ' - "$1"
-}
-
verify_packs () {
for p in .git/objects/pack/*.pack
do
@@ -2481,7 +2464,7 @@ test_expect_success PIPE 'R: copy using cat-file' '
read blob_id type size <&3 &&
echo "$blob_id $type $size" >response &&
- head_c $size >blob <&3 &&
+ test_copy_bytes $size >blob <&3 &&
read newline <&3 &&
cat <<-EOF &&
@@ -2524,7 +2507,7 @@ test_expect_success PIPE 'R: print blob mid-commit' '
EOF
read blob_id type size <&3 &&
- head_c $size >actual <&3 &&
+ test_copy_bytes $size >actual <&3 &&
read newline <&3 &&
echo
@@ -2559,7 +2542,7 @@ test_expect_success PIPE 'R: print staged blob within commit' '
echo "cat-blob $to_get" &&
read blob_id type size <&3 &&
- head_c $size >actual <&3 &&
+ test_copy_bytes $size >actual <&3 &&
read newline <&3 &&
echo deleteall
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 48884d5..90856d6 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -961,3 +961,17 @@ test_env () {
done
)
}
+
+# Read up to "$1" bytes (or to EOF) from stdin and write them to stdout.
+test_copy_bytes () {
+ perl -e '
+ my $len = $ARGV[1];
+ while ($len > 0) {
+ my $s;
+ my $nread = sysread(STDIN, $s, $len);
+ die "cannot read: $!" unless defined($nread);
+ print $s;
+ $len -= $nread;
+ }
+ ' - "$1"
+}
--
2.9.0.317.g65b4e7c
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html