hi Ralph,
thank you for that clarification.
the function works a little faster with them.
file_info(){ echo -n ${1:=/dev/stdin}$'\t'; ( tee < "${1}" >( file
--mime-type -b -e compress -e tar -e elf - >&3; cat >/dev/null ) >( md5sum
>&3 ) >( sha1sum >&3 ) >/dev/null; ) 3>&1 | tr '\n' '\t'; echo; }
pv big.tar > /dev/null
1,71GiO 0:00:00 [5,32GiB/s]
[==============================================================================================================>]
100%
time ( for i in $( seq 24 ); do file_info big.tar ; done )
big.tar application/x-gtar 53f0d0240e5ddc94266087ec96ebb802236fa0bc
- 6989542fabd98b04086524d1106b7907 -
...
big.tar application/x-gtar 53f0d0240e5ddc94266087ec96ebb802236fa0bc
- 6989542fabd98b04086524d1106b7907 -
real 3m2,712s
user 0m14,988s
sys 1m13,303s
file_info(){ echo -n ${1:=/dev/stdin}$'\t'; ( trap "" pipe; tee < "${1}" >(
md5sum >&3 ) >( sha1sum >&3 ) | file --mime-type -b -e compress -e tar -e
elf - ) 3>&1 | tr '\n' '\t'; echo; }
pv big.tar > /dev/null
1,71GiO 0:00:00 [5,37GiB/s]
[==============================================================================================================>]
100%
time ( for i in $( seq 24 ); do file_info big.tar ; done )
big.tar application/x-gtar 53f0d0240e5ddc94266087ec96ebb802236fa0bc
- 6989542fabd98b04086524d1106b7907 -
...
big.tar application/x-gtar 53f0d0240e5ddc94266087ec96ebb802236fa0bc
- 6989542fabd98b04086524d1106b7907 -
real 2m36,013s
user 0m9,349s
sys 0m50,257s
Le jeu. 19 déc. 2019 à 11:59, Ralph Corderoy <ralph@xxxxxxxxxxxxxxx> a
écrit :
> Hi Pascal,
>
> > file_info(){
> > echo -n ${1:=/dev/stdin}$'\t'
> > (
> > tee < "${1}" \
> > >( file --mime-type -b -e compress -e tar -e elf - >&3 ) \
> > >( md5sum >&3 ) \
> > >( sha1sum >&3 ) \
> > >/dev/null
> > ) 3>&1 |
> > tr '\n' '\t'
> > echo
> > }
> >
> > it no longer works because the data flow is quickly interrupted by tee
> > which does not consume all the data.
>
> You're missing the reason why. tee(1) receives a SIGPIPE because it
> writes to a pipe that's closed. Adding a cat(1) is a waste of CPU, as
> is discarding tee's stdout instead of using it for one of the workers.
>
> Examine these differences.
>
> $ seq 31415 | wc
> 31415 31415 177384
> $ seq 31415 | tee >(sed q) >(wc) > >(tr -d 42 | wc); sleep 1
> 1
> 14139 14109 62130
> 12773 12774 65536
> $ seq 31415 | (trap '' pipe; tee >(sed q) >(wc) > >(tr -d 42 | wc));
> sleep 1
> 1
> 31415 31369 142504
> 31415 31415 177384
> $
>
> Note the output of the commands can be in any order, and intermingle if
> they're long enough.
>
> tee(1) has -p and --output-error but they're not as specific as stating
> SIGPIPE is expected for just one worker.
>
> --
> Cheers, Ralph.
>
