On Fri, Aug 17, 2012 at 10:17:59AM -0400, Seth Vidal wrote:
>
>
>
> On Fri, 17 Aug 2012, Cris Rhea wrote:
>
> >
> >
> >As you suspected, urlgrabber has horrid performance compared to wget.
> >
> >What's the next step?
>
> Figure out what it is that's taking all the time/timeout in urlgrabber -
> strace is probably going to be your pal there.
Did show anything more interesting than the snip of strace I posted in
the first message. Here's the interesting part of strac'ing urlgrabber:
0.000111 gettimeofday({1345213462, 342323}, NULL) = 0
0.000235 clock_gettime(CLOCK_MONOTONIC, {147703, 456218722}) = 0
0.000105 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 5
0.000106 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)
0.000091 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
0.000090 connect(5, {sa_family=AF_INET, sin_port=htons(42898), sin_addr=inet_addr("129.176.212.87")}, 16) = -1 EINPROGRESS (Operation now in progress)
0.000146 clock_gettime(CLOCK_MONOTONIC, {147703, 456753378}) = 0
0.000099 poll([{fd=5, events=POLLOUT|POLLWRNORM}], 1, 299963) = 1 ([{fd=5, revents=POLLERR|POLLHUP}])
62.999460 getsockopt(5, SOL_SOCKET, SO_ERROR, [8589934702], [4]) = 0
0.000134 close(5) = 0
0.000118 clock_gettime(CLOCK_MONOTONIC, {147766, 456564196}) = 0
0.000120 sendto(4, "PASV\r\n", 6, MSG_NOSIGNAL, NULL, 0) = 6
0.000160 clock_gettime(CLOCK_MONOTONIC, {147766, 456846415}) = 0
0.000103 clock_gettime(CLOCK_MONOTONIC, {147766, 456945981}) = 0
0.000117 clock_gettime(CLOCK_MONOTONIC, {147766, 457064697}) = 0
0.000268 poll([{fd=4, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 3600000) = 1 ([{fd=4, revents=POLLIN|POLLRDNORM}])
0.000287 recvfrom(4, "227 Entering Passive Mode (129,1"..., 16384, 0, NULL,
I was HOPEFUL that this group would see this and say "That's a known bug
XYZ in PDQ package. The update is out in patch 124.23."
Clearly not a yum specific issue, if yum just uses urlgrabber....
> >PS: Don't want to go off-topic for this list, but why the comments on ftp?
>
> ftp is a pain to firewall for, most of the ftp daemons aren't actively
> maintained anymore, in general it is just pain.
>
> http servers are diverse, good and well-maintained. It is trivial to
> firewall for them and they have much more flexibility than ftp servers do.
>
> In general, I try to discourage folks from using ftp if they have a
> choice.
>
> -sv
I could see why some might prefer to use a web server for access to a
SW repo as in this case. There are still some things I can do faster/easier
and with better granularity/control using one of the better FTP packages
(e.g., vsftpd) than with a web server.
Thanks for your help and comments. Unless someone on this list has
info on the "smoking gun", I'll try to track down issues with urlgrabber.
--- Cris
--
Cristopher J. Rhea
Mayo Clinic - Research Computing Facility
200 First St SW, Rochester, MN 55905
crhea@xxxxxxxx
(507) 284-0587
_______________________________________________
Yum mailing list
Yum@xxxxxxxxxxxxxxxxx
http://lists.baseurl.org/mailman/listinfo/yum
