On Mon, Jul 28, 2003 at 10:02:51AM +0400, Grigory Bakunov wrote: > On ðÎÄ, 2003-07-28 at 04:00, Michael Stenner wrote: > > OK, I have a new urlgrabber.py. Here's what's changed: > > > > * user-agent support in both 1.x series and 2.x series (it was in 2.x > > but I changed how it is done). > > * auth support for http[s] in the 2.x series. > > > Wow! Nice-Nice-Nice! Thanks :) > How about to add 'reget' support for both http and ftp? I'll look into that. > And i think it's a good idea to make separate project 'urlgrabber' > cause not only yum project need a good downloader. Hehe. That was the main reason I changed the User-agent code. It was previously hardcoded as "yum/2.x" in urlgrabber.py. It is now settable via a function that you call immediately after importing so the calling program can make it whatever it likes. Perhaps I'll set it up as a miniproject (http://linux.duke.edu/projects/). The good/bad about urlgrabber: good: * it acts like a simple stateless file retriever - no real setup or teardown is required (although it might be polite to close keepalive connections) * all urls are treated identically - ftp, http, https, file * support for conditional retries, progress meters, throttling, http keepalive (with the keepalive module) bad: * certain types of downloads just don't work: dynamic data mostly, but that includes directory listings, etc. This may be surmountable, but I haven't thought about it much because it's ok for yum. > Thanks! Glad you like it :) -Michael -- Michael Stenner Office Phone: 919-660-2513 Duke University, Dept. of Physics mstenner@xxxxxxxxxxxx Box 90305, Durham N.C. 27708-0305
