Re: PATCH simuleneous download for packages

[Tim Lauridsen <tim.lauridsen@xxxxxxxxxxxxxx>]

On 04/26/2009 09:28 PM, pranjal pandit wrote:

Hello,
The patch attached performs simultaneous downloads of packages by
creating two different urlgrab objects and passing them different
package names to download.

The download is fine but there is a problem in display in case of
simultaneous downloads of  more than one packages. The display
percentages and download speed is shown incorrect. The progress object
used is of class TextMeter() from the module urlgrabber.progress.

In case of a single package download, the code works perfectly fine.

--Regards
--Pranjal Pandit
Computer Science Engineering Department
NIT HAMIRPUR
  

diff --git a/cli.py b/cli.py
index 0374667..cb8507c 100644
--- a/cli.py
+++ b/cli.py
@@ -25,9 +25,9 @@ import sys
 import time
 import random
 import logging
+import threading
 from optparse import OptionParser
 import rpm
-
 import output
 import shell
 import yum
@@ -62,7 +62,7 @@ class CliError(yum.Errors.YumBaseError):
         yum.Errors.YumBaseError.__init__(self)
         self.args = args
 
-class YumBaseCli(yum.YumBase, output.YumOutput):
+class YumBaseCli(yum.YumBase, output.YumOutput,threading.Thread):
     """This is the base class for yum cli.
        Inherits from yum.YumBase and output.YumOutput """
        
@@ -98,7 +98,11 @@ class YumBaseCli(yum.YumBase, output.YumOutput):
         self.registerCommand(yumcommands.RepoListCommand())
         self.registerCommand(yumcommands.HelpCommand())
         self.registerCommand(yumcommands.ReInstallCommand())        
-
+        self.helpDownloadPkgs=yum.HDP()
  

yum.HDP() look very ugly to me, use something longer and more descriptive in mixed Case (MyDammUsefulClass)

+        self.yB=yum.YumBase()
  

Why do you define a yum.yumBase() class inside a class there inherit from yum.yumBase, you can just use self

+        threading.Thread.__init__(self)
+    def run(self):
+        self.yB.downloadPkgs2(self.helpDownloadPkgs)
     def registerCommand(self, command):
         for name in command.getNames():
             if self.yum_cli_commands.has_key(name):
@@ -396,8 +400,22 @@ class YumBaseCli(yum.YumBase, output.YumOutput):
 
         self.verbose_logger.log(yum.logginglevels.INFO_2,
             _('Downloading Packages:'))
-        problems = self.downloadPkgs(downloadpkgs, callback_total=self.download_callback_total_cb) 
-
+        
+        self.yB.downloadPkgs1(downloadpkgs,self.helpDownloadPkgs)
+
+        if self.yB.numberOfPackages() is 1:
+            self.yB.downloadPkgs2(self.helpDownloadPkgs)
+        else:    
+            call1=YumBaseCli()
+            call2=YumBaseCli()
  

here you make 2 new instances if the class you already are inside, very confusing and a memory nightmare
now you have 3 YumBaseCli classes and 3 Extra YumBase clasess running.

+            cl=[]
+            cl.append(call1)
+            cl.append(call2)
+            call1.start()
+            call2.start()
+            for c in cl:
+                c.join()
+        problems = self.yB.downloadPkgs3()
         if len(problems) > 0:
             errstring = ''
             errstring += _('Error Downloading Packages:\n')
diff --git a/po/yum.pot b/po/yum.pot
index 9e01e13..c868dc9 100644
--- a/po/yum.pot
+++ b/po/yum.pot
  

yumex.pot is a genratend files, no need to send patches against that.

There is a lot of other issues, it makes the code very confusing. :(

I need to be convinced that it is worth the effort.

I think you are attacking the "problem" in the wrong way. you should try making a plugin instead there is downloading
the packages multi threaded in a "predownload" hook. The you need no changes to yumbase and yum will just skip the downloading because all the packages is in the cache already and continue with the transaction. You don't even need to
use urlgrapper, you can use something else like pycurl, i think that i already supports multiple downloads, so you dont need to use python threads :)
When it is rocking then it might be an idea to itegrate it into yumbase, but it in lower level than you current attempt.
So it can be useful to Yum API users also and not just yum cli users.

Hope that i don't sound to refusing, it was not my intention :)

Tim

_______________________________________________
Yum mailing list
Yum@xxxxxxxxxxxxxxxxx
http://lists.baseurl.org/mailman/listinfo/yum