On Friday 13 March 2009 03:50:42 Adrian LIkins wrote:
> makkalot wrote:
> > ********* CAUTION A VERY LONG POST ********************
> >
> > Func Facts :
> >
> > What is it ?
> > The Facts are modules like minion modules but a little bit different.
> > Facts are like variables of database tables.
> > name,temperature,kernel_version etc. Why to need facts ? The main idea
> > behind facts is to be able to send queries to minions like you do to
> > database or ORM. Example , 'run update method on machines which are f9 >=
> > ... You can do that with current minion modules of course but you have to
> > send at least 2 queries to your minions. Another idea behind fact is to
> > do more things with less connection hits.
> >
> > Python API :
> > To use Func Facts you can use OverlordQueryProxy (i dont like the name
> > if you have better please tell), as it name says it is a proxy object
> > which acts like
> > Overlord and a little bit more. When writing the facts code i didnt touch
> > the Overlord class because it is already overloaded, and i think that
> > should be the future behaviour of new added bits to func. Keeping
> > Overlord as a little bit abstract may be a good idea. Examples are
> > better:
>
> Subclassing the Overlord class is probably a good idea. At least for
> the common cases, it wouldn't hurt
> to have some canned subclasses ready to use.
>
> I'm drawing a blank on better names at the moment
cool.
>
> > #using object as Overlord
> > In [1]: from func.minion.facts.overlord_query import OverlordQueryProxy
> >
> > In [2]: fc = OverlordQueryProxy("*")
> >
> > In [3]: fc.echo.run_string("func blips")
> > Out[3]: {'localhost.localdomain': 'func blips'}
> >
> > #to see the availible fact methods i have written a little minion module
> > to show them ..
> > [root@fedorabig func]# func "*" call fact list_fact_modules
> > {'localhost.localdomain': ['hardware', 'fact_module']}
> >
> > [root@fedorabig func]# func "*" call fact list_fact_methods
> > {'localhost.localdomain': ['hardware.cpu_model',
> > 'kernel',
> > 'cpumodel',
> > 'hardware.kernel_version',
> > 'cpuvendor',
> > 'hardware.run_level',
> > 'hardware.cpu_vendor',
> > 'hardware.os_name',
> > 'runlevel',
> > 'os']}
> >
> > [root@fedorabig func]# func "*" call fact show_fact_module "hardware"
> > {'localhost.localdomain': {'description': 'A modules that supplies
> > hardware facts',
> > 'name': 'hardware',
> > 'version': '0.0.1'}}
> >
> >
> > [root@fedorabig func]# func "*" call fact show_fact_method "runlevel"
> > {'localhost.localdomain': {'description': 'Shows the runlevel of the
> > system', 'name': 'runlevel',
> > 'tag': 'runlevel',
> > 'usage': 'Can be used with all keywords'}}
>
> Sounds reasonable API to me, mirrors the normal func module/method
> introspection.
Well i put those just for little demonstration otherwise they are kind of
useless.
>
> > The facts are methods that mostly (at least now) doesnt accept any
> > arguments, so if you want to see the value of an existing fact just
> >
> > [root@fedorabig func]# func "*" call fact call_fact "runlevel"
> > {'localhost.localdomain': '5'}
>
> What would be an example of fact methods that need args?
Well i wanted to keep facts as simple as possible, it maybe more useful
if you think about them just as properties of the minion. Can u give me an,
example for sending arguments and why it will be useful.
>
> > The ORM queries:
> >
> > When Michael mentioned about facts idea i wanted to have something that
> > can let me do some crazy chaining queries so i now i should explain a
> > little bit the query stuff of the facts. With func facts you can run very
> > complex queries. But lets go from easaier to hard :
> >
> > In [3]: fc = OverlordQueryProxy("*")
> > In [6]: result=fc.filter(runlevel=4).echo.run_string("Hey func") #run
> > that on machine that has runlevel 4
> > In [8]: fc.display_active(result)
> > Out[8]: {}
> >
> > We have no results because my runlevel is different.
> >
> > In [11]: fc = OverlordQueryProxy("*")
> >
> > In [12]:
> > result=fc.filter(runlevel__gt=4,runlevel__lt=6).echo.run_string("Hey
> > func")
> >
> >
> > In [13]: fc.display_active(result)
> > Out[13]: {'localhost.localdomain': 'Hey func'}
> >
> > The query above is for facts that are greater than 4 and less than 5 :)
> > and we have a match
> >
> > Lets do the same thing with async :
> >
> > In [14]: fc = OverlordQueryProxy("*",async=True)
> >
> > In [15]:
> > result=fc.filter(runlevel__gt=4,runlevel__lt=6).echo.run_string("Hey
> > func")
> >
> > In [16]: result
> > Out[16]: '*-echo-run_string-1236893793.6649389'
> >
> > In [17]: fc.job_status(result)
> > Out[17]: (1, {'localhost.localdomain': 'Hey func'})
> >
> > Also you can do chaining when querying like that
> >
> > In [11]: fc = OverlordQueryProxy("*")
> >
> > In [12]: result =
> > fc.filter(runlevel=5).filter(os__icontains="fedora").echo.run_string("Hey
> > func")
> >
> > In [13]: fc.display_active(result)
> > Out[13]: {'localhost.localdomain': 'Hey func'}
>
> Whats the difference between passing two facts to check to filter,
> and chaining two fact checks together?
No difference wanted to show just the chaining.
>
> > Sometimes you may need some more complex queries by OR'in and 'AND'in the
> > stuff deeply , facts try to support that :
> >
> > In [20]: fc = OverlordQueryProxy("*")
> >
> > In [21]: result = fc.set_complexq(Q(os__icontains="fedora")|
> > Q(os__icontains="ubuntu")).echo.run_string("Hey func")
> >
> > In [23]: fc.display_active(result)
> > Out[23]: {'localhost.localdomain': 'Hey func'}
> >
> > A last example with heavy chaining :)
> >
> > In [24]: fc = OverlordQueryProxy("*")
> >
> > In [25]: result = fc.set_complexq(Q(os__icontains="fedora")|
> > Q(os__icontains="ubuntu")).filter(runlevel=5).echo.run_string("Hey func")
> >
> > In [26]: fc.display_active(result)
> > Out[26]: {'localhost.localdomain': 'Hey func'}
>
> Not sure what I think about the Q (query? ) api. My first thought is
> that maybe it should just
> be a little domain specific language for that, but thats probably a bad
> idea. The approach you
> take lets you construct the queries a little more programmatically than
> just query strings.
> What do the filter/queries look like over the wire? (shouldn't
> matter much, just curious)
Well the Q stuff was just an example what u can do programatically using
facts, most of the users wont need them and probably wont know about them.
If you take a look into filter code you will see that under the hood there are
Q classes. Therefore if you need something like filter_or() method it is very
easy to add a new one and probably that is the way it should be so the
statement above will become :
fc.filter_or(os__icontains="fedora",os__icontains="ubuntu").echo.run_string("Heyfunc")
>
> > Func Keywords :
> > Func Facts support some keywords that are really useful and makes the
> > world really easy (as you saw above). The "__" is very important when
> > using facts because that is how the things are parsed on other side. For
> > example if you want to have the os that contains name fedora You write
> >
> > fact_method__keyword = some_value --> so os__icontains = "fedora"
> >
> > The current keywords are as follow :
> > "","gt","gte","lt","lte",'contains','icontains','iexact','startswith'
> >
> > Adding new keywords is pretty easy just go to :
> > func.minion.facts.minion_query.QueryKeyword and add a method in that
> > format def keyword_nameOFyouKeyword(self,overlord_value,fact_value)
> > that is all you need to do.
>
> Cool.
>
> > Writing Fact Modules:
> > Fact modules are loaded as almost the same way as minion modules so
> > writing them is also that easy. Go to func.minion.facts.modules and add
> > your module there if there is no problems it should loaded next time you
> > restart your server. IMPORTANT : facts methods doesnt accept currently
> > any arguments, think about
> > them as properties of your system ...
> >
> > Example :
> >
> > class HardwareFacts(fact_module.BaseFactModule):
> >
> > version = "0.0.1"
> > description = "A modules that supplies hardware facts"
> >
> > ##snip snip ..
> >
> > def run_level(self):
> > """
> > The runlevel of the system
> > """
> > return str(self.host.defaultRunlevel)
> >
> > #for easier acces be creful should be unique
> > run_level.tag = "runlevel"
> > run_level.description = "Shows the runlevel of the system"
> > run_level.usage = "Can be used with all keywords"
> >
> > An important thing to note here is the run_level.tag value ,by assigning
> > that value you are able to use your method as tag_value__keyword = blip
> > if you dont assing the "tag" attribute it is ok again just when calling
> > you should do it that way : hardware.run_level__keyword = blip
> > Btw, i like the tag idea, be careful not to have 2 tags with same name,
> > the last one will be availible :)
>
> Hmm, kind of worried about the namespace issues there. Those kind of
> things tend to cause hard
> to find bugs.
It wont be a hard to add a method in minion fact module to
check_for_duplicates, so when someone add a new fact and has a
duplicate can catch that ?
>
> > For more info you can check the examples ...
> >
> >
> > Under The Hood :
> > Well how the facts is actually working? What i did was really simle if
> > you use OverlordQueryProxy and used the magical methods like filter and
> > set_complexq , OverlordQueryProxy serializes the current query (ORs and
> > ANDs) and inserts it as an argument to be sent to the minions. The server
> > side (minion)
> > deserializes it executes the logic parts (True and True etc) and if the
> > result is True it continues to executes the actual parts (minion methods)
> > if not returns back to overlord as signing __fact__ as False, so we know
> > that minion doesnt match our query. By doing that we can do lots of thing
> > with one connection or HIT.
>
> Cool.
>
> > Note : If you have lots of queries it may useful to run the query
> > "async" you may get timeout.
> >
> >
> > Todo :
> > -Solving bugs (fixing when writing that post :) )
> > - Implement it so it can work from commandline, not sure about syntax
> > - Maybe making Python Api more sweet
>
> I would like to see the api a little sweeter. Could we do something
> like implement __gt__, __lt__, etc
> on the filter/query objects? We still have a serializing step in the
> middle though. That may require implementing
> minion and overlord versions of the filter/query types though, which
> might make adding types on the fly
> a bit weird though.
Can u give me an example what kind of usage do you think about having
__gt__, __lt__ ? What will be the benefits ?
>
> Adrian
_______________________________________________
Func-list mailing list
Func-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/func-list
