makkalot wrote:
********* CAUTION A VERY LONG POST ********************
Func Facts :
What is it ?
The Facts are modules like minion modules but a little bit different. Facts
are like variables of database tables. name,temperature,kernel_version etc.
Why to need facts ? The main idea behind facts is to be able to send queries
to minions like you do to database or ORM. Example , 'run update method on
machines which are f9 >= ... You can do that with current minion modules of
course but you have to send at least 2 queries to your minions. Another idea
behind fact is to do more things with less connection hits.
Python API :
To use Func Facts you can use OverlordQueryProxy (i dont like the name if
you have better please tell), as it name says it is a proxy object which acts
like
Overlord and a little bit more. When writing the facts code i didnt touch the
Overlord class because it is already overloaded, and i think that should be
the future behaviour of new added bits to func. Keeping Overlord as a little
bit abstract may be a good idea. Examples are better:
Subclassing the Overlord class is probably a good idea. At least for
the common cases, it wouldn't hurt
to have some canned subclasses ready to use.
I'm drawing a blank on better names at the moment.
#using object as Overlord
In [1]: from func.minion.facts.overlord_query import OverlordQueryProxy
In [2]: fc = OverlordQueryProxy("*")
In [3]: fc.echo.run_string("func blips")
Out[3]: {'localhost.localdomain': 'func blips'}
#to see the availible fact methods i have written a little minion module to
show them ..
[root@fedorabig func]# func "*" call fact list_fact_modules
{'localhost.localdomain': ['hardware', 'fact_module']}
[root@fedorabig func]# func "*" call fact list_fact_methods
{'localhost.localdomain': ['hardware.cpu_model',
'kernel',
'cpumodel',
'hardware.kernel_version',
'cpuvendor',
'hardware.run_level',
'hardware.cpu_vendor',
'hardware.os_name',
'runlevel',
'os']}
[root@fedorabig func]# func "*" call fact show_fact_module "hardware"
{'localhost.localdomain': {'description': 'A modules that supplies hardware
facts',
'name': 'hardware',
'version': '0.0.1'}}
[root@fedorabig func]# func "*" call fact show_fact_method "runlevel"
{'localhost.localdomain': {'description': 'Shows the runlevel of the system',
'name': 'runlevel',
'tag': 'runlevel',
'usage': 'Can be used with all keywords'}}
Sounds reasonable API to me, mirrors the normal func module/method
introspection.
The facts are methods that mostly (at least now) doesnt accept any arguments,
so if you want to see the value of an existing fact just
[root@fedorabig func]# func "*" call fact call_fact "runlevel"
{'localhost.localdomain': '5'}
What would be an example of fact methods that need args?
The ORM queries:
When Michael mentioned about facts idea i wanted to have something that can
let me do some crazy chaining queries so i now i should explain a little bit
the query stuff of the facts. With func facts you can run very complex queries.
But lets go from easaier to hard :
In [3]: fc = OverlordQueryProxy("*")
In [6]: result=fc.filter(runlevel=4).echo.run_string("Hey func") #run that on
machine that has runlevel 4
In [8]: fc.display_active(result)
Out[8]: {}
We have no results because my runlevel is different.
In [11]: fc = OverlordQueryProxy("*")
In [12]: result=fc.filter(runlevel__gt=4,runlevel__lt=6).echo.run_string("Hey
func")
In [13]: fc.display_active(result)
Out[13]: {'localhost.localdomain': 'Hey func'}
The query above is for facts that are greater than 4 and less than 5 :) and we
have a match
Lets do the same thing with async :
In [14]: fc = OverlordQueryProxy("*",async=True)
In [15]: result=fc.filter(runlevel__gt=4,runlevel__lt=6).echo.run_string("Hey
func")
In [16]: result
Out[16]: '*-echo-run_string-1236893793.6649389'
In [17]: fc.job_status(result)
Out[17]: (1, {'localhost.localdomain': 'Hey func'})
Also you can do chaining when querying like that
In [11]: fc = OverlordQueryProxy("*")
In [12]: result =
fc.filter(runlevel=5).filter(os__icontains="fedora").echo.run_string("Hey func")
In [13]: fc.display_active(result)
Out[13]: {'localhost.localdomain': 'Hey func'}
Whats the difference between passing two facts to check to filter,
and chaining two fact checks together?
Sometimes you may need some more complex queries by OR'in and 'AND'in the stuff
deeply , facts try to support that :
In [20]: fc = OverlordQueryProxy("*")
In [21]: result = fc.set_complexq(Q(os__icontains="fedora")|
Q(os__icontains="ubuntu")).echo.run_string("Hey func")
In [23]: fc.display_active(result)
Out[23]: {'localhost.localdomain': 'Hey func'}
A last example with heavy chaining :)
In [24]: fc = OverlordQueryProxy("*")
In [25]: result = fc.set_complexq(Q(os__icontains="fedora")|
Q(os__icontains="ubuntu")).filter(runlevel=5).echo.run_string("Hey func")
In [26]: fc.display_active(result)
Out[26]: {'localhost.localdomain': 'Hey func'}
Not sure what I think about the Q (query? ) api. My first thought is
that maybe it should just
be a little domain specific language for that, but thats probably a bad
idea. The approach you
take lets you construct the queries a little more programmatically than
just query strings.
What do the filter/queries look like over the wire? (shouldn't
matter much, just curious)
Func Keywords :
Func Facts support some keywords that are really useful and makes the world
really easy (as you saw above). The "__" is very important when using
facts because that is how the things are parsed on other side. For example if
you want to have the os that contains name fedora You write
fact_method__keyword = some_value --> so os__icontains = "fedora"
The current keywords are as follow :
"","gt","gte","lt","lte",'contains','icontains','iexact','startswith'
Adding new keywords is pretty easy just go to :
func.minion.facts.minion_query.QueryKeyword and add a method in that format
def keyword_nameOFyouKeyword(self,overlord_value,fact_value)
that is all you need to do.
Cool.
Writing Fact Modules:
Fact modules are loaded as almost the same way as minion modules so writing
them is also that easy. Go to func.minion.facts.modules and add your module
there if there is no problems it should loaded next time you restart your
server. IMPORTANT : facts methods doesnt accept currently any arguments, think
about
them as properties of your system ...
Example :
class HardwareFacts(fact_module.BaseFactModule):
version = "0.0.1"
description = "A modules that supplies hardware facts"
##snip snip ..
def run_level(self):
"""
The runlevel of the system
"""
return str(self.host.defaultRunlevel)
#for easier acces be creful should be unique
run_level.tag = "runlevel"
run_level.description = "Shows the runlevel of the system"
run_level.usage = "Can be used with all keywords"
An important thing to note here is the run_level.tag value ,by assigning that
value you are able to use your method as tag_value__keyword = blip
if you dont assing the "tag" attribute it is ok again just when calling you
should do it that way : hardware.run_level__keyword = blip
Btw, i like the tag idea, be careful not to have 2 tags with same name, the
last one will be availible :)
Hmm, kind of worried about the namespace issues there. Those kind of
things tend to cause hard
to find bugs.
For more info you can check the examples ...
Under The Hood :
Well how the facts is actually working? What i did was really simle if you
use OverlordQueryProxy and used the magical methods like filter and
set_complexq , OverlordQueryProxy serializes the current query (ORs and ANDs)
and inserts it as an argument to be sent to the minions. The server side
(minion)
deserializes it executes the logic parts (True and True etc) and if the result
is True it continues to executes the actual parts (minion methods) if not
returns back to overlord as signing __fact__ as False, so we know that minion
doesnt match our query. By doing that we can do lots of thing with one
connection or HIT.
Cool.
Note : If you have lots of queries it may useful to run the query "async" you
may get timeout.
Todo :
-Solving bugs (fixing when writing that post :) )
- Implement it so it can work from commandline, not sure about syntax
- Maybe making Python Api more sweet
I would like to see the api a little sweeter. Could we do something
like implement __gt__, __lt__, etc
on the filter/query objects? We still have a serializing step in the
middle though. That may require implementing
minion and overlord versions of the filter/query types though, which
might make adding types on the fly
a bit weird though.
Adrian
_______________________________________________
Func-list mailing list
Func-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/func-list
