On 10/31/2014 12:28 PM, Aravinda wrote:
Since HTIME file is available we need not maintain changelogs in two
different places as we do now(backend and working dir), We can provide
backend changelog names to consumer instead of working dir names. One
problem with this is for parsing these changelogs they have to use one
more RPC call. How about returning FOP details directly instead of
changelog file names?
Nicely put Aravinda.
That's exactly what's planned for Gnotify (lowlevel APIs), where the
consumer does not have to _know_ the changelog format. The fops are
notified (via callback) in a way similar to inotify(7) in an event
buffer. Avoiding maintaining state given that we keep performance on par
would be a huge plus point. Further, we could relax ordering and have
callback invoked from multiple threads for cases when ordering does not
really matter.
If we use db(sqlite) internally in API itself then we can do multiple
optimization.
1. If multiple data operations in requested time for a file then
return only one
2. If a file/gfid is created and deleted in the requested time range
then skip from the output.
3. If a file/gfid is deleted in the given time then do not return data
fops.
Other challenge is getting cluster view of these notifications.(and
RENAME issues)
Correct. Cluster view needs a good amount of thinking w.r.t
functionality, performance, etc.
--
regards
Aravinda
On 10/31/2014 11:10 AM, Kotresh Hiremath Ravishankar wrote:
Hi Venky,
If I understand correctly, the idea is to provide RPC based history API?
If yes, how can that solve the different clients/consumers having to
maintain
state of how much it has processed?
Thanks and Regards,
Kotresh H R
----- Original Message -----
From: "Venky Shankar" <vshankar@xxxxxxxxxx>
To: "Ajeet Jha" <ajha@xxxxxxxxxx>
Cc: "Kotresh Hiremath Ravishankar" <khiremat@xxxxxxxxxx>,
gluster-devel@xxxxxxxxxxx
Sent: Wednesday, October 29, 2014 6:23:40 PM
Subject: HTIME API
Hey Ajeet,
Since changelog translator maintains a meta file (HTIME) containing a
list of consumable changelogs, we could leverage this to provide an
API invokable by libgfchangelog (once the interaction is RPC based)
to get a list of changelogs for a given time period (similar to what
history API does today).
The benefit that this provides is that the consumers[1] need not
maintain state (the API takes care of that) of the amount of
changelogs [to be] processed and make use of this API to get freshly
consumable changelogs.
Thoughts?
[1]: esp for lowlevel API
(http://gluster.org/community/documentation/index.php/Features/Gnotify)
Venky
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-devel
