Re: HTIME API

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 10/31/2014 12:28 PM, Aravinda wrote:
Since HTIME file is available we need not maintain changelogs in two different places as we do now(backend and working dir), We can provide backend changelog names to consumer instead of working dir names. One problem with this is for parsing these changelogs they have to use one more RPC call. How about returning FOP details directly instead of changelog file names?

Nicely put Aravinda.

That's exactly what's planned for Gnotify (lowlevel APIs), where the consumer does not have to _know_ the changelog format. The fops are notified (via callback) in a way similar to inotify(7) in an event buffer. Avoiding maintaining state given that we keep performance on par would be a huge plus point. Further, we could relax ordering and have callback invoked from multiple threads for cases when ordering does not really matter.


If we use db(sqlite) internally in API itself then we can do multiple optimization.

1. If multiple data operations in requested time for a file then return only one 2. If a file/gfid is created and deleted in the requested time range then skip from the output. 3. If a file/gfid is deleted in the given time then do not return data fops.

Other challenge is getting cluster view of these notifications.(and RENAME issues)

Correct. Cluster view needs a good amount of thinking w.r.t functionality, performance, etc.


--
regards
Aravinda

On 10/31/2014 11:10 AM, Kotresh Hiremath Ravishankar wrote:
Hi Venky,

If I understand correctly, the idea is to provide RPC based history API?
If yes, how can that solve the different clients/consumers having to maintain
state of how much it has processed?

Thanks and Regards,
Kotresh H R

----- Original Message -----
From: "Venky Shankar" <vshankar@xxxxxxxxxx>
To: "Ajeet Jha" <ajha@xxxxxxxxxx>
Cc: "Kotresh Hiremath Ravishankar" <khiremat@xxxxxxxxxx>, gluster-devel@xxxxxxxxxxx
Sent: Wednesday, October 29, 2014 6:23:40 PM
Subject: HTIME API

Hey Ajeet,

Since changelog translator maintains a meta file (HTIME) containing a list of consumable changelogs, we could leverage this to provide an API invokable by libgfchangelog (once the interaction is RPC based) to get a list of changelogs for a given time period (similar to what history API does today).

The benefit that this provides is that the consumers[1] need not maintain state (the API takes care of that) of the amount of changelogs [to be] processed and make use of this API to get freshly consumable changelogs.

Thoughts?

[1]: esp for lowlevel API (http://gluster.org/community/documentation/index.php/Features/Gnotify)

     Venky
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-devel

_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-devel




[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux