I understand that the following ideas may require some infrastructural changes and thus it may be tempting to dismiss the idea summarily without consideration. I am hopeful that through the discussion perhaps there are some techniques that I missed or value to discussing these within a wider context to see if such functionality might have general applicability.
1/ Would there be value in allowing an application to provide an opaque set of data that provides application specific identification information at the cpg layer? Even if the information were to be restricted to a maximum size of data this would potentially provide basic membership credentials of each user of the group defined by the application layer. For example, it is often handy to have members of a group be full participants with capabilities to both read and write data, but for purposes of monitoring and statistics gathering it is also handy to have members join that can only participate as readers. If each application could provide their intent and all group members could read that intent on an application level then they might be able to make more informed decisions without the need for additional messages or secondary groups.
One possible implementation would be to augment the cpg interface library to provide a secondary interface with further information for cpg_join(), such as cpg_join_id(cpg_handle_t handle, const struct cpg_name *group, void *data, size_t len) that allowed the application to provide a token set of information that was passed through to each of the participating group members and identifies the use of the group by the application. Although the change has impacts on 'req_lib_cpg_join' structure to hold the data and would have implications on several internal APIs in order to store and pass the data to allow the data to be ultimately extracted by the application on a membership callback there may not need to be a large impact to the external API. Even if the data was restricted to a single fixed size token it would allow a permissions mask to to provided.
2. The second consideration is the value of a header mode for receiving information. The CPG internal system utilizes a number of valuable concepts ultimately buried into the underlying communications layer header. For example, the unique token of the sequence number provides an excellent reference for data. Repeating this type of header at the application layer would not be necessary if an API was provided that allowed the header data to be available for read only use by the application and might be achieved without additional movement of data. The concept of a receive header mode have been used for the IP stack and are utilized by tools such as tcpdump.
3. I am curious if any thought has been given to providing additional statistics on a histogram of message latency in the system. For example, for each message the sender could record the latency for receipt of the reflected response at the conclusion of the ring. Unfortunately measuring time between nodes is more difficult due to the lack of a global clock and often leads to misleading times if negative values are not properly compensated. Perhaps there are already mechanisms that might provide insight into message volume and inter-node message latency on a aggregate for the node and on a per group basis. It is also interesting to be able to differentiate between the deliver of the message by the daemon to the application space versus the applications ultimate retrieval of the information by properly exorcising the receive routine when analysing transient conditions caused by fast senders and slow or stalled receivers.
Perhaps there more elegant techniques available that would provide value to all cpg users. If the problems are solved once at the lower layer, perhaps many applications above could utilize a common proven tool.
Thanks for your considerations.
dan
_______________________________________________ discuss mailing list discuss@xxxxxxxxxxxx http://lists.corosync.org/mailman/listinfo/discuss
