Hello all, my name is Marios Kogias and I am a student at the National Technical University of Athens. As part of my diploma thesis and my participation in Google Summer of Code 2014 (in the LTTng organization) I am working on a low-overhead tracing infrastructure for distributed systems. I am also collaborating with the Synnefo team (https://www.synnefo.org/) and especially with Vangelis Koukis, Constantinos Venetsanopoulos and Filippos Giannakos (cc) Some time ago, we started experimenting with RADOS instrumentation using LTTng and we noticed that there are similar endeavours in the Ceph github repository [1]. However, unlike your approach, we are following an annotation-based tracing schema, which enables us to track a specific request from the time it enters the system at higher levels till it is finally served by RADOS. In general, we try to implement the tracing semantics described in the Dapper paper [2] in order to trace the causal relationships between the different processing phases that an IO request may trigger. Our target is an end-to-end visualisation of the request's route in the system, accompanied by information concerning latencies in each processing phase. Thanks to LTTng this can happen with a minimal overhead and in realtime. In order to visualize the results we have integrated Twitter's Zipkin [3], (which is a tracing system entirely based on Dapper) with LTTng. You can find a proof of concept of what we've done so far here: http://snf-551656.vm.okeanos.grnet.gr:8080/traces/0b554b8a48cb3e84?serviceName=MOSDOp In the above link you can see the trace of a write request served by a RADOS pool with replication level set to 3 (two replicas). We'd love to have early feedback and comments from you guys too, so that we can incorporate useful recommendations. You can find all the relevant code here[5][6]. If you have any questions or you wish to experiment with the project please do not hesitate to contact us. Kind regards, Marios [1]https://github.com/ceph/ceph/tree/wip-lttng [2]http://static.googleusercontent.com/media/research.google.com/el//pubs/archive/36356.pdf [3]http://twitter.github.io/zipkin/ [4] https://github.com/marioskogias/blkin [5] https://github.com/marioskogias/babeltrace-plugins -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
