Re: more human readable log to track request or using mapreduce for data statistics

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 26/03/2015, at 09.05, 池信泽 <xmdxcxz@xxxxxxxxx> wrote:

hi,ceph:

Currently, the command ”ceph --admin-daemon
/var/run/ceph/ceph-osd.0.asok dump_historic_ops“ may return as below:

{ "description": "osd_op(client.4436.1:11617
rb.0.1153.6b8b4567.000000000192 [] 2.8eb4757c ondisk+write e92)",
         "received_at": "2015-03-25 19:41:47.146145",
         "age": "2.186521",
         "duration": "1.237882",
         "type_data": [
               "commit sent; apply or cleanup",
               { "client": "client.4436",
                 "tid": 11617},
               [
                   { "time": "2015-03-25 19:41:47.150803",
                     "event": "event1"},
                   { "time": "2015-03-25 19:41:47.150873",
                     "event": "event2"},
                   { "time": "2015-03-25 19:41:47.150895",
                     "event": "event3"},
                   { "time": "2015-03-25 19:41:48.384027",
                     "event": "event4"}]]}
Seems like JSON format
So consider doing your custom conversion by some means of CLI


I think this message is not so suitable for grep log or using
mapreduce for data statistics. Such as, I want to know
the write request avg latency for each rbd everyday. If we could
output the all latency in just one line, it would be very easy to
achieve it.

Such as, the output log maybe something like this:
2015-03-26 03:30:53.859759 osd=osd.0 pg=2.11 op=(client.4436.1:11617
rb.0.1153.6b8b4567.000000000192 [] 2.8eb4757c ondisk+write e92)
received_at=1427355253 age=2.186521 duration=1.237882 tid=11617
client=client.4436 event1=20ms event2=300ms event3=400ms event4=100ms.

The above:

duration means: the time between (reply_to_client_stamp -
request_received_stamp)
event1 means: the time between (the event1_stamp - request_received_stamp)
...
event4 means: the time between (the event4_stamp - request_received_stamp)

Now, If we output the every log as above. it would be every easy to
know the write request avg latency for each rbd everyday.
Or if I use grep it is more easy to find out which stage is the bottleneck.

--
Regards,
xinze
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Attachment: signature.asc
Description: Message signed with OpenPGP using GPGMail

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux