ceph_monitor - monitor your cluster with parallel python

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Cephers!

 

I’ve created small tool to help track memory/cpu/io usage. It’s useful for me so I thought I could share with you: https://github.com/aiicore/ceph_monitor

 

In general this is a python script, that uses parallel python to run a function on remote host. Data is gathered from all hosts and presented on console or added to sqlite database, then can be plotted with e.g. gnuplot. You can define osd ranges, that you want to monitor, or monitor certain process, e.g. osds only from pool that has ssds.

 

The main concept is that monitor don’t know and don’t care on which host osd’s are running, it treats them as a whole set.

 

Script uses psutil to get data related to processes (mon/osd/rgw/whatever). In near feature I’d like to add modes that can modify process behavior e.g. psutil has .nice .ionice .cpu_affinity methods, that could be useful in some tests. Basically with parallel python you can run any function remotely, so tuning SO by changing some /proc/* files can be done too.

 

You can add labels to data to see when what happens.

 

Sample plot: https://raw.githubusercontent.com/aiicore/ceph_monitor/master/examples/avg_cpu_mem.png

Simple test: https://github.com/aiicore/ceph_monitor/blob/master/examples/example_test_with_rados.sh

 

Short readme:  https://github.com/aiicore/ceph_monitor

Full readme: https://github.com/aiicore/ceph_monitor/blob/master/readme.txt

 

I encourage You to use and develop it, if not just please read the full readme text, maybe you’ll  come up with a better idea based on my this concept and something interesting will happen.

 

p.s. This currently works with python 2.6 and psutil 0.6.1 on centos 6.6.  If you find any bug – report it on my github as an issue.

 

!!! Security notice !!!

Parallel python supports SHA authentication  – my version currently runs WITHOUT this so in certain environments it could be dangerous (you could run any function from untrusted client). For now use it only in test/dev isolated clusters.

 

Regards,

Igor.

 

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux