network Ports Linked to each OSD process

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

Iam trying to do some network control on the storage nodes. For this, I
need to know the ports opened for communication by each OSD processes.

I got to know from the link
http://ceph.com/docs/master/rados/configuration/network-config-ref/ , that
each OSD process requires 3 ports and they from port 6800 it is reserved
for OSD processes.

However, when I do a ceph osd dump command, it lists 4 ports in use for
each of the OSDs:

*root at cephnode2:~# ceph osd dump | grep osd*
*max_osd 4*
*osd.0 up   in  weight 1 up_from 71 up_thru 71 down_at 68
last_clean_interval [4,70) 10.223.169.166:6800/83380
<http://10.223.169.166:6800/83380> 10.223.169.166:6810/1083380
<http://10.223.169.166:6810/1083380> 10.223.169.166:6811/1083380
<http://10.223.169.166:6811/1083380> 10.223.169.166:6812/1083380
<http://10.223.169.166:6812/1083380> exists,up
fdbbc6eb-7d9f-4ad8-a8c3-caf995422528*
*osd.1 up   in  weight 1 up_from 7 up_thru 71 down_at 0 last_clean_interval
[0,0) 10.223.169.201:6800/83569 <http://10.223.169.201:6800/83569>
10.223.169.201:6801/83569 <http://10.223.169.201:6801/83569>
10.223.169.201:6802/83569 <http://10.223.169.201:6802/83569>
10.223.169.201:6803/83569 <http://10.223.169.201:6803/83569> exists,up
db545fd7-071f-4671-b1c4-c57221f894a3*
*osd.2 up   in  weight 1 up_from 64 up_thru 64 down_at 61
last_clean_interval [12,60) 10.223.169.166:6805/92402
<http://10.223.169.166:6805/92402> 10.223.169.166:6806/92402
<http://10.223.169.166:6806/92402> 10.223.169.166:6807/92402
<http://10.223.169.166:6807/92402> 10.223.169.166:6808/92402
<http://10.223.169.166:6808/92402> exists,up
594b73b9-1908-4757-b914-d887d850b386*
*osd.3 up   in  weight 1 up_from 17 up_thru 71 down_at 0
last_clean_interval [0,0) 10.223.169.201:6805/84590
<http://10.223.169.201:6805/84590> 10.223.169.201:6806/84590
<http://10.223.169.201:6806/84590> 10.223.169.201:6807/84590
<http://10.223.169.201:6807/84590> 10.223.169.201:6808/84590
<http://10.223.169.201:6808/84590> exists,up
37536050-ef92-4eba-95a7-e7a099c6d059*
*root at cephnode2:~# *



I also, listed the ports listening on the above highlighted OSD  process
using lsof

 root at cephnode2:~/nethogs# lsof -i | grep ceph | grep 83380
ntpd        1627  ntp   19u  IPv4   33890      0t0  UDP
cephnode2.iind.intel.com:ntp
*ceph-osd   83380 root    4u  IPv4 4881747      0t0  TCP *:6800 (LISTEN)*
*ceph-osd   83380 root    5u  IPv4 5045544      0t0  TCP
cephnode2.iind.intel.com:6810 <http://cephnode2.iind.intel.com:6810>
(LISTEN)*
*ceph-osd   83380 root    6u  IPv4 5045545      0t0  TCP
cephnode2.iind.intel.com:6811 <http://cephnode2.iind.intel.com:6811>
(LISTEN)*
*ceph-osd   83380 root    7u  IPv4 5045546      0t0  TCP
cephnode2.iind.intel.com:6812 <http://cephnode2.iind.intel.com:6812>
(LISTEN)*
*ceph-osd   83380 root    8u  IPv4 4881751      0t0  TCP *:6804 (LISTEN)*
ceph-osd   83380 root   19u  IPv4 5101954      0t0  TCP
cephnode2.iind.intel.com:6800->computeich.iind.intel.com:60781 (ESTABLISHED)
ceph-osd   83380 root   23u  IPv4 5013387      0t0  TCP
cephnode2.iind.intel.com:41878->cephnode4.iind.intel.com:6803 (ESTABLISHED)
ceph-osd   83380 root   25u  IPv4 5037728      0t0  TCP
cephnode2.iind.intel.com:44251->cephnode4.iind.intel.com:6802 (ESTABLISHED)
ceph-osd   83380 root   83u  IPv4 5025954      0t0  TCP
cephnode2.iind.intel.com:47863->cephnode4.iind.intel.com:6808 (ESTABLISHED)
ceph-osd   83380 root  111u  IPv4 4850005      0t0  TCP
cephnode2.iind.intel.com:43189->cephnode2.iind.intel.com:6807 (ESTABLISHED)
ceph-osd   83380 root  112u  IPv4 4850839      0t0  TCP
cephnode2.iind.intel.com:59738->cephnode2.iind.intel.com:6808 (ESTABLISHED)
ceph-osd   83380 root  130u  IPv4 5037729      0t0  TCP
cephnode2.iind.intel.com:41902->cephnode4.iind.intel.com:6807 (ESTABLISHED)
ceph-osd   83380 root  152u  IPv4 5013621      0t0  TCP
cephnode2.iind.intel.com:34798->cephmon.iind.intel.com:6789 (ESTABLISHED)
ceph-osd   83380 root  159u  IPv4 5040569      0t0  TCP
cephnode2.iind.intel.com:6811->cephnode4.iind.intel.com:35321 (ESTABLISHED)
ceph-osd   83380 root  160u  IPv4 5040570      0t0  TCP
cephnode2.iind.intel.com:6812->cephnode4.iind.intel.com:42682 (ESTABLISHED)
ceph-osd   83380 root  161u  IPv4 5043767      0t0  TCP
cephnode2.iind.intel.com:6812->cephnode4.iind.intel.com:42683 (ESTABLISHED)
ceph-osd   83380 root  162u  IPv4 5038664      0t0  TCP
cephnode2.iind.intel.com:6811->cephnode4.iind.intel.com:35324 (ESTABLISHED)


In the above list, it looks like it is listening to some additional
ports(6810-6812) from what is listed in the "ceph osd dump" command.

I would like to know, if there is any straight way of listing the ports
used by each OSD process.

Also, I would also like to understand the networking architecture of Ceph
in more detail. Is there any link/doc for the same?

Thanks in Advance,
Sharmila
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140523/35b269b3/attachment.htm>


[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux