Re: {Disarmed} Re: Public Network Meaning

Andrija Panic <andrija.panic@xxxxxxxxx> · Sat, 14 Mar 2015 19:35:53 +0100

Georgios,
no need to put ANYTHING if you don't plan to split client-to-OSD vs OSD-OSD-replication on 2 different Network Cards/Networks - for pefromance reasons.

if you have only 1 network - simply DONT configure networks at all inside your CEPH.conf file...

if you have 2 x 1G cards in servers, then you may use first 1G for client traffic, and second 1G for OSD-to-OSD replication...

best

On 14 March 2015 at 19:33, Georgios Dimitrakakis <giorgis@xxxxxxxxxxxx> wrote:
Andrija,

Thanks for you help!

In my case I just have one 192.* network, so should I put that for both?

Besides monitors do I have to list OSDs as well?

Thanks again!

Best,

George

This is how I did it, and then retart each OSD one by one, but

monritor with ceph -s, when ceph is healthy, proceed with next OSD

restart...

Make sure the networks are fine on physical nodes, that you can ping

in between...

[global]

x

x

x

x

x

x

#############################################

### REPLICATION NETWORK ON SEPARATE 10G NICs

# replication network

cluster network = MAILSCANNER WARNING: NUMERICAL LINKS ARE OFTEN

MALICIOUS: 10.44.251.0/24 [29]

# public/client network

public network = MAILSCANNER WARNING: NUMERICAL LINKS ARE OFTEN

MALICIOUS: 10.44.253.0/16 [30]

#############################################

[mon.xx]

mon_addr = x.x.x.x:6789

host = xx

[mon.yy]

mon_addr = x.x.x.x:6789

host = yy

[mon.zz]

mon_addr = x.x.x.x:6789

host = zz

On 14 March 2015 at 19:14, Georgios Dimitrakakis  wrote:

I thought that it was easy but apparently its not!

I have the following in my conf file

mon_host = 192.168.1.100,192.168.1.101,192.168.1.102

public_network = MAILSCANNER WARNING: NUMERICAL LINKS ARE OFTEN

MALICIOUS: 192.168.1.0/24 [26]

mon_initial_members = fu,rai,jin

but still the 15.12.6.21 link is being saturated....

Any ideas why???

Should I put cluster network as well??

Should I put each OSD in the CONF file???

Regards,

George

Andrija,

thanks a lot for the useful info!

I would also like to thank "Kingrat" at the IRC channel for his

useful advice!

I was under the wrong impression that public is the one used for

RADOS.

So I thought that public=external=internet and therefore I used

that

one in my conf.

I understand now that I should have specified in CEPH Publics

Network what I call

"internal" and which is the one that all machines are talking

directly to each other.

Thanks you all for the feedback!

Regards,

George

Public network is clients-to-OSD traffic - and if you have NOT

explicitely defined cluster network, than also OSD-to-OSD

replication

takes place over same network.

Otherwise, you can define public and cluster(private) network -

so OSD

replication will happen over dedicated NICs (cluster network)

and thus

speed up.

If i.e. replica count on pool is 3, that means, each 1GB of

data

writen to some particualr OSD, will generate 3 x 1GB of more

writes,

to the replicas... - which ideally will take place over

separate NICs

to speed up things...

On 14 March 2015 at 17:43, Georgios Dimitrakakis  wrote:

Hi all!!

What is the meaning of public_network in ceph.conf?

Is it the network that OSDs are talking and transferring

data?

I have two nodes with two IP addresses each. One for internal

network MAILSCANNER WARNING: NUMERICAL LINKS ARE OFTEN

MALICIOUS:

MAILSCANNER WARNING: NUMERICAL LINKS ARE OFTEN MALICIOUS:

192.168.1.0/24 [1] [1]

and one external 15.12.6.*

I see the following in my logs:

osd.0 is down since epoch 2204, last address MAILSCANNER

WARNING: