On 09/08/2015 08:13 AM, Willi Fehler wrote: > Hi Chris, > > I tried to reconfigure my cluster but my MONs are still using the wrong > network. The new ceph.conf was pushed to all nodes and ceph was restarted. If your monitors are already deployed, you will need to move them to the new network manually. Once deployed, the monitors no longer care for ceph.conf for their addresses, but will use the monmap instead - only clients will look into ceph.conf to figure out where the monitors are. You will need to follow the procedure to add/rm monitors [1]. HTH. -Joao [1] http://ceph.com/docs/master/rados/operations/add-or-rm-mons/ > > [root@linsrv001 ~]# netstat -tulpen > Aktive Internetverbindungen (Nur Server) > Proto Recv-Q Send-Q Local Address Foreign Address > State Benutzer Inode PID/Program name > tcp 0 0 10.10.10.1:6789 0.0.0.0:* > LISTEN 0 19969 1793/ceph-mon > > [root@linsrv001 ~]# cat /etc/hosts > 127.0.0.1 localhost localhost.localdomain localhost4 > localhost4.localdomain4 > ::1 localhost localhost.localdomain localhost6 > localhost6.localdomain6 > 192.168.0.5 linsrv001 > 192.168.0.6 linsrv002 > 192.168.0.7 linsrv003 > > [root@linsrv001 ~]# cat /etc/ceph/ceph.conf > [global] > fsid = 7a8cc185-d7f1-4dd5-9fe6-42cfd5d3a5b7 > mon_initial_members = linsrv001, linsrv002, linsrv003 > mon_host = 192.168.0.5,192.168.0.6,192.168.0.7 > auth_cluster_required = cephx > auth_service_required = cephx > auth_client_required = cephx > filestore_xattr_use_omap = true > osd_pool_default_size = 2 > public_network = 192.168.0.0/24 > cluster_network = 10.10.10.0/24 > > [osd] > osd recovery max active = 1 > osd max backfills = 1 > filestore max sync interval = 30 > filestore min sync interval = 29 > filestore flusher = false > filestore queue max ops = 10000 > filestore op threads = 2 > osd op threads = 2 > > [client] > rbd cache = true > rbd cache writethrough until flush = true > > Regards - Willi > > Am 08.09.15 um 08:53 schrieb Willi Fehler: >> Hi Chris, >> >> thank you for your support. I will try to reconfigure my settings. >> >> Regards - Willi >> >> Am 08.09.15 um 08:43 schrieb Chris Taylor: >>> Willi, >>> >>> Looking at your conf file a second time, it looks like you have the >>> MONs on the same boxes as the OSDs. Is this correct? In my cluster >>> the MONs are on separate boxes. >>> >>> I'm making an assumption with your public_network, but try changing your >>> mon_host = 10.10.10.1,10.10.10.2,10.10.10.3 >>> to >>> mon_host = 192.168.0.1,192.168.0.2,192.168.0.3 >>> >>> You might also need to change your hosts file to reflect the correct >>> names and IP addresses also. >>> >>> >>> >>> My ceph.conf: >>> >>> [global] >>> fsid = d960d672-e035-413d-ba39-8341f4131760 >>> mon_initial_members = ceph-mon1, ceph-mon2, ceph-mon3 >>> mon_host = 10.20.0.11,10.20.0.12,10.20.0.13 >>> auth_cluster_required = cephx >>> auth_service_required = cephx >>> auth_client_required = cephx >>> filestore_xattr_use_omap = true >>> osd_pool_default_size = 2 >>> public_network = 10.20.0.0/24 >>> cluster_network = 10.21.0.0/24 >>> >>> [osd] >>> osd recovery max active = 1 >>> osd max backfills = 1 >>> filestore max sync interval = 30 >>> filestore min sync interval = 29 >>> filestore flusher = false >>> filestore queue max ops = 10000 >>> filestore op threads = 2 >>> osd op threads = 2 >>> >>> [client] >>> rbd cache = true >>> rbd cache writethrough until flush = true >>> >>> >>> >>> >>> On 09/07/2015 10:20 PM, Willi Fehler wrote: >>>> Hi Chris, >>>> >>>> could you please send me your ceph.conf? I tried to set "mon addr" >>>> but it looks like that it was ignored all the time. >>>> >>>> Regards - Willi >>>> >>>> >>>> Am 07.09.15 um 20:47 schrieb Chris Taylor: >>>>> My monitors are only connected to the public network, not the >>>>> cluster network. Only the OSDs are connected to the cluster network. >>>>> >>>>> Take a look at the diagram here: >>>>> http://ceph.com/docs/master/rados/configuration/network-config-ref/ >>>>> >>>>> -Chris >>>>> >>>>> On 09/07/2015 03:15 AM, Willi Fehler wrote: >>>>>> Hi, >>>>>> >>>>>> any ideas? >>>>>> >>>>>> Many thanks, >>>>>> Willi >>>>>> >>>>>> Am 07.09.15 um 08:59 schrieb Willi Fehler: >>>>>>> Hello, >>>>>>> >>>>>>> I'm trying to setup my first Ceph Cluster on Hammer. >>>>>>> >>>>>>> [root@linsrv002 ~]# ceph -v >>>>>>> ceph version 0.94.3 (95cefea9fd9ab740263bf8bb4796fd864d9afe2b) >>>>>>> >>>>>>> [root@linsrv002 ~]# ceph -s >>>>>>> cluster 7a8cc185-d7f1-4dd5-9fe6-42cfd5d3a5b7 >>>>>>> health HEALTH_OK >>>>>>> monmap e1: 3 mons at >>>>>>> {linsrv001=10.10.10.1:6789/0,linsrv002=10.10.10.2:6789/0,linsrv003=10.10.10.3:6789/0} >>>>>>> election epoch 256, quorum 0,1,2 >>>>>>> linsrv001,linsrv002,linsrv003 >>>>>>> mdsmap e60: 1/1/1 up {0=linsrv001=up:active}, 2 up:standby >>>>>>> osdmap e622: 9 osds: 9 up, 9 in >>>>>>> pgmap v1216: 384 pgs, 3 pools, 2048 MB data, 532 objects >>>>>>> 6571 MB used, 398 GB / 404 GB avail >>>>>>> 384 active+clean >>>>>>> >>>>>>> My issue is that I have two networks a public network >>>>>>> 192.168.0.0/24 and a cluster network 10.10.10.0/24 and my >>>>>>> monitors should listen on 192.168.0.0/24. Later I want to use >>>>>>> CephFS over the public network. >>>>>>> >>>>>>> [root@linsrv002 ~]# cat /etc/ceph/ceph.conf >>>>>>> [global] >>>>>>> fsid = 7a8cc185-d7f1-4dd5-9fe6-42cfd5d3a5b7 >>>>>>> mon_initial_members = linsrv001, linsrv002, linsrv003 >>>>>>> mon_host = 10.10.10.1,10.10.10.2,10.10.10.3 >>>>>>> auth_cluster_required = cephx >>>>>>> auth_service_required = cephx >>>>>>> auth_client_required = cephx >>>>>>> filestore_xattr_use_omap = true >>>>>>> mon_clock_drift_allowed = 1 >>>>>>> public_network = 192.168.0.0/24 >>>>>>> cluster_network = 10.10.10.0/24 >>>>>>> >>>>>>> [root@linsrv002 ~]# cat /etc/hosts >>>>>>> 127.0.0.1 localhost localhost.localdomain localhost4 >>>>>>> localhost4.localdomain4 >>>>>>> ::1 localhost localhost.localdomain localhost6 >>>>>>> localhost6.localdomain6 >>>>>>> 10.10.10.1 linsrv001 >>>>>>> 10.10.10.2 linsrv002 >>>>>>> 10.10.10.3 linsrv003 >>>>>>> >>>>>>> I've deployed my first cluster with ceph-deploy. What should I do >>>>>>> to have :6789 to be listen on the public network? >>>>>>> >>>>>>> Regards - Willi >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> ceph-users mailing list >>>>>> ceph-users@xxxxxxxxxxxxxx >>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> ceph-users mailing list >>>>> ceph-users@xxxxxxxxxxxxxx >>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>> >>> >> >> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |