That was it!
Sorry the 10.20.4.x NICs weren't configured correctly on those two nodes. I'll admit this one was definitely my mistake.
Thanks for pointing it out.
Tom
2013/7/9 Gregory Farnum <greg@xxxxxxxxxxx>
Do the nodes in question actually have those IPs assigned? I wonder ifOn Tue, Jul 9, 2013 at 3:08 AM, Tom Verdaat <tom@xxxxxxxxxx> wrote:
> Hi all,
>
> I've set up a new Ceph cluster for testing and it doesn't seem to be working
> out-of-the-box. If I check the status it tells me that of the 3 defined
> OSD's, only 1 is in:
>
>> health HEALTH_WARN 392 pgs degraded; 392 pgs stuck unclean
>> monmap e1: 3 mons at
>> {controller-01=10.20.3.110:6789/0,controller-02=10.20.3.111:6789/0,controller-03=10.20.3.112:6789/0},
>> election epoch 6, quorum 0,1,2 controller-01,controller-02,controller-03
>> osdmap e20: 3 osds: 1 up, 1 in
>> pgmap v35: 392 pgs: 392 active+degraded; 0 bytes data, 37444 KB used,
>> 15312 MB / 15348 MB avail
>> mdsmap e1: 0/0/1 up
>
>
> Turns out this is true because if I run "service ceph restart" on my OSD
> nodes, osd.0 will restart just fine but osd.1 and osd.2 give me the
> follwoing error:
>
>> Starting Ceph osd.0 on storage-02...
>>
>> starting osd.1 at :/0 osd_data /var/lib/ceph/osd/ceph-1
>> /var/lib/ceph/osd/ceph-1/journal
>> 2013-07-09 11:54:26.497639 7f5b18813780 -1 accepter.accepter.bind unable
>> to bind to 10.20.4.121:7100 on any port in range 6800-7100: Cannot assign
>> requested address
>> failed: 'ulimit -n 8192; /usr/bin/ceph-osd -i 1 --pid-file
>> /var/run/ceph/osd.1.pid -c /etc/ceph/ceph.conf '
>
>
> My ceph.conf just has a very limited configuration. The OSD section
> basically contains:
>
>> [osd]
>> public network=10.20.3.0/24
>> cluster network=10.20.4.0/24
>> [osd.0]
>> host = storage-01
>> public addr = 10.20.3.120
>> cluster addr = 10.20.4.120
>> [osd.1]
>> host = storage-02
>> public addr = 10.20.3.121
>> cluster addr = 10.20.4.121
>> [osd.2]
>> host = storage-03
>> public addr = 10.20.3.122
>> cluster addr = 10.20.4.122
>
>
> A quick Google search on that port binding error doesn't really yield and
> results so I'm reaching out to you guys. Any thoughts on how to fix this?
maybe the NIC isn't set up right or something.
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
