Hello Pratik, I'm using virtual disks as OSDs. I prefer virtual disks over directories because this resembles the production environment a bit better. I'm using VirtualBox for virtualisation. The OSDs are dynamic disks, not pre-allocated, but this shouldn't be a problem, right? I don't have the diskspace on my iMac to have all 6 OSDs pre-allocated :). I've made the virtual OSD disks 10 GB each, by the way, so that should be enough for a first test, imho. Met vriendelijke groet/With kind regards, Tijn Buijs Cloud.nl logo tijn at cloud.nl <mailto:tijn at cloud.nl> | T. 0800-CLOUDNL / +31 (0)162 820 000 | F. +31 (0)162 820 001 Cloud.nl B.V. | Minervum 7092D | 4817 ZK Breda | www.cloud.nl <http://www.cloud.nl> On 04/08/14 14:51, Pratik Rupala wrote: > Hi, > > You mentioned that you have 3 hosts which are VMs. Are you using > simple directories as OSDs or virtual disks as OSDs? > > I had same problem few days back where enough space was not available > from OSD for the cluster. > > Try to increase the size of disks if you are using virtual disks and > if you are using directories as OSDs then check whether you have > enough space on root device using df -h command on OSD node. > > Regards, > Pratik > > On 8/4/2014 4:11 PM, Tijn Buijs wrote: >> Hi Everybody, >> >> My idea was that maybe I was inpatient or something, so I let my Ceph >> cluster running over the weekend. So from friday 15:00 until now (it >> is monday morning 11:30 here now) it kept on running. And it didn't >> help :). It still needs to create 192 PGs. >> I've reinstalled my entier cluster a few times now. I switched over >> from CentOS 6.5 to Ubuntu 14.04.1 LTS and back to CentOS again, and >> every time I get exactly the same results. The PGs are getting in the >> incomplete, stuck inactive, stuk unclean state. What am I doing >> wrong? :). >> >> For the moment I'm running with 6 OSDs evenly divided over 3 hosts >> (so each host has 2 OSDs). I've only got 1 monitor configured in my >> current cluster. I hit some other problem when trying to add monitor >> 2 and 3 again. And to not complicate things with multiple problems at >> the same time I've switched back to only 1 monitor. The cluster >> should work that way, right? >> >> To make things clear for everybody, here is the output of ceph health >> and ceph -s: >> $ ceph health >> HEALTH_WARN 192 pgs incomplete; 192 pgs stuck inactive; 192 pgs stuck >> unclean >> $ ceph -s >> cluster 43d5f48b-d034-4f50-bec8-5c4f3ad8276f >> health HEALTH_WARN 192 pgs incomplete; 192 pgs stuck inactive; >> 192 pgs stuck unclean >> monmap e1: 1 mons at {ceph-mon1=10.28.28.71:6789/0}, election >> epoch 1, quorum 0 ceph-mon1 >> osdmap e20: 6 osds: 6 up, 6 in >> pgmap v40: 192 pgs, 3 pools, 0 bytes data, 0 objects >> 197 MB used, 30456 MB / 30653 MB avail >> 192 incomplete >> >> I hope somebody has an idea for me to try :). >> >> Met vriendelijke groet/With kind regards, >> >> Tijn Buijs >> >> Cloud.nl logo >> >> tijn at cloud.nl <mailto:tijn at cloud.nl> | T. 0800-CLOUDNL / +31 (0)162 >> 820 000 | F. +31 (0)162 820 001 >> Cloud.nl B.V. | Minervum 7092D | 4817 ZK Breda | www.cloud.nl >> <http://www.cloud.nl> >> On 31/07/14 17:19, Alfredo Deza wrote: >>> >>> >>> >>> On Thu, Jul 31, 2014 at 10:36 AM, Tijn Buijs <tijn at cloud.nl >>> <mailto:tijn at cloud.nl>> wrote: >>> >>> Hello everybody, >>> >>> At cloud.nl <http://cloud.nl> we are going to use Ceph. So I >>> find it a good idea to get some handson experience with it, so I >>> can work with it :). So I'm installing a testcluster in a few >>> VirtualBox machines on my iMac, which runs OS X 10.9.4 >>> offcourse. I know I will get a lousy performance, but that's not >>> the objective here. The objective is to get some experience with >>> Ceph, to see how it works. >>> >>> But I hit an issue during the initial setup of the cluster. When >>> I'm done installing everything and following the howto's on >>> ceph.com <http://ceph.com> (the preflight >>> <http://ceph.com/docs/master/start/quick-start-preflight/> and >>> the Storage Cluster quick start >>> <http://ceph.com/docs/master/start/quick-ceph-deploy/>) I need >>> to run ceph health to see that everything is running perfectly. >>> But it doesn't run perfectly, I get the following output: >>> ceph at ceph-admin:~$ ceph health >>> HEALTH_WARN 192 pgs incomplete; 192 pgs stuck inactive; 192 pgs >>> stuck unclean >>> >>> And it stays at this information, it never ever changes. So >>> everything is really stuck. But I don't know what is stuck >>> exactly and how I can fix it. Some more info about my cluster: >>> ceph at ceph-admin:~$ ceph -s >>> cluster d31586a5-6dd6-454e-8835-0d6d9e204612 >>> health HEALTH_WARN 192 pgs incomplete; 192 pgs stuck >>> inactive; 192 pgs stuck unclean >>> monmap e3: 3 mons at >>> {ceph-mon1=10.28.28.18:6789/0,ceph-mon2=10.28.28.31:6789/0,ceph-mon3=10.28.28.50:6789/0 >>> <http://10.28.28.18:6789/0,ceph-mon2=10.28.28.31:6789/0,ceph-mon3=10.28.28.50:6789/0>}, >>> election epoch 4, quorum 0,1,2 ceph-mon1,ceph-mon2,ceph-mon3 >>> osdmap e25: 6 osds: 6 up, 6 in >>> pgmap v56: 192 pgs, 3 pools, 0 bytes data, 0 objects >>> 197 MB used, 30455 MB / 30653 MB avail >>> 192 creating+incomplete >>> >>> I'm running on Ubuntu 14.04.1 LTS Server. I did try to get it >>> running on CentOS 6.5 too (CentOS 6.5 is my actual distro of >>> choice, but Ceph has more affinity with Ubuntu, so I tried that >>> too), but I got exactly the same results. >>> >>> But because this is my first install of Ceph I don't know the >>> exact debug commands and stuff. I'm willing to get this working, >>> but I just don't know how :). Any help is appreciated :). >>> >>> >>> Did you use ceph-deploy? (the link to the quick start guide makes me >>> think you did) >>> >>> If that was the case, did you get any warnings/errors at all? >>> >>> ceph-deploy is very verbose because some of these things are hard to >>> debug. Mind sharing that output? >>> >>> >>> Met vriendelijke groet/With kind regards, >>> >>> Tijn Buijs >>> >>> Cloud.nl logo >>> >>> tijn at cloud.nl <mailto:tijn at cloud.nl> | T. 0800-CLOUDNL / +31 >>> (0)162 820 000 <tel:%2B31%20%280%29162%20820%20000> | F. +31 >>> (0)162 820 001 <tel:%2B31%20%280%29162%20820%20001> >>> Cloud.nl B.V. | Minervum 7092D | 4817 ZK Breda | www.cloud.nl >>> <http://www.cloud.nl> >>> >>> _______________________________________________ >>> ceph-users mailing list >>> ceph-users at lists.ceph.com <mailto:ceph-users at lists.ceph.com> >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >>> >> >> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users at lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140805/2ac6867a/attachment.htm> -------------- next part -------------- A non-text attachment was scrubbed... Name: logo_cloud_nl.jpg Type: image/jpeg Size: 14056 bytes Desc: not available URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140805/2ac6867a/attachment.jpg> -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/jpeg Size: 14056 bytes Desc: not available URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140805/2ac6867a/attachment.jpeg> -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/jpeg Size: 14056 bytes Desc: not available URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140805/2ac6867a/attachment-0001.jpeg>
 |