I am not sure about your background knowledge of ceph, but if you are starting. Maybe first try and get ceph working in a virtual environment, that should not be to much of a problem. Then try migrating it to your container. Now you are probably fighting to many issues at the same time. -----Original Message----- From: Varun Singh [mailto:varun.singh@xxxxxxxxx] Sent: 22 April 2019 07:46 To: ceph-users@xxxxxxxx Subject: Re: Ceph inside Docker containers inside VirtualBox On Fri, Apr 19, 2019 at 6:53 PM Varun Singh <varun.singh@xxxxxxxxx> wrote: > > On Fri, Apr 19, 2019 at 10:44 AM Varun Singh <varun.singh@xxxxxxxxx> wrote: > > > > On Thu, Apr 18, 2019 at 9:53 PM Siegfried Höllrigl > > <siegfried.hoellrigl@xxxxxxxxxx> wrote: > > > > > > Hi ! > > > > > > I am not 100% sure, but i think, --net=host does not propagate > > > /dev/ inside the conatiner. > > > > > > From the Error Message : > > > > > > 2019-04-18 07:30:06 /opt/ceph-container/bin/entrypoint.sh: ERROR- > > > The device pointed by OSD_DEVICE (/dev/vdd) doesn't exist ! > > > > > > > > > I whould say, you should add something like --device=/dev/vdd to the docker run command for the osd. > > > > > > Br > > > > > > > > > Am 18.04.2019 um 14:46 schrieb Varun Singh: > > > > Hi, > > > > I am trying to setup Ceph through Docker inside a VM. My host > > > > machine is Mac. My VM is an Ubuntu 18.04. Docker version is > > > > 18.09.5, build e8ff056. > > > > I am following the documentation present on ceph/daemon Docker > > > > Hub page. The idea is, if I spawn docker containers as mentioned > > > > on the page, I should get a ceph setup without KV store. I am > > > > not worried about KV store as I just want to try it out. > > > > Following are the commands I am firing to bring the containers up: > > > > > > > > Monitor: > > > > docker run -d --net=host -v /etc/ceph:/etc/ceph -v > > > > /var/lib/ceph/:/var/lib/ceph/ -e MON_IP=10.0.2.15 -e > > > > CEPH_PUBLIC_NETWORK=10.0.2.0/24 ceph/daemon mon > > > > > > > > Manager: > > > > docker run -d --net=host -v /etc/ceph:/etc/ceph -v > > > > /var/lib/ceph/:/var/lib/ceph/ ceph/daemon mgr > > > > > > > > OSD: > > > > docker run -d --net=host --pid=host --privileged=true -v > > > > /etc/ceph:/etc/ceph -v /var/lib/ceph/:/var/lib/ceph/ -v > > > > /dev/:/dev/ -e OSD_DEVICE=/dev/vdd ceph/daemon osd > > > > > > > > From the above commands I am able to spawn monitor and manager > > > > properly. I verified this by firing this command on both monitor > > > > and manager containers: > > > > sudo docker exec d1ab985 ceph -s > > > > > > > > I get following outputs for both: > > > > > > > > cluster: > > > > id: 14a6e40a-8e54-4851-a881-661a84b3441c > > > > health: HEALTH_OK > > > > > > > > services: > > > > mon: 1 daemons, quorum serverceph-VirtualBox (age 62m) > > > > mgr: serverceph-VirtualBox(active, since 56m) > > > > osd: 0 osds: 0 up, 0 in > > > > > > > > data: > > > > pools: 0 pools, 0 pgs > > > > objects: 0 objects, 0 B > > > > usage: 0 B used, 0 B / 0 B avail > > > > pgs: > > > > > > > > However when I try to bring up OSD using above command, it > > > > doesn't work. Docker logs show this output: > > > > 2019-04-18 07:30:06 /opt/ceph-container/bin/entrypoint.sh: static: > > > > does not generate config > > > > 2019-04-18 07:30:06 /opt/ceph-container/bin/entrypoint.sh: > > > > ERROR- The device pointed by OSD_DEVICE (/dev/vdd) doesn't exist ! > > > > > > > > I am not sure why the doc asks to pass /dev/vdd to OSD_DEVICE env var. > > > > I know there are five different ways to spawning the OSD, but I > > > > am not able to figure out which one would be suitable for a > > > > simple deployment. If you could please let me know how to spawn > > > > OSDs using Docker, it would help a lot. > > > > > > > > > > > > Thanks Br, I will try this out today. > > > > -- > > Regards, > > Varun Singh > > Hi, > So following your suggestion I tried following two commands: > 1. I added --device=/dev/vdd switch without removing OSD_DEVICE env > var. This resulted in same error before docker run -d --net=host > --pid=host --privileged=true --device=/dev/vdd -v /etc/ceph:/etc/ceph > -v /var/lib/ceph/:/var/lib/ceph/ -v /dev/:/dev/ -e OSD_DEVICE=/dev/vdd > ceph/daemon osd > > > 2. Then I removed OSD_DEVICE env var and just added --device=/dev/vdd > switch docker run -d --net=host --pid=host --privileged=true > --device=/dev/vdd -v /etc/ceph:/etc/ceph -v > /var/lib/ceph/:/var/lib/ceph/ -v /dev/:/dev/ ceph/daemon osd > > OSD_DEVICE related error went away and I think ceph created an OSD > successfully. But it wasn't able to connect to cluster. Is it because > I did not give and network related information? I get the following > error now: > > 2019-04-18 08:30:47 /opt/ceph-container/bin/entrypoint.sh: static: > does not generate config > 2019-04-18 08:30:47 /opt/ceph-container/bin/entrypoint.sh: > Bootstrapped OSD(s) found; using OSD directory > 2019-04-18 08:30:47 /opt/ceph-container/bin/entrypoint.sh: Creating > osd > 2019-04-18 08:30:52.944 7f897ca6d700 -1 auth: unable to find a keyring > on /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or > directory > 2019-04-18 08:30:52.944 7f897ca6d700 -1 AuthRegistry(0x7f8978063e78) > no keyring found at /var/lib/ceph/bootstrap-osd/ceph.keyring, > disabling cephx > 2019-04-18 08:30:52.964 7f897ca6d700 -1 auth: unable to find a keyring > on /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or > directory > 2019-04-18 08:30:52.964 7f897ca6d700 -1 AuthRegistry(0x7f89780bcad8) > no keyring found at /var/lib/ceph/bootstrap-osd/ceph.keyring, > disabling cephx > 2019-04-18 08:30:52.964 7f897ca6d700 -1 auth: unable to find a keyring > on /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or > directory > 2019-04-18 08:30:52.964 7f897ca6d700 -1 AuthRegistry(0x7f897ca6be78) > no keyring found at /var/lib/ceph/bootstrap-osd/ceph.keyring, > disabling cephx > 2019-04-18 08:30:52.976 7f897585d700 -1 monclient(hunting): > handle_auth_bad_method server allowed_methods [2] but i only support > [1] > 2019-04-18 08:30:52.976 7f897ca6d700 -1 monclient: authenticate NOTE: > no keyring found; disabled cephx authentication [errno 95] error > connecting to the cluster > > > I tried googling the error and came across a couple of filed bugs but > no solution. Any help in fixing this would be great. > > -- > Regards, > Varun Singh Hi, Any help here would be deeply appreciated. -- Regards, Varun Singh -- Confidentiality Notice and Disclaimer: This email (including any attachments) contains information that may be confidential, privileged and/or copyrighted. If you are not the intended recipient, please notify the sender immediately and destroy this email. Any unauthorized use of the contents of this email in any manner whatsoever, is strictly prohibited. If improper activity is suspected, all available information may be used by the sender for possible disciplinary action, prosecution, civil claim or any remedy or lawful purpose. Email transmission cannot be guaranteed to be secure or error-free, as information could be intercepted, lost, arrive late, or contain viruses. The sender is not liable whatsoever for damage resulting from the opening of this message and/or the use of the information contained in this message and/or attachments. Expressions in this email cannot be treated as opined by the sender company management they are solely expressed by the sender unless authorized. _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |