The ansible deploy is quite a pain to get set up properly, but it does
work to get the whole stack working under Docker. It uses the following
script on Ubuntu to start the OSD containers:
/usr/bin/docker run \
--rm \
--net=host \
--privileged=true \
--pid=host \
--memory=64386m \
--cpus=1 \
-v /dev:/dev \
-v /etc/localtime:/etc/localtime:ro \
-v /var/lib/ceph:/var/lib/ceph:z \
-v /etc/ceph:/etc/ceph:z \
-v /var/run/ceph:/var/run/ceph:z \
--security-opt apparmor:unconfined \
-e OSD_BLUESTORE=1 \
-e OSD_DMCRYPT=0 \
-e CLUSTER=ceph \
-v /run/lvm/lvmetad.socket:/run/lvm/lvmetad.socket \
-e CEPH_DAEMON=OSD_CEPH_VOLUME_ACTIVATE \
-e OSD_ID="$1" \
--name=ceph-osd-"$1" \
\
docker.io/ceph/daemon:latest
Hi !
I am not 100% sure, but i think, --net=host does not propagate /dev/
inside the conatiner.
From the Error Message :
2019-04-18 07:30:06 /opt/ceph-container/bin/entrypoint.sh: ERROR- The
device pointed by OSD_DEVICE (/dev/vdd) doesn't exist !
I whould say, you should add something like --device=/dev/vdd to the
docker run command for the osd.
Br
Am 18.04.2019 um 14:46 schrieb Varun Singh:
Hi,
I am trying to setup Ceph through Docker inside a VM. My host machine
is Mac. My VM is an Ubuntu 18.04. Docker version is 18.09.5, build
e8ff056.
I am following the documentation present on ceph/daemon Docker Hub
page. The idea is, if I spawn docker containers as mentioned on the
page, I should get a ceph setup without KV store. I am not worried
about KV store as I just want to try it out. Following are the
commands I am firing to bring the containers up:
Monitor:
docker run -d --net=host -v /etc/ceph:/etc/ceph -v
/var/lib/ceph/:/var/lib/ceph/ -e MON_IP=10.0.2.15 -e
CEPH_PUBLIC_NETWORK=10.0.2.0/24 ceph/daemon mon
Manager:
docker run -d --net=host -v /etc/ceph:/etc/ceph -v
/var/lib/ceph/:/var/lib/ceph/ ceph/daemon mgr
OSD:
docker run -d --net=host --pid=host --privileged=true -v
/etc/ceph:/etc/ceph -v /var/lib/ceph/:/var/lib/ceph/ -v /dev/:/dev/ -e
OSD_DEVICE=/dev/vdd ceph/daemon osd
From the above commands I am able to spawn monitor and manager
properly. I verified this by firing this command on both monitor and
manager containers:
sudo docker exec d1ab985 ceph -s
I get following outputs for both:
cluster:
id: 14a6e40a-8e54-4851-a881-661a84b3441c
health: HEALTH_OK
services:
mon: 1 daemons, quorum serverceph-VirtualBox (age 62m)
mgr: serverceph-VirtualBox(active, since 56m)
osd: 0 osds: 0 up, 0 in
data:
pools: 0 pools, 0 pgs
objects: 0 objects, 0 B
usage: 0 B used, 0 B / 0 B avail
pgs:
However when I try to bring up OSD using above command, it doesn't
work. Docker logs show this output:
2019-04-18 07:30:06 /opt/ceph-container/bin/entrypoint.sh: static:
does not generate config
2019-04-18 07:30:06 /opt/ceph-container/bin/entrypoint.sh: ERROR- The
device pointed by OSD_DEVICE (/dev/vdd) doesn't exist !
I am not sure why the doc asks to pass /dev/vdd to OSD_DEVICE env var.
I know there are five different ways to spawning the OSD, but I am not
able to figure out which one would be suitable for a simple
deployment. If you could please let me know how to spawn OSDs using
Docker, it would help a lot.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
