Sorry for the late reply - been traveling. I'm doing exactly that right now, using the ceph-docker container. It's just in my test rack for now, but hardware arrived this week to seed the production version. I'm using separate containers for each daemon, including a container for each OSD. I've got a bit of cloudinit logic to loop over all disks in a machine, fire off a "prepare" container if the disk isn't partitioned, then start an "activate" container to bring the OSD up. Works pretty well; I can power on a new machine, and get a stack of new OSDs about 5 minutes later. I've opted to not allow ANY containers to run on local disk, and we're setting up appropriate volume plugins (for NFS, CephFS, and Ceph RBDs) now. IN THEORY (so far, so good) volumes will be dynamically mapped into the container at startup. This should let us orchestrate containers with Swarm or Kubernetes, and give us the same volumes wherever they land. In a few weeks we'll start experimenting with a vxlan network plugin as well, to allow similar flexibility with IPs and subnets. Once that's done, our registry will become just another container (though we'll need a master registry, with storage on local disk SOMEWHERE, to be able to handle a cold-boot of the Ceph containers). I'm curious where you're going with your environment. To me, Ceph+Docker seems like a nice match; if others are doing this, we should definitely pool experiences. -- Mike Shuey On Thu, May 26, 2016 at 7:00 AM, EnDSgUy EnDSgUy <endsguy@xxxxxxxxxxx> wrote: > Hello All, > > I am looking for some help to design the Ceph for the cluster of 7 > machines running on CoreOS with fleet and docker. I am still thinking > what's the best way for the moment. > > Has anybody done something similair and could advise on his > experiences? > > The primary purpose is > - be able to store "docker data containers" and only them in a > redundant way (so let a specific directory be mounted to specific > container). So Ceph should be availabe in a container. > - ideally other types of containers should be running without > redundancy just on the hard drive > - docker images (registry) should also be stored in a redundant way > > Has anybody done something similair? > > > Dmitry > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |