Solved! I stumbled into the solution while switching from block device to a file. I was being bit by running mkcephfs multiple times -- it wasn't really failing on the journal, it was failing because the OSD data disk had been initialized before. I couldn't see that until I used a file for the journal and then I see log output like: === osd.0 === 2012-11-08 16:41:37.677620 7ffc3cfcd780 -1 provided osd id 0 != superblock's -1 2012-11-08 16:41:37.678726 7ffc3cfcd780 -1 ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-0: (22) Invalid argument I unmounted the OSD's that had been touched before, reformatted them, and then remounted. I setup ceph.conf to use block devices for the journals, and then everything proceeded normally. So the final relevant bits from my ceph.conf file look like: [osd] osd journal size = 0 journal dio = true journal aio = true [osd.0] host = ceph1 osd journal = /dev/sda5 [osd.1] host = ceph1 osd journal = /dev/sda6 ... Thanks, - Travis On Thu, Nov 8, 2012 at 10:08 AM, Travis Rhoden <trhoden@xxxxxxxxx> wrote: > One more thing -- Google search says this is harmless -- I see quite a > few of these in syslog: > > hdparm: sending ioctl 2285 to a partition! -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html