I used another host for osd (with that same name), but now with Debian 7.4
ceph-deploy osd prepare ceph-node0:sdb --dmcrypt
[ceph_deploy.cli][INFO ] Invoked (1.3.5): /usr/bin/ceph-deploy osd prepare ceph-node0:sdb --dmcrypt
[ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks ceph-node0:/dev/sdb:
[ceph-node0][DEBUG ] connected to host: ceph-node0
[ceph-node0][DEBUG ] detect platform information from remote host
[ceph-node0][DEBUG ] detect machine type
[ceph_deploy.osd][INFO ] Distro info: debian 7.4 wheezy
[ceph_deploy.osd][DEBUG ] Deploying osd to ceph-node0
[ceph-node0][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph-node0][WARNIN] osd keyring does not exist yet, creating one
[ceph-node0][DEBUG ] create a keyring file
[ceph-node0][INFO ] Running command: udevadm trigger --subsystem-match=block --action=""> [ceph_deploy.osd][DEBUG ] Preparing host ceph-node0 disk /dev/sdb journal None activate False
[ceph-node0][INFO ] Running command: ceph-disk-prepare --fs-type xfs --dmcrypt --dmcrypt-key-dir /etc/ceph/dmcrypt-keys --cluster ceph -- /dev/sdb
[ceph-node0][WARNIN] INFO:ceph-disk:Will colocate journal with data on /dev/sdb
[ceph-node0][WARNIN] ceph-disk: Error: partition 1 for /dev/sdb does not appear to exist
[ceph-node0][DEBUG ] Information: Moved requested sector from 34 to 2048 in
[ceph-node0][DEBUG ] order to align on 2048-sector boundaries.
[ceph-node0][DEBUG ] The operation has completed successfully.
[ceph-node0][DEBUG ] Information: Moved requested sector from 10485761 to 10487808 in
[ceph-node0][DEBUG ] order to align on 2048-sector boundaries.
[ceph-node0][DEBUG ] Warning: The kernel is still using the old partition table.
[ceph-node0][DEBUG ] The new table will be used at the next reboot.
[ceph-node0][DEBUG ] The operation has completed successfully.
[ceph-node0][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy.osd][ERROR ] Failed to execute command: ceph-disk-prepare --fs-type xfs --dmcrypt --dmcrypt-key-dir /etc/ceph/dmcrypt-keys --cluster ceph -- /dev/sdb
[ceph_deploy][ERROR ] GenericError: Failed to create 1 OSDs
This subcommand fails on OSD host -> ceph-disk-prepare
I run this command on ceph-node0, and...
ceph-disk-prepare --fs-type xfs --dmcrypt --dmcrypt-key-dir /etc/ceph/dmcrypt-keys --cluster ceph -- /dev/sdb
ceph-disk: Error: Device /dev/sdb2 is in use by a device-mapper mapping (dm-crypt?): dm-0
I can reply this error with 100% guarantee.
I used Debian 7.4 and Ubuntu 13.04 for test.
Best Regards,
Michael Lukzak
| Hi, I tried to use whole new blank disk to create two separate partition (one for data and second for journal) and use dmcrypt, but there is a problem with use this. It's looks like there is a problem with mounting or formatting partitions. OS is Ubuntu 13.04 with ceph v0.72 (emperor) I used command: ceph-deploy osd prepare ceph-node0:sdb --dmcrypt --dmcrypt-key-dir=/root --fs-type=xfs [ceph-node0][WARNIN] INFO:ceph-disk:Will colocate journal with data on /dev/sdb [ceph-node0][DEBUG ] Creating new GPT entries. [ceph-node0][DEBUG ] Information: Moved requested sector from 34 to 2048 in [ceph-node0][DEBUG ] order to align on 2048-sector boundaries. [ceph-node0][DEBUG ] The operation has completed successfully. [ceph-node0][DEBUG ] Information: Moved requested sector from 10485761 to 10487808 in [ceph-node0][DEBUG ] order to align on 2048-sector boundaries. [ceph-node0][DEBUG ] The operation has completed successfully. [ceph-node0][DEBUG ] meta-data="" isize=2048 agcount=4, agsize=720831 blks [ceph-node0][DEBUG ] = sectsz=512 attr=2, projid32bit=0 [ceph-node0][DEBUG ] data = bsize=4096 blocks=2883323, imaxpct=25 [ceph-node0][DEBUG ] = sunit=0 swidth=0 blks [ceph-node0][DEBUG ] naming =version 2 bsize=4096 ascii-ci=0 [ceph-node0][DEBUG ] log =internal log bsize=4096 blocks=2560, version=2 [ceph-node0][DEBUG ] = sectsz=512 sunit=0 blks, lazy-count=1 [ceph-node0][DEBUG ] realtime =none extsz=4096 blocks=0, rtextents=0 [ceph-node0][DEBUG ] The operation has completed successfully. [ceph_deploy.osd][DEBUG ] Host ceph-node0 is now ready for osd use. Here it's look like all was good, but on the host ceph-node0 (where is disk sdb) is a problem. Here are dump from syslog (at ceph-node0) Mar 17 14:03:02 ceph-node0 kernel: [ 68.645938] sd 2:0:1:0: [sdb] Cache data unavailable Mar 17 14:03:02 ceph-node0 kernel: [ 68.645943] sd 2:0:1:0: [sdb] Assuming drive cache: write through Mar 17 14:03:02 ceph-node0 kernel: [ 68.708930] sdb: sdb1 sdb2 Mar 17 14:03:02 ceph-node0 kernel: [ 68.996013] bio: create slab <bio-1> at 1 Mar 17 14:03:03 ceph-node0 kernel: [ 69.613407] SGI XFS with ACLs, security attributes, realtime, large block/inode numbers, no debug enabled Mar 17 14:03:03 ceph-node0 kernel: [ 69.619904] XFS (dm-0): Mounting Filesystem Mar 17 14:03:03 ceph-node0 kernel: [ 69.658693] XFS (dm-0): Ending clean mount Mar 17 14:03:04 ceph-node0 kernel: [ 70.745337] sd 2:0:1:0: [sdb] Cache data unavailable Mar 17 14:03:04 ceph-node0 kernel: [ 70.745342] sd 2:0:1:0: [sdb] Assuming drive cache: write through Mar 17 14:03:04 ceph-node0 kernel: [ 70.750667] sdb: sdb1 sdb2 Mar 17 14:04:05 ceph-node0 udevd[515]: timeout: killing '/bin/bash -c 'while [ ! -e /dev/mapper/d92421e6-27c3-498a-9754-b5c10a281500 ];do sleep 1; done'' [1903] Mar 17 14:04:05 ceph-node0 udevd[515]: '/bin/bash -c 'while [ ! -e /dev/mapper/d92421e6-27c3-498a-9754-b5c10a281500 ];do sleep 1; done'' [1903] terminated by signal 9 (Killed) Mar 17 14:05:07 ceph-node0 udevd[515]: timeout: killing '/bin/bash -c 'while [ ! -e /dev/mapper/d92421e6-27c3-498a-9754-b5c10a281500 ];do sleep 1; done'' [2215] Mar 17 14:05:07 ceph-node0 udevd[515]: '/bin/bash -c 'while [ ! -e /dev/mapper/d92421e6-27c3-498a-9754-b5c10a281500 ];do sleep 1; done'' [2215] terminated by signal 9 (Killed) Two partitions (sdb1 and sdb2) are created, but it looks like is a problem with mounting or formating it? I can't figure out. parted show that sdb1 and sdb2 exists, but in collumn filesystem is empty 2 1049kB 5369MB 5368MB ceph journal 1 5370MB 17,2GB 11,8GB ceph data Keys for dmcrypt are stored in /root So lets try without switch --dmcrypt ceph-deploy osd prepare ceph-node0:sdb --fs-type=xfs [ceph_deploy.cli][INFO ] Invoked (1.3.5): /usr/bin/ceph-deploy osd prepare ceph-node0:sdb --fs-type=xfs [ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks ceph-node0:/dev/sdb: [ceph-node0][DEBUG ] connected to host: ceph-node0 [ceph-node0][DEBUG ] detect platform information from remote host [ceph-node0][DEBUG ] detect machine type [ceph_deploy.osd][INFO ] Distro info: Ubuntu 13.04 raring [ceph_deploy.osd][DEBUG ] Deploying osd to ceph-node0 [ceph-node0][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [ceph-node0][INFO ] Running command: udevadm trigger --subsystem-match=block --action=""> [ceph_deploy.osd][DEBUG ] Preparing host ceph-node0 disk /dev/sdb journal None activate False [ceph-node0][INFO ] Running command: ceph-disk-prepare --fs-type xfs --cluster ceph -- /dev/sdb [ceph-node0][WARNIN] INFO:ceph-disk:Will colocate journal with data on /dev/sdb [ceph-node0][DEBUG ] Information: Moved requested sector from 34 to 2048 in [ceph-node0][DEBUG ] order to align on 2048-sector boundaries. [ceph-node0][DEBUG ] The operation has completed successfully. [ceph-node0][DEBUG ] Information: Moved requested sector from 10485761 to 10487808 in [ceph-node0][DEBUG ] order to align on 2048-sector boundaries. [ceph-node0][DEBUG ] The operation has completed successfully. [ceph-node0][DEBUG ] meta-data="" isize=2048 agcount=4, agsize=720831 blks [ceph-node0][DEBUG ] = sectsz=512 attr=2, projid32bit=0 [ceph-node0][DEBUG ] data = bsize=4096 blocks=2883323, imaxpct=25 [ceph-node0][DEBUG ] = sunit=0 swidth=0 blks [ceph-node0][DEBUG ] naming =version 2 bsize=4096 ascii-ci=0 [ceph-node0][DEBUG ] log =internal log bsize=4096 blocks=2560, version=2 [ceph-node0][DEBUG ] = sectsz=512 sunit=0 blks, lazy-count=1 [ceph-node0][DEBUG ] realtime =none extsz=4096 blocks=0, rtextents=0 [ceph-node0][DEBUG ] The operation has completed successfully. [ceph_deploy.osd][DEBUG ] Host ceph-node0 is now ready for osd use. Two partition sdb1 and sdb2 are created and mounted properly on host ceph-node0. Here is dump from syslog (at ceph-node0). Mar 17 14:08:20 ceph-node0 kernel: [ 385.330968] sd 2:0:1:0: [sdb] Cache data unavailable Mar 17 14:08:20 ceph-node0 kernel: [ 385.330973] sd 2:0:1:0: [sdb] Assuming drive cache: write through Mar 17 14:08:20 ceph-node0 kernel: [ 385.335410] sdb: sdb2 Mar 17 14:08:21 ceph-node0 kernel: [ 386.845878] sd 2:0:1:0: [sdb] Cache data unavailable Mar 17 14:08:21 ceph-node0 kernel: [ 386.845883] sd 2:0:1:0: [sdb] Assuming drive cache: write through Mar 17 14:08:21 ceph-node0 kernel: [ 386.851324] sdb: sdb1 sdb2 Mar 17 14:08:22 ceph-node0 kernel: [ 387.469774] XFS (sdb1): Mounting Filesystem Mar 17 14:08:22 ceph-node0 kernel: [ 387.492869] XFS (sdb1): Ending clean mount Mar 17 14:08:23 ceph-node0 kernel: [ 388.549737] sd 2:0:1:0: [sdb] Cache data unavailable Mar 17 14:08:23 ceph-node0 kernel: [ 388.549742] sd 2:0:1:0: [sdb] Assuming drive cache: write through Mar 17 14:08:23 ceph-node0 kernel: [ 388.564160] sdb: sdb1 sdb2 Mar 17 14:08:23 ceph-node0 kernel: [ 388.922841] XFS (sdb1): Mounting Filesystem Mar 17 14:08:23 ceph-node0 kernel: [ 388.974655] XFS (sdb1): Ending clean mount And dump from parted (now at sdb1 we see that filesystem is present (xfs)). 2 1049kB 5369MB 5368MB ceph journal 1 5370MB 17,2GB 11,8GB xfs ceph data Ceph show that new OSD was arrived, there is no problem to activate and use it. Ok, so my question is - what problem is with dmcrypt? -- Pozdrowienia, Michael Lukzak |
--
Pozdrowienia,
Michael Lukzak
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
