I tried to use whole new blank disk to create two separate partition (one for data and second for journal)
and use dmcrypt, but there is a problem with use this. It's looks like there is a problem with mounting or
formatting partitions.
OS is Ubuntu 13.04 with ceph v0.72 (emperor)
I used command:
ceph-deploy osd prepare ceph-node0:sdb --dmcrypt --dmcrypt-key-dir=/root --fs-type=xfs
[ceph-node0][WARNIN] INFO:ceph-disk:Will colocate journal with data on /dev/sdb
[ceph-node0][DEBUG ] Creating new GPT entries.
[ceph-node0][DEBUG ] Information: Moved requested sector from 34 to 2048 in
[ceph-node0][DEBUG ] order to align on 2048-sector boundaries.
[ceph-node0][DEBUG ] The operation has completed successfully.
[ceph-node0][DEBUG ] Information: Moved requested sector from 10485761 to 10487808 in
[ceph-node0][DEBUG ] order to align on 2048-sector boundaries.
[ceph-node0][DEBUG ] The operation has completed successfully.
[ceph-node0][DEBUG ] meta-data="" isize=2048 agcount=4, agsize=720831 blks
[ceph-node0][DEBUG ] = sectsz=512 attr=2, projid32bit=0
[ceph-node0][DEBUG ] data = bsize=4096 blocks=2883323, imaxpct=25
[ceph-node0][DEBUG ] = sunit=0 swidth=0 blks
[ceph-node0][DEBUG ] naming =version 2 bsize=4096 ascii-ci=0
[ceph-node0][DEBUG ] log =internal log bsize=4096 blocks=2560, version=2
[ceph-node0][DEBUG ] = sectsz=512 sunit=0 blks, lazy-count=1
[ceph-node0][DEBUG ] realtime =none extsz=4096 blocks=0, rtextents=0
[ceph-node0][DEBUG ] The operation has completed successfully.
[ceph_deploy.osd][DEBUG ] Host ceph-node0 is now ready for osd use.
Here it's look like all was good, but on the host ceph-node0 (where is disk sdb) is a problem.
Here are dump from syslog (at ceph-node0)
Mar 17 14:03:02 ceph-node0 kernel: [ 68.645938] sd 2:0:1:0: [sdb] Cache data unavailable
Mar 17 14:03:02 ceph-node0 kernel: [ 68.645943] sd 2:0:1:0: [sdb] Assuming drive cache: write through
Mar 17 14:03:02 ceph-node0 kernel: [ 68.708930] sdb: sdb1 sdb2
Mar 17 14:03:02 ceph-node0 kernel: [ 68.996013] bio: create slab <bio-1> at 1
Mar 17 14:03:03 ceph-node0 kernel: [ 69.613407] SGI XFS with ACLs, security attributes, realtime, large block/inode numbers, no debug enabled
Mar 17 14:03:03 ceph-node0 kernel: [ 69.619904] XFS (dm-0): Mounting Filesystem
Mar 17 14:03:03 ceph-node0 kernel: [ 69.658693] XFS (dm-0): Ending clean mount
Mar 17 14:03:04 ceph-node0 kernel: [ 70.745337] sd 2:0:1:0: [sdb] Cache data unavailable
Mar 17 14:03:04 ceph-node0 kernel: [ 70.745342] sd 2:0:1:0: [sdb] Assuming drive cache: write through
Mar 17 14:03:04 ceph-node0 kernel: [ 70.750667] sdb: sdb1 sdb2
Mar 17 14:04:05 ceph-node0 udevd[515]: timeout: killing '/bin/bash -c 'while [ ! -e /dev/mapper/d92421e6-27c3-498a-9754-b5c10a281500 ];do sleep 1; done'' [1903]
Mar 17 14:04:05 ceph-node0 udevd[515]: '/bin/bash -c 'while [ ! -e /dev/mapper/d92421e6-27c3-498a-9754-b5c10a281500 ];do sleep 1; done'' [1903] terminated by signal 9 (Killed)
Mar 17 14:05:07 ceph-node0 udevd[515]: timeout: killing '/bin/bash -c 'while [ ! -e /dev/mapper/d92421e6-27c3-498a-9754-b5c10a281500 ];do sleep 1; done'' [2215]
Mar 17 14:05:07 ceph-node0 udevd[515]: '/bin/bash -c 'while [ ! -e /dev/mapper/d92421e6-27c3-498a-9754-b5c10a281500 ];do sleep 1; done'' [2215] terminated by signal 9 (Killed)
Two partitions (sdb1 and sdb2) are created, but it looks like is a problem with mounting or formating it? I can't figure out.
parted show that sdb1 and sdb2 exists, but in collumn filesystem is empty
2 1049kB 5369MB 5368MB ceph journal
1 5370MB 17,2GB 11,8GB ceph data
Keys for dmcrypt are stored in /root
So lets try without switch --dmcrypt
ceph-deploy osd prepare ceph-node0:sdb --fs-type=xfs
[ceph_deploy.cli][INFO ] Invoked (1.3.5): /usr/bin/ceph-deploy osd prepare ceph-node0:sdb --fs-type=xfs
[ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks ceph-node0:/dev/sdb:
[ceph-node0][DEBUG ] connected to host: ceph-node0
[ceph-node0][DEBUG ] detect platform information from remote host
[ceph-node0][DEBUG ] detect machine type
[ceph_deploy.osd][INFO ] Distro info: Ubuntu 13.04 raring
[ceph_deploy.osd][DEBUG ] Deploying osd to ceph-node0
[ceph-node0][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph-node0][INFO ] Running command: udevadm trigger --subsystem-match=block --action=""> [ceph_deploy.osd][DEBUG ] Preparing host ceph-node0 disk /dev/sdb journal None activate False
[ceph-node0][INFO ] Running command: ceph-disk-prepare --fs-type xfs --cluster ceph -- /dev/sdb
[ceph-node0][WARNIN] INFO:ceph-disk:Will colocate journal with data on /dev/sdb
[ceph-node0][DEBUG ] Information: Moved requested sector from 34 to 2048 in
[ceph-node0][DEBUG ] order to align on 2048-sector boundaries.
[ceph-node0][DEBUG ] The operation has completed successfully.
[ceph-node0][DEBUG ] Information: Moved requested sector from 10485761 to 10487808 in
[ceph-node0][DEBUG ] order to align on 2048-sector boundaries.
[ceph-node0][DEBUG ] The operation has completed successfully.
[ceph-node0][DEBUG ] meta-data="" isize=2048 agcount=4, agsize=720831 blks
[ceph-node0][DEBUG ] = sectsz=512 attr=2, projid32bit=0
[ceph-node0][DEBUG ] data = bsize=4096 blocks=2883323, imaxpct=25
[ceph-node0][DEBUG ] = sunit=0 swidth=0 blks
[ceph-node0][DEBUG ] naming =version 2 bsize=4096 ascii-ci=0
[ceph-node0][DEBUG ] log =internal log bsize=4096 blocks=2560, version=2
[ceph-node0][DEBUG ] = sectsz=512 sunit=0 blks, lazy-count=1
[ceph-node0][DEBUG ] realtime =none extsz=4096 blocks=0, rtextents=0
[ceph-node0][DEBUG ] The operation has completed successfully.
[ceph_deploy.osd][DEBUG ] Host ceph-node0 is now ready for osd use.
Two partition sdb1 and sdb2 are created and mounted properly on host ceph-node0.
Here is dump from syslog (at ceph-node0).
Mar 17 14:08:20 ceph-node0 kernel: [ 385.330968] sd 2:0:1:0: [sdb] Cache data unavailable
Mar 17 14:08:20 ceph-node0 kernel: [ 385.330973] sd 2:0:1:0: [sdb] Assuming drive cache: write through
Mar 17 14:08:20 ceph-node0 kernel: [ 385.335410] sdb: sdb2
Mar 17 14:08:21 ceph-node0 kernel: [ 386.845878] sd 2:0:1:0: [sdb] Cache data unavailable
Mar 17 14:08:21 ceph-node0 kernel: [ 386.845883] sd 2:0:1:0: [sdb] Assuming drive cache: write through
Mar 17 14:08:21 ceph-node0 kernel: [ 386.851324] sdb: sdb1 sdb2
Mar 17 14:08:22 ceph-node0 kernel: [ 387.469774] XFS (sdb1): Mounting Filesystem
Mar 17 14:08:22 ceph-node0 kernel: [ 387.492869] XFS (sdb1): Ending clean mount
Mar 17 14:08:23 ceph-node0 kernel: [ 388.549737] sd 2:0:1:0: [sdb] Cache data unavailable
Mar 17 14:08:23 ceph-node0 kernel: [ 388.549742] sd 2:0:1:0: [sdb] Assuming drive cache: write through
Mar 17 14:08:23 ceph-node0 kernel: [ 388.564160] sdb: sdb1 sdb2
Mar 17 14:08:23 ceph-node0 kernel: [ 388.922841] XFS (sdb1): Mounting Filesystem
Mar 17 14:08:23 ceph-node0 kernel: [ 388.974655] XFS (sdb1): Ending clean mount
And dump from parted (now at sdb1 we see that filesystem is present (xfs)).
2 1049kB 5369MB 5368MB ceph journal
1 5370MB 17,2GB 11,8GB xfs ceph data
Ceph show that new OSD was arrived, there is no problem to activate and use it.
Ok, so my question is - what problem is with dmcrypt?
--
Pozdrowienia,
Michael Lukzak
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
