Re: ceph-deploy and journal on separate disk

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 

>>>>So that is after running `disk zap`. What does it say after using ceph-deploy >>>>and failing?

 

After ceph-disk -v prepare /dev/sdaa /dev/sda1:

 

root@ceph001:~# parted /dev/sdaa print

Model: ATA ST3000DM001-1CH1 (scsi)

Disk /dev/sdaa: 3001GB

Sector size (logical/physical): 512B/4096B

Partition Table: gpt

 

Number  Start   End     Size    File system  Name       Flags

1      1049kB  3001GB  3001GB  xfs          ceph data

 

And

 

root@ceph001:~# parted /dev/sda1 print

Model: Unknown (unknown)

Disk /dev/sda1: 10.7GB

Sector size (logical/physical): 512B/512B

Partition Table: gpt

 

Number  Start  End  Size  File system  Name  Flags

 

With the same errors:

 

root@ceph001:~# ceph-disk -v prepare /dev/sdaa /dev/sda1

DEBUG:ceph-disk:Journal /dev/sda1 is a partition

WARNING:ceph-disk:OSD will not be hot-swappable if journal is not the same device as the osd data

DEBUG:ceph-disk:Creating osd partition on /dev/sdaa

Information: Moved requested sector from 34 to 2048 in

order to align on 2048-sector boundaries.

The operation has completed successfully.

DEBUG:ceph-disk:Creating xfs fs on /dev/sdaa1

meta-data="" isize=2048   agcount=32, agsize=22892700 blks

         =                       sectsz=512   attr=2, projid32bit=0

data     =                       bsize=4096   blocks=732566385, imaxpct=5

         =                       sunit=0      swidth=0 blks

naming   =version 2              bsize=4096   ascii-ci=0

log      =internal log           bsize=4096   blocks=357698, version=2

         =                       sectsz=512   sunit=0 blks, lazy-count=1

realtime =none                   extsz=4096   blocks=0, rtextents=0

DEBUG:ceph-disk:Mounting /dev/sdaa1 on /var/lib/ceph/tmp/mnt.UkJbwx with options noatime

mount: /dev/sdaa1: more filesystems detected. This should not happen,

       use -t <type> to explicitly specify the filesystem type or

       use wipefs(8) to clean up the device.

 

mount: you must specify the filesystem type

ceph-disk: Mounting filesystem failed: Command '['mount', '-o', 'noatime', '--', '/dev/sdaa1', '/var/lib/ceph/tmp/mnt.UkJbwx']' returned non-zero exit status 32

 

 

From: Alfredo Deza [mailto:alfredo.deza@xxxxxxxxxxx]
Sent: Wednesday, August 14, 2013 7:44 PM
To: Pavel Timoschenkov
Cc: Samuel Just; ceph-users@xxxxxxxx
Subject: Re: ceph-deploy and journal on separate disk

 

 

 

On Wed, Aug 14, 2013 at 10:47 AM, Pavel Timoschenkov <Pavel@xxxxxxxxxxxxxxxxxxxxxxxxxxx> wrote:

 

 

From: Alfredo Deza [mailto:alfredo.deza@xxxxxxxxxxx]
Sent: Wednesday, August 14, 2013 5:41 PM
To: Pavel Timoschenkov
Cc: Samuel Just; ceph-users@xxxxxxxx


Subject: Re: ceph-deploy and journal on separate disk

 

 

 

On Wed, Aug 14, 2013 at 7:41 AM, Pavel Timoschenkov <Pavel@xxxxxxxxxxxxxxxxxxxxxxxxxxx> wrote:

>>>It looks like at some point the filesystem is not passed to the options. Would >>>you mind running the `ceph-disk-prepare` command again but with

>>>the --verbose flag?

>>>I think that from the output above (correct it if I am mistaken) that would be >>>something like:

>>>ceph-disk-prepare --verbose -- /dev/sdaa /dev/sda1

 

Hi.

If I’m running:

ceph-deploy disk zap ceph001:sdaa ceph001:sda1

and

ceph-disk -v prepare /dev/sdaa /dev/sda1, get the same errors:

======================================================

root@ceph001:~# ceph-disk -v prepare /dev/sdaa /dev/sda1

DEBUG:ceph-disk:Journal /dev/sda1 is a partition

WARNING:ceph-disk:OSD will not be hot-swappable if journal is not the same device as the osd data

DEBUG:ceph-disk:Creating osd partition on /dev/sdaa

Information: Moved requested sector from 34 to 2048 in

order to align on 2048-sector boundaries.

The operation has completed successfully.

DEBUG:ceph-disk:Creating xfs fs on /dev/sdaa1

meta-data="" isize=2048   agcount=32, agsize=22892700 blks

         =                       sectsz=512   attr=2, projid32bit=0

data     =                       bsize=4096   blocks=732566385, imaxpct=5

         =                       sunit=0      swidth=0 blks

naming   =version 2              bsize=4096   ascii-ci=0

log      =internal log           bsize=4096   blocks=357698, version=2

         =                       sectsz=512   sunit=0 blks, lazy-count=1

realtime =none                   extsz=4096   blocks=0, rtextents=0

DEBUG:ceph-disk:Mounting /dev/sdaa1 on /var/lib/ceph/tmp/mnt.EGTIq2 with options noatime

mount: /dev/sdaa1: more filesystems detected. This should not happen,

       use -t <type> to explicitly specify the filesystem type or

       use wipefs(8) to clean up the device.

 

mount: you must specify the filesystem type

ceph-disk: Mounting filesystem failed: Command '['mount', '-o', 'noatime', '--', '/dev/sdaa1', '/var/lib/ceph/tmp/mnt.EGTIq2']' returned non-zero exit status 32

 

If executed this command separately for both disks - looks like ok:

 

For sdaa:

 

root@ceph001:~# ceph-disk -v prepare /dev/sdaa

INFO:ceph-disk:Will colocate journal with data on /dev/sdaa

DEBUG:ceph-disk:Creating journal partition num 2 size 1024 on /dev/sdaa

Information: Moved requested sector from 34 to 2048 in

order to align on 2048-sector boundaries.

The operation has completed successfully.

DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/d1389210-6e02-4460-9cb2-0e31e4b0924f

DEBUG:ceph-disk:Creating osd partition on /dev/sdaa

Information: Moved requested sector from 2097153 to 2099200 in

order to align on 2048-sector boundaries.

The operation has completed successfully.

DEBUG:ceph-disk:Creating xfs fs on /dev/sdaa1

meta-data="" isize=2048   agcount=32, agsize=22884508 blks

         =                       sectsz=512   attr=2, projid32bit=0

data     =                       bsize=4096   blocks=732304241, imaxpct=5

         =                       sunit=0      swidth=0 blks

naming   =version 2              bsize=4096   ascii-ci=0

log      =internal log           bsize=4096   blocks=357570, version=2

         =                       sectsz=512   sunit=0 blks, lazy-count=1

realtime =none                   extsz=4096   blocks=0, rtextents=0

DEBUG:ceph-disk:Mounting /dev/sdaa1 on /var/lib/ceph/tmp/mnt.K3q9v5 with options noatime

DEBUG:ceph-disk:Preparing osd data dir /var/lib/ceph/tmp/mnt.K3q9v5

DEBUG:ceph-disk:Creating symlink /var/lib/ceph/tmp/mnt.K3q9v5/journal -> /dev/disk/by-partuuid/d1389210-6e02-4460-9cb2-0e31e4b0924f

DEBUG:ceph-disk:Unmounting /var/lib/ceph/tmp/mnt.K3q9v5

The operation has completed successfully.

DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sdaa

 

For sda1:

 

root@ceph001:~# ceph-disk -v prepare /dev/sda1

DEBUG:ceph-disk:OSD data device /dev/sda1 is a partition

DEBUG:ceph-disk:Creating xfs fs on /dev/sda1

meta-data="" isize=2048   agcount=4, agsize=655360 blks

         =                       sectsz=512   attr=2, projid32bit=0

data     =                       bsize=4096   blocks=2621440, imaxpct=25

         =                       sunit=0      swidth=0 blks

naming   =version 2              bsize=4096   ascii-ci=0

log      =internal log           bsize=4096   blocks=2560, version=2

         =                       sectsz=512   sunit=0 blks, lazy-count=1

realtime =none                   extsz=4096   blocks=0, rtextents=0

DEBUG:ceph-disk:Mounting /dev/sda1 on /var/lib/ceph/tmp/mnt.G30zPD with options noatime

DEBUG:ceph-disk:Preparing osd data dir /var/lib/ceph/tmp/mnt.G30zPD

DEBUG:ceph-disk:Unmounting /var/lib/ceph/tmp/mnt.G30zPD

DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sda1

 

 

The separate commands (e.g. `ceph-disk -v prepare /dev/sda1`) works because then the journal is on the same device as the OSD data, so the execution is different to get them to a working state.

I suspect that there are left over partitions in /dev/sdaa that are causing this to fail, I *think* that we could pass the `-t` flag with the filesystem and prevent this.

Just to be sure, could you list all the partitions on /dev/sdaa (if /dev/sdaa is the whole device)?

Something like:

    sudo parted /dev/sdaa print

Or if you prefer any other way that could tell use what are all the partitions in that device.

 

 

After

ceph-deploy disk zap ceph001:sdaa ceph001:sda1

 

root@ceph001:~# parted /dev/sdaa print

Model: ATA ST3000DM001-1CH1 (scsi)

Disk /dev/sdaa: 3001GB

Sector size (logical/physical): 512B/4096B

Partition Table: gpt

 

Number  Start  End  Size  File system  Name  Flags

 

root@ceph001:~# parted /dev/sda1 print

Model: Unknown (unknown)

Disk /dev/sda1: 10.7GB

Sector size (logical/physical): 512B/512B

Partition Table: gpt

So that is after running `disk zap`. What does it say after using ceph-deploy and failing?

 

Number  Start  End  Size  File system  Name  Flags

 

 

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]

  Powered by Linux