Re: Shared WAL/DB device partition for multiple OSDs?

Hervé Ballans <herve.ballans@xxxxxxxxxxxxx> · Thu, 23 Aug 2018 17:32:18 +0200

    Le 23/08/2018 à 16:13, Alfredo Deza a
      écrit :

        What you mean is that, at this stage, I must directly declare the UUID paths
in value of --block.db (i.e. replace /dev/nvme0n1p1 with its PARTUUID), that
is ?

      No, this all looks correct. How does the ceph-volume.log and
ceph-volume-systemd.log look when you are booting up for the OSDs that
aren't coming up?

Anything useful in there?

    ceph-volume.log (extract)

    [2018-08-20
      11:26:29,430][ceph_volume.process][INFO  ] Running command:
      systemctl start ceph-osd@1

      [2018-08-20 11:26:32,268][ceph_volume.main][INFO  ] Running
      command: ceph-volume  lvm trigger
      1-4a9954ce-0a0f-432b-a91d-eaacb45287d4

      [2018-08-20 11:26:32,269][ceph_volume.process][INFO  ] Running
      command: /sbin/lvs --noheadings --readonly --separator=";" -o
      lv_tags,lv_path,lv_name,vg_name,lv_uuid

      [2018-08-20 11:26:32,347][ceph_volume.process][INFO  ] stdout
ceph.block_device=/dev/ceph-425a36ca-8b60-471d-80ed-19726714ea3a/osd-block-9380cd27-c0fe-4ede-9ed3-d09eff545037,ceph.block_uuid=kkykJn-EmUa-EvZj-Ffr7-dk06-VUG0-nhnUvB,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=838506b7-e0c6-4022-9e17-2d1cf9458be6,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.db_device=/dev/nvme0n1p4,ceph.db_uuid=ec4cf738-1627-4ddf-a7ef-91810fb8aaab,ceph.encrypted=0,ceph.osd_fsid=9380cd27-c0fe-4ede-9ed3-d09eff545037,ceph.osd_id=3,ceph.type=block,ceph.vdo=0";"/dev/ceph-425a36ca-8b60-471d-80ed-19726714ea3a/osd-block-9380cd27-c0fe-4ede-9ed3-d09eff545037";"osd-block-9380cd27-c0fe-4ede-9ed3-d09eff545037";"ceph-425a36ca-8b60-471d-80ed-19726714ea3a";"kkykJn-EmUa-EvZj-Ffr7-dk06-VUG0-nhnUvB

      [2018-08-20 11:26:32,347][ceph_volume.process][INFO  ] stdout
ceph.block_device=/dev/ceph-597bbc7c-33ba-4020-a1c6-fe4297ad0421/osd-block-5d4af2fc-388c-4795-9d1a-53ad8aba56d8,ceph.block_uuid=WdXALj-0bCs-Ndfy-xGIN-q1rF-hcmg-utIiNB,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=838506b7-e0c6-4022-9e17-2d1cf9458be6,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.db_device=/dev/nvme0n1p8,ceph.db_uuid=9642a688-0991-4a19-b49a-994a22edaf60,ceph.encrypted=0,ceph.osd_fsid=5d4af2fc-388c-4795-9d1a-53ad8aba56d8,ceph.osd_id=7,ceph.type=block,ceph.vdo=0";"/dev/ceph-597bbc7c-33ba-4020-a1c6-fe4297ad0421/osd-block-5d4af2fc-388c-4795-9d1a-53ad8aba56d8";"osd-block-5d4af2fc-388c-4795-9d1a-53ad8aba56d8";"ceph-597bbc7c-33ba-4020-a1c6-fe4297ad0421";"WdXALj-0bCs-Ndfy-xGIN-q1rF-hcmg-utIiNB

      [2018-08-20 11:26:32,347][ceph_volume.process][INFO  ] stdout
ceph.block_device=/dev/ceph-5dbc3f3a-d781-4d22-9e9c-8c7a74ce39fb/osd-block-b8e82f22-e993-4458-984b-90232b8b3d55,ceph.block_uuid=Q1Aeu5-bm6V-m9yB-O3vX-SFvX-8oHo-SNkrfe,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=838506b7-e0c6-4022-9e17-2d1cf9458be6,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.db_device=/dev/nvme0n1p3,ceph.db_uuid=c7bb1f33-65f7-4116-ae27-0d73dde82208,ceph.encrypted=0,ceph.osd_fsid=b8e82f22-e993-4458-984b-90232b8b3d55,ceph.osd_id=2,ceph.type=block,ceph.vdo=0";"/dev/ceph-5dbc3f3a-d781-4d22-9e9c-8c7a74ce39fb/osd-block-b8e82f22-e993-4458-984b-90232b8b3d55";"osd-block-b8e82f22-e993-4458-984b-90232b8b3d55";"ceph-5dbc3f3a-d781-4d22-9e9c-8c7a74ce39fb";"Q1Aeu5-bm6V-m9yB-O3vX-SFvX-8oHo-SNkrfe

      [2018-08-20 11:26:32,347][ceph_volume.process][INFO  ] stdout
ceph.block_device=/dev/ceph-70e7b5f7-408d-4810-9d92-d6679d621db0/osd-block-02540fff-5478-4a67-bf5c-679c72150e8d,ceph.block_uuid=jp7YJ5-vNcj-93at-49LK-bSWT-ke4m-Y0Brzy,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=838506b7-e0c6-4022-9e17-2d1cf9458be6,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.db_device=/dev/nvme0n1p5,ceph.db_uuid=b5ea7d3a-8c90-4163-a2af-a4dfa045ced6,ceph.encrypted=0,ceph.osd_fsid=02540fff-5478-4a67-bf5c-679c72150e8d,ceph.osd_id=4,ceph.type=block,ceph.vdo=0";"/dev/ceph-70e7b5f7-408d-4810-9d92-d6679d621db0/osd-block-02540fff-5478-4a67-bf5c-679c72150e8d";"osd-block-02540fff-5478-4a67-bf5c-679c72150e8d";"ceph-70e7b5f7-408d-4810-9d92-d6679d621db0";"jp7YJ5-vNcj-93at-49LK-bSWT-ke4m-Y0Brzy

      [2018-08-20 11:26:32,347][ceph_volume.process][INFO  ] stdout
ceph.block_device=/dev/ceph-766bd78c-ed1a-4e27-8b4d-7adc4c4f2f0d/osd-block-98bfb597-009b-4e88-bc5e-dd22587d21fe,ceph.block_uuid=enriOd-To2e-lLWi-Dc91-keUr-1tf7-4Hvrj2,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=838506b7-e0c6-4022-9e17-2d1cf9458be6,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.db_device=/dev/nvme0n1p1,ceph.db_uuid=99870da1-b7d1-479d-ad63-14407c593278,ceph.encrypted=0,ceph.osd_fsid=98bfb597-009b-4e88-bc5e-dd22587d21fe,ceph.osd_id=0,ceph.type=block,ceph.vdo=0";"/dev/ceph-766bd78c-ed1a-4e27-8b4d-7adc4c4f2f0d/osd-block-98bfb597-009b-4e88-bc5e-dd22587d21fe";"osd-block-98bfb597-009b-4e88-bc5e-dd22587d21fe";"ceph-766bd78c-ed1a-4e27-8b4d-7adc4c4f2f0d";"enriOd-To2e-lLWi-Dc91-keUr-1tf7-4Hvrj2

      [2018-08-20 11:26:32,347][ceph_volume.process][INFO  ] stdout
ceph.block_device=/dev/ceph-8df436c9-1302-4f1f-b185-c060f5e177d5/osd-block-2e5b3463-5904-4aee-9ae1-7d31d8576dc8,ceph.block_uuid=nqgDsS-dz7R-Ddlf-j31m-Q98Y-Qjua-Lmi5vq,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=838506b7-e0c6-4022-9e17-2d1cf9458be6,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.db_device=/dev/nvme0n1p10,ceph.db_uuid=d7ac0b16-a18c-4e8a-973f-829970e5a5ec,ceph.encrypted=0,ceph.osd_fsid=2e5b3463-5904-4aee-9ae1-7d31d8576dc8,ceph.osd_id=9,ceph.type=block,ceph.vdo=0";"/dev/ceph-8df436c9-1302-4f1f-b185-c060f5e177d5/osd-block-2e5b3463-5904-4aee-9ae1-7d31d8576dc8";"osd-block-2e5b3463-5904-4aee-9ae1-7d31d8576dc8";"ceph-8df436c9-1302-4f1f-b185-c060f5e177d5";"nqgDsS-dz7R-Ddlf-j31m-Q98Y-Qjua-Lmi5vq

      [2018-08-20 11:26:32,348][ceph_volume.process][INFO  ] stdout
ceph.block_device=/dev/ceph-a23b8ade-25ae-43b2-ae42-8990bae6f389/osd-block-b7100200-9eef-4c85-b855-b5a0a435354c,ceph.block_uuid=xa3cq8-cGm1-1aKI-RC5M-mcqw-zadE-tdcHhK,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=838506b7-e0c6-4022-9e17-2d1cf9458be6,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.db_device=/dev/nvme0n1p6,ceph.db_uuid=fd88634e-7381-40e4-9aec-1df115ef64cf,ceph.encrypted=0,ceph.osd_fsid=b7100200-9eef-4c85-b855-b5a0a435354c,ceph.osd_id=5,ceph.type=block,ceph.vdo=0";"/dev/ceph-a23b8ade-25ae-43b2-ae42-8990bae6f389/osd-block-b7100200-9eef-4c85-b855-b5a0a435354c";"osd-block-b7100200-9eef-4c85-b855-b5a0a435354c";"ceph-a23b8ade-25ae-43b2-ae42-8990bae6f389";"xa3cq8-cGm1-1aKI-RC5M-mcqw-zadE-tdcHhK

      [2018-08-20 11:26:32,348][ceph_volume.process][INFO  ] stdout
ceph.block_device=/dev/ceph-cb9bd619-dfd8-4a09-9b60-ecfb50b2f13c/osd-block-bcb9d7e6-44ea-449b-ad97-1aa5f880dfdd,ceph.block_uuid=S51Kgl-r9QD-P7Dn-PYzp-6mXi-TYfD-RGxefs,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=838506b7-e0c6-4022-9e17-2d1cf9458be6,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.db_device=/dev/nvme0n1p2,ceph.db_uuid=3ee5298a-2cbd-424b-b26d-e9eb04fdbc84,ceph.encrypted=0,ceph.osd_fsid=bcb9d7e6-44ea-449b-ad97-1aa5f880dfdd,ceph.osd_id=1,ceph.type=block,ceph.vdo=0";"/dev/ceph-cb9bd619-dfd8-4a09-9b60-ecfb50b2f13c/osd-block-bcb9d7e6-44ea-449b-ad97-1aa5f880dfdd";"osd-block-bcb9d7e6-44ea-449b-ad97-1aa5f880dfdd";"ceph-cb9bd619-dfd8-4a09-9b60-ecfb50b2f13c";"S51Kgl-r9QD-P7Dn-PYzp-6mXi-TYfD-RGxefs

      [2018-08-20 11:26:32,348][ceph_volume.process][INFO  ] stdout
ceph.block_device=/dev/ceph-db6fc227-11fb-4510-8a8b-29c16c7d9bfb/osd-block-ba351d69-5c48-418e-a377-4034f503af93,ceph.block_uuid=kpvBCz-Ikuu-52r0-0FQ2-NGR1-hlGP-bHpMi4,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=838506b7-e0c6-4022-9e17-2d1cf9458be6,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.db_device=/dev/nvme0n1p7,ceph.db_uuid=61bc909c-0930-49b7-90ad-c5e2d27eed9e,ceph.encrypted=0,ceph.osd_fsid=ba351d69-5c48-418e-a377-4034f503af93,ceph.osd_id=6,ceph.type=block,ceph.vdo=0";"/dev/ceph-db6fc227-11fb-4510-8a8b-29c16c7d9bfb/osd-block-ba351d69-5c48-418e-a377-4034f503af93";"osd-block-ba351d69-5c48-418e-a377-4034f503af93";"ceph-db6fc227-11fb-4510-8a8b-29c16c7d9bfb";"kpvBCz-Ikuu-52r0-0FQ2-NGR1-hlGP-bHpMi4

      [2018-08-20 11:26:32,348][ceph_volume.process][INFO  ] stdout
ceph.block_device=/dev/ceph-ee343712-359b-4972-bbdf-c66ae5d4bf4a/osd-block-913e65e3-62d9-48f8-a0ef-45315cf64593,ceph.block_uuid=ZctH0P-9zjl-zvTf-xHcd-bIIV-bMGa-vS4wOP,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=838506b7-e0c6-4022-9e17-2d1cf9458be6,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.db_device=/dev/nvme0n1p9,ceph.db_uuid=70e9e629-2c82-4aaa-bd73-abf22ebc9736,ceph.encrypted=0,ceph.osd_fsid=913e65e3-62d9-48f8-a0ef-45315cf64593,ceph.osd_id=8,ceph.type=block,ceph.vdo=0";"/dev/ceph-ee343712-359b-4972-bbdf-c66ae5d4bf4a/osd-block-913e65e3-62d9-48f8-a0ef-45315cf64593";"osd-block-913e65e3-62d9-48f8-a0ef-45315cf64593";"ceph-ee343712-359b-4972-bbdf-c66ae5d4bf4a";"ZctH0P-9zjl-zvTf-xHcd-bIIV-bMGa-vS4wOP

      [2018-08-20 11:26:32,349][ceph_volume][ERROR ] exception caught by
      decorator

      Traceback (most recent call last):

        File
      "/usr/lib/python2.7/dist-packages/ceph_volume/decorators.py", line
      59, in newfunc

          return f(*a, **kw)

        File "/usr/lib/python2.7/dist-packages/ceph_volume/main.py",
      line 153, in main

          terminal.dispatch(self.mapper, subcommand_args)

        File "/usr/lib/python2.7/dist-packages/ceph_volume/terminal.py",
      line 182, in dispatch

          instance.main()

        File
      "/usr/lib/python2.7/dist-packages/ceph_volume/devices/lvm/main.py",
      line 38, in main

          terminal.dispatch(self.mapper, self.argv)

        File "/usr/lib/python2.7/dist-packages/ceph_volume/terminal.py",
      line 182, in dispatch

          instance.main()

        File
      "/usr/lib/python2.7/dist-packages/ceph_volume/decorators.py", line
      16, in is_root

          return func(*a, **kw)

        File
      "/usr/lib/python2.7/dist-packages/ceph_volume/devices/lvm/trigger.py",
      line 70, in main

          Activate(['--auto-detect-objectstore', osd_id,
      osd_uuid]).main()

        File
      "/usr/lib/python2.7/dist-packages/ceph_volume/devices/lvm/activate.py",
      line 320, in main

          self.activate(args)

        File
      "/usr/lib/python2.7/dist-packages/ceph_volume/decorators.py", line
      16, in is_root

          return func(*a, **kw)

        File
      "/usr/lib/python2.7/dist-packages/ceph_volume/devices/lvm/activate.py",
      line 230, in activate

          raise RuntimeError('could not find osd.%s with fsid %s' %
      (osd_id, osd_fsid))

      RuntimeError: could not find osd.1 with fsid
      4a9954ce-0a0f-432b-a91d-eaacb45287d4

    ceph-volume-systemd.log (extract)

    [2018-08-20
      11:26:26,386][systemd][INFO  ] raw systemd input received:
      lvm-6-ba351d69-5c48-418e-a377-4034f503af93

      [2018-08-20 11:26:26,386][systemd][INFO  ] raw systemd input
      received: lvm-3-9380cd27-c0fe-4ede-9ed3-d09eff545037

      [2018-08-20 11:26:26,386][systemd][INFO  ] raw systemd input
      received: lvm-1-bcb9d7e6-44ea-449b-ad97-1aa5f880dfdd

      [2018-08-20 11:26:26,386][systemd][INFO  ] raw systemd input
      received: lvm-4-02540fff-5478-4a67-bf5c-679c72150e8d

      [2018-08-20 11:26:26,386][systemd][INFO  ] raw systemd input
      received: lvm-0-98bfb597-009b-4e88-bc5e-dd22587d21fe

      [2018-08-20 11:26:26,386][systemd][INFO  ] raw systemd input
      received: lvm-8-913e65e3-62d9-48f8-a0ef-45315cf64593

      [2018-08-20 11:26:26,386][systemd][INFO  ] raw systemd input
      received: lvm-5-b7100200-9eef-4c85-b855-b5a0a435354c

      [2018-08-20 11:26:26,386][systemd][INFO  ] raw systemd input
      received: lvm-7-5d4af2fc-388c-4795-9d1a-53ad8aba56d8

      [2018-08-20 11:26:26,386][systemd][INFO  ] parsed sub-command:
      lvm, extra data: 6-ba351d69-5c48-418e-a377-4034f503af93

      [2018-08-20 11:26:26,386][systemd][INFO  ] parsed sub-command:
      lvm, extra data: 3-9380cd27-c0fe-4ede-9ed3-d09eff545037

      [2018-08-20 11:26:26,386][systemd][INFO  ] parsed sub-command:
      lvm, extra data: 1-bcb9d7e6-44ea-449b-ad97-1aa5f880dfdd

      [2018-08-20 11:26:26,387][systemd][INFO  ] parsed sub-command:
      lvm, extra data: 4-02540fff-5478-4a67-bf5c-679c72150e8d

      [2018-08-20 11:26:26,387][systemd][INFO  ] parsed sub-command:
      lvm, extra data: 0-98bfb597-009b-4e88-bc5e-dd22587d21fe

      [2018-08-20 11:26:26,387][systemd][INFO  ] parsed sub-command:
      lvm, extra data: 8-913e65e3-62d9-48f8-a0ef-45315cf64593

      [2018-08-20 11:26:26,387][systemd][INFO  ] parsed sub-command:
      lvm, extra data: 5-b7100200-9eef-4c85-b855-b5a0a435354c

      [2018-08-20 11:26:26,386][systemd][INFO  ] raw systemd input
      received: lvm-9-2e5b3463-5904-4aee-9ae1-7d31d8576dc8

      [2018-08-20 11:26:26,386][systemd][INFO  ] raw systemd input
      received: lvm-2-b8e82f22-e993-4458-984b-90232b8b3d55

      [2018-08-20 11:26:26,387][systemd][INFO  ] parsed sub-command:
      lvm, extra data: 7-5d4af2fc-388c-4795-9d1a-53ad8aba56d8

      [2018-08-20 11:26:26,386][systemd][INFO  ] raw systemd input
      received: lvm-1-4a9954ce-0a0f-432b-a91d-eaacb45287d4

      [2018-08-20 11:26:26,387][systemd][INFO  ] parsed sub-command:
      lvm, extra data: 1-4a9954ce-0a0f-432b-a91d-eaacb45287d4

      [2018-08-20 11:26:26,387][systemd][INFO  ] parsed sub-command:
      lvm, extra data: 9-2e5b3463-5904-4aee-9ae1-7d31d8576dc8

      [2018-08-20 11:26:26,387][systemd][INFO  ] parsed sub-command:
      lvm, extra data: 2-b8e82f22-e993-4458-984b-90232b8b3d55

      [2018-08-20 11:26:26,458][ceph_volume.process][INFO  ] Running
      command: ceph-volume lvm trigger
      3-9380cd27-c0fe-4ede-9ed3-d09eff545037

      [2018-08-20 11:26:26,458][ceph_volume.process][INFO  ] Running
      command: ceph-volume lvm trigger
      2-b8e82f22-e993-4458-984b-90232b8b3d55

      [2018-08-20 11:26:26,458][ceph_volume.process][INFO  ] Running
      command: ceph-volume lvm trigger
      5-b7100200-9eef-4c85-b855-b5a0a435354c

      [2018-08-20 11:26:26,458][ceph_volume.process][INFO  ] Running
      command: ceph-volume lvm trigger
      6-ba351d69-5c48-418e-a377-4034f503af93

      [2018-08-20 11:26:26,458][ceph_volume.process][INFO  ] Running
      command: ceph-volume lvm trigger
      4-02540fff-5478-4a67-bf5c-679c72150e8d

      [2018-08-20 11:26:26,459][ceph_volume.process][INFO  ] Running
      command: ceph-volume lvm trigger
      8-913e65e3-62d9-48f8-a0ef-45315cf64593

      [2018-08-20 11:26:26,459][ceph_volume.process][INFO  ] Running
      command: ceph-volume lvm trigger
      0-98bfb597-009b-4e88-bc5e-dd22587d21fe

      [2018-08-20 11:26:26,459][ceph_volume.process][INFO  ] Running
      command: ceph-volume lvm trigger
      7-5d4af2fc-388c-4795-9d1a-53ad8aba56d8

      [2018-08-20 11:26:26,459][ceph_volume.process][INFO  ] Running
      command: ceph-volume lvm trigger
      1-4a9954ce-0a0f-432b-a91d-eaacb45287d4

      [2018-08-20 11:26:26,459][ceph_volume.process][INFO  ] Running
      command: ceph-volume lvm trigger
      9-2e5b3463-5904-4aee-9ae1-7d31d8576dc8

      [2018-08-20 11:26:26,459][ceph_volume.process][INFO  ] Running
      command: ceph-volume lvm trigger
      1-bcb9d7e6-44ea-449b-ad97-1aa5f880dfdd

      [2018-08-20 11:26:27,068][ceph_volume.process][INFO  ] stderr
      -->  RuntimeError: could not find osd.1 with fsid
      4a9954ce-0a0f-432b-a91d-eaacb45287d4

      [2018-08-20 11:26:27,086][systemd][WARNING] command returned
      non-zero exit status: 1

      [2018-08-20 11:26:27,086][systemd][WARNING] failed activating OSD,
      retries left: 30

        This sounds very odd. Could you clarify where block and block.db are?
Also useful here would be to take a look at
/var/log/ceph/ceph-volume-systemd.log and ceph-volume.log to
see how ceph-volume is trying to get this OSD up and running.

Also useful would be to check `ceph-volume lvm list` to verify that
regardless of the name change, it recognizes the correct partition
mapped to the OSD

Oops !

# ceph-volume lvm list
-->  KeyError: 'devices'

      Can you re-run this like:

    CEPH_VOLUME_DEBUG=1 ceph-volume lvm list

And paste the output? I think this has been fixed since, but want to
double check

    # ceph --version

        ceph version 12.2.7 (3ec878d1e53e1aeb47a9f619c49d9e7c0aa384d5)
        luminous (stable)
    #
        CEPH_VOLUME_DEBUG=1 ceph-volume lvm list

        Traceback (most recent call last):

          File "/usr/sbin/ceph-volume", line 6, in <module>

            main.Volume()

          File "/usr/lib/python2.7/dist-packages/ceph_volume/main.py",
        line 37, in __init__

            self.main(self.argv)

          File
        "/usr/lib/python2.7/dist-packages/ceph_volume/decorators.py",
        line 59, in newfunc

            return f(*a, **kw)

          File "/usr/lib/python2.7/dist-packages/ceph_volume/main.py",
        line 153, in main

            terminal.dispatch(self.mapper, subcommand_args)

          File
        "/usr/lib/python2.7/dist-packages/ceph_volume/terminal.py", line
        182, in dispatch

            instance.main()

          File
        "/usr/lib/python2.7/dist-packages/ceph_volume/devices/lvm/main.py",
        line 38, in main

            terminal.dispatch(self.mapper, self.argv)

          File
        "/usr/lib/python2.7/dist-packages/ceph_volume/terminal.py", line
        182, in dispatch

            instance.main()

          File
        "/usr/lib/python2.7/dist-packages/ceph_volume/devices/lvm/listing.py",
        line 300, in main

            self.list(args)

          File
        "/usr/lib/python2.7/dist-packages/ceph_volume/decorators.py",
        line 16, in is_root

            return func(*a, **kw)

          File
        "/usr/lib/python2.7/dist-packages/ceph_volume/devices/lvm/listing.py",
        line 120, in list

            pretty_report(report)

          File
        "/usr/lib/python2.7/dist-packages/ceph_volume/devices/lvm/listing.py",
        line 53, in pretty_report

            device_metadata_item_template.format(tag_name='devices',
        value=','.join(device['devices'])))

        KeyError: 'devices'

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com