Re: Disk device path changed - cephadm faild to apply osd service

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Do you really need device paths in your configuration? You could use other criteria like disk sizes, vendors, rotational flag etc. If you really want device paths you'll probably need to ensure they're persistent across reboots via udev rules.

Zitat von Kilian Ries <mail@xxxxxxxxxxxxxx>:

Hi,


it seems that after reboot / OS update my disk labels / device paths may have changed. Since then i get an error like this:



CEPHADM_APPLY_SPEC_FAIL: Failed to apply 1 service(s): osd.osd-12-22_hdd-2



###


RuntimeError: cephadm exited with an error code: 1, stderr:Non-zero exit code 1 from /bin/docker run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:9e2fd45a080aea67d1935d7d9a9025b6db2e8be9173186e068a79a0da5a54ada -e NODE_NAME=ceph-osd07.intern -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_OSDSPEC_AFFINITY=osd-12-22_hdd-2 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/01578d80-6c97-46ba-9327-cb2b13980916:/var/run/ceph:z -v /var/log/ceph/01578d80-6c97-46ba-9327-cb2b13980916:/var/log/ceph:z -v /var/lib/ceph/01578d80-6c97-46ba-9327-cb2b13980916/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmp2cvmr5lf:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpb38cuw7q:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:9e2fd45a080aea67d1935d7d9a9 025b6db2e8be9173186e068a79a0da5a54ada lvm batch --no-auto /dev/sdm /dev/sdn /dev/sdo /dev/sdp /dev/sdq --db-devices /dev/sdg --yes --no-systemd

/bin/docker: stderr Traceback (most recent call last):

/bin/docker: stderr   File "/usr/sbin/ceph-volume", line 11, in <module>

/bin/docker: stderr load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')()

/bin/docker: stderr File "/usr/lib/python3.6/site-packages/ceph_volume/main.py", line 41, in __init__

/bin/docker: stderr     self.main(self.argv)

/bin/docker: stderr File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 59, in newfunc

/bin/docker: stderr     return f(*a, **kw)

/bin/docker: stderr File "/usr/lib/python3.6/site-packages/ceph_volume/main.py", line 153, in main

/bin/docker: stderr     terminal.dispatch(self.mapper, subcommand_args)

/bin/docker: stderr File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch

/bin/docker: stderr     instance.main()

/bin/docker: stderr File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/main.py", line 46, in main

/bin/docker: stderr     terminal.dispatch(self.mapper, self.argv)

/bin/docker: stderr File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 192, in dispatch

/bin/docker: stderr     instance = mapper.get(arg)(argv[count:])

/bin/docker: stderr File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/batch.py", line 348, in __init__

/bin/docker: stderr     self.args = parser.parse_args(argv)

/bin/docker: stderr File "/usr/lib64/python3.6/argparse.py", line 1734, in parse_args

/bin/docker: stderr     args, argv = self.parse_known_args(args, namespace)

/bin/docker: stderr File "/usr/lib64/python3.6/argparse.py", line 1766, in parse_known_args

/bin/docker: stderr namespace, args = self._parse_known_args(args, namespace)

/bin/docker: stderr File "/usr/lib64/python3.6/argparse.py", line 1954, in _parse_known_args

/bin/docker: stderr positionals_end_index = consume_positionals(start_index)

/bin/docker: stderr File "/usr/lib64/python3.6/argparse.py", line 1931, in consume_positionals

/bin/docker: stderr     take_action(action, args)

/bin/docker: stderr File "/usr/lib64/python3.6/argparse.py", line 1824, in take_action

/bin/docker: stderr argument_values = self._get_values(action, argument_strings)

/bin/docker: stderr File "/usr/lib64/python3.6/argparse.py", line 2279, in _get_values

/bin/docker: stderr value = [self._get_value(action, v) for v in arg_strings]

/bin/docker: stderr File "/usr/lib64/python3.6/argparse.py", line 2279, in <listcomp>

/bin/docker: stderr value = [self._get_value(action, v) for v in arg_strings]

/bin/docker: stderr File "/usr/lib64/python3.6/argparse.py", line 2294, in _get_value

/bin/docker: stderr     result = type_func(arg_string)

/bin/docker: stderr File "/usr/lib/python3.6/site-packages/ceph_volume/util/arg_validators.py", line 116, in __call__

/bin/docker: stderr     return self._format_device(self._is_valid_device())

/bin/docker: stderr File "/usr/lib/python3.6/site-packages/ceph_volume/util/arg_validators.py", line 127, in _is_valid_device

/bin/docker: stderr     super()._is_valid_device(raise_sys_exit=False)

/bin/docker: stderr File "/usr/lib/python3.6/site-packages/ceph_volume/util/arg_validators.py", line 104, in _is_valid_device

/bin/docker: stderr     super()._is_valid_device()

/bin/docker: stderr File "/usr/lib/python3.6/site-packages/ceph_volume/util/arg_validators.py", line 69, in _is_valid_device

/bin/docker: stderr     super()._is_valid_device()

/bin/docker: stderr File "/usr/lib/python3.6/site-packages/ceph_volume/util/arg_validators.py", line 47, in _is_valid_device

/bin/docker: stderr raise RuntimeError("Device {} has partitions.".format(self.dev_path))

/bin/docker: stderr RuntimeError: Device /dev/sdq has partitions.

Traceback (most recent call last):

File "/var/lib/ceph/01578d80-6c97-46ba-9327-cb2b13980916/cephadm.0317efb4d3a353d5a77e82f4a4f52582f06970d6aba66473daecf92e26ee3a51", line 9309, in <module>

    main()

File "/var/lib/ceph/01578d80-6c97-46ba-9327-cb2b13980916/cephadm.0317efb4d3a353d5a77e82f4a4f52582f06970d6aba66473daecf92e26ee3a51", line 9297, in main

    r = ctx.func(ctx)

File "/var/lib/ceph/01578d80-6c97-46ba-9327-cb2b13980916/cephadm.0317efb4d3a353d5a77e82f4a4f52582f06970d6aba66473daecf92e26ee3a51", line 1941, in _infer_config

    return func(ctx)

File "/var/lib/ceph/01578d80-6c97-46ba-9327-cb2b13980916/cephadm.0317efb4d3a353d5a77e82f4a4f52582f06970d6aba66473daecf92e26ee3a51", line 1872, in _infer_fsid

    return func(ctx)

File "/var/lib/ceph/01578d80-6c97-46ba-9327-cb2b13980916/cephadm.0317efb4d3a353d5a77e82f4a4f52582f06970d6aba66473daecf92e26ee3a51", line 1969, in _infer_image

    return func(ctx)

File "/var/lib/ceph/01578d80-6c97-46ba-9327-cb2b13980916/cephadm.0317efb4d3a353d5a77e82f4a4f52582f06970d6aba66473daecf92e26ee3a51", line 1859, in _validate_fsid

    return func(ctx)

File "/var/lib/ceph/01578d80-6c97-46ba-9327-cb2b13980916/cephadm.0317efb4d3a353d5a77e82f4a4f52582f06970d6aba66473daecf92e26ee3a51", line 5366, in command_ceph_volume

    out, err, code = call_throws(ctx, c.run_cmd())

File "/var/lib/ceph/01578d80-6c97-46ba-9327-cb2b13980916/cephadm.0317efb4d3a353d5a77e82f4a4f52582f06970d6aba66473daecf92e26ee3a51", line 1661, in call_throws

    raise RuntimeError('Failed command: %s' % ' '.join(command))


###



/dev/sdg is my boot device at the moment wich was formerly /dev/sda.


Is it safe to edit the ceph orch yaml file and change the device pathes to the new format? Like this:


ceph orch ls --service_name=<service-name> --export > myservice.yaml


vi (change device pathes in spec -> data_devices -> path | db_devices -> path)


ceph orch apply -i myservice.yaml [--dry-run]



Is that ok / expected behaviour? Or is there a better way?


However can see that mgr detects new devices in ceph orch log:


mgr.ceph-mon03.lrfomu [INF] Detected new or changed devices on ceph-osd07



Regards,

Kilian


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx


_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux