Ah, ceph-volume.log pointed out the actual problem: RuntimeError: Cannot use device (/dev/storage/bluestore). A vg/lv path or an existing device is needed When I changed "--data /dev/storage/bluestore" to "--data storage/bluestore", everything worked fine. I agree that the ceph-deploy logs are a bit confusing. I submitted a PR to add a brief note to the quick-start guide, in case anyone else makes the same mistake: https://github.com/ceph/ceph/pull/23879 Thanks for the assistance! -- David On Sun, Sep 2, 2018 at 7:44 AM Alfredo Deza <adeza@xxxxxxxxxx> wrote: > > There should be useful logs from ceph-volume in > /var/log/ceph/ceph-volume.log that might show a bit more here. > > I would also try the command that fails directly on the server (sans > ceph-deploy) to see what is it that is actually failing. Seems like > the ceph-deploy log output is a bit out of order (some race condition > here maybe) > > > On Sun, Sep 2, 2018 at 2:53 AM, David Wahler <dwahler@xxxxxxxxx> wrote: > > Hi all, > > > > I'm attempting to get a small Mimic cluster running on ARM, starting > > with a single node. Since there don't seem to be any Debian ARM64 > > packages in the official Ceph repository, I had to build from source, > > which was fairly straightforward. > > > > After installing the .deb packages that I built and following the > > quick-start guide > > (http://docs.ceph.com/docs/mimic/start/quick-ceph-deploy/), things > > seemed to be working fine at first, but I got this error when > > attempting to create an OSD: > > > > rock64@rockpro64-1:~/my-cluster$ ceph-deploy osd create --data > > /dev/storage/bluestore rockpro64-1 > > [ceph_deploy.conf][DEBUG ] found configuration file at: > > /home/rock64/.cephdeploy.conf > > [ceph_deploy.cli][INFO ] Invoked (2.0.1): /usr/bin/ceph-deploy osd > > create --data /dev/storage/bluestore rockpro64-1 > > [ceph_deploy.cli][INFO ] ceph-deploy options: > > [ceph_deploy.cli][INFO ] verbose : False > > [ceph_deploy.cli][INFO ] bluestore : None > > [ceph_deploy.cli][INFO ] cd_conf : > > <ceph_deploy.conf.cephdeploy.Conf instance at 0x7fa9c0f9e0> > > [ceph_deploy.cli][INFO ] cluster : ceph > > [ceph_deploy.cli][INFO ] fs_type : xfs > > [ceph_deploy.cli][INFO ] block_wal : None > > [ceph_deploy.cli][INFO ] default_release : False > > [ceph_deploy.cli][INFO ] username : None > > [ceph_deploy.cli][INFO ] journal : None > > [ceph_deploy.cli][INFO ] subcommand : create > > [ceph_deploy.cli][INFO ] host : rockpro64-1 > > [ceph_deploy.cli][INFO ] filestore : None > > [ceph_deploy.cli][INFO ] func : <function > > osd at 0x7fa9ca0c80> > > [ceph_deploy.cli][INFO ] ceph_conf : None > > [ceph_deploy.cli][INFO ] zap_disk : False > > [ceph_deploy.cli][INFO ] data : > > /dev/storage/bluestore > > [ceph_deploy.cli][INFO ] block_db : None > > [ceph_deploy.cli][INFO ] dmcrypt : False > > [ceph_deploy.cli][INFO ] overwrite_conf : False > > [ceph_deploy.cli][INFO ] dmcrypt_key_dir : > > /etc/ceph/dmcrypt-keys > > [ceph_deploy.cli][INFO ] quiet : False > > [ceph_deploy.cli][INFO ] debug : False > > [ceph_deploy.osd][DEBUG ] Creating OSD on cluster ceph with data > > device /dev/storage/bluestore > > [rockpro64-1][DEBUG ] connection detected need for sudo > > [rockpro64-1][DEBUG ] connected to host: rockpro64-1 > > [rockpro64-1][DEBUG ] detect platform information from remote host > > [rockpro64-1][DEBUG ] detect machine type > > [rockpro64-1][DEBUG ] find the location of an executable > > [ceph_deploy.osd][INFO ] Distro info: debian buster/sid sid > > [ceph_deploy.osd][DEBUG ] Deploying osd to rockpro64-1 > > [rockpro64-1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf > > [rockpro64-1][WARNIN] osd keyring does not exist yet, creating one > > [rockpro64-1][DEBUG ] create a keyring file > > [rockpro64-1][DEBUG ] find the location of an executable > > [rockpro64-1][INFO ] Running command: sudo /usr/sbin/ceph-volume > > --cluster ceph lvm create --bluestore --data /dev/storage/bluestore > > [rockpro64-1][DEBUG ] Running command: /usr/bin/ceph-authtool --gen-print-key > > [rockpro64-1][WARNIN] --> RuntimeError: command returned non-zero > > exit status: 22 > > [rockpro64-1][DEBUG ] Running command: /usr/bin/ceph --cluster ceph > > --name client.bootstrap-osd --keyring > > /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new > > 4903fff3-550c-4ce3-aa7d-97193627c6c0 > > [rockpro64-1][DEBUG ] --> Was unable to complete a new OSD, will > > rollback changes > > [rockpro64-1][DEBUG ] Running command: /usr/bin/ceph --cluster ceph > > --name client.bootstrap-osd --keyring > > /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.0 > > --yes-i-really-mean-it > > [rockpro64-1][DEBUG ] stderr: no valid command found; 10 closest matches: > > [rockpro64-1][DEBUG ] osd tier add-cache <poolname> <poolname> <int[0-]> > > [rockpro64-1][DEBUG ] osd tier remove-overlay <poolname> > > [rockpro64-1][DEBUG ] osd out <ids> [<ids>...] > > [rockpro64-1][DEBUG ] osd in <ids> [<ids>...] > > [rockpro64-1][DEBUG ] osd down <ids> [<ids>...] > > [rockpro64-1][DEBUG ] stderr: osd unset > > full|pause|noup|nodown|noout|noin|nobackfill|norebalance|norecover|noscrub|nodeep-scrub|notieragent|nosnaptrim > > [rockpro64-1][DEBUG ] osd require-osd-release luminous|mimic > > {--yes-i-really-mean-it} > > [rockpro64-1][DEBUG ] osd erasure-code-profile ls > > [rockpro64-1][DEBUG ] osd set > > full|pause|noup|nodown|noout|noin|nobackfill|norebalance|norecover|noscrub|nodeep-scrub|notieragent|nosnaptrim|sortbitwise|recovery_deletes|require_jewel_osds|require_kraken_osds > > {--yes-i-really-mean-it} > > [rockpro64-1][DEBUG ] osd erasure-code-profile get <name> > > [rockpro64-1][DEBUG ] Error EINVAL: invalid command > > [rockpro64-1][ERROR ] RuntimeError: command returned non-zero exit status: 1 > > [ceph_deploy.osd][ERROR ] Failed to execute command: > > /usr/sbin/ceph-volume --cluster ceph lvm create --bluestore --data > > /dev/storage/bluestore > > [ceph_deploy][ERROR ] GenericError: Failed to create 1 OSDs > > > > I'm not very familiar with Ceph yet; does anyone have any > > troubleshooting suggestions? I found a previous issue > > (https://tracker.ceph.com/issues/23918) suggesting this error could be > > caused by mismatched package versions, but as far as I can tell, > > everything on my system is consistent: > > > > rock64@rockpro64-1:~/my-cluster$ sudo ceph version > > ceph version 13.2.1 (5533ecdc0fda920179d7ad84e0aa65a127b20d77) mimic (stable) > > rock64@rockpro64-1:~/my-cluster$ dpkg -l | grep -i ceph > > ii ceph > > 13.2.1-1 arm64 distributed > > storage and file system > > ii ceph-base > > 13.2.1-1 arm64 common ceph > > daemon libraries and management tools > > ii ceph-common > > 13.2.1-1 arm64 common utilities > > to mount and interact with a ceph storage cluster > > ii ceph-deploy > > 2.0.1 all Ceph-deploy is > > an easy to use configuration tool > > ii ceph-mds > > 13.2.1-1 arm64 metadata server > > for the ceph distributed file system > > ii ceph-mgr > > 13.2.1-1 arm64 manager for the > > ceph distributed storage system > > ii ceph-mon > > 13.2.1-1 arm64 monitor server > > for the ceph storage system > > ii ceph-osd > > 13.2.1-1 arm64 OSD server for > > the ceph storage system > > ii libcephfs2 > > 13.2.1-1 arm64 Ceph distributed > > file system client library > > ii python-cephfs > > 13.2.1-1 arm64 Python 2 > > libraries for the Ceph libcephfs library > > ii python-rados > > 13.2.1-1 arm64 Python 2 > > libraries for the Ceph librados library > > ii python-rbd > > 13.2.1-1 arm64 Python 2 > > libraries for the Ceph librbd library > > ii python-rgw > > 13.2.1-1 arm64 Python 2 > > libraries for the Ceph librgw library > > > > Thanks, > > -- David > > _______________________________________________ > > ceph-users mailing list > > ceph-users@xxxxxxxxxxxxxx > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |