Willem Jan, hello.
On 16 Aug 2018, at 12:07, Willem Jan Withagen wrote:
In the mean time I have uploaded a PR to fix this in the manual, which
should read:
gpart create -s GPT ada1
gpart add -t freebsd-zfs -l osd.1 ada1
zpool create osd.1 gpt/osd.1
zfs create -o mountpoint=/var/lib/ceph/osd/osd.1 osd.1
I'm still having some difficulties with these instructions, I'm afraid.
First, I'm slightly confused by the zfs create command there, which
seems to mention a pool without a filesystem. Something like zfs create
osd.1/foo would create the filesystem in the pool osd.1, but just 'zfs
create osd.1' mentions a filesystem but no filesystem, and:
# zfs create -o mountpoint=/tmp/foo osd.0
cannot create 'osd.0': missing dataset name
What I've done is
# zpool create -m/var/lib/ceph/osd/osd.0 osd.0 gpt/zd000 gpt/zd001
That creates a zfs (root) filesystem mounted at /var/lib/ceph/osd/osd.0:
# zfs list osd.0
NAME USED AVAIL REFER MOUNTPOINT
osd.0 388K 7.02T 88K /var/lib/ceph/osd/osd.0
I'm following the setup instructions in
<http://docs.ceph.com/docs/master/install/manual-freebsd-deployment/>,
which (I believe) result in:
# ceph-authtool --create-keyring /tmp/ceph.mon.keyring \
--gen-key -n mon.0 --cap mon 'allow *'
# ceph-authtool --create-keyring
/usr/local/etc/ceph/ceph.client.admin.keyring \
--gen-key -n client.admin --set-uid=0 --cap mon 'allow *' \
--cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *'
# ceph-authtool /tmp/ceph.mon.keyring \
--import-keyring /usr/local/etc/ceph/ceph.client.admin.keyring
# monmaptool --create --add pochhammer 172.20.202.119 \
--fsid 08b41ee1-a080-11e8-9017-a0369fa1e52c /tmp/monmap
# mkdir /var/lib/ceph/mon/ceph-pochhammer
# chown ceph /var/lib/ceph/mon/ceph-pochhammer
# chmod 644 /tmp/ceph.mon.keyring
# sudo -u ceph ceph-mon --mkfs -i pochhammer \
--monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring
I'm experimenting here, and aiming to create a test monitor and a couple
of OSDs on a single machine.
When I attempt to start the service, I get:
# service ceph start
=== mon.pochhammer ===
Starting Ceph mon.pochhammer on pochhammer...
2018-08-20 12:19:39.676389 9ee4480 -1 WARNING: 'mon addr' config option
172.20.202.119:0/0 does not match monmap file
continuing with monmap configuration
2018-08-20 12:19:39.677483 9ee4480 -1 auth: error reading file:
/var/lib/ceph/mon/ceph-pochhammer/keyring: can't open
/var/lib/ceph/mon/ceph-pochhammer/keyring: (2) No such file or directory
2018-08-20 12:19:39.677487 9ee4480 -1 mon.pochhammer@-1(probing) e0
unable to load initial keyring
/etc/ceph/ceph.mon.pochhammer.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
2018-08-20 12:19:39.677492 9ee4480 -1 failed to initialize
failed: 'ulimit -n 32768; /usr/local/bin/ceph-mon -i pochhammer
--pid-file /var/run/ceph/mon.pochhammer.pid -c
/usr/local/etc/ceph/ceph.conf --cluster ceph --setuser ceph --setgroup
ceph '
There are a couple of things about this that puzzle me.
1. I don't have a file /var/lib/ceph/mon/ceph-pochhammer/keyring. After
the setup instructions, I have a /tmp/ceph.mon.keyring and /tmp/monmap,
but
# ls /var/lib/ceph/mon/ceph-pochhammer
done kv_backend store.db/
Have I failed to copy something into place? Similarly, I don't have any
keyring copied into that directory.
2. I'm not sure what monmap file this is examining, as reported in the
second file, presumably not the one in /tmp. Also, is there a way to
dump the contents of the/a monmap file? -- I can hexdump it, but that's
not particularly illuminating.
I'm feeling a bit stuck at this point..., and feel I've surely missed
some steps.
Best wishes,
Norman
My ceph.conf is minimally changed from the default:
[global]
fsid = 08b41ee1-a080-11e8-9017-a0369fa1e52c
public network = 172.20.202.0/24
pid file = /var/run/ceph/$name.pid
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
cephx cluster require signatures = true
cephx service require signatures = false
osd pool default size = 2
osd pool default min size = 2
osd pool default pg num = 350
osd pool default pgp num = 350
[mon]
mon initial members = pochhammer
[mon.pochhammer]
host = pochhammer
mon addr = 172.20.202.119
[mds]
[osd]
[client]
rbd cache = true
[client.radosgw.gateway]
--
Norman Gray : https://nxg.me.uk
SUPA School of Physics and Astronomy, University of Glasgow, UK
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
