How can there be a "catastrophic reason" if you have "no active,
production workload"...? Do as you please. I am also having 1
replication for temp en tests. But if you have only one osd why use
ceph? Choose the correct 'tool' for the job.
-----Original Message-----
From: Kenneth Van Alstyne [
mailto:kvanalstyne@xxxxxxxxxxxxxxx]
Sent: 16 January 2019 15:04
To: ceph-users
Subject: [ceph-users] Filestore OSD on CephFS?
Disclaimer: Even I will admit that I know this is going to sound like a
silly/crazy/insane question, but I have a reason for wanting to do this
and asking the question. Its also worth noting that no active,
production workload will be used on this cluster, so Im worried
more about data integrity than performance of availability.
Can anyone think of any catastrophic reason why I cannot use an existing
clusters CephFS filesystem as a single OSD for a small cluster? Ive
tested it and it seems to work with the following caveats:
- 50% performance degradation (due to double write penalty since journal
and OSD data both are on the same backing cluster)
- Max object name and namespace length limits, which can be overcome
with the following OSD parameters:
- osd max object name len = 256
- osd max object namespace len = 64
- Due to above name/namespace length limits, cluster should be limited
to RBD (which is exactly what I want to do)
Some details of my cluster are below if anyone cares and Im getting a
consistent, solid roughly 50% of the underlying clusters performance
benchmarks using rados bench:
# ceph --cluster cephfs status
cluster:
id: 0f8904ce-754b-48d4-aa58-7ee6fe9e2cca
health: HEALTH_OK
services:
mon: 1 daemons, quorum storage
mgr: storage(active)
osd: 1 osds: 1 up, 1 in
rbd-mirror: 1 daemon active
data:
pools: 1 pools, 32 pgs
objects: 10 objects, 133 B
usage: 12 MiB used, 87 GiB / 87 GiB avail
pgs: 32 active+clean
io:
client: 85 B/s wr, 0 op/s rd, 0 op/s wr
# ceph --cluster cephfs versions
{
"mon": {
"ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e)
mimic (stable)": 1
},
"mgr": {
"ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e)
mimic (stable)": 1
},
"osd": {
"ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e)
mimic (stable)": 1
},
"mds": {},
"rbd-mirror": {
"ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e)
mimic (stable)": 1
},
"overall": {
"ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e)
mimic (stable)": 4
}
}
# ceph --cluster cephfs osd df
ID CLASS WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS
0 hdd 0.08510 1.00000 87 GiB 16 MiB 87 GiB 0.02 1.00 32
TOTAL 87 GiB 16 MiB 87 GiB 0.02
MIN/MAX VAR: 1.00/1.00 STDDEV: 0
# ceph --cluster cephfs df
GLOBAL:
SIZE AVAIL RAW USED %RAW USED
87 GiB 87 GiB 16 MiB 0.02
POOLS:
NAME ID USED %USED MAX AVAIL OBJECTS
rbd 1 133 B 0 83 GiB 10
# df -h /var/lib/ceph/osd/cephfs-0/
Filesystem Size Used Avail Use% Mounted on
10.0.0.1:/ceph-remote 87G 12M 87G 1% /var/lib/ceph
Thanks,
--
Kenneth Van Alstyne
Systems Architect
Knight Point Systems, LLC
Service-Disabled Veteran-Owned Business
1775 Wiehle Avenue Suite 101 | Reston, VA 20190
c: 228-547-8045 f: 571-266-3106
www.knightpoint.com
DHS EAGLE II Prime Contractor: FC1 SDVOSB Track GSA Schedule 70 SDVOSB:
GS-35F-0646S GSA MOBIS Schedule: GS-10F-0404Y ISO 9001 / ISO 20000 / ISO
27001 / CMMI Level 3
Notice: This e-mail message, including any attachments, is for the sole
use of the intended recipient(s) and may contain confidential and
privileged information. Any unauthorized review, copy, use, disclosure,
or distribution is STRICTLY prohibited. If you are not the intended
recipient, please contact the sender by reply e-mail and destroy all
copies of the original message.
