On 2023-12-14 07:48, Marcus Pedersén wrote:
Hi all,
I am looking in to ceph and cephfs and in my
head I am comparing with gluster.
The way I have been running gluster over the years
is either a replicated or replicated-distributed clusters.
Here are my observations but I am far from an expert in either Ceph or
Gluster.
Gluster works very well with 2 servers containing 2 big RAID disk arrays.
Ceph on the other hand has MON,MGR,MDS... that can run on multiple
servers, and should be for redundancy, but the OSDs should be lots of
small servers with very few disks attached.
It kind of seems that the perfect OSD would be a disk with a raspberry
pi attached and a 2.5Gb nic.
Something really cheap and replaceable.
So putting Ceph on 2 big servers with RAID arrays is likely a very bad idea.
I am hoping that someone picks up Gluster because it fits the storage
requirements for organizations who start measuring their storage in TB
as opposed to EB.
The small setup we have had has been a replicated cluster
with one arbiter and two fileservers.
These fileservers has been configured with RAID6 and
that raid has been used as the brick.
If disaster strikes and one fileserver burns up
there is still the other fileserver and as it is RAIDed
I can loose two disks on this machine before I
start to loose data.
.... thinking ceph and similar setup ....
The idea is to have one "admin" node and two fileservers.
The admin node will run mon, mgr and mds.
The storage nodes will run mon, mgr, mds and 8x osd (8 disks),
with replication = 2.
The problem is that I can not get my head around how
to think when disaster strikes.
So one fileserver burns up, there is still the other
fileserver and from my understanding the ceph system
will start to replicate the files on the same fileserver
and when this is done disks can be lost on this server
without loosing data.
But to be able to have this security on hardware it
means that the ceph cluster can never be more then 50% full
or this will not work, right?
... and it becomes similar if we have three fileservers,
then the cluster can never be more then 2/3 full?
I am not sure if I missunderstand how ceph works or
that ceph works bad on smaller systems like this?
I would appreciate if somebody with better knowledge
would be able to help me out with this!
Many thanks in advance!!
Marcus
---
När du skickar e-post till SLU så innebär detta att SLU behandlar dina personuppgifter. För att läsa mer om hur detta går till, klicka här <https://www.slu.se/om-slu/kontakta-slu/personuppgifter/>
E-mailing SLU will result in SLU processing your personal data. For more information on how this is done, click here <https://www.slu.se/en/about-slu/contact-slu/personal-data/>
________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users
--
Alvin Starr || land: (647)478-6285
Netvel Inc. || Cell: (416)806-0133
alvin@xxxxxxxxxx ||
________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users
