Thanks for your prompt reply and that clears my doubt. 4 of the OSDs in 2 different nodes goes down daily by getting errors in multipath failure. All the four paths are going to failure mode that makes the OSD down. My query is that as the ceph cluster is overloaded with IOPS does the multipaths get failed or it is the issue with the SAN device (DELL Unity) itself? On November 3, 2023 at 11:38 AM Janne Johansson <icepic.dz@xxxxxxxxx> wrote: > Den tors 2 nov. 2023 kl 23:46 skrev V A Prabha <prabhav@xxxxxxx>: > > > > Is it possible to move the OSDs safe (making the OSDs out and move the > > content > > to other OSDs and remove it and map it fresh to other nodes which is less > > loaded) > > > > As the client feels that using 3 replicas and holding these much spare > > storage , > > we are not using the storage in an optimal way? > > These two sentences don't really add up. > > Ceph has replica=3 as a default in order for drives and hosts to be > able to crash, so that you can recover without losing redundancy. As > soon as you lose redundancy, there is no "safe" anything, you are > immediately in danger of losing data so that you can never get it back > from ceph. > If the client thinks you are wasting space, then they must not care > for the data, because ANY random hiccup, any broken sector somewhere > becomes a data-loss event if the other copy is being moved, or that > server is having maintenance or whatever. With only two copies of the > data, you can never reboot a server, upgrades means the cluster stops > serving data. > > The joke in the 80s (might be older than that of course) was: > "Data is binary, either it is important and backed up, or it is not important" > > Ceph chooses to treat your data as important. You can lower the > expectations by reducing replicas, or reduce perf with erasure coding > (but I understand that this whole thread is about poor total > performance of both client traffic and scrubs and so on), but the > defaults are there to protect you from any random bit flip on one of > the disks and this will happen. Not "perhaps", with enough drives > and/or enough time, this is a certainty. Your design of the storage > decides how your will survive such an event. > > If you have three copies of all data, you can be doing (planned or > unplanned) maintenance on one OSD box, notice an error somewhere on > another box and still recover from this using the third copy. > > With only two replicas, you can't. The maintenance OSD host will have > missed IO that went on while it was away, and the second copy is known > bad. You could hope that you never need to do maintenance, but with > few exceptions, hosts will reboot at times, whether you plan it or > not. > > > > -- > May the most significant bit of your life be positive. Thanks & Regards, Ms V A Prabha / श्रीमती प्रभा वी ए Joint Director / संयुक्त निदेशक Centre for Development of Advanced Computing(C-DAC) / प्रगत संगणन विकास केन्द्र(सी-डैक) Tidel Park”, 8th Floor, “D” Block, (North &South) / “टाइडल पार्क”,8वीं मंजिल, “डी” ब्लॉक, (उत्तर और दक्षिण) No.4, Rajiv Gandhi Salai / नं.4, राजीव गांधी सलाई Taramani / तारामणि Chennai / चेन्नई – 600113 Ph.No.:044-22542226/27 Fax No.: 044-22542294 ------------------------------------------------------------------------------------------------------------ [ C-DAC is on Social-Media too. Kindly follow us at: Facebook: https://www.facebook.com/CDACINDIA & Twitter: @cdacindia ] This e-mail is for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies and the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email is strictly prohibited and appropriate legal action will be taken. ------------------------------------------------------------------------------------------------------------ _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |