> Op 9 augustus 2016 om 17:44 schreef Martin Palma <martin@xxxxxxxx>: > > > Hi Wido, > > thanks for your advice. > Just keep in mind, you should update the CRUSHMap in one big bang. The cluster will be calculating and peering for 1 or 2 min and afterwards you should see all PGs active+X. Then the waiting game starts, get coffee, some sleep and wait for it to finish. By throttling recovery you prevent this to become slow for the clients. Wido > Best, > Martin > > On Tue, Aug 9, 2016 at 10:05 AM, Wido den Hollander <wido@xxxxxxxx> wrote: > > > >> Op 8 augustus 2016 om 16:45 schreef Martin Palma <martin@xxxxxxxx>: > >> > >> > >> Hi all, > >> > >> we are in the process of expanding our cluster and I would like to > >> know if there are some best practices in doing so. > >> > >> Our current cluster is composted as follows: > >> - 195 OSDs (14 Storage Nodes) > >> - 3 Monitors > >> - Total capacity 620 TB > >> - Used 360 TB > >> > >> We will expand the cluster by other 14 Storage Nodes and 2 Monitor > >> nodes. So we are doubling the current deployment: > >> > >> - OSDs: 195 --> 390 > >> - Total capacity: 620 TB --> 1250 TB > >> > >> During the expansion we would like to minimize the client impact and > >> data movement. Any suggestions? > >> > > > > There are a few routes you can take, I would suggest that you: > > > > - set max backfills to 1 > > - set max recovery to 1 > > > > Now, add the OSDs to the cluster, but NOT to the CRUSHMap. > > > > When all the OSDs are online, inject a new CRUSHMap where you add the new OSDs to the data placement. > > > > $ ceph osd setcrushmap -i <new crushmap> > > > > The OSDs will now start to migrate data, but this is throttled by the max recovery and backfill settings. > > > > Wido > > > >> Best, > >> Martin > >> _______________________________________________ > >> ceph-users mailing list > >> ceph-users@xxxxxxxxxxxxxx > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |