Hello, Wido I appreciate your answer. And some more discussion goes below. On Tue, Jul 5, 2011 at 1:54 AM, Wido den Hollander <wido@xxxxxxxxx> wrote: > Hi, > > On Tue, 2011-07-05 at 00:25 +0800, Fusheng Han wrote: >> Hi, all >> >> After reading the RADOS paper, I have some questions about >> availability of RADOS. Nowhere I can find to discuss, so I'm here. >> When adding new host to the cluster, some placement group will be >> mapped to the new one. After the cluster info incremental propagates >> to all the OSDs and clients, the client write operation will be >> directed to the PG whose primary is at the new host. Before the new >> host get the data migration down, it can not service these requests. >> And due to the limitation of network bandwidth, the data migration may >> take long. There is a long time that the new host can not service. I >> got confused. > > Yes, during migration a PG will become unavailable for a short period of > time. In a large cluster you have a large number of PGs where each PG > doesn't contain that much data, which makes this period short. > > What kind of bandwith are you talking about? Ceph/RADOS is intended to > run in datacenters where you have low latency high bandwith (1G) > networks. Migrating a PG would take that much time in such environments. What I mentioned is the network bandwidth limitation even with 1Gbps NIC. Taking an imagine cluster for example: Node: 10 Disk: 10TB per node (1TB per disk, 10 disks per node) Utilization: 80% (i.e. totally 80TB data, 8TB data per node) While adding a new node, 7.2TB (= 80TB / 11) data will be migrated to the new one. With 1Gbps bandwidth, for best case, it will cost 7200 seconds (= 7.2TB / 1Gbps) to complete migration. For the last placement group migrated to the new node, it will become unavailable for 7200 seconds. Fusheng Han > > Wido > >> Do I miss something important in this paper? >> >> Thank you all >> Fusheng Han >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
