回复: Re: Ceph as a HDFS alternative?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



RBD now support “read_from_replica=localize”, which may reduce some network traffic. But CephFS seems not to support this.
But I think it is not easy to tell Hadoop to schedule jobs taking the data location into account.

发件人: Serkan Çoban<mailto:cobanserkan@xxxxxxxxx>
发送时间: 2021年8月26日 17:11
收件人: zhang listar<mailto:zhanglinuxstar@xxxxxxxxx>
抄送: ceph-users<mailto:ceph-users@xxxxxxx>
主题:  Re: Ceph as a HDFS alternative?

Ceph cannot scale like HDFS. There are 10K-20K node HDFS clusters in production.
There is no data locality concept if you use CEPH, every IO will be
served from the network.

On Thu, Aug 26, 2021 at 12:04 PM zhang listar <zhanglinuxstar@xxxxxxxxx> wrote:
>
> Hi, all.
>
> I want to use ceph instead of HDFS in big data analysis senario, does ceph
> have some potential problems when the cluster becoming big? say 100PB or
> 500PB?
>
> As far I know, there are some cons:
>
>    1.
>
>    no short circuit read, so we need fast network say 10G or better 50G?
>    2.
>
>    not exactly du, but it is acceptable for applications.
>    3.
>
>    ceph can't handle slow disk as HDFS does for example heged read or write.
>
> Is that right? or there are many other cons?
>
> Thanks in advance.
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux