Re: How to make CephFS a tiered file system?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello Samuel,

you can use the docs from
https://docs.ceph.com/en/latest/cephfs/file-layouts/ to find out how to put
a file on a different pool. Example: "setfattr -n ceph.dir.layout.pool -v
poolname foldername"

Ceph itself does unfortunately not provide tiering out of the box. You
therefore have to write a script das scans your storage and migrates files.
This will cause a lot of IOs and CPU/RAM on the MDS, so make sure it won't
affect your regular operations.

Pseudo code like:

for each file in cephfs do
   if file.date < xxxx then // or whatever you want
      copy file to new temporary file in pool
      remove old file
      rename temp file to old file location

--
Martin Verges
Managing director

Mobile: +49 174 9335695
E-Mail: martin.verges@xxxxxxxx
Chat: https://t.me/MartinVerges

croit GmbH, Freseniusstr. 31h, 81247 Munich
CEO: Martin Verges - VAT-ID: DE310638492
Com. register: Amtsgericht Munich HRB 231263

Web: https://croit.io
YouTube: https://goo.gl/PGE1Bx


On Mon, 19 Jul 2021 at 23:28, huxiaoyu@xxxxxxxxxxxx <huxiaoyu@xxxxxxxxxxxx>
wrote:

> Dear Cepher,
>
> I have a requirement to use CephFS as a tiered file system, i.e. the data
> will be first stored onto an all-flash pool (using SSD OSDs), and then
> automatically moved to an EC coded pool (using HDD OSDs) according to
> threshold on file creation time (or access time). The reason for such a
> file system is due to the fact that, files are created and most likely
> accessed within the first 6 months or 1 year, and after that period, those
> files have much less chance to be accessed and thus could be moved to a
> slower and cheap pool.
>
> Does CephFS already support such a tiered feature? and if yes, how to
> implement such feature with a pool of all SSD pool and a pool of EC-coded
> HDD pool?
>
> Any suggestion, ideas, comments are highly appreciated,
>
> best regards,
>
> samuel
>
>
>
> huxiaoyu@xxxxxxxxxxxx
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux