Dear Andreas, On 2019-10-24 19:51, Andreas Dilger wrote: > On Oct 24, 2019, at 4:43 AM, Paul Menzel <pmenzel@xxxxxxxxxxxxx> > wrote: >> In our cluster, we offer scratch space for temporary files. As >> these files are temporary, we do not need any safety requirements >> – especially not those when the system crashes or shuts down. So >> no `sync` is for example needed. >> >> Are there file systems catering to this need? I couldn’t find any? >> Maybe I missed some options for existing file systems. > > How big do you need the scratch filesystem to be? Is it local to > the node or does it need to be shared between nodes? In this case local. > If it needs to be large and shared between nodes then Lustre is > typically used for this. If it is local and relatively small you > could consider using tmpfs backed by swab on an NVMe flash device > (M.2 or U.2, Optane if you can afford it) inside the node. > > That way you get RAM-like performance for many files, with a larger > capacity than RAM when needed (tmpfs can use swap). > > You might consider to mount a new tmpfs filesystem per job (no > formatting is needed for tmpfs), and then unmount it when the job is > done, so that the old files are automatically cleaned up. That is a good idea, but probably not practical for 10 TB. Out of curiosity, what is the limit for “relatively small” in your experience? Kind regards, Paul
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature
