On Oct 24, 2019, at 4:43 AM, Paul Menzel <pmenzel@xxxxxxxxxxxxx> wrote: > > Dear Linux folks, > > > In our cluster, we offer scratch space for temporary files. As > these files are temporary, we do not need any safety > requirements – especially not those when the system crashes or > shuts down. So no `sync` is for example needed. > > Are there file systems catering to this need? I couldn’t find > any? Maybe I missed some options for existing file systems. How big do you need the scratch filesystem to be? Is it local to the node or does it need to be shared between nodes? If it needs to be large and shared between nodes then Lustre is typically used for this. If it is local and relatively small you could consider using tmpfs backed by swab on an NVMe flash device (M.2 or U.2, Optane if you can afford it) inside the node. That way you get RAM-like performance for many files, with a larger capacity than RAM when needed (tmpfs can use swap). You might consider to mount a new tmpfs filesystem per job (no formatting is needed for tmpfs), and then unmount it when the job is done, so that the old files are automatically cleaned up. Cheers, Andreas
Attachment:
signature.asc
Description: Message signed with OpenPGP
