thanks. I'm evaluating cephfs for a computer science dept. We have users that run week-long AI training jobs. They use standard packages, which they probably don't want to modify. At the moment we use NFS. It uses synchronous I/O, so if somethings goes wrong, the users' jobs pause until we reboot, and then continue. However there's an obvious performance penalty for this. ________________________________ From: Gregory Farnum <gfarnum@xxxxxxxxxx> Sent: Thursday, December 8, 2022 2:08 AM To: Dhairya Parmar <dparmar@xxxxxxxxxx> Cc: Charles Hedrick <hedrick@xxxxxxxxxxx>; ceph-users@xxxxxxx <ceph-users@xxxxxxx> Subject: Re: Re: what happens if a server crashes with cephfs? More generally, as Manuel noted you can (and should!) make use of fsync et al for data safety. Ceph’s async operations are not any different at the application layer from how data you send to the hard drive can sit around in volatile caches until a consistency point like fsync is invoked. -Greg On Wed, Dec 7, 2022 at 10:02 PM Dhairya Parmar <dparmar@xxxxxxxxxx<mailto:dparmar@xxxxxxxxxx>> wrote: Hi Charles, There are many scenarios where the write/close operation can fail but generally failures/errors are logged (normally every time) to help debug the case. Therefore there are no silent failures as such except you encountered a very rare bug. - Dhairya On Wed, Dec 7, 2022 at 11:38 PM Charles Hedrick <hedrick@xxxxxxxxxxx<mailto:hedrick@xxxxxxxxxxx>> wrote: > I believe asynchronous operations are used for some operations in cephfs. > That means the server acknowledges before data has been written to stable > storage. Does that mean there are failure scenarios when a write or close > will return an error? fail silently? > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx> > To unsubscribe send an email to ceph-users-leave@xxxxxxx<mailto:ceph-users-leave@xxxxxxx> > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx> To unsubscribe send an email to ceph-users-leave@xxxxxxx<mailto:ceph-users-leave@xxxxxxx> _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |