Hi Dave,
Thank you for getting back to me. Please see my response inline.
>> What's does "false-negative" mean? I don't know what your application is trying to do, so I don't know what your failure criteria is...
Assume the following :
1) No failures (application crash OR disk / sector errors OR any form of disk or memory corruption)
2) xfs file-system is mounted with dirsync option.
3) A directory in XFS with 1.2 million files. fsync() was called on each file after creation.
4) An application with two threads (Thread-A and Thread-B) that uses this directory. Thread-A iterates through the directory and lists the names of all the files in the directory using readdir calls. The Thread-B concurrently deletes some files from the directory.
5) Files can be only deleted from the directory. No new files will be created once the directory listing is started and we have a coordination layer to ensure this.
Given this context:
- False-positive means a file that was deleted by Thread-B is seen by Thread-A.
- False-negative means a file that is already in the directory is not seen by Thread-A (due to concurrent deletes of some other files in the directory).
Does this make sense ? I know that false-positives are possible in this context. Is it possible to see false negatives in this context?
Thank you. I really appreciate your answers.
Emalayan
From: Dave Chinner <david@xxxxxxxxxxxxx>
Sent: April 5, 2018 6:18:08 PM
To: Vairavanathan, Emalayan
Cc: linux-xfs@xxxxxxxxxxxxxxx
Subject: Re: [Question] Behaviour of directory listing in XFS with concurrent deletes
On Thu, Apr 05, 2018 at 11:53:18PM +0000, Vairavanathan, Emalayan wrote:
> Hi There,
>
>
> We have an application that iterates through a large directory
> (that has around 1.2 million files) and lists the names of all
> the files in the directory.
>
> Files stored in the directory could be concurrently deleted by
> another thread in the application while the application lists the
> files.
XFS complies with Posix requirements for stable directory offsets.
i.e. that dirents don't change offset when other dirents are added
or removed. Hence sequential readdir calls will always start off at
the dirent offset the last call finished at.
However, this guarantees nothing about individual dirent state
when concurrent modifications occur between readdir calls.
> Our application can tolerate false positives when listing a
> directory. However false negatives in directory listing *cannot*
> be tolerated.
>
> To this end could someone tell me how directory listing in XFS
> behaves with concurrent deletes? Is it possible to see false
> negatives ?
What's does "false-negative" mean? I don't know what your
application is trying to do, so I don't know what your failure
criteria is....
Cheers,
Dave.
--
Dave Chinner
david@xxxxxxxxxxxxx
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
