On Mon, May 28, 2007 at 11:30:32AM +1000, Neil Brown wrote: > > Thanks everyone for your input. There was some very valuable > observations in the various emails. > I will try to pull most of it together and bring out what seem to be > the important points. > > > 1/ A BIO_RW_BARRIER request should never fail with -EOPNOTSUP. Sounds good to me, but how do we test to see if the underlying device supports barriers? Do we just assume that they do and only change behaviour if -o nobarrier is specified in the mount options? > 2/ Maybe barriers provide stronger semantics than are required. > > All write requests are synchronised around a barrier write. This is > often more than is required and apparently can cause a measurable > slowdown. > > Also the FUA for the actual commit write might not be needed. It is > important for consistency that the preceding writes are in safe > storage before the commit write, but it is not so important that the > commit write is immediately safe on storage. That isn't needed until > a 'sync' or 'fsync' or similar. The use of barriers in XFS assumes the commit write to be on stable storage before it returns. One of the ordering guarantees that we need is that the transaction (commit write) is on disk before the metadata block containing the change in the transaction is written to disk and the current barrier behaviour gives us that. > One possible alternative is: > - writes can overtake barriers, but barrier cannot overtake writes. No, that breaks the above usage of a barrier.... > - flush before the barrier, not after. > > This is considerably weaker, and hence cheaper. But I think it is > enough for all filesystems (providing it is still an option to call > blkdev_issue_flush on 'fsync'). No, not enough for XFS. > Another alternative would be to tag each bio was being in a > particular barrier-group. Then bio's in different groups could > overtake each other in either direction, but a BARRIER request must > be totally ordered w.r.t. other requests in the barrier group. > This would require an extra bio field, and would give the filesystem > more appearance of control. I'm not yet sure how much it would > really help... And that assumes the filesystem is tracking exact dependencies between I/Os. Such a mechanism would probably require filesystems to be redesigned to use this, but I can see how it would be useful for doing things like ensuring ordering between just an inode and it's data writes. What would the overhead of having to support several hundred thousand different barrier groups be (i.e. one per dirty inode in a system)? > I think the implementation priorities here are: Depending on the answer to my first question: 0/ implement a specific test for filesystems to run at mount time to determine if barriers are supported or not. > 1/ implement a zero-length BIO_RW_BARRIER option. > 2/ Use it (or otherwise) to make all dm and md modules handle > barriers (and loop?). > 3/ Devise and implement appropriate fall-backs with-in the block layer > so that -EOPNOTSUP is never returned. > 4/ Remove unneeded cruft from filesystems (and elsewhere). Sounds like a good start. ;) Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group -- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
