On Thu, Apr 12, 2018 at 10:01:22PM +1000, Dave Chinner wrote: > On Thu, Apr 12, 2018 at 07:09:14AM -0400, Jeff Layton wrote: > > When there is a writeback error, what should be done with the dirty > > page(s)? Right now, we usually just mark them clean and carry on. Is > > that the right thing to do? > > There isn't a right thing. Whatever we do will be wrong for someone. That's the problem. The best that could be done (and it's not enough) would be to have a mode which does with the PG folks want (or what they *think* they want). It seems what they want is to have an error result in the page being marked clean. When they discover the outcome (OOM-city and the unability to unmount a file system on a failed drive), then they will complain to us *again*, at which point we can tell them that want they really want is another variation on O_PONIES, and welcome to the real world and real life. Which is why, even if they were to pay someone to implement what they want, I'm not sure we would want to accept it upstream --- or distro's might consider it a support nightmare, and refuse to allow that mode to be enabled on enterprise distro's. But at least, it will have been some PG-based company who will have implemented it, so they're not wasting other people's time or other people's resources... We could try to get something like what Google is doing upstream, which is to have the I/O errors sent to userspace via a netlink channel (without changing anything else about how buffered writeback is handled in the face of errors). Then userspace applications could switch to Direct I/O like all of the other really serious userspace storage solutions I'm aware of, and then someone could try to write some kind of HDD health monitoring system that tries to do the right thing when a disk is discovered to have developed some media errors or something more serious (e.g., a head failure). That plus some kind of RAID solution is I think the only thing which is really realistic for a typical PG site. It's certainly that's what *I* would do if I didn't decide to use a hosted cloud solution, such as Cloud SQL for Postgres, and let someone else solve the really hard problems of dealing with real-world HDD failures. :-) - Ted
