Btw, one issue with using direct I/O is that need to synchronize with page cache access from the server itself. For pNFS we can do that as we track outstanding layouts. Without layouts it will be more work as we'll need a different data structure tracking grant for bypassing the server. Or just piggy back on layouts anyway as that's what they are doing.
