On Thu, Jun 28, 2018 at 09:05:04PM -0600, Andreas Dilger wrote: > > > On Jun 28, 2018, at 7:45 PM, Theodore Y. Ts'o <tytso@xxxxxxx> wrote: > > > > On Thu, Jun 28, 2018 at 06:07:47PM -0600, Andreas Dilger wrote: > >>> But does an RDMA operation actually do a block allocation? Really? > >>> And if it is willing to do a block allocation, why is it not willing > >>> to bump i_size? > >> > >> It's not that the RDMA does block allocation, but rather that the RDMA > >> always transfers and writes the full PAGE_SIZE of data, even if i_size > >> is less than the end of the last block. This simplifies the RDMA code > >> so that it can always write the data instead of having to stop at i_size. Every time we write past i_size we need to extend the i_size, why is this case different ? > > > > Right. So there are two choices: > > > > 1) Keep the blocks beyond i_size marked as uninitialized. You > > transfer and write the full PAGE_SIZE of data, but it simply will > > never be available to the user. Yes, that's for extent mapped files. > > > > 2) Zero the page, write it out to the file, and then extend i_size and > > mark the extents as uninitialized. Except at that point you do not really need to mark the extent as unitialized, the blocks are allocated and written to and i_size is extended. That's how it needs to be done for indirect block mapped files. > > The end of the page would already be zeroed before write. > > > Why is it that Lustre is choosing to keep i_size where it is, but to > > mark the blocks beyond it as initialized? > > This isn't about initialized vs. uninitialized extents. It is only about > allocated vs. unallocated blocks, possibly with block-mapped files. There > is no way to have uninitialized blocks with a block-mapped file. > > The code is checking whether there are any blocks allocated beyond i_size, > and if there are, without the patch it considers i_size broken and extends > it to the end of the last allocated block. The patch allows a small number > of blocks to be allocated beyond i_size without triggering this heuristic. I do not think that ext4 has any capacity to allocate initialized blocks beyond i_size without actually writing to it and if we write beyong i_size then we need to extend i_size so if we see anything like that it's a problem that needs fixing. I think it's as simple as that. Unless you can show me where in upstream ext4 we can get to this situation I am strongly for fixing e2fsck. But maybe I am missing something. > > The only difference vs. the previous code is that it correctly calculates > what the PAGE_SIZE aligned block number is (the old code assumed that lblock > was the base-1 block number instead of the base-0 block number that it is). Right, but the test is broken when PAGE_SIZE > 4k that's how I found out in the first place. See me previous email about f_eofblocks and f_pgsize_gt_blksize failures. Thanks! -Lukas
