Re: ENOSPC returned during writepages

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



在 2008-08-21四的 22:15 +0530,Aneesh Kumar K.V写道:
> On Wed, Aug 20, 2008 at 11:13:39AM +0530, Aneesh Kumar K.V wrote:
> > Hi,
> > 
> > I am getting this even with the latest patch queue. The test program is
> > a modified fsstress with fallocate support.
> > 
> > mpage_da_map_blocks block allocation failed for inode 377954 at logical
> > offset 313 with max blocks 4 with error -28
> > mpage_da_map_blocks block allocation failed for inode 336367 at logical
> > offset 74 with max blocks 9 with error -28
> > mpage_da_map_blocks block allocation failed for inode 345560 at logical
> > offset 542 with max blocks 7 with error -28
> > This should not happen.!! Data will be lost
> > mpage_da_map_blocks block allocation failed for inode 355317 at logical
> > offset 152 with max blocks 10 with error -28
> > This should not happen.!! Data will be lost
> > mpage_da_map_blocks block allocation failed for inode 395261 at logical
> > offset 462 with max blocks 1 with error -28
> > This should not happen.!! Data will be lost
> > mpage_da_map_blocks block allocation failed for inode 323784 at logical
> > offset 313 with max blocks 11 with error -28
> > This should not happen.!! Data will be lost
> > 
> 
> With this patch i am not seeing error. It does the below
> 
> a) use ext4_claim_free_blocks that also update the free blocks count
> b) Later after block allocation update the free blocks count if we
> allocated less with non-delayed mode
> c) Switch to non delay mode if we are low on free blocks. 
> 
I had sent a patch to do c) yesterday, I noticed that we can't switch to
non delayed mode if the inode already have some delalloc dirty pages.


> @@ -2462,11 +2464,21 @@ static int ext4_da_write_begin(struct file *file, struct address_space *mapping,
>  	unsigned from, to;
>  	struct inode *inode = mapping->host;
>  	handle_t *handle;
> +	s64 free_blocks;
> +	struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
> 
>  	index = pos >> PAGE_CACHE_SHIFT;
>  	from = pos & (PAGE_CACHE_SIZE - 1);
>  	to = from + len;
> 
> +	free_blocks = percpu_counter_read_positive(&sbi->s_freeblocks_counter);
> +	if (free_blocks < (4 * (FBC_BATCH * nr_cpu_ids))) {
> +		/* switch to non delalloc mode */
> +		*fsdata = (void *)1;
> +		return ext4_write_begin(file, mapping, pos,
> +					len, flags, pagep, fsdata);
> +	}
> +	*fsdata = (void *)0;

No, calling ext4_write_begin() directly won't work, as it start a
handle, do the block allocation , and leave the handle there.   It
expect later the write_end aops  to file the inode to the transaction
list, and close that handle. 

With your change, the aops write_end still points to the
ext4_da_write_end(), which doesn't match the ext4_write_begin. We need
to switch the aop write_begin/write_end  function pointers all together.


I updated my patch, I have a few place I want to polish, since you are
looking at this , comments pls.


ext4: fall back to non delalloc mode if filesystem is almost full
From: Mingming Cao <cmm@xxxxxxxxxx>

In the case of filesystem is close to full (free blocks is below 
the watermark FBC_BATCH * NRCPUS *4) and there is no delalloc dirty
pages in the page cache for the given inode, fall back to non delalloc
 mode for the inode to avoid possible later ENOSPC.

Signed-off-by: Mingming Cao <cmm@xxxxxxxxxx>
---
 fs/ext4/ext4.h  |    2 -
 fs/ext4/inode.c |   75 ++++++++++++++++++++++++++++++++++++++++++++++----------
 fs/ext4/namei.c |    4 +-
 3 files changed, 65 insertions(+), 16 deletions(-)

Index: linux-2.6.27-rc3/fs/ext4/inode.c
===================================================================
--- linux-2.6.27-rc3.orig/fs/ext4/inode.c	2008-08-20 15:20:10.000000000 -0700
+++ linux-2.6.27-rc3/fs/ext4/inode.c	2008-08-20 17:59:20.000000000 -0700
@@ -2391,6 +2391,25 @@
 	return ret;
 }
 
+/*
+ * In case of filesystem is almost full and delalloc could not
+ * get enough free blocks to reserve to prevent later ENOSPC,
+ * let's fall back to the nondelalloc mode
+ */
+static int ext4_fall_back_to_nondelalloc(struct file *file,
+				struct address_space *mapping,
+				loff_t pos, unsigned len, unsigned flags,
+				struct page **pagep, void **fsdata)
+{
+	struct inode *inode = mapping->host;
+
+	/* turn off delalloc for this inode*/
+	ext4_set_aops(inode, 0);
+
+	return mapping->a_ops->write_begin(file, mapping, pos, len,
+					   flags, pagep, fsdata);
+}
+
 static int ext4_da_write_begin(struct file *file, struct address_space *mapping,
 				loff_t pos, unsigned len, unsigned flags,
 				struct page **pagep, void **fsdata)
@@ -2402,6 +2421,15 @@
 	struct inode *inode = mapping->host;
 	handle_t *handle;
 
+	/*
+	 * Fall back to nondelalloc mode if fs is about run out of free
+	 * blocks. We only do this if there is no delalloc dirty pages, those
+	 * dirty pages still need delalloc writepages aops to flush to disk.
+	 */
+	if (!ext4_has_delalloc_pages(inode) && ext4_free_blocks_low(inode))
+		ret= ext4_fall_back_to_nondelalloc(file,mapping,pos,
+						  len, flags, pagep,fsdata);
+
 	index = pos >> PAGE_CACHE_SHIFT;
 	from = pos & (PAGE_CACHE_SIZE - 1);
 	to = from + len;
@@ -2435,8 +2463,8 @@
 		page_cache_release(page);
 	}
 
-	if (ret == -ENOSPC && ext4_should_retry_alloc(inode->i_sb, &retries))
-		goto retry;
+	if (ret == -ENOSPC && (ext4_should_retry_alloc(inode->i_sb, &retries))
+			goto retry;
 out:
 	return ret;
 }
@@ -3008,16 +3036,37 @@
 	.is_partially_uptodate  = block_is_partially_uptodate,
 };
 
-void ext4_set_aops(struct inode *inode)
+#define	EXT4_MIN_FREE_BLOCKS	(NR_CPUS * 4 * NR_CPUS*4)
+
+static int ext4_free_blocks_low(struct inode* inode)
+{
+	return (ext4_has_free_blocks(EXT4_SB(inode->i_sb),
+			 EXT4_MIN_FREE_BLOCKS) > EXT4_MIN_FREE_BLOCKS);
+}
+
+static int ext4_has_delalloc_pages(struct inode* inode)
+{
+	return EXT4_I(inode)->i_reserved_data_blocks;
+}
+
+void ext4_set_aops(struct inode *inode, int delalloc)
 {
-	if (ext4_should_order_data(inode) &&
-		test_opt(inode->i_sb, DELALLOC))
-		inode->i_mapping->a_ops = &ext4_da_aops;
-	else if (ext4_should_order_data(inode))
+	if (test_opt(inode->i_sb, DELALLOC)) {
+		if (delalloc && !ext4_has_delalloc_pages(inode)
+			&& ext4_free_blocks_low(inode))
+			delalloc = 0;
+
+		if (delalloc) {
+			inode->i_mapping->a_ops = &ext4_da_aops;
+			return;
+		} else
+			printk(KERN_INFO "filesystem is close to full, "
+				"delayed allocation is turned off for "
+				" inode %lu\n", inode->i_ino);
+	}
+
+	if (ext4_should_order_data(inode))
 		inode->i_mapping->a_ops = &ext4_ordered_aops;
-	else if (ext4_should_writeback_data(inode) &&
-		 test_opt(inode->i_sb, DELALLOC))
-		inode->i_mapping->a_ops = &ext4_da_aops;
 	else if (ext4_should_writeback_data(inode))
 		inode->i_mapping->a_ops = &ext4_writeback_aops;
 	else
@@ -4011,7 +4060,7 @@
 	if (S_ISREG(inode->i_mode)) {
 		inode->i_op = &ext4_file_inode_operations;
 		inode->i_fop = &ext4_file_operations;
-		ext4_set_aops(inode);
+		ext4_set_aops(inode, 1);
 	} else if (S_ISDIR(inode->i_mode)) {
 		inode->i_op = &ext4_dir_inode_operations;
 		inode->i_fop = &ext4_dir_operations;
@@ -4020,7 +4069,7 @@
 			inode->i_op = &ext4_fast_symlink_inode_operations;
 		else {
 			inode->i_op = &ext4_symlink_inode_operations;
-			ext4_set_aops(inode);
+			ext4_set_aops(inode, 1);
 		}
 	} else {
 		inode->i_op = &ext4_special_inode_operations;
@@ -4783,7 +4832,7 @@
 		EXT4_I(inode)->i_flags |= EXT4_JOURNAL_DATA_FL;
 	else
 		EXT4_I(inode)->i_flags &= ~EXT4_JOURNAL_DATA_FL;
-	ext4_set_aops(inode);
+	ext4_set_aops(inode, 1);
 
 	jbd2_journal_unlock_updates(journal);
 
Index: linux-2.6.27-rc3/fs/ext4/ext4.h
===================================================================
--- linux-2.6.27-rc3.orig/fs/ext4/ext4.h	2008-08-20 15:41:36.000000000 -0700
+++ linux-2.6.27-rc3/fs/ext4/ext4.h	2008-08-20 15:41:56.000000000 -0700
@@ -1070,7 +1070,7 @@
 extern void ext4_truncate (struct inode *);
 extern void ext4_set_inode_flags(struct inode *);
 extern void ext4_get_inode_flags(struct ext4_inode_info *);
-extern void ext4_set_aops(struct inode *inode);
+extern void ext4_set_aops(struct inode *inode, int delalloc);
 extern int ext4_writepage_trans_blocks(struct inode *);
 extern int ext4_meta_trans_blocks(struct inode *, int nrblocks, int idxblocks);
 extern int ext4_chunk_trans_blocks(struct inode *, int nrblocks);
Index: linux-2.6.27-rc3/fs/ext4/namei.c
===================================================================
--- linux-2.6.27-rc3.orig/fs/ext4/namei.c	2008-08-20 15:42:13.000000000 -0700
+++ linux-2.6.27-rc3/fs/ext4/namei.c	2008-08-20 15:42:41.000000000 -0700
@@ -1738,7 +1738,7 @@
 	if (!IS_ERR(inode)) {
 		inode->i_op = &ext4_file_inode_operations;
 		inode->i_fop = &ext4_file_operations;
-		ext4_set_aops(inode);
+		ext4_set_aops(inode, 1);
 		err = ext4_add_nondir(handle, dentry, inode);
 	}
 	ext4_journal_stop(handle);
@@ -2210,7 +2210,7 @@
 
 	if (l > sizeof (EXT4_I(inode)->i_data)) {
 		inode->i_op = &ext4_symlink_inode_operations;
-		ext4_set_aops(inode);
+		ext4_set_aops(inode, 1);
 		/*
 		 * page_symlink() calls into ext4_prepare/commit_write.
 		 * We have a transaction open.  All is sweetness.  It also sets


--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Reiser Filesystem Development]     [Ceph FS]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite National Park]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Device Mapper]     [Linux Media]

  Powered by Linux