On Mon, Jul 01, 2019 at 10:25:04PM +1000, Dave Chinner wrote:
> On Fri, Jun 21, 2019 at 12:57:39PM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <darrick.wong@xxxxxxxxxx>
> >
> > Use the libxfs AG initialization functions to write out the new
> > filesystem instead of open-coding everything.
> >
> > Signed-off-by: Darrick J. Wong <darrick.wong@xxxxxxxxxx>
> .....
> > @@ -4087,8 +3770,16 @@ main(
> > /*
> > * Initialise all the static on disk metadata.
> > */
> > + INIT_LIST_HEAD(&buffer_list);
> > for (agno = 0; agno < cfg.agcount; agno++)
> > - initialise_ag_headers(&cfg, mp, sbp, agno, &worst_freelist);
> > + initialise_ag_headers(&cfg, mp, sbp, agno, &worst_freelist,
> > + &buffer_list);
> > +
> > + if (libxfs_buf_delwri_submit(&buffer_list)) {
> > + fprintf(stderr, _("%s: writing AG headers failed\n"),
> > + progname);
> > + exit(1);
> > + }
>
> The problem I came across with this "one big delwri list" construct
> when adding delwri lists for batched AIO processing is that the
> memory footprint for high AG count filesystems really blows out. Did
> you check what happens when you create a filesystem with a few tens
> of thousands of AGs?
I did, and then amended this patch to delwri_submit every ~16 or so AGs.
:)
I haven't resent the patch since I figure xfsprogs 5.3 is a ways off...
--D
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@xxxxxxxxxxxxx
