On Wed, May 15, 2024 at 1:34 PM Luis Chamberlain <mcgrof@xxxxxxxxxx> wrote: > > RFC to see if we have a breakout session today at LSFMM. > > After the TAO talk today it occurred to me that it might make sense > to review how we're measuring memory fragmentation today. We're looking > to add automation support into kdevops for this to help compare and > contrast memory fragmentation behaviour with one kernel against another. > A while ago, while mTHP was being evaluated I asked genearlly how we > could measure fragmentation with a simple one value, and John Hubbard > had one recommendation [0], working that proved we could simplify things > [1] but we also could just use the existing fragmentation index and only > consider the values where this is concerned for fragmentation and not > lack of memory. It begs the question of how folks are measuring memory > fragmentation today in production, and if they have any desirable > changes. The first approach being considered is to reproduce the > workloads Mel Gorman had written and used for mmtests and leverage those > on kdevops, perhaps modernize them, but before we do so it seems > reviewing how we measure fragmentation today might be useful to others > too. > > As for mmtests integration into kdevops, first order of business are > just a few distro-friendly updates [2], for the next steps after that > though it would be great to review the above. > > [0] https://lore.kernel.org/all/5ac6a387-0ca7-45ca-bebc-c3bdd48452cb@xxxxxxxxxx/T/#u > [1] https://lkml.kernel.org/r/20240314005710.2964798-1-mcgrof@xxxxxxxxxx > [2] https://lore.kernel.org/kdevops/20240319044621.2682968-1-mcgrof@xxxxxxxxxx/ Please correct me if I'm wrong -- I don't think we can use a single measure to describe fragmentation in an actionable way. IMO, we would need at least multiple values, e.g., fragmentation index for each non-zero order, to describe how fragmented the memory is with respect to the order of interest. Of course we could encode multiple fragmentation indices into a single value, but that's not really one measure. Fragmentation index of an order can tell whether reclaim+compaction can theoretically result in a free area of that order. As an average, fragmentation index can't tell which actionable unit area, e.g., pageblock, would be the best candidate for reclaim and/or compaction. That would require a ranking model, e.g., a cost function and weights for reclaim and compaction operations, and calculations of the cost to produce a free area of a requested order for each pageblock, i.e., a 2-dimensional measure costs_to_produce_free_area[NR_non_zero_orders][NR_pageblocks].
