On Fri, 7 Jan 2022, Eric Wheeler wrote: > On Fri, 7 Jan 2022, Martin K. Petersen wrote: > > Eric, > > > > > Even new new RAID controlers that _do_ provide `io_opt` still do _not_ > > > indicate partial_stripes_expensive (which is an mdraid feature, but Martin > > > please correct me if I'm wrong here). > > > > partial_stripes_expensive is a bcache thing, I am not sure why it needs > > a separate flag. It is implied, although I guess one could argue that > > RAID0 is a special case since partial writes are not as painful as with > > parity RAID. > > I'm guessing bcache used did some optimization for > queue->limits.raid_partial_stripes_expensive because md raid5 code sets > this flag. At least when using Linux md as the RAID5 implementation it > gets configured automatically: > raid5.c: mddev->queue->limits.raid_partial_stripes_expensive = 1; > > https://elixir.bootlin.com/linux/latest/source/drivers/md/raid5.c#L7729 > > Interestingly only bcache uses it, but md does set it. Ok so `git blame` shows that Kent added this to md/raid5.c in c78afc6261b (Kent Overstreet 2013-07-11 22:39:53 -0700 7526) mddev->queue->limits.raid_partial_stripes_expensive = 1; bcache/md: Use raid stripe size Now that we've got code for raid5/6 stripe awareness, bcache just needs to know about the stripes and when writing partial stripes is expensive - we probably don't want to enable this optimization for raid1 or 10, even though they have stripes. So add a flag to queue_limits. Kent, Martin: Do you think we should leave the md-specific raid_partial_stripes_expensive setting and require users of RAID controllers to set the bit themselves in bcache---or---remove all raid_partial_stripes_expensive code and always treat writes as "expensive" when `opt_io` is defined? -- Eric Wheeler > > > The SCSI spec states that submitting an I/O that is smaller than io_min > > "may incur delays in processing the command". And similarly, submitting > > a command larger than io_opt "may incur delays in processing the > > command". > > > > IOW, the spec says "don't write less than an aligned multiple of the > > stripe chunk size" and "don't write more than an aligned full > > stripe". That leaves "aligned multiples of the stripe chunk size but > > less than the full stripe width" unaccounted for. And I guess that's > > what the bcache flag is trying to capture. > > Maybe any time io_opt is provided then partial_stripes_expensive should be > flagged too and any code to the contrary should be removed? > > Question: Does anyone have a reason to keep partial_stripes_expensive in > the kernel at all? > > > SCSI doesn't go into details about RAID levels and other implementation > > details which is why the wording is deliberately vague. But obviously > > the expectation is that partial stripe writes are slower than full. > > > > In my book any component in the stack that sees either io_min or io_opt > > should try very hard to send I/Os that are aligned multiples of those > > values. I am not opposed to letting users manually twiddle the > > settings. But I do think that we should aim for the stack doing the > > right thing when it sees io_opt reported on a device. > > Agreed, thanks for the feedback! > > -Eric > > > > > > -- > > Martin K. Petersen Oracle Linux Engineering > > >
