On Mon, 2016-04-04 at 17:01 -0600, Chris Friesen wrote: > On 04/04/2016 04:29 PM, Nicholas A. Bellinger wrote: > > On Mon, 2016-04-04 at 09:20 -0600, Chris Friesen wrote: > >> On 04/02/2016 07:15 PM, Nicholas A. Bellinger wrote: > >>> On Fri, 2016-04-01 at 12:35 -0600, Chris Friesen wrote: > >> > >>>>>> On a slightly different note, is there any way to throttle or limit the overall > >>>>>> bandwidth consumed by the iSCSI target in the kernel? I'd like to ensure that > >>>>>> the iSCSI traffic doesn't completely swamp the host accesses to the same block > >>>>>> device. > >>>>>> > >>>>>> I suppose I could do networking-based traffic shaping, but are there any > >>>>>> controls in the block IO subsystem? > >>>>>> > >>>>> > >>>>> On a individual block_device backend basis, block cgroups is the > >>>>> preferred method for doing this. > >>>>> > >>>>> Note that any rate limiting imposed by block cgroups is subject to the > >>>>> current se_node_acl->queue_depth enforced across all LUNs within a given > >>>>> iscsi session. > >>>> > >>>> > >>>> How would I use cgroups with the kernel iSCSI target? Is there a set of kernel > >>>> threads dedicated to the iSCSI target that I can assign to a particular group? > >>>> > >>> > >>> block cgroups can set I/O throttling (bandwidth + IOPs) limits for any > >>> normal struct block_device. > >>> > >>> These values are configured via blkio.throttle.* resource class, and the > >>> limits are imposed independently of the block_device's association with > >>> target_core_mod backend driver export. Namely: > >>> > >>> blkio.throttle.write_iops_device > >>> blkio.throttle.read_iops_device > >>> blkio.throttle.write_bps_device > >>> blkio.throttle.read_bdp_device > >>> > >>> Some examples using these values, plus more blkio.* info is here: > >>> http://events.linuxfoundation.org/sites/events/files/slides/cgroups_0.pdf. > >>> > >> > >> I understand how to set up the cgroups, but don't I need to ensure that the IO > >> operations happen in the context of a particular cgroup? If so, how do I ensure > >> that the in-kernel iSCSI target traffic happens in the context of the specified > >> group? Are there a set of kernel threads dedicated to processing iSCSI requests? > >> > > > > block cgroups does I/O ratelimiting at struct block_device level. > > > > Eg: The process cgroup (which AFAICT is what your thinking about) is > > separate from block cgroups, and doesn't need to be explicitly enabled > > in order for block cgroup to function. > > > > I'm still confused. > > I'm not trying to globally throttle IO on a particular block device. I'm trying > to control how much IO the iSCSI target in the kernel is allowed to drive on a > particular block device. > > The goal is to ensure that the iSCSI target does not consume all of the > available bandwidth for a particular block device. I want to ensure that some > of the bandwidth for that device is available to other processes on the host > (for management purposes) rather than being consumed by a greedy iSCSI initiator. > AFAIK, different traffic classes for a single block device is not supported by block cgroups. Also given target_core_iblock claims a given block_device for exclusive access, you can't actually use the same block device for a mounted filesystem, LVM, MD, etc, once it's been registered for target usage. > In an ideal world I would like a set of rules that say things like: > 1) if there is contention, ensure that the host is guaranteed X percent of the > available /dev/sdb IOPS > 2) if there is contention, do not allow the iSCSI target traffic to consume more > than Y percent of /dev/sdb's write traffic > Yeah, that doesn't exist in standlone block groups. You could probably use network cgroups to limit bandwidth at the socket level for iscsi_target_mod, but that's going to be across all LUNs in a session, and not at individual LUN level. -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
