Re: [LSF/MM/BPF BoF]: extend UBLK to cover real storage hardware

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Ming,

Ming Lei <ming.lei@xxxxxxxxxx> writes:

> On Mon, Feb 13, 2023 at 02:13:59PM -0500, Stefan Hajnoczi wrote:
>> On Mon, Feb 13, 2023 at 11:47:31AM +0800, Ming Lei wrote:
>> > On Wed, Feb 08, 2023 at 07:17:10AM -0500, Stefan Hajnoczi wrote:
>> > > On Wed, Feb 08, 2023 at 10:12:19AM +0800, Ming Lei wrote:
>> > > > On Mon, Feb 06, 2023 at 03:27:09PM -0500, Stefan Hajnoczi wrote:
>> > > > > On Mon, Feb 06, 2023 at 11:00:27PM +0800, Ming Lei wrote:
>> > > > > > Hello,
>> > > > > > 
>> > > > > > So far UBLK is only used for implementing virtual block device from
>> > > > > > userspace, such as loop, nbd, qcow2, ...[1].
>> > > > > 
>> > > > > I won't be at LSF/MM so here are my thoughts:
>> > > > 
>> > > > Thanks for the thoughts, :-)
>> > > > 
>> > > > > 
>> > > > > > 
>> > > > > > It could be useful for UBLK to cover real storage hardware too:
>> > > > > > 
>> > > > > > - for fast prototype or performance evaluation
>> > > > > > 
>> > > > > > - some network storages are attached to host, such as iscsi and nvme-tcp,
>> > > > > > the current UBLK interface doesn't support such devices, since it needs
>> > > > > > all LUNs/Namespaces to share host resources(such as tag)
>> > > > > 
>> > > > > Can you explain this in more detail? It seems like an iSCSI or
>> > > > > NVMe-over-TCP initiator could be implemented as a ublk server today.
>> > > > > What am I missing?
>> > > > 
>> > > > The current ublk can't do that yet, because the interface doesn't
>> > > > support multiple ublk disks sharing single host, which is exactly
>> > > > the case of scsi and nvme.
>> > > 
>> > > Can you give an example that shows exactly where a problem is hit?
>> > > 
>> > > I took a quick look at the ublk source code and didn't spot a place
>> > > where it prevents a single ublk server process from handling multiple
>> > > devices.
>> > > 
>> > > Regarding "host resources(such as tag)", can the ublk server deal with
>> > > that in userspace? The Linux block layer doesn't have the concept of a
>> > > "host", that would come in at the SCSI/NVMe level that's implemented in
>> > > userspace.
>> > > 
>> > > I don't understand yet...
>> > 
>> > blk_mq_tag_set is embedded into driver host structure, and referred by queue
>> > via q->tag_set, both scsi and nvme allocates tag in host/queue wide,
>> > that said all LUNs/NSs share host/queue tags, current every ublk
>> > device is independent, and can't shard tags.
>> 
>> Does this actually prevent ublk servers with multiple ublk devices or is
>> it just sub-optimal?
>
> It is former, ublk can't support multiple devices which share single host
> because duplicated tag can be seen in host side, then io is failed.
>

I have trouble following this discussion. Why can we not handle multiple
block devices in a single ublk user space process?

>From this conversation it seems that the limiting factor is allocation
of the tag set of the virtual device in the kernel? But as far as I can
tell, the tag sets are allocated per virtual block device in
`ublk_ctrl_add_dev()`?

It seems to me that a single ublk user space process shuld be able to
connect to multiple storage devices (for instance nvme-of) and then
create a ublk device for each namespace, all from a single ublk process.

Could you elaborate on why this is not possible?

Best regards,
Andreas Hindborg



[Index of Archives]     [Linux RAID]     [Linux SCSI]     [Linux ATA RAID]     [IDE]     [Linux Wireless]     [Linux Kernel]     [ATH6KL]     [Linux Bluetooth]     [Linux Netdev]     [Kernel Newbies]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Device Mapper]

  Powered by Linux