On 10/24/24 11:56 AM, Pavel Begunkov wrote:
> On 10/24/24 18:19, Jens Axboe wrote:
>> On 10/24/24 10:06 AM, Pavel Begunkov wrote:
>>> On 10/24/24 16:45, Jens Axboe wrote:
> ...
>>>> bv = kmsg->bvec;
>>>> for_each_iov {
>>>> struct iovec iov;
>>>>
>>>> unsafe_get_user(iov.iov_base, &user_iovec->iov_base, foo);
>>>> unsafe_get_user(iov.iov_len, &user_iovec->iov_len, foo);
>>>>
>>>> import_to_bvec(bv, &iov);
>>>>
>>>> user_iovec++;
>>>> bv++;
>>>> }
>>>>
>>>> if it can be done at prep time, because then there's no need to store
>>>> the iovec at all, it's already stable, just in bvecs. And this avoids
>>>> overlapping iovec/bvec memory, and it avoids doing two iterations for
>>>> import. Purely a poc, just tossing out ideas.
>>>>
>>>> But I haven't looked too closely at your series yet. In any case,
>>>> whatever ends up working for you, will most likely be find for the
>>>> bundled zerocopy send (non-vectored) as well, and I can just put it on
>>>> top of that.
>>>>
>>>>> And you just made one towards delaying the imu resolution, which
>>>>> is conceptually the right thing to do because of the mess with
>>>>> links, just like it is with fixed files. That's why it need to
>>>>> copy the iovec at the prep stage and resolve at the issue time.
>>>>
>>>> Indeed, prep time is certainly the place to do it. And the above
>>>> incremental import should work fine then, as we won't care abot
>>>> user_iovec being stable being prep.
>>>
>>> Seems like you're agreeing but then stating the opposite, there
>>> is some confusion. I'm saying that IMHO the right API wise way
>>> is resolving an imu at issue time, just like it's done for fixed
>>> files, and what your recent series did for send zc.
>>
>> Yeah early morning confusion I guess. And I do agree in principle,
>> though for registered buffers, those have to be registered upfront
>> anyway, so no confusion possible with prep vs issue there. For provided
>> buffers, it only matters for the legacy ones, which generally should not
>> be used. Doesn't change the fact that you're technically correct, the
>> right time to resolve them would be at issue time.
>
> I'm talking about sendmsg with iovec. Registered buffers should
> be registered upfront, that's right, but iovec should be copied
> at prep, and finally resolved into bvecs incl the imu/buffer lookup
> at the issue time. And those are two different points in time,
> maybe because of links, draining or anything else. And if they
> should be at different moments, there is no way to do it while
> copying iovec.
Oh I totally follow, the incremental approach would only work if it can
be done at prep time. If at issue time, then it has to turn an existing
iovec array into the appropriate bvec array. And that's where you'd have
to do some clever bits to avoid holding both a full bvec and iovec array
in memory, which would be pretty wasteful/inefficient. If done at issue
time, then there's no way around a second iteration :/
--
Jens Axboe
