On 3/21/22 10:26 PM, Matthew Wilcox wrote:
> On Mon, Mar 21, 2022 at 10:08:47PM +0800, JeffleXu wrote:
>> reqs_lock is also used to protect the check of cache->flags. Please
>> refer to patch 4 [1] of this patchset.
>
> Yes, that's exactly what I meant by "bad idea".
>
>> ```
>> + /*
>> + * Enqueue the pending request.
>> + *
>> + * Stop enqueuing the request when daemon is dying. So we need to
>> + * 1) check cache state, and 2) enqueue request if cache is alive.
>> + *
>> + * The above two ops need to be atomic as a whole. @reqs_lock is used
>> + * here to ensure that. Otherwise, request may be enqueued after xarray
>> + * has been flushed, in which case the orphan request will never be
>> + * completed and thus netfs will hang there forever.
>> + */
>> + read_lock(&cache->reqs_lock);
>> +
>> + /* recheck dead state under lock */
>> + if (test_bit(CACHEFILES_DEAD, &cache->flags)) {
>> + read_unlock(&cache->reqs_lock);
>> + ret = -EIO;
>> + goto out;
>> + }
>
> So this is an error path. We're almost always going to take the xa_lock
> immediately after taking the read_lock. In other words, you've done two
> atomic operations instead of one.
Right.
>
>> + xa_lock(xa);
>> + ret = __xa_alloc(xa, &id, req, xa_limit_32b, GFP_KERNEL);
>> + if (!ret)
>> + __xa_set_mark(xa, id, CACHEFILES_REQ_NEW);
>> + xa_unlock(xa);
>> +
>> + read_unlock(&cache->reqs_lock);
>> ```
>>
>> It's mainly used to protect against the xarray flush.
>>
>> Besides, IMHO read-write lock shall be more performance friendly, since
>> most cases are the read side.
>
> That's almost never true. rwlocks are usually a bad idea because you
> still have to bounce the cacheline, so you replace lock contention
> (which you can see) with cacheline contention (which is harder to
> measure). And then you have questions about reader/writer fairness
> (should new readers queue behind a writer if there's one waiting, or
> should a steady stream of readers be able to hold a writer off
> indefinitely?)
Interesting, I didn't notice it before. Thanks for explaining it.
BTW what I want is just
```
PROCESS 1 PROCESS 2
========= =========
#lock #lock
set DEAD state if (not DEAD)
flush xarray enqueue into xarray
#unlock #unlock
```
I think it is a generic paradigm. So it seems that the spinlock inside
xarray is already adequate for this job?
--
Thanks,
Jeffle
