On Dec 7, 2014, at 4:57 AM, Loïc Pefferkorn <loic@xxxxxxxx> wrote:
>
> On Tue, Dec 02, 2014 at 02:11:33PM -0700, Andreas Dilger wrote:
>> On Nov 28, 2014, at 11:50 AM, Greg KH <gregkh@xxxxxxxxxxxxxxxxxxx> wrote:
>>> On Thu, Nov 27, 2014 at 07:34:10PM +0100, Loïc Pefferkorn wrote:
>>>> Hello Greg,
>>>>
>>>> After some investigation, I think that removing these wrappers is not going to improve the code readability:
>>>>
>>>> On Wed, Nov 26, 2014 at 12:54:43PM -0800, Greg KH wrote:
>>>>> On Wed, Nov 26, 2014 at 05:15:48PM +0100, Loic Pefferkorn wrote:
>>>>>> Add __acquires() and __releases() function annotations, to fix sparse warnings related to lock context imbalance.
>>>>>>
>>>>>> diff --git a/drivers/staging/lustre/lustre/libcfs/hash.c b/drivers/staging/lustre/lustre/libcfs/hash.c
>>>>>> index 32da783..7c6e2a3 100644
>>>>>> --- a/drivers/staging/lustre/lustre/libcfs/hash.c
>>>>>> +++ b/drivers/staging/lustre/lustre/libcfs/hash.c
>>>>>> @@ -126,18 +126,21 @@ cfs_hash_nl_unlock(union cfs_hash_lock *lock, int exclusive) {}
>>>>>>
>>>>>> static inline void
>>>>>> cfs_hash_spin_lock(union cfs_hash_lock *lock, int exclusive)
>>>>>> + __acquires(&lock->spin)
>>>>>> {
>>>>>> spin_lock(&lock->spin);
>>>>>> }
>>>>>>
>>>>>> static inline void
>>>>>> cfs_hash_spin_unlock(union cfs_hash_lock *lock, int exclusive)
>>>>>> + __releases(&lock->spin)
>>>>>> {
>>>>>> spin_unlock(&lock->spin);
>>>>>> }
>>>>>
>>>>> Ugh, how horrid, please just delete these functions and push down the
>>>>> spin_lock/unlock calls down into the places these are called.
>>>>
>>>> cfs_hash_spin_lock() and cfs_hash_spin_unlock() are referenced by function pointers later in the same file:
>>>>
>>>> 165 /** no bucket lock, one spinlock to protect everything */
>>>> 166 static cfs_hash_lock_ops_t cfs_hash_nbl_lops = {
>>>> 167 .hs_lock = cfs_hash_spin_lock,
>>>> 168 .hs_unlock = cfs_hash_spin_unlock,
>>>> 169 .hs_bkt_lock = cfs_hash_nl_lock,
>>>> 170 .hs_bkt_unlock = cfs_hash_nl_unlock,
>>>> 171 };
>>>> 172
>>>> 173 /** spin bucket lock, rehash is enabled */
>>>> 174 static cfs_hash_lock_ops_t cfs_hash_bkt_spin_lops = {
>>>> 175 .hs_lock = cfs_hash_rw_lock,
>>>> 176 .hs_unlock = cfs_hash_rw_unlock,
>>>> 177 .hs_bkt_lock = cfs_hash_spin_lock,
>>>> 178 .hs_bkt_unlock = cfs_hash_spin_unlock,
>>>> 179 };
>>>
>>> That's even worse than I imagined. Putting sparse markings on these
>>> function calls is just papering over nonsense. Please work on
>>> unwinding the mess so that you don't need callbacks for locks,
>>> that is an abstraction that isn't needed.
>>
>> Greg,
>> in this case these abstractions are not old layering. This is part of
>> scalable hashing code that we use all over Lustre, since it needs to
>> handle thousands/millions of clients/connections/files/locks/RPCs
>> or whatever in a lot of places. Having a single interface to define
>> and use these hash functions simplifies the code and avoids bugs,
>> like list_* functions do everywhere else in the kernel.
>>
>> Depending on how the hash table is declared, it may have read/write
>> locks or normal spinlocks. The hash table size may be static, or it
>> may grow when the buckets get too full (since the numbers of items
>> being hashed may differ by many orders of magnitude on different filesystems) so the actual locking functions used depend on how the
>> hash table is set up.
>>
>> I don't think it is possible to unwind the locking of cfs_hash_* code
>> since they are accessed via methods depending on the hash.
>>
>> Cheers, Andreas
>>
>>>>>> static inline void
>>>>>> cfs_hash_rw_lock(union cfs_hash_lock *lock, int exclusive)
>>>>>> + __acquires(&lock->rw)
>>>>>> {
>>>>>> if (!exclusive)
>>>>>> read_lock(&lock->rw);
>>>>>> @@ -147,6 +150,7 @@ cfs_hash_rw_lock(union cfs_hash_lock *lock, int exclusive)
>>>>>>
>>>>>> static inline void
>>>>>> cfs_hash_rw_unlock(union cfs_hash_lock *lock, int exclusive)
>>>>>> + __releases(&lock->rw)
>>>>>> {
>>>>>> if (!exclusive)
>>>>>> read_unlock(&lock->rw);
>>>>>
>>>>>
>>>>> Same for these.
>>>>
>>>> Likewise for cfs_hash_rw_lock() and cfs_hash_rw_unlock():
>>>>
>>>> 173 /** spin bucket lock, rehash is enabled */
>>>> 174 static cfs_hash_lock_ops_t cfs_hash_bkt_spin_lops = {
>>>> 175 .hs_lock = cfs_hash_rw_lock,
>>>> 176 .hs_unlock = cfs_hash_rw_unlock,
>>>> 177 .hs_bkt_lock = cfs_hash_spin_lock,
>>>> 178 .hs_bkt_unlock = cfs_hash_spin_unlock,
>>>> 179 };
>>>> 180
>>>> 181 /** rw bucket lock, rehash is enabled */
>>>> 182 static cfs_hash_lock_ops_t cfs_hash_bkt_rw_lops = {
>>>> 183 .hs_lock = cfs_hash_rw_lock,
>>>> 184 .hs_unlock = cfs_hash_rw_unlock,
>>>> 185 .hs_bkt_lock = cfs_hash_rw_lock,
>>>> 186 .hs_bkt_unlock = cfs_hash_rw_unlock,
>>>> 187 };
>>>
>>> Same here, ick ick ick.
>>>
>>>>>> diff --git a/drivers/staging/lustre/lustre/libcfs/libcfs_lock.c b/drivers/staging/lustre/lustre/libcfs/libcfs_lock.c
>>>>>> index 2c199c7..1e529fc 100644
>>>>>> --- a/drivers/staging/lustre/lustre/libcfs/libcfs_lock.c
>>>>>> +++ b/drivers/staging/lustre/lustre/libcfs/libcfs_lock.c
>>>>>> @@ -91,6 +91,7 @@ EXPORT_SYMBOL(cfs_percpt_lock_alloc);
>>>>>> */
>>>>>> void
>>>>>> cfs_percpt_lock(struct cfs_percpt_lock *pcl, int index)
>>>>>> + __acquires(pcl->pcl_locks[index])
>>>>>> {
>>>>>> int ncpt = cfs_cpt_number(pcl->pcl_cptab);
>>>>>> int i;
>>>>>> @@ -125,6 +126,7 @@ EXPORT_SYMBOL(cfs_percpt_lock);
>>>>>> /** unlock a CPU partition */
>>>>>> void
>>>>>> cfs_percpt_unlock(struct cfs_percpt_lock *pcl, int index)
>>>>>> + __releases(pcl->pcl_locks[index])
>>>>>> {
>>>>>> int ncpt = cfs_cpt_number(pcl->pcl_cptab);
>>>>>> int i;
>>>>>> diff --git a/drivers/staging/lustre/lustre/libcfs/linux/linux-tracefile.c b/drivers/staging/lustre/lustre/libcfs/linux/linux-tracefile.c
>>>>>> index 976c61e..257669b 100644
>>>>>> --- a/drivers/staging/lustre/lustre/libcfs/linux/linux-tracefile.c
>>>>>> +++ b/drivers/staging/lustre/lustre/libcfs/linux/linux-tracefile.c
>>>>>> @@ -151,6 +151,7 @@ cfs_trace_buf_type_t cfs_trace_buf_idx_get(void)
>>>>>> * for details.
>>>>>> */
>>>>>> int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
>>>>>> + __acquires(&tcd->tc_lock)
>>>>>> {
>>>>>> __LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
>>>>>> if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
>>>>>> @@ -165,6 +166,7 @@ int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
>>>>>> }
>>>>>>
>>>>>> void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
>>>>>> + __releases(&tcd->tcd_lock)
>>>>>> {
>>>>>> __LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
>>>>>> if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
>>>>>> diff --git a/drivers/staging/lustre/lustre/obdclass/cl_object.c b/drivers/staging/lustre/lustre/obdclass/cl_object.c
>>>>>> index ce96bd2..8577f97 100644
>>>>>> --- a/drivers/staging/lustre/lustre/obdclass/cl_object.c
>>>>>> +++ b/drivers/staging/lustre/lustre/obdclass/cl_object.c
>>>>>> @@ -193,6 +193,7 @@ static spinlock_t *cl_object_attr_guard(struct cl_object *o)
>>>>>> * cl_object_attr_get(), cl_object_attr_set().
>>>>>> */
>>>>>> void cl_object_attr_lock(struct cl_object *o)
>>>>>> + __acquires(cl_object_attr_guard(o))
>>>>>> {
>>>>>> spin_lock(cl_object_attr_guard(o));
>>>>>> }
>>>>>> @@ -202,6 +203,7 @@ EXPORT_SYMBOL(cl_object_attr_lock);
>>>>>> * Releases data-attributes lock, acquired by cl_object_attr_lock().
>>>>>> */
>>>>>> void cl_object_attr_unlock(struct cl_object *o)
>>>>>> + __releases(cl_object_attr_guard(o))
>>>>>> {
>>>>>> spin_unlock(cl_object_attr_guard(o));
>>>>>> }
>>>>>
>>>>> Same thing here.
>>>>
>>>> These ones are easy to replace, but the naming scheme of all functions in cl_object.c is consistent,
>>>> from my point of view it ease code reading where they are called, for example in lustre/lustre/osc/osc_request.c:
>>>>
>>>> before:
>>>>
>>>> 1827 if (valid != 0) {
>>>> 1828 cl_object_attr_lock(obj);
>>>> 1829 cl_object_attr_set(env, obj, attr, valid);
>>>> 1830 cl_object_attr_unlock(obj);
>>>>
>>>> after:
>>>>
>>>> 1827 if (valid != 0) {
>>>> 1828 spin_lock(cl_object_attr_guard(obj));
>>>> 1829 cl_object_attr_set(env, obj, attr, valid);
>>>> 1830 spin_unlock(cl_object_attr_guard(obj));
>>>>
>>>>
>>>> But I'm here for learning, and I would be grateful to have your opinion.
>>>
>>> Don't hide "implementation of locks" in functions like this, it only
>>> causes problems. This code has layers of layers of layers of
>>> abstractions due to it wanting to be originally ported to other
>>> operating systems and lots of different kernel versions of Linux itself.
>>> Unwinding and removing those layers is a good thing to do, don't paper
>>> over the nonsense by putting sparse markings on pointless functions.
>>>
>>> thanks,
>>>
>>> greg k-h
>>
>>
>> Cheers, Andreas
>
> Hello,
>
> Thanks for your answer. The annotations look like the best thing to do?
Yes, the annotations still make sense.
Cheers, Andreas
_______________________________________________
devel mailing list
devel@xxxxxxxxxxxxxxxxxxxxxx
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
