On Tue, Sep 08, 2020 at 09:53:11PM +0200, Pavel Machek wrote:
> Hi!
>
> > On machines with much memory (> 2 TByte) and log_mtts_per_seg == 0, a
> > max_order of 31 will be passed to mlx_buddy_init(), which results in
> > s = BITS_TO_LONGS(1 << 31) becoming a negative value, leading to
> > kvmalloc_array() failure when it is converted to size_t.
> >
> > mlx4_core 0000:b1:00.0: Failed to initialize memory region table, aborting
> > mlx4_core: probe of 0000:b1:00.0 failed with error -12
> >
> > Fix this issue by changing the left shifting operand from a signed literal to
> > an unsigned one.
>
> Will we still have problems with > 4 TByte machines?
AFAIK we're safe since max_buddy is calculated as such
/* In drivers/net/ethernet/mellanox/mlx4/mr.c */
err = mlx4_buddy_init(&mr_table->mtt_buddy,
ilog2((u32)dev->caps.num_mtts /
(1 << log_mtts_per_seg)));
Also, num_mtts is capped at 2^31
/* In drivers/net/ethernet/mellanox/mlx4/profile.c */
/*
* We want to scale the number of MTTs with the size of the
* system memory, since it makes sense to register a lot of
* memory on a system with a lot of memory. As a heuristic,
* make sure we have enough MTTs to cover twice the system
* memory (with PAGE_SIZE entries).
*
* This number has to be a power of two and fit into 32 bits
* due to device limitations, so cap this at 2^31 as well.
* That limits us to 8TB of memory registration per HCA with
* 4KB pages, which is probably OK for the next few months.
*/
si_meminfo(&si);
request->num_mtt =
roundup_pow_of_two(max_t(unsigned, request->num_mtt,
min(1UL << (31 - log_mtts_per_seg),
(si.totalram << 1) >> log_mtts_per_seg)));
Best,
Shung-Hsi Yu
> Should the computation be done in u64?
>
> Best regards,
> Pavel
>
> > Fixes: 225c7b1feef1 ("IB/mlx4: Add a driver Mellanox ConnectX InfiniBand adapters")
> > Signed-off-by: Shung-Hsi Yu <shung-hsi.yu@xxxxxxxx>
> > Signed-off-by: David S. Miller <davem@xxxxxxxxxxxxx>
> > Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>
>
> > +++ b/drivers/net/ethernet/mellanox/mlx4/mr.c
> > @@ -114,7 +114,7 @@ static int mlx4_buddy_init(struct mlx4_buddy *buddy, int max_order)
> > goto err_out;
> >
> > for (i = 0; i <= buddy->max_order; ++i) {
> > - s = BITS_TO_LONGS(1 << (buddy->max_order - i));
> > + s = BITS_TO_LONGS(1UL << (buddy->max_order - i));
> > buddy->bits[i] = kvmalloc_array(s, sizeof(long), GFP_KERNEL | __GFP_ZERO);
> > if (!buddy->bits[i])
> > goto err_out_free;
>
> --
> (english) http://www.livejournal.com/~pavelmachek
> (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
