On 2017-03-23 12:06 PM, Yishai Hadas wrote:
On 3/23/2017 2:51 PM, Jarod Wilson wrote:
Not sure if it's intentional, but in my adventure of the past day or two
to get rdma-core v13 building across all the arches I need to build it
for, I discovered that the mlx5 bits silently don't build on s390x. No
build failure, they just aren't attempted, so far as I can see. The only
sign of failure was when rpm %files globs failed to locate mlx5 bits.
How about mlx4, was it built successfully ? we expect the same behavior
from build point of view as both and others providers depend on
HAVE_COHERENT_DMA from CMakeLists.txt
Any option that you saw below message in the log ?
"Architecture NOT able to do coherent DMA (check libibverbs/arch.h) some
providers disabled!")"
Yep, it's there, lost it in all the verbosity. Okay, that explains that.
I noted that mlx4 has some s390x-specific trickery in
providers/mlx4/mmio.h, and wondered if something similar was required
for mlx5, or if "does not work on s390x and isn't expected to" is the
norm. Currently, my builds are excluding mlx5 support on s390x due to
the current state.
The patch of mlx4 for s390x in mlx4/mmio.h came from IBM people few
years ago, I believe that similar one is needed for mlx5 otherwise it
won't be able to write to PCI memory, so excluding mlx5 for s390 makes
sense. However, mlx4 should be in, is it really the case when you build
the s390 package ?
Apparently not:
$ rpm -qpl libibverbs-13-1.el7.s390x.rpm
/etc/libibverbs.d
/etc/libibverbs.d/hfi1verbs.driver
/etc/libibverbs.d/ipathverbs.driver
/etc/libibverbs.d/rxe.driver
/usr/bin/rxe_cfg
/usr/lib64/libibverbs
/usr/lib64/libibverbs.so.1
/usr/lib64/libibverbs.so.1.1.13
/usr/lib64/libibverbs/libhfi1verbs-rdmav2.so
/usr/lib64/libibverbs/libipathverbs-rdmav2.so
/usr/lib64/libibverbs/librxe-rdmav2.so
/usr/share/doc/rdma-core-13/libibverbs.md
/usr/share/doc/rdma-core-13/rxe.md
/usr/share/man/man7/rxe.7.gz
/usr/share/man/man8/rxe_cfg.8.gz
Looks like only rxe, ipathverbs and hfi1verbs got built. Didn't realize
that many others (including mlx4) weren't getting built until now. Oops.
--
Jarod Wilson
jarod@xxxxxxxxxx
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
