OK guys... I built a serial version of that application. Therefore, the MPI is not present any more. Good news is that I was able to reproduce the illegal instruction. I built the application with -g -ggdb on the frontend and then ran it from the compute node and attached GDB once the process has been started. Please see the GDB output (gdb) c Continuing. Program received signal SIGILL, Illegal instruction. 0x00000000008fdb6e in zdot_kernel_8 () (gdb) bt #0 0x00000000008fdb6e in zdot_kernel_8 () #1 0x00000000008fdd70 in zdotc_k () #2 0x00000000008404fe in zdotc_ () #3 0x000000000082e1ce in zpotf2 (uplo=<value optimized out>, n=64, a=..., lda=1847, info=0, _uplo=<value optimized out>) at zpotf2.f:118 #4 0x000000000080a529 in zpotrf (uplo=<value optimized out>, n=1847, a=..., lda=1847, info=0, _uplo=<value optimized out>) at zpotrf.f:129 #5 0x00000000004b9590 in cdiag (h=..., s=..., n=1847, nm=1847, nml=1847, w=..., z=..., neigvec=0, iscf=1, ierror=0) at /export/apps/siesta/siesta-4.0/Src/cdiag.F:480 #6 0x000000000042d3c1 in diagk (nspin=1, nuo=1847, no=3694, maxspn=1, maxnh=827341, maxnd=827341, maxo=1847, numh=..., listhptr=..., listh=..., numd=..., listdptr=..., listd=..., h=..., s=..., getd=.TRUE., getpsi=.FALSE., fixspin=.FALSE., qtot=554.00000000000125, qs=..., temp=0.0019824056039010785, e1=1, e2=-1, xij=..., indxuo=..., nk=6, kpoint=..., wk=..., eo=..., qo=..., dnew=..., enew=..., ef=0, efs=..., entropy=0, haux=..., saux=..., psi=..., dk=..., ek=..., aux=..., nuotot=1847, occtol=9.9999999999999998e-13, iscf=1) at /export/apps/siesta/siesta-4.0/Src/diagk.F:171 #7 0x0000000000421a1e in diagon (no=3694, maxspn=1, maxuo=1847, maxnh=827341, maxnd=827341, maxo=1847, numh=..., listhptr=..., listh=..., numd=..., listdptr=..., listd=..., h=..., s=..., qtot=554.00000000000125, fixspin=.FALSE., qs=..., temp=0.0019824056039010785, e1=1, e2=-1, gamma=.FALSE., xij=..., indxuo=..., kpoint=..., eo=..., dnew=..., enew=..., ef=0, efs=..., entropy=0, iscf=1, neigwanted=1847) at /export/apps/siesta/siesta-4.0/Src/diagon.F:289 #8 0x00000000004d2896 in m_compute_dm::compute_dm (iscf=1) at /export/apps/siesta/siesta-4.0/Src/compute_dm.F:120 #9 0x00000000004f10e6 in m_siesta_forces::siesta_forces () at /export/apps/siesta/siesta-4.0/Src/siesta_forces.F:132 #10 0x00000000006af149 in siesta () at /export/apps/siesta/siesta-4.0/Src/siesta.F:30 #11 0x000000000092609a in main () #12 0x0000003d1721ecdd in __libc_start_main () from /lib64/libc.so.6 #13 0x0000000000403a29 in _start () (gdb) disas Dump of assembler code for function zdot_kernel_8: 0x00000000008fda20 <+0>: cmp $0x27f,%rdi 0x00000000008fda27 <+7>: jle 0x8fdb18 <zdot_kernel_8+248> 0x00000000008fda2d <+13>: xor %eax,%eax 0x00000000008fda2f <+15>: vzeroupper 0x00000000008fda32 <+18>: vxorpd %xmm0,%xmm0,%xmm0 0x00000000008fda36 <+22>: vxorpd %xmm1,%xmm1,%xmm1 0x00000000008fda3a <+26>: vxorpd %xmm2,%xmm2,%xmm2 0x00000000008fda3e <+30>: vxorpd %xmm3,%xmm3,%xmm3 0x00000000008fda42 <+34>: vxorpd %xmm4,%xmm4,%xmm4 0x00000000008fda46 <+38>: vxorpd %xmm5,%xmm5,%xmm5 0x00000000008fda4a <+42>: vxorpd %xmm6,%xmm6,%xmm6 0x00000000008fda4e <+46>: vxorpd %xmm7,%xmm7,%xmm7 0x00000000008fda52 <+50>: data32 data32 data32 data32 nopw %cs:0x0(%rax,%rax,1) 0x00000000008fda60 <+64>: prefetcht0 0x200(%rsi,%rax,8) 0x00000000008fda68 <+72>: vmovups (%rsi,%rax,8),%xmm8 0x00000000008fda6d <+77>: vmovups 0x10(%rsi,%rax,8),%xmm9 0x00000000008fda73 <+83>: prefetcht0 0x200(%rdx,%rax,8) 0x00000000008fda7b <+91>: vmovups (%rdx,%rax,8),%xmm12 0x00000000008fda80 <+96>: vmovups 0x10(%rdx,%rax,8),%xmm13 0x00000000008fda86 <+102>: vmovups 0x20(%rsi,%rax,8),%xmm10 0x00000000008fda8c <+108>: vmovups 0x30(%rsi,%rax,8),%xmm11 0x00000000008fda92 <+114>: vmovups 0x20(%rdx,%rax,8),%xmm14 0x00000000008fda98 <+120>: vmovups 0x30(%rdx,%rax,8),%xmm15 0x00000000008fda9e <+126>: vfmadd231pd %xmm8,%xmm12,%xmm0 0x00000000008fdaa3 <+131>: vfmadd231pd %xmm9,%xmm13,%xmm1 0x00000000008fdaa8 <+136>: vpermilpd $0x1,%xmm13,%xmm13 0x00000000008fdaae <+142>: vpermilpd $0x1,%xmm12,%xmm12 0x00000000008fdab4 <+148>: vfmadd231pd %xmm10,%xmm14,%xmm2 0x00000000008fdab9 <+153>: vfmadd231pd %xmm11,%xmm15,%xmm3 0x00000000008fdabe <+158>: vpermilpd $0x1,%xmm14,%xmm14 0x00000000008fdac4 <+164>: vpermilpd $0x1,%xmm15,%xmm15 0x00000000008fdaca <+170>: vfmadd231pd %xmm8,%xmm12,%xmm4 0x00000000008fdacf <+175>: add $0x8,%rax 0x00000000008fdad3 <+179>: vfmadd231pd %xmm9,%xmm13,%xmm5 0x00000000008fdad8 <+184>: vfmadd231pd %xmm10,%xmm14,%xmm6 ---Type <return> to continue, or q <return> to quit--- Quit (gdb) x/i $pc => 0x8fdb6e <zdot_kernel_8+334>: vfmadd231pd %xmm8,%xmm12,%xmm0 So, the instruction is vfmadd231pd Any idea? Regards, Mahmood
