The ABINIT Users Mailing List ( CLOSED )

[Alain Jacques <ajs.jacques@gmail.com>]

Aaron wrote:

4101f9fe0902110725t420dadd0x6b8ea1d44f86a2d@mail.gmail.com" type="cite">

Maybe the fact that /home/opt/intel/impi/3.1//bin/mpicc is called comes from a mess in my setup.  I have compiled and installed abinit-5.4.4p serial version in cluster, and I did not do any clean job before compiling abinit-5.6.4 parallel version. I just created a new directory named abinit-5.6.4 and do all parallel compilation in it. Could the previous installation affect the new compilation?

PATH environment variable is like this after cleaning from pgi and pathscale links:

/home/opt/intel/impi/3.1/bin64:/home/opt/intel//fce/10.1.015/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/mfeng/bin:/home/opt/bin:/sbin:/usr/sbin:/home/opt/vasp/bin:/home/opt/crystal98/utils:/home/opt/tetr_lev00/bin:/home/opt/wien2k:/home/opt/wien2k/SRC_structeditor/bin:/home/mfeng/abinit/abinit-5.4.4p/tmp/src/main

and LD_LIBRARY_PATH:

/home/opt/intel/impi/3.1/lib64:/home/opt/intel/mkl/10.0.3.020/lib/em64t:/home/opt/intel//fce/10.1.015/lib

I'll maybe have to discuss with Yann about the pros or cons of not using the mpicc/mpif90 by default in the path - but if he has done that, experience has shown that is for good reasons.
Just for comparison, my setup before compiling abinit-5.6.4 look like (I'm using icc, ifort 10.1 64bit in the usual /opt/intel/... tree, openmpi-1.2.8 home built in /opt/openmpi-1.2.8_intel-10.1_m64):

echo $PATH
/opt/intel/fce/10.1.021/bin:/opt/intel/cce/10.1.021/bin:/opt/openmpi-1.2.8_intel-10.1_m64/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin
echo $LD_LIBRARY_PATH
/opt/openmpi-1.2.8_intel-10.1_m64/lib:/opt/intel/fce/10.1.021/lib:/opt/intel/cce/10.1.021/lib

It is not to tell you that your lines are not okay but it would be nice to include the path to icc binaries and library path to Intel provided libs.

4101f9fe0902110725t420dadd0x6b8ea1d44f86a2d@mail.gmail.com" type="cite">

"which mpicc" returns:    /home/opt/intel/impi/3.1/bin64/mpicc

"which mpif90" returns:    /home/opt/intel/impi/3.1/bin64/mpif90

"mpicc --showme" returns:    

/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/crt1.o: In function `_start':

(.text+0x20): undefined reference to `main'

collect2: ld returned 1 exit status

"mpif90 --showme" returns:

/usr/lib/gcc/x86_64-redhat-linux/4.1.2/libgfortranbegin.a(fmain.o): In function `main':

(.text+0xa): undefined reference to `MAIN__'

collect2: ld returned 1 exit status

"mpicc -v" returns:

mpigcc for the Intel(R) MPI Library 3.1 for Linux

Using built-in specs.

Target: x86_64-redhat-linux

Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --enable-plugin --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --with-cpu=generic --host=x86_64-redhat-linux

Thread model: posix

gcc version 4.1.2 20070925 (Red Hat 4.1.2-33)

 /usr/libexec/gcc/x86_64-redhat-linux/4.1.2/collect2 --eh-frame-hdr --build-id -m elf_i386 --hash-style=gnu -dynamic-linker /lib/ld-linux.so.2 crt1.o crti.o /usr/lib/gcc/x86_64-redhat-linux/4.1.2/32/crtbegin.o -L/home/opt/intel/impi/3.1/lib -L/home/opt/intel/mkl/10.0.3.020/lib/em64t -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2/32 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2/32 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib -L/lib/../lib -L/usr/lib/../lib -rpath /home/opt/intel/impi/3.1/lib -rpath /opt/intel/mpi-rt/3.1 -lmpi -lmpigf -lmpigi -lrt -lpthread -ldl -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-redhat-linux/4.1.2/32/crtend.o crtn.o

/usr/bin/ld: crt1.o: No such file: No such file or directory

Can I tell my system administrator that MPI in cluster was built with gcc?

Independently of abinit, there is something wrong with your mpi setup. You picked 64bit mpicc and mpif90 ... nice. But it should not report missing references in crt1.o and it should not rely on gcc-4.1.2 and not not depend on /usr/lib/gcc/x86_64-redhat-linux/4.1.2/libgfortranbegin.a if it has been configured to work with Intel icc/ifort compilers. I am not working with Intel MPI 3.1 but with openmpi-1.2.8 built with icc/ifort 10.1 and, for comparison, the corresponding output on my system is

mpicc -showme
icc -I/opt/openmpi-1.2.8_intel-10.1_m64/include -pthread -L/opt/openmpi-1.2.8_intel-10.1_m64/lib -lmpi -lopen-rte -lopen-pal -lrt -ldl -Wl,--export-dynamic -lnsl -lutil
mpif90 -showme
ifort -I/opt/openmpi-1.2.8_intel-10.1_m64/include -pthread -I/opt/openmpi-1.2.8_intel-10.1_m64/lib -L/opt/openmpi-1.2.8_intel-10.1_m64/lib -lmpi_f90 -lmpi_f77 -lmpi -lopen-rte -lopen-pal -lrt -ldl -Wl,--export-dynamic -lnsl -lutil

As you can see, mpicc and mpif90 are "wrappers" around icc and ifort. Now it is maybe possible that Intel MPI 3.1 can work with gcc/gfortran but I don't see why you would not use Intel compilers as you have them and your gcc/gfortran-4.1.2 is quite outdated. It would be interesting to have your system admin opinion about your mpicc/mpif90 -v output.

4101f9fe0902110725t420dadd0x6b8ea1d44f86a2d@mail.gmail.com" type="cite">

I have compiled and run the parallel C code you gave me. It completed successfully and showed correct result.

I am rather surprised.

4101f9fe0902110725t420dadd0x6b8ea1d44f86a2d@mail.gmail.com" type="cite">

Then I tried many configure settings as following:

 ./configure FC="/home/opt/intel/impi/3.1/bin64/mpif90" CC="/home/opt/intel/impi/3.1/bin64/mpicc" --enable-mpi=yes

Configuration could completed using above setting, but errors came out when "make":

cd blas && make FC="/home/opt/intel/impi/3.1/bin64/mpif90" FCFLAGS="  -g -O3 -fschedule-insns2 -malign-double -march=nocona -mmmx -msse -msse2 -msse3 -mfpmath=sse" AR="ar" ARFLAGS="rv" RANLIB="ranlib"

make[5]: Entering directory `/home/mfeng/abinit/abinit-5.6.4/prereqs/linalg/blas'

/home/opt/intel/impi/3.1/bin64/mpif90 -g -O3 -fschedule-insns2 -malign-double -march=nocona -mmmx -msse -msse2 -msse3 -mfpmath=sse -c caxpy.f

caxpy.f:0: error: -malign-double makes no sense in the 64bit mode

make[5]: *** [caxpy.o] Error 1

make[5]: Leaving directory `/home/mfeng/abinit/abinit-5.6.4/prereqs/linalg/blas'

make[4]: *** [build-stamp] Error 2

make[4]: Leaving directory `/home/mfeng/abinit/abinit-5.6.4/prereqs/linalg'

make[3]: *** [package-ready] Error 2

make[3]: Leaving directory `/home/mfeng/abinit/abinit-5.6.4/prereqs/linalg'

make[2]: *** [all-recursive] Error 1

make[2]: Leaving directory `/home/mfeng/abinit/abinit-5.6.4/prereqs'

make[1]: *** [all-recursive] Error 1

make[1]: Leaving directory `/home/mfeng/abinit/abinit-5.6.4'

make: *** [all] Error 2

The offending flag malign-double is only valid for 32bit executables, not 64bit; it should definitely not be there. Again Aaron there is a mixture here of 32 bit / 64 bit flags and tricks for gfortran, not ifort . You may have to configure with --enable-64bit-flags.

4101f9fe0902110725t420dadd0x6b8ea1d44f86a2d@mail.gmail.com" type="cite">

 ./configure FC="/home/opt/intel/impi/3.1/bin64/mpif90" CC="/home/opt/intel/impi/3.1/bin64/mpiicc" --enable-mpi=yes

Configuration goes wrong as before. It says "c compiler cannot create executables"

 ./configure FC="/home/opt/intel/impi/3.1/bin64/mpiifort" --enable-mpi=yes --disable-netcdf

Configuration completed but errors came out when make went here:

......

Making all in netcdf

make[3]: Entering directory `/home/mfeng/abinit/abinit-5.6.4/plugins/netcdf'

make -f ../../plugins/netcdf/netcdf.mk

make[4]: Entering directory `/home/mfeng/abinit/abinit-5.6.4/plugins/netcdf'

gzip -cd /home/mfeng/.abinit/tarballs/netcdf-3.6.3.tar.gz | tar xf -

touch uncompress-stamp

netcdf-3.6.3 has been uncompressed.

mkdir tmp

...

I am curious about why --disable-netcdf has no effect on configuration and make. Now I am continue trying to fix this problem.

Because you didn't disable etsf-io and it requires netcdf to be built. Add --disable-etsf-io --disble-netcdf to get rid of both libs.
You can find a sample of my abinit configure recipe on http://www.etsf.be/downloads/ETSF_softlib/page2/page2.html


Kind regards,

Alain

4101f9fe0902110725t420dadd0x6b8ea1d44f86a2d@mail.gmail.com" type="cite">

Thank you for your suggestion. And thanks to Yann too. I hope you can give me more advices about make process.

Best regards,