The ABINIT Users Mailing List ( CLOSED )

[Jyh-Shyong <c00jsh00@nchc.org.tw>]

Hi,

thanks for the information, with flag "--assume 2underscores", libGPU.a 
is recognized now, however, new
problem occurs:



....

/opt/mvapich2/intel/bin/mpif90 -I../modules 
-I/opt/mvapich2/intel/include -I. -I. -O2 -assume 2underscores 
-L/opt/mvapich2/intel/lib -L/opt/lapack/intel -L/opt/lapack/intel 
-L../../libXC/src -o conv_check conv_check.o memory.o base.o 
../libbigdft.a -L/opt/cuda/lib/ -lcudart -lcublas libGPU.a -lxc -llapack 
-lblas -lmpich -libverbs -lpthread
conv_check.o: In function `MAIN__':
conv_check.f90:(.text+0xea7): undefined reference to `convrot_n_per__'
conv_check.f90:(.text+0x1ed6): undefined reference to `fill_mod_arr__'
conv_check.f90:(.text+0x2e38): undefined reference to `ana_rot_per__'
conv_check.f90:(.text+0x44c8): undefined reference to 
`adjust_keys_for_gpu__'
conv_check.f90:(.text+0x5365): undefined reference to 
`adjust_keys_for_gpu__'
conv_check.f90:(.text+0x63ae): undefined reference to `syn_rot_per__'
memory.o: In function `memory_occupation__':
../profiling/memory.f90:(.text+0x289): undefined reference to `mpi_abort__'
../profiling/memory.f90:(.text+0x549): undefined reference to `mpi_abort__'
../libbigdft.a(base.o): In function `module_base_mp_herk_simple_':
base.F90:(.text+0xd33): undefined reference to `cublas_cherk_'
../libbigdft.a(base.o): In function `module_base_mp_syrk_double_':
base.F90:(.text+0xd93): undefined reference to `cublas_dsyrk_'
../libbigdft.a(base.o): In function `module_base_mp_syrk_simple_':
base.F90:(.text+0xdf3): undefined reference to `cublas_ssyrk_'
../libbigdft.a(base.o): In function `module_base_mp_c_gemm_simple_':
base.F90:(.text+0xf05): undefined reference to `cublas_cgemm_'
../libbigdft.a(base.o): In function `module_base_mp_gemm_double_':
base.F90:(.text+0xfe5): undefined reference to `cublas_dgemm_'
../libbigdft.a(base.o): In function `module_base_mp_gemm_simple_':
base.F90:(.text+0x10c5): undefined reference to `cublas_sgemm_'
../libbigdft.a(base.o): In function `module_base_mp_nrm2_simple_':
base.F90:(.text+0x114e): undefined reference to `cublas_snrm2_'
../libbigdft.a(base.o): In function `module_base_mp_dot_simple_':
base.F90:(.text+0x117e): undefined reference to `cublas_sdot_'
../libbigdft.a(base.o): In function `module_base_mp_c_axpy_simple_':
base.F90:(.text+0x11ae): undefined reference to `cublas_caxpy_'
../libbigdft.a(base.o): In function `module_base_mp_axpy_simple_':
base.F90:(.text+0x11de): undefined reference to `cublas_saxpy_'
../libbigdft.a(base.o): In function `module_base_mp_c_trmm_simple_':
base.F90:(.text+0x12b5): undefined reference to `cublas_ctrmm_'
../libbigdft.a(base.o): In function `module_base_mp_trmm_simple_':
base.F90:(.text+0x13d5): undefined reference to `cublas_strmm_'
../libbigdft.a(base.o): In function `module_base_mp_c_scal_simple_':
base.F90:(.text+0x144e): undefined reference to `cublas_cscal_'
../libbigdft.a(base.o): In function `module_base_mp_scal_simple_':
base.F90:(.text+0x147e): undefined reference to `cublas_sscal_'
make[1]: *** [conv_check] Error 1
make[1]: Leaving directory `/work/chem/bigdft-1.3.0/src/CUDA'
make: *** [check-am] Error 2


Here is the configure options:

export FC=/opt/mvapich2/intel/bin/mpif90
export FCFLAGS=""
export LDFLAGS=" -L/opt/mvapich2/intel/lib"
export LIBS="-lmpich -libverbs -lpthread"
export CC=icc
export CPPFLAGS="-I/opt/mvapich2/intel/include"
export CXX=icpc
export MPI_INCLUDE="-I/opt/mvapich2/intel/include"
export MPI_LDFLAGS="-L/opt/mvapich2/intel/lib"
export MPI_LIBS="-lmpich -lfmpich"
export MPI_NAME="mpich"
./configure 
-prefix=/package/chem/workdir/abinit-5.8.4/tmp/plugins/bigdft 
--with-blas --with-blas-path=/opt/lapack/intel --with-lapack 
--with-lapack-path=/opt/lapack/intel --enable-mpi --enable-cuda-gpu 
--with-cuda-path=/opt/cuda --with-lib-cutils=/opt/cuda/lib


Flags --assume 2underscores was added my editing Makefile.

Thanks for your help.

Jyh-Shyong Ho



Luigi Genovese 提到:
> Hi,
>
> as Matthieu Ospici pointed out, I have just realised that I did not
> read the output of your configure... sorry for that.
>
> Another option which sould be added to the fortran compiler flags for
> supporting cuda routines is the one which adds a second underscore to
> functions, whic in ifort is "-assume 2underscores".
>
> Sorry for the incomplete answer
>
> Luigi
>
>
>
> On Sun, Sep 20, 2009 at 8:01 PM, Luigi Genovese
> <luigi.genovese@gmail.com> wrote:
>   
> > Hi,
> >
> > to compile BigDFT with the CUDA routines you have to add
> > --enable-cuda-gpu
> > in the configure script options, and optionally provide the path of
> > the cuda libraries if it is not the default value.
> > See configure --help for more details.
> >
> > However, your GPU must support double precision (NVIDIA release number
> > 1.3) to run BigDFT on. Simple precision is not supported.
> >
> > In the 1.3.0 version, not so much is add for what concern the
> > documentation for GPU. Do not hesitate to communicate us the problems
> > you may have.
> >
> > Regards
> >
> > Luigi Genovese
> >
> > On Sun, Sep 20, 2009 at 10:26 AM, Jyh-Shyong Ho <c00jsh00@nchc.org.tw> wrote:
> >     
> > > Hi,
> > >
> > > BigDFT is one of plugin programs of abinit 5.8.4, now BigDFT can be built with
> > > NVIDIA CUDA support. I wonder if anyone has successfully built BigDFT with
> > > CUDA?
> > >
> > > I downloaded the latest version bigdft-1.3 and tried to install it on my
> > > computer with CUDA 2.3, but failed. If someone can provide me some hint where
> > > might be wrong, it will be very appreciated.
> > >
> > > In directory bigdft-1.3.0/src/CUDA, I was able to build libGPU.a, however, I
> > > failed to built check_conv which uses libGPU.a:
> > >
> > > ...
> > > chem@gch:/work/chem/bigdft-1.3.0/src/CUDA> make check
> > > make  conv_check GPUham
> > > make[1]: Entering directory `/work/chem/bigdft-1.3.0/src/CUDA'
> > > ifort -I../modules  -I. -I. -O2  -L/opt/lapack/intel -L/opt/lapack/intel
> > > -L../../libXC/src -o conv_check conv_check.o memory.o base.o  ../libbigdft.a
> > > -L/opt/cuda/lib/ -lcudart -lcublas libGPU.a -lxc -llapack -lblas  -lpthread
> > > conv_check.o: In function `MAIN__':
> > > conv_check.f90:(.text+0x4f0): undefined reference to `set_gpu_double_'
> > > conv_check.f90:(.text+0x1417): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x144c): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x1488): undefined reference to `gpu_send_'
> > > conv_check.f90:(.text+0x1703): undefined reference to `gpu_receive_'
> > > conv_check.f90:(.text+0x171a): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x1731): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x230d): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x2342): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x2377): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x23ac): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x23e8): undefined reference to `gpu_send_'
> > > conv_check.f90:(.text+0x269e): undefined reference to `gpu_receive_'
> > > conv_check.f90:(.text+0x26b5): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x26cc): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x26e3): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x26fa): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x2fc3): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x2ff8): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x3034): undefined reference to `gpu_send_'
> > > conv_check.f90:(.text+0x32bc): undefined reference to `gpu_receive_'
> > > conv_check.f90:(.text+0x32d3): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x32ea): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x441b): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x4458): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x448f): undefined reference to `gpu_send_'
> > > conv_check.f90:(.text+0x475b): undefined reference to `gpu_receive_'
> > > conv_check.f90:(.text+0x4772): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x4789): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x47a0): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x52aa): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x52e7): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x532c): undefined reference to `gpu_send_'
> > > conv_check.f90:(.text+0x55df): undefined reference to `gpu_receive_'
> > > conv_check.f90:(.text+0x55f6): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x560d): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x5624): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x6539): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x656e): undefined reference to `gpu_allocate_'
> > > conv_check.f90:(.text+0x65aa): undefined reference to `gpu_send_'
> > > conv_check.f90:(.text+0x6832): undefined reference to `gpu_receive_'
> > > conv_check.f90:(.text+0x6849): undefined reference to `gpu_deallocate_'
> > > conv_check.f90:(.text+0x6860): undefined reference to `gpu_deallocate_'
> > > memory.o: In function `memory_occupation_':
> > > ../profiling/memory.f90:(.text+0x289): undefined reference to `mpi_abort_'
> > > ../profiling/memory.f90:(.text+0x549): undefined reference to `mpi_abort_'
> > > base.o: In function `module_base_mp_herk_simple_':
> > > ../modules/base.F90:(.text+0xd33): undefined reference to `cublas_cherk_'
> > > base.o: In function `module_base_mp_syrk_double_':
> > > ../modules/base.F90:(.text+0xd93): undefined reference to `cublas_dsyrk_'
> > > base.o: In function `module_base_mp_syrk_simple_':
> > > ../modules/base.F90:(.text+0xdf3): undefined reference to `cublas_ssyrk_'
> > > base.o: In function `module_base_mp_c_gemm_simple_':
> > > ../modules/base.F90:(.text+0xf05): undefined reference to `cublas_cgemm_'
> > > base.o: In function `module_base_mp_gemm_double_':
> > > ../modules/base.F90:(.text+0xfe5): undefined reference to `cublas_dgemm_'
> > > base.o: In function `module_base_mp_gemm_simple_':
> > > ../modules/base.F90:(.text+0x10c5): undefined reference to `cublas_sgemm_'
> > > base.o: In function `module_base_mp_nrm2_simple_':
> > > ../modules/base.F90:(.text+0x114e): undefined reference to `cublas_snrm2_'
> > > base.o: In function `module_base_mp_dot_simple_':
> > > ../modules/base.F90:(.text+0x117e): undefined reference to `cublas_sdot_'
> > > base.o: In function `module_base_mp_c_axpy_simple_':
> > > ../modules/base.F90:(.text+0x11ae): undefined reference to `cublas_caxpy_'
> > > base.o: In function `module_base_mp_axpy_simple_':
> > > ../modules/base.F90:(.text+0x11de): undefined reference to `cublas_saxpy_'
> > > base.o: In function `module_base_mp_c_trmm_simple_':
> > > ../modules/base.F90:(.text+0x12b5): undefined reference to `cublas_ctrmm_'
> > > base.o: In function `module_base_mp_trmm_simple_':
> > > ../modules/base.F90:(.text+0x13d5): undefined reference to `cublas_strmm_'
> > > base.o: In function `module_base_mp_c_scal_simple_':
> > > ../modules/base.F90:(.text+0x144e): undefined reference to `cublas_cscal_'
> > > base.o: In function `module_base_mp_scal_simple_':
> > > ../modules/base.F90:(.text+0x147e): undefined reference to `cublas_sscal_'
> > > ../libbigdft.a(convolut_common_interface_cuda.o): In function
> > > `gpu_precond_helper_stream_':
> > > convolut_common_interface_cuda.f90:(.text+0x11b): undefined reference to
> > > `gpuprecond_stream_'
> > > ../libbigdft.a(convolut_common_interface_cuda.o): In function
> > > `gpu_locham_helper_stream_':
> > > convolut_common_interface_cuda.f90:(.text+0x3b4): undefined reference to
> > > `gpulocham_stream_'
> > > ../libbigdft.a(convolut_common_interface_cuda.o): In function
> > > `gpu_locden_helper_stream_':
> > > convolut_common_interface_cuda.f90:(.text+0x6ee): undefined reference to
> > > `gpulocden_stream_'
> > > ../libbigdft.a(convolut_common_interface_cuda.o): In function
> > > `local_partial_density_gpu_':
> > > convolut_common_interface_cuda.f90:(.text+0x9ee): undefined reference to
> > > `gpu_send_'
> > > convolut_common_interface_cuda.f90:(.text+0xbc2): undefined reference to
> > > `gpu_receive_'
> > > convolut_common_interface_cuda.f90:(.text+0xbee): undefined reference to
> > > `create_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0xc70): undefined reference to
> > > `mem_copy_f_to_c_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0xca1): undefined reference to
> > > `gpu_send_pi_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0xde4): undefined reference to
> > > `gpulocden_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0xe82): undefined reference to
> > > `gpu_receive_pi_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0xea0): undefined reference to
> > > `mem_copy_c_to_f_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0xea7): undefined reference to
> > > `launch_all_streams_'
> > > ../libbigdft.a(convolut_common_interface_cuda.o): In function
> > > `preconditionall_gpu_':
> > > convolut_common_interface_cuda.f90:(.text+0xfc4): undefined reference to
> > > `gpu_send_'
> > > convolut_common_interface_cuda.f90:(.text+0x120e): undefined reference to
> > > `gpu_receive_'
> > > convolut_common_interface_cuda.f90:(.text+0x1322): undefined reference to
> > > `create_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x135a): undefined reference to
> > > `mem_copy_f_to_c_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x1387): undefined reference to
> > > `gpu_send_pi_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x144d): undefined reference to
> > > `gpuprecond_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x147e): undefined reference to
> > > `gpu_receive_pi_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x149c): undefined reference to
> > > `mem_copy_c_to_f_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x14b6): undefined reference to
> > > `launch_all_streams_'
> > > ../libbigdft.a(convolut_common_interface_cuda.o): In function
> > > `local_hamiltonian_gpu_':
> > > convolut_common_interface_cuda.f90:(.text+0x15d1): undefined reference to
> > > `gpu_send_'
> > > convolut_common_interface_cuda.f90:(.text+0x181e): undefined reference to
> > > `gpu_receive_'
> > > convolut_common_interface_cuda.f90:(.text+0x1869): undefined reference to
> > > `create_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x18c0): undefined reference to
> > > `mem_copy_f_to_c_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x18fd): undefined reference to
> > > `gpu_send_pi_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x1904): undefined reference to
> > > `launch_all_streams_'
> > > convolut_common_interface_cuda.f90:(.text+0x19fe): undefined reference to
> > > `create_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x1ac9): undefined reference to
> > > `gpulocham_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x1b04): undefined reference to
> > > `gpu_receive_pi_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x1b2b): undefined reference to
> > > `mem_copy_c_to_f_stream_'
> > > convolut_common_interface_cuda.f90:(.text+0x1b4d): undefined reference to
> > > `launch_all_streams_'
> > > ../libbigdft.a(convolut_common_interface_cuda.o): In function `free_gpu_':
> > > convolut_common_interface_cuda.f90:(.text+0x1b79): undefined reference to
> > > `gpu_deallocate_'
> > > convolut_common_interface_cuda.f90:(.text+0x1b89): undefined reference to
> > > `gpu_deallocate_'
> > > convolut_common_interface_cuda.f90:(.text+0x1b99): undefined reference to
> > > `gpu_deallocate_'
> > > convolut_common_interface_cuda.f90:(.text+0x1ba9): undefined reference to
> > > `gpu_deallocate_'
> > > convolut_common_interface_cuda.f90:(.text+0x1bb9): undefined reference to
> > > `gpu_deallocate_'
> > > ../libbigdft.a(convolut_common_interface_cuda.o):convolut_common_interface_cuda.f90:(.text+0x1bc8):
> > > more undefined references to `gpu_deallocate_' follow
> > > ../libbigdft.a(convolut_common_interface_cuda.o): In function
> > > `adjust_keys_for_gpu_':
> > > convolut_common_interface_cuda.f90:(.text+0x2348): undefined reference to
> > > `gpu_int_allocate_'
> > > convolut_common_interface_cuda.f90:(.text+0x2368): undefined reference to
> > > `gpu_int_send_'
> > > ../libbigdft.a(convolut_common_interface_cuda.o): In function
> > > `prepare_gpu_for_locham_':
> > > convolut_common_interface_cuda.f90:(.text+0x247f): undefined reference to
> > > `set_gpu_double_'
> > > convolut_common_interface_cuda.f90:(.text+0x2b47): undefined reference to
> > > `gpu_int_allocate_'
> > > convolut_common_interface_cuda.f90:(.text+0x2b6d): undefined reference to
> > > `gpu_int_send_'
> > > convolut_common_interface_cuda.f90:(.text+0x2dc4): undefined reference to
> > > `gpu_allocate_'
> > > convolut_common_interface_cuda.f90:(.text+0x2e12): undefined reference to
> > > `gpu_allocate_'
> > > convolut_common_interface_cuda.f90:(.text+0x2e35): undefined reference to
> > > `gpu_allocate_'
> > > convolut_common_interface_cuda.f90:(.text+0x2e58): undefined reference to
> > > `gpu_allocate_'
> > > convolut_common_interface_cuda.f90:(.text+0x2e92): undefined reference to
> > > `gpu_allocate_'
> > > ../libbigdft.a(convolut_common_interface_cuda.o):convolut_common_interface_cuda.f90:(.text+0x2ec6):
> > > more undefined references to `gpu_allocate_' follow
> > > ../libbigdft.a(convolut_common_interface_cuda.o): In function
> > > `prepare_gpu_for_locham_':
> > > convolut_common_interface_cuda.f90:(.text+0x2f3b): undefined reference to
> > > `cpu_pinned_allocation_'
> > > convolut_common_interface_cuda.f90:(.text+0x2f5b): undefined reference to
> > > `cpu_pinned_allocation_'
> > > make[1]: *** [conv_check] Error 1
> > > make[1]: Leaving directory `/work/chem/bigdft-1.3.0/src/CUDA'
> > > make: *** [check-am] Error 2
> > >
> > >
> > > Most undefined references are included in ligGPU.a, however, for some known
> > > reason, ligGPU.a was not recognized.
> > >
> > > Thanks for any help.
> > >
> > > Jyh-Shyong Ho, Ph.D.
> > > Research Scientist
> > > National Center for High Performance Computing
> > > Hsinchu, Taiwan, ROC
> > >
> > >       
>
>