The ABINIT Users Mailing List ( CLOSED )

[Jyh-Shyong Ho <c00jsh00@nchc.org.tw>]

Hi,

BigDFT is one of plugin programs of abinit 5.8.4, now BigDFT can be built with
NVIDIA CUDA support. I wonder if anyone has successfully built BigDFT with
CUDA?

I downloaded the latest version bigdft-1.3 and tried to install it on my
computer with CUDA 2.3, but failed. If someone can provide me some hint where
might be wrong, it will be very appreciated.

In directory bigdft-1.3.0/src/CUDA, I was able to build libGPU.a, however, I
failed to built check_conv which uses libGPU.a:

...
chem@gch:/work/chem/bigdft-1.3.0/src/CUDA> make check
make  conv_check GPUham
make[1]: Entering directory `/work/chem/bigdft-1.3.0/src/CUDA'
ifort -I../modules  -I. -I. -O2  -L/opt/lapack/intel -L/opt/lapack/intel
-L../../libXC/src -o conv_check conv_check.o memory.o base.o  ../libbigdft.a
-L/opt/cuda/lib/ -lcudart -lcublas libGPU.a -lxc -llapack -lblas  -lpthread
conv_check.o: In function `MAIN__':
conv_check.f90:(.text+0x4f0): undefined reference to `set_gpu_double_'
conv_check.f90:(.text+0x1417): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x144c): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x1488): undefined reference to `gpu_send_'
conv_check.f90:(.text+0x1703): undefined reference to `gpu_receive_'
conv_check.f90:(.text+0x171a): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x1731): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x230d): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x2342): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x2377): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x23ac): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x23e8): undefined reference to `gpu_send_'
conv_check.f90:(.text+0x269e): undefined reference to `gpu_receive_'
conv_check.f90:(.text+0x26b5): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x26cc): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x26e3): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x26fa): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x2fc3): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x2ff8): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x3034): undefined reference to `gpu_send_'
conv_check.f90:(.text+0x32bc): undefined reference to `gpu_receive_'
conv_check.f90:(.text+0x32d3): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x32ea): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x441b): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x4458): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x448f): undefined reference to `gpu_send_'
conv_check.f90:(.text+0x475b): undefined reference to `gpu_receive_'
conv_check.f90:(.text+0x4772): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x4789): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x47a0): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x52aa): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x52e7): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x532c): undefined reference to `gpu_send_'
conv_check.f90:(.text+0x55df): undefined reference to `gpu_receive_'
conv_check.f90:(.text+0x55f6): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x560d): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x5624): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x6539): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x656e): undefined reference to `gpu_allocate_'
conv_check.f90:(.text+0x65aa): undefined reference to `gpu_send_'
conv_check.f90:(.text+0x6832): undefined reference to `gpu_receive_'
conv_check.f90:(.text+0x6849): undefined reference to `gpu_deallocate_'
conv_check.f90:(.text+0x6860): undefined reference to `gpu_deallocate_'
memory.o: In function `memory_occupation_':
../profiling/memory.f90:(.text+0x289): undefined reference to `mpi_abort_'
../profiling/memory.f90:(.text+0x549): undefined reference to `mpi_abort_'
base.o: In function `module_base_mp_herk_simple_':
../modules/base.F90:(.text+0xd33): undefined reference to `cublas_cherk_'
base.o: In function `module_base_mp_syrk_double_':
../modules/base.F90:(.text+0xd93): undefined reference to `cublas_dsyrk_'
base.o: In function `module_base_mp_syrk_simple_':
../modules/base.F90:(.text+0xdf3): undefined reference to `cublas_ssyrk_'
base.o: In function `module_base_mp_c_gemm_simple_':
../modules/base.F90:(.text+0xf05): undefined reference to `cublas_cgemm_'
base.o: In function `module_base_mp_gemm_double_':
../modules/base.F90:(.text+0xfe5): undefined reference to `cublas_dgemm_'
base.o: In function `module_base_mp_gemm_simple_':
../modules/base.F90:(.text+0x10c5): undefined reference to `cublas_sgemm_'
base.o: In function `module_base_mp_nrm2_simple_':
../modules/base.F90:(.text+0x114e): undefined reference to `cublas_snrm2_'
base.o: In function `module_base_mp_dot_simple_':
../modules/base.F90:(.text+0x117e): undefined reference to `cublas_sdot_'
base.o: In function `module_base_mp_c_axpy_simple_':
../modules/base.F90:(.text+0x11ae): undefined reference to `cublas_caxpy_'
base.o: In function `module_base_mp_axpy_simple_':
../modules/base.F90:(.text+0x11de): undefined reference to `cublas_saxpy_'
base.o: In function `module_base_mp_c_trmm_simple_':
../modules/base.F90:(.text+0x12b5): undefined reference to `cublas_ctrmm_'
base.o: In function `module_base_mp_trmm_simple_':
../modules/base.F90:(.text+0x13d5): undefined reference to `cublas_strmm_'
base.o: In function `module_base_mp_c_scal_simple_':
../modules/base.F90:(.text+0x144e): undefined reference to `cublas_cscal_'
base.o: In function `module_base_mp_scal_simple_':
../modules/base.F90:(.text+0x147e): undefined reference to `cublas_sscal_'
../libbigdft.a(convolut_common_interface_cuda.o): In function
`gpu_precond_helper_stream_':
convolut_common_interface_cuda.f90:(.text+0x11b): undefined reference to
`gpuprecond_stream_'
../libbigdft.a(convolut_common_interface_cuda.o): In function
`gpu_locham_helper_stream_':
convolut_common_interface_cuda.f90:(.text+0x3b4): undefined reference to
`gpulocham_stream_'
../libbigdft.a(convolut_common_interface_cuda.o): In function
`gpu_locden_helper_stream_':
convolut_common_interface_cuda.f90:(.text+0x6ee): undefined reference to
`gpulocden_stream_'
../libbigdft.a(convolut_common_interface_cuda.o): In function
`local_partial_density_gpu_':
convolut_common_interface_cuda.f90:(.text+0x9ee): undefined reference to
`gpu_send_'
convolut_common_interface_cuda.f90:(.text+0xbc2): undefined reference to
`gpu_receive_'
convolut_common_interface_cuda.f90:(.text+0xbee): undefined reference to
`create_stream_'
convolut_common_interface_cuda.f90:(.text+0xc70): undefined reference to
`mem_copy_f_to_c_stream_'
convolut_common_interface_cuda.f90:(.text+0xca1): undefined reference to
`gpu_send_pi_stream_'
convolut_common_interface_cuda.f90:(.text+0xde4): undefined reference to
`gpulocden_stream_'
convolut_common_interface_cuda.f90:(.text+0xe82): undefined reference to
`gpu_receive_pi_stream_'
convolut_common_interface_cuda.f90:(.text+0xea0): undefined reference to
`mem_copy_c_to_f_stream_'
convolut_common_interface_cuda.f90:(.text+0xea7): undefined reference to
`launch_all_streams_'
../libbigdft.a(convolut_common_interface_cuda.o): In function
`preconditionall_gpu_':
convolut_common_interface_cuda.f90:(.text+0xfc4): undefined reference to
`gpu_send_'
convolut_common_interface_cuda.f90:(.text+0x120e): undefined reference to
`gpu_receive_'
convolut_common_interface_cuda.f90:(.text+0x1322): undefined reference to
`create_stream_'
convolut_common_interface_cuda.f90:(.text+0x135a): undefined reference to
`mem_copy_f_to_c_stream_'
convolut_common_interface_cuda.f90:(.text+0x1387): undefined reference to
`gpu_send_pi_stream_'
convolut_common_interface_cuda.f90:(.text+0x144d): undefined reference to
`gpuprecond_stream_'
convolut_common_interface_cuda.f90:(.text+0x147e): undefined reference to
`gpu_receive_pi_stream_'
convolut_common_interface_cuda.f90:(.text+0x149c): undefined reference to
`mem_copy_c_to_f_stream_'
convolut_common_interface_cuda.f90:(.text+0x14b6): undefined reference to
`launch_all_streams_'
../libbigdft.a(convolut_common_interface_cuda.o): In function
`local_hamiltonian_gpu_':
convolut_common_interface_cuda.f90:(.text+0x15d1): undefined reference to
`gpu_send_'
convolut_common_interface_cuda.f90:(.text+0x181e): undefined reference to
`gpu_receive_'
convolut_common_interface_cuda.f90:(.text+0x1869): undefined reference to
`create_stream_'
convolut_common_interface_cuda.f90:(.text+0x18c0): undefined reference to
`mem_copy_f_to_c_stream_'
convolut_common_interface_cuda.f90:(.text+0x18fd): undefined reference to
`gpu_send_pi_stream_'
convolut_common_interface_cuda.f90:(.text+0x1904): undefined reference to
`launch_all_streams_'
convolut_common_interface_cuda.f90:(.text+0x19fe): undefined reference to
`create_stream_'
convolut_common_interface_cuda.f90:(.text+0x1ac9): undefined reference to
`gpulocham_stream_'
convolut_common_interface_cuda.f90:(.text+0x1b04): undefined reference to
`gpu_receive_pi_stream_'
convolut_common_interface_cuda.f90:(.text+0x1b2b): undefined reference to
`mem_copy_c_to_f_stream_'
convolut_common_interface_cuda.f90:(.text+0x1b4d): undefined reference to
`launch_all_streams_'
../libbigdft.a(convolut_common_interface_cuda.o): In function `free_gpu_':
convolut_common_interface_cuda.f90:(.text+0x1b79): undefined reference to
`gpu_deallocate_'
convolut_common_interface_cuda.f90:(.text+0x1b89): undefined reference to
`gpu_deallocate_'
convolut_common_interface_cuda.f90:(.text+0x1b99): undefined reference to
`gpu_deallocate_'
convolut_common_interface_cuda.f90:(.text+0x1ba9): undefined reference to
`gpu_deallocate_'
convolut_common_interface_cuda.f90:(.text+0x1bb9): undefined reference to
`gpu_deallocate_'
../libbigdft.a(convolut_common_interface_cuda.o):convolut_common_interface_cuda.f90:(.text+0x1bc8):
more undefined references to `gpu_deallocate_' follow
../libbigdft.a(convolut_common_interface_cuda.o): In function
`adjust_keys_for_gpu_':
convolut_common_interface_cuda.f90:(.text+0x2348): undefined reference to
`gpu_int_allocate_'
convolut_common_interface_cuda.f90:(.text+0x2368): undefined reference to
`gpu_int_send_'
../libbigdft.a(convolut_common_interface_cuda.o): In function
`prepare_gpu_for_locham_':
convolut_common_interface_cuda.f90:(.text+0x247f): undefined reference to
`set_gpu_double_'
convolut_common_interface_cuda.f90:(.text+0x2b47): undefined reference to
`gpu_int_allocate_'
convolut_common_interface_cuda.f90:(.text+0x2b6d): undefined reference to
`gpu_int_send_'
convolut_common_interface_cuda.f90:(.text+0x2dc4): undefined reference to
`gpu_allocate_'
convolut_common_interface_cuda.f90:(.text+0x2e12): undefined reference to
`gpu_allocate_'
convolut_common_interface_cuda.f90:(.text+0x2e35): undefined reference to
`gpu_allocate_'
convolut_common_interface_cuda.f90:(.text+0x2e58): undefined reference to
`gpu_allocate_'
convolut_common_interface_cuda.f90:(.text+0x2e92): undefined reference to
`gpu_allocate_'
../libbigdft.a(convolut_common_interface_cuda.o):convolut_common_interface_cuda.f90:(.text+0x2ec6):
more undefined references to `gpu_allocate_' follow
../libbigdft.a(convolut_common_interface_cuda.o): In function
`prepare_gpu_for_locham_':
convolut_common_interface_cuda.f90:(.text+0x2f3b): undefined reference to
`cpu_pinned_allocation_'
convolut_common_interface_cuda.f90:(.text+0x2f5b): undefined reference to
`cpu_pinned_allocation_'
make[1]: *** [conv_check] Error 1
make[1]: Leaving directory `/work/chem/bigdft-1.3.0/src/CUDA'
make: *** [check-am] Error 2
   

Most undefined references are included in ligGPU.a, however, for some known
reason, ligGPU.a was not recognized.

Thanks for any help.

Jyh-Shyong Ho, Ph.D.
Research Scientist
National Center for High Performance Computing
Hsinchu, Taiwan, ROC