The ABINIT Users Mailing List ( CLOSED )

[Zeila Zanolli <zeila.zanolli@uclouvain.be>]

Hi everybody, 

the mpw values in the serial and parallel output files are different:

serial: mpw = 30332

7cpu:  mpw =   4333 

where mpw is the maximum number of plane waves.

Hence it seems that the parallelization is done on the plane waves (and indeed is  30332./7 ~ 4333 )

However, in the input file it is   npfft = 1, that is no parallelization on the plane waves.

So... on what is actually performed the parallelization?

On passing, in the log file of the parallel calculation there is the following WARNING:

 getng : WARNING -

  The second and third dimension of the FFT grid,    0    0  were imposed to be multiple of the number of processors for the FFT,    1

 For input ecut=  2.000000E+01 best grid ngfft=      96      40     270

       max ecut=  2.055907E+01

 getng: value of mgfft=     270 and nfft=     1036800

 getng: values of ngfft(4),ngfft(5),ngfft(6)      97      41     270

 getmpw: optimal value of mpw=    4333

while, in the serial mode, it is:

 getng is called for the coarse grid:

 For input ecut=  2.000000E+01 best grid ngfft=      96      40     270

       max ecut=  2.055907E+01

 getng: value of mgfft=     270 and nfft=     1036800

 getng: values of ngfft(4),ngfft(5),ngfft(6)      97      41     270

 getmpw: optimal value of mpw=   30332

can this warning be relevant to the problem?

Thanks again for the help!

  Zeila

On 12 Jan 2010, at 18:25, Zeila Zanolli wrote:

Hi all and thanks for answering.

The peculiarity of my system is that I only have 1 k-point (so I cannot parallelize over k-points), many atoms (~60), a lot of fft points (nfft =   1036800 )

answering to Marc:

the average number of scf steps is similar in both serial and parallel:

serial => 15.4714   

parallel 7cpu => 13.0714

to Francois:

- I know i'm lucky :)

- in the PAW calculation I have reduced the unit cell with respect to the previous T-M calculation that had memory problems.

So now the memory needed is very little:

parallel: ~ 1. Gb per cpu

serial:     ~ 1.4 Gb 

- The input is what I've already posted (only the coordinates and the unit cell are missing)

I'm attaching the difference of the input and output files.

I'm also attaching the number of steps needed to get convergence for each broyden step for both the parallel and serial case

- I've submitted the jobs to check the timing .... i'll let you know as soon as the jobs are done

- I'm running abinit 6.0.0

to Emmanuel:

yes, i've thought about that and I'll do it.

However I was wondering if and how I could exploit more efficiently the bands/fft parallelization.

cheers,

   Zeila

<diff_input>

<diff_out>

<scf_step_parall>

<scf_step_serial>

On 12 Jan 2010, at 16:51, BOTTIN Francois wrote:

Hi Zeila,

This is a very strange behaviour. Never seen until now.

No luck! after a memory problem you have a scaling problem ;)

Is it possible that a relation exists between them (a swap problem due to a too large memory use)?

I don't see anything in your input file which can lead to this very poor scaling!

However, the input file is truncated. Please, would you send your whole input file,

and also the two output files, in order to see the diff between them.

Indeed, there are a lot of variables which can differ between the two calculations

(istwfk, nstep during one Broyden, printing,...)

You can also compare the various part of the timing between the sequential and parallel calculations.

If the part scales linearly, the timing is identical.

This could be useful to detect the one(s) which doesn't scale.

Note that you have to set timopt -3 to have a full description of the timing in parallel.

To have these results more quickly, you can also launch your sequential and parallel calculations

for 1 hour (or 2 Broyden steps).

Regards,

Francois

NB: What is the version of Abinit?

Zeila Zanolli a écrit :

Dear all,

I'm trying my luck with the relaxation of a ~60 atoms molecule with abinit with PAW.

I've set-up a calculation in serial mode and one in parallel mode, using the band/fft parallelization.

However, the parallel calculation is taking the same time as the serial one, as you can see here:

serial:    63 broyden steps in 41 hours

7 cpus:  64 broyden steps in 40 hours

I've set the parallel flags using the paral_kgp = -n 

I have also have a spin-polarized system and in this case the parallel calculation was not proceeding at all:

the calculation stopped at the broyden step 0, after the 3^rd iteration.

The serial calculation for the same spin-pol system is, instead, proceeding without problems (ok - it is just extremely slow: 19 broyden steps in ~ 40 hours).

Here are my input files:

#--------------- for the parallel calculation ---------------------

paral_kgb 1

npkpt  1

npband 7       npfft  1  bandpp 4

nband 112

wfoptalg 4  nloalg 4  fftalg 401  intxc 0  fft_opt_lob 2

#-------------- the following is the same in the parallel and serial calculation ------

kptopt 1

ngkpt 1 1 1

nshiftk 1

shiftk 0 0 0

ntime 1000

tolmxf 1.e-5

ionmov 2

tolrff 1.e-2

ixc 1

ecut 20

pawecutdg 40

nstep 100

diemac 1

Same input for the spin-polarized case, just with the 'spin' variables in addition:

nsppol 2

and initialization of the spin.

Any help is appreciated!

Thanks,

    Zeila

---------------------------------------------------------------------------------------------

Dr. Zeila Zanolli

Université Catholique de Louvain (UCL)

Unité Physico-Chimie et de Physique des Matériaux (PCPM) Place Croix du Sud, 1 (Boltzmann)

B-1348 Louvain-la-Neuve, Belgium

Phone: +32 (0)10 47 3501 Mobile: +32 (0)487 556699

Fax: +32 (0)10 47 3452

e-mail: zeila.zanolli@uclouvain.be <mailto:zeila.zanolli@uclouvain.be>

web: http://www.nano2hybrids.net/

---------------------------------------------------------------------------------------------

-- 

##############################################################

Francois Bottin                    tel: 01 69 26 41 73

CEA/DIF                            fax: 01 69 26 70 77

BP 12 Bruyeres-le-Chatel         email: Francois.Bottin@cea.fr

##############################################################

---------------------------------------------------------------------------------------------

Dr. Zeila Zanolli

Université Catholique de Louvain (UCL)

Unité Physico-Chimie et de Physique des Matériaux (PCPM) 

Place Croix du Sud, 1 (Boltzmann)

B-1348 Louvain-la-Neuve, Belgium

Phone: +32 (0)10 47 3501 

Mobile: +32 (0)487 556699

Fax: +32 (0)10 47 3452

e-mail: zeila.zanolli@uclouvain.be

web: http://www.nano2hybrids.net/

---------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------

Dr. Zeila Zanolli

Université Catholique de Louvain (UCL)

Unité Physico-Chimie et de Physique des Matériaux (PCPM) 

Place Croix du Sud, 1 (Boltzmann)

B-1348 Louvain-la-Neuve, Belgium

Phone: +32 (0)10 47 3501 

Mobile: +32 (0)487 556699

Fax: +32 (0)10 47 3452

e-mail: zeila.zanolli@uclouvain.be

web: http://www.nano2hybrids.net/

---------------------------------------------------------------------------------------------