The ABINIT Users Mailing List ( CLOSED )

[Matteo Giantomassi <gmatteo@pcpm.ucl.ac.be>]

Dear Philippe and Fabien,

I have introduced the findggp subroutine to find the "irreducible G-pairs",
namely the set of G-pairs that allow to reconstruct the full dielectric
matrix, for a given external q-point, using symmetry properties. 

This algorithm will lead to a speed up of the code if the irreducible
polarizability is calculated using the spectral method where 
a Hilbert transform must be evaluated for each matrix element.
 
Unfortunately it's not possible to improve the performance of the 
screening part if \chi_0 is obtained through the Adler-Wiser expression 
using the FFT technique to evaluate the oscillator matrix elements 
between pseudo-wavefunctions,
It might be useful, instead, in case of PAW+GW calculations. 

In my development version, I'm using the output of findggp 
to check the correctness of new pieces of code I'm presently 
developing, and I'm also working on the optimization of findggp. 
In my stable release the optimized version of findggp should 
be called only if the spectral method is employed, thus avoiding 
the worsening on the performance you are observing.

The problem is that the next stable release of abinit (5.4.4) 
will not contain these modifications; we agreed with Xavier, indeed, 
that they will be included starting with version 5.5
As a consequence, the GW part of abinit-5.4.4 will 
contain pieces of code that are still under development, sorry for this.
For the moment the best solution consists in commenting the incriminated 
loops as you have done.

Continue below

> Hi Fabien,
> 
> In waiting Matteo, I can clarify.
> As you know, GW computations are very time demanding, and the 
> parallelism that has been introduced is of high interest.
> 
> You are perfectly right : the arrays are filled-in but they are never
> used.
> Humm... ok, 5.4.2 is a dev. version, and these arrays will be probably
> used in a next version.

> 
> For me, the scaling of findggp seems more like O(N^4) : the pairs of Gi
> 
> are compared with other pairs
> that have already been found....
> taking into account that this part is replicated on all the cpus, (i.e.
> 
> is not parallelized), my tests on ZrO2 (with N ~1000)
> show that in practice it becomes the most time consuming part when you
> 
> increase the number of cpus (Amdahl's law).
> For me it limits the scaling to some few cpus, whatever the machine I
> try.
> 

In my opinion the scaling should be somewhere between N^3 and N^4, 
it's not so easy to estimate the scaling of the innner loop on the pairs 
that have been found since it depends on the degree of symmetry of 
the system 
I agree that findgpp could be quite CPU demanding if the number of
G vectors in \chi_0 is large, but it's possible to optimize the 
inner loop by ordering the pairs in shell as well as precalculating 
some arrays.
The most important problem consists in the fact that findggp 
cannot be easily parallelized, at least for a fixed q point! 

By the way, how many G vectors are you using to describe the dielectric 
matrix? I've done tests on systems with a relatively large unit cell 
(up to 800 G vectors for \chi_0) but they have never 
required more than few minutes to find the irreducible G-pairs!  
I guess the cutoff of 20-30 Ha concerns the description of the wavefunctions. 
Could you report the value of npweps used in your tests?

Regards, 
Matteo

> As you said, for the moment it seems that these arrays are not used by
> 
> abinit ; that's why it's safe to put
> a "return" at the beginning of findggp, well done !
> Then the screening calculations scale again when you increase the number
> 
> of cpus, up to 50 cpus,
> youpi, ah merci abinit !!!
> 
> Well, I'm sure that Matteo has some good reasons to introduce the 
> computations of gptab, gptabo, ipf2p,
> probably to gain something like a factor of 2 or 3, but as a user, I 
> would say that this could have a negative impact
> on // performance of abinit.
> 
> Regards,
> 
>  Philippe
> 
> Fabien Bruneval a écrit :
> > Dear Philippe,
>
> > I'm maybe not the right person to answer. But, just rushing trough the
> 
> > new implementation (in version 5.4.2), it seems that the outputs of 
> > the subroutine findggp are not used anywhere in the following. The 
> > arrays gptab, gptabo, and ip2fp are built up and then their usage is
> 
> > everywhere commented out.
> >
> > But, the scaling looks like N^3 where N is npweps, which is usually 
> > rather small (~100-500). It should not hit the calculation to
> much...
> >
> > Matteo, can you tell your opinion on this?
> >
> >
> > Fabien
> >
> >
> >
> > Philippe Blaise wrote:
> >>
> >> Since a few days I'm benchmarking the parallel performances of abinit
> 
> >> 5.4.2
> >> for my GW calculations.
> >>
> >> Contrary to the 'sigma' part of the calculation, the 'screening' part
> 
> >> seems to be
> >> very inefficient in parallel (~50 cpus / ZrO2 / 500 bands, 20-30 Ha
> 
> >> for the cutoffs),
> >> and it is mainly due to the loop on the Gi, Gj pairs done in the 
> >> FINDGGP subroutine.
> >> This loop computes indexes during hours ! Its complexity seems to be
> 
> >> in O(N^4) where
> >> N is the number of Gi i=1,...,N.
> >>
> >> For the moment, I have decided to comment this loop and to do an 
> >> unconditionnal
> >> return, as if they were no symmetry to take into account.
> >> This is of course a work-around, but it's far more efficient in 
> >> parallel,
> >> even if there is more matrix elements to compute for chi0 (approx a
> 
> >> factor of 2 of 3 times).
> >>
> >> Nevertheless I see a comment in the file findggp.F90
> >>
> >> ! TODO Calculate SG outside the loop, requires more memory but should
> 
> >> be faster!
> >>
> >> that sounds good to me. Will it be implemented in a next version ?
> >>
> >> Of course, any comment from the GW experts/developpers is welcome.
> >>
> >> Regards,
> >>
> >>  Philippe
> >>
> >>
> >
> >
> 
> 



----------------
Matteo Giantomassi
PCPM/FSA/UCL

-------------------------------------------------
This mail sent through IMP: atlas.pcpm.ucl.ac.be