Problem installing Planck 2018 likelihood on NERSC

Use of Healpix, camb, CLASS, cosmomc, compilers, etc.
Post Reply
Sunny Vagnozzi
Posts: 51
Joined: August 15 2016
Affiliation: Kavli Institute for Cosmology (KICC), University of Cambridge

Problem installing Planck 2018 likelihood on NERSC

Post by Sunny Vagnozzi » September 17 2019

Hi all (especially Antony and/or Karim),

I am encountering problems when installing plc-3.0.1 on Cori @NERSC. Specifically, inside plc-3.0.1 the steps I am following are:

Code: Select all

./waf distclean
./waf configure --install_all_deps --lapack_mkl=$MKLROOT --ifort --icc
./waf install
However, already at the ./waf configure part I get the following error:

Code: Select all

No suitable fortran compiler found (cause: 'list index out of range')
This is strange since I explicitly used the --ifort flag, and previously waf managed to successfully find ifort ("Checking for program 'ifort': /opt/intel/compilers_and_libraries_2019.3.199/linux/bin/intel64/ifort") and detect its version ("Check ifort version: 19.0.3.199"). I tried also using the flag --lapack_mkl_version=10.3 but the error remained. Note that echo $MKLROOT returns me /opt/intel/compilers_and_libraries_2019.3.199/linux/mkl.

I suspect this might be due to an incompatibility with Intel 2019, but I might be wrong. I remember already with plc-2.0 there had been problems of compatibility with Intel 2018 (at the time I temporarily fixed this by unloading Intel 2018 and loading Intel 2016, by doing something like module unload intel/18.0.2.199 ; module load intel/16.0.3.210. After that, when NERSC decided to completely remove Intel 2016 I got plc-2.0 to work by using the new version Karim provided (plc-2.1_py3). So I wouldn't be surprised if similar incompatibilities are at play here...

Did anyone ever encounter such a problem, and if so how did you fix it?

Cheers,
Sunny

Sunny Vagnozzi
Posts: 51
Joined: August 15 2016
Affiliation: Kavli Institute for Cosmology (KICC), University of Cambridge

Re: Problem installing Planck 2018 likelihood on NERSC

Post by Sunny Vagnozzi » September 18 2019

Update: I tried installing again using Intel 2018 and python 3 by doing:

Code: Select all

module unload intel ; module load intel/18.0.3.222 ; module unload python ; module load python/3.7-anaconda-2019.07
but I still get the same error:

Code: Select all

module unload intel ; module load intel/18.0.3.222 ; module unload python ; module load python/3.7-anaconda-2019.07
which gets triggered after the following:

Code: Select all

retrieve ifort link line                 : not found 
I explicitly used the flag --ifort, and waf managed to detect ifort successfully:

Code: Select all

Checking for program 'ifort'             : /opt/intel/compilers_and_libraries_2018.3.222/linux/bin/intel64/ifort 
Can the problem be instead the failure to detect gfortran?

Any help would be greatly appreciated! Cheers,
Sunny

Karim Benabed
Posts: 11
Joined: December 18 2008
Affiliation: Institut d'Astrophysique de Paris

Re: Problem installing Planck 2018 likelihood on NERSC

Post by Karim Benabed » September 25 2019

Hi

In order to allow one to link the PLC library with any code in any language that was built and linked with whatever compiler and linker, the PLC installer needs to obtain the recipe that describes how to link with the fortran runtime libraries (CAMspec, bflike and plik_lite are fortran codes). This information is used when building PLC and propagated by the clik-config tool that allows anyone to compile a code that calls the PLC functions. Look for example at the result of

Code: Select all

 bin/clik-config --libs
Obtaining the list of runtime libraries and their locations is straightforward for gfortran, but more difficult for ifort. In this case the list of libraries depends on the architecture and version of the compiler... The PLC waf installer tries to obtain the information at configure time or uses the information provided using the option --fortran_flagline.
The Makefile that can be used instead of waf to build PLC has no way to guess the list of libraries and their locations. This information must be entered by hand, by modifying the line 40 and 41 of the file.

The "best" way I found to allow the waf installer to retrieve the needed information was to ask ifort what is the recipe it uses, using the --dryrun option, and then parse the result of the command to obtain the correct list of library and their locations.
This is fragile. It has broken in the past, and the current ifort 19 installation at NERSC breaks it again by asking ifort to use a ld tool located at a particular path instead of the default one. This confuses the current version of the PLC installer...

I have improved the parser to solve this particular issue and also returns an hopefully clearer error message in case of failure, but it is really a whack-a-mole solution and not a very satisfying response to the problem... If anyone has a better idea on how to solve the problem in a more robust way, I will be very happy to implement it !

I am currently checking that I did not break anything for other version of ifort or for gfortran. When that is done, I will release a new version of the code (along with some other bug fixes).

Meanwhile, if you replace the file waf_tools/try_ifort.py by the one here: https://nextcloud.planck.fr/s/txazptte95JZ2ex, you should be able to compile on NERSC with ifort 19.

Sunny Vagnozzi
Posts: 51
Joined: August 15 2016
Affiliation: Kavli Institute for Cosmology (KICC), University of Cambridge

Re: Problem installing Planck 2018 likelihood on NERSC

Post by Sunny Vagnozzi » September 26 2019

Hi Karim,

thanks a lot for getting back to me on this! As you said, it is a fragile solution to the problem and will likely break at some point when NERSC performs some major maintenance.

In the meantime, I thought I would post an alternative solution (credits go to Martina Gerbino) which worked for me on Cori:

-First of all, unload Intel 2019 and replace it with Intel 2018, and similarly unload Python 2 and replace it with Python 3. In practice what I do is to define the following alias in my .bashrc.ext:

Code: Select all

alias forplc3='module unload intel ; module load intel/18.0.3.222 ; module unload python ; module load python/3.7-anaconda-2019.07'
and after sourcing the .bashrc.ext I run forplc3
-next, inside plc-3.01/ I make the following changes. Modify waf_tools/try_icc.py by going to the function do_icc(ctx) and replacing

Code: Select all

  for pth in list(rL) + ["/lib","/lib64"]:
    ctx.env.append_value("LIBPATH",pth)
    ctx.env.append_value("RPATH",pth) 
with

Code: Select all

  for pth in list(rL) + ["/lib","/lib64"]:
    ctx.env.append_value("LIBPATH_cc_runtime",pth)
    ctx.env.append_value("RPATH_cc_runtime",pth)

  ctx.env.append_value("LIB_cc_runtime",list(rl)+["pthread"]) 
Similarly, modify waf_tools/try_ifort.py by going to the function ifort_conf(ctx) and replacing

Code: Select all

       L = set([ll.strip() for ll in re.findall("^\s+-L(.+)\s*\\\\", re.split("^\s*ld\s*\\\\",llge,flags=re.M)[1],flags=re.M)  if ("ifort" in ll.lower()) or ("intel" in ll.lower())])
       l = set([ll.strip() for ll in re.findall("^\s+-l(.+)\s*\\\\", re.split("^\s*ld\s*\\\\",llge,flags=re.M)[1],flags=re.M)]) 
with

Code: Select all

       L = set([ll.strip() for ll in re.findall("-L(.+)\s*\\\\", llge.split("ld ")[1]) if ("ifort" in ll.lower()) or ("intel" in ll.lower())])
       l = set([ll.strip() for ll in re.findall("-l(.+)\s*\\\\", llge.split("ld ")[1])]) 
-then I compile as usual, i.e.

Code: Select all

./waf distclean
./waf configure --install_all_deps --ifort --icc --lapack_mkl=$MKLROOT
./waf install
This worked for me, although of course it relies on reloading Intel 2018, and I have the suspicion that during one of the major maintenances at NERSC in 2020, Intel 2018 will be completely removed (usually they only keep the Intel version from the current and previous year). So this solution is also quite short-term and ideally someone would come up with something better and more longer-lasting...

Cheers,
Sunny

Post Reply