Installing Planck likelihood

Use of Healpix, camb, CLASS, cosmomc, compilers, etc.
Post Reply
Albin Nilsson
Posts: 13
Joined: October 05 2016
Affiliation: NCBJ, Warsaw

Installing Planck likelihood

Post by Albin Nilsson » October 05 2016

I am trying to install the Planck likelihood using waf, but I am getting an error which I have no idea what to do with.
* The system is Ubuntu 16.04
* MKL is installed in /opt/intel/mkl, this directory is saved in the link $MKLROOT
The command './waf configure --lapack_mkl=/opt/intel/mkl --install_all_deps
'gives me the following error:

Code: Select all

Asking python-config for pyembed '--cflags --libs --ldflags' flags : yes 
Getting pyembed flags from python-config                           : yes 
Checking for library dl                                            : yes 
checking for RTLD_DEFAULT in dl                                    : no 
Checking for program 'pmc-config'                                  : not found 
Checking for library pmc                                           : not found 
Check mkl version                                                  : 10.3 
create specific mkl lib                                            : Command 'gcc -shared -Bdynamic  -u dposv_ -u dtrsv_ -u dpotrf_ -u dpotrs_ -u dpotri_ -u dtrtri_ -u dtrmm_ -u dtrmv_ -u dgeqrf_ -u dormqr_ -u dsyev_ -u dgesvd_ -u dsymv_ -u dgemv_ -u dgemm_ -u dsyrk_ -u dsyr2k_ -u daxpy_ -u dtrsm_ -u dsymm_ -u dsyr_ -u ddot_ -u dsyevd_ -u dlamch_ -u dsyevr_  -Wl,--start-group /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.a /opt/intel/mkl/lib/intel64/libmkl_intel_thread.a /opt/intel/mkl/lib/intel64/libmkl_core.a  -Wl,--end-group -L/home/koldrakan/Cosmological_Codes/planck/cosmology/likelihood/plc-2.0 -liomp5 -lpthread -lm -o "/home/koldrakan/Cosmological_Codes/planck/cosmology/likelihood/plc-2.0/lib/libclik_mkl.so" ' returned 1
Does anyone have any idea what to do about this? Any help would be greatly appreciated

Karim Benabed
Posts: 8
Joined: December 18 2008
Affiliation: Institut d'Astrophysique de Paris

Installing Planck likelihood

Post by Karim Benabed » October 06 2016

Hi

That's the part where a self contained mkl dynamical lib is created to avoid having to link to different libs depending on your particular architecture.

What version of the mkl are you using ?

What happens if you run :

gcc -shared -Bdynamic -u dposv_ -u dtrsv_ -u dpotrf_ -u dpotrs_ -u dpotri_ -u dtrtri_ -u dtrmm_ -u dtrmv_ -u dgeqrf_ -u dormqr_ -u dsyev_ -u dgesvd_ -u dsymv_ -u dgemv_ -u dgemm_ -u dsyrk_ -u dsyr2k -u daxpy_ -u dtrsm_ -u dsymm_ -u dsyr_ -u ddot_ -u dsyevd_ -u dlamch_ -u dsyevr_ -Wl,–start-group /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.a /opt/intel/mkl/lib/intel64/libmkl_intel_thread.a /opt/intel/mkl/lib/intel64/libmkl_core.a -Wl,–end-group -liomp5 -lpthread -lm -o libclik_mkl_TEST.so

Albin Nilsson
Posts: 13
Joined: October 05 2016
Affiliation: NCBJ, Warsaw

Installing Planck likelihood

Post by Albin Nilsson » October 06 2016

Hi, thank you for replying.

I am running MKL version 10.3. The command you provided throws the following:

Code: Select allp>

/usr/bin/ld: cannot find -liomp5
collect2: error: ld returned 1 exit status
so presumably I am missing liomp5?

Karim Benabed
Posts: 8
Joined: December 18 2008
Affiliation: Institut d'Astrophysique de Paris

Installing Planck likelihood

Post by Karim Benabed » October 06 2016

could you send me the full result of the ./waf configure command ?

Which C and F90 compiler are you using ?

Albin Nilsson
Posts: 13
Joined: October 05 2016
Affiliation: NCBJ, Warsaw

Installing Planck likelihood

Post by Albin Nilsson » October 06 2016

I am using gcc and gfortran, but newest versions from the Ubuntu repositories.
I have actually tried this on a second machine now, and it encounters the same problem.
I run the following command (MKL is installed in the default directory /opt/intel):

Code: Select allp>

./waf configure –install_all_deps –lapack_mkl=/opt/intel/
The output is the following:

Code: Select allp>

Setting top to                           : /home/koldrakan/Cosmological_Codes/planck/cosmology/PR2/likelihood/plc−2.0 
Setting out to                           : /home/koldrakan/Cosmological_Codes/planck/cosmology/PR2/likelihood/plc−2.0/build 
Checking for program 'gcc, cc'           : /usr/bin/gcc 
Checking for program 'ar'                : /usr/bin/ar 
Check gcc version                        : 5.4.0 
Compile a test code with gcc             : 
yes 
Setting architecture flag to             : ['-m64'] 
Checking for $CC option to support OpenMP : -fopenmp 
Checking for program 'ifort'              : not found 
ifort not found, defaulting to gfortran (cause: 'Could not find the program ['ifort']
(complete log in /home/koldrakan/Cosmological_Codes/planck/cosmology/PR2/likelihood/plc−2.0/build/config.log)') 
Checking for program 'gfortran, g77'      : /usr/bin/gfortran 
Check gfortran version                    : not found, let's hope for the best... 
Compile a test code with gfortran         : yes 
retrieve gfortran link line               : ok 
fortran link line                         : -L/home/koldrakan/Cosmological_Codes/planck/cosmology/PR2/likelihood/plc−2.0 -lgfortran -lgomp 
Setting install root to                   : /home/koldrakan/Cosmological_Codes/planck/cosmology/PR2/likelihood/plc−2.0 
Setting install bin directory to          : /home/koldrakan/Cosmological_Codes/planck/cosmology/PR2/likelihood/plc−2.0/bin 
Setting install lib directory to          : /home/koldrakan/Cosmological_Codes/planck/cosmology/PR2/likelihood/plc−2.0/lib 
Setting install include directory to      : /home/koldrakan/Cosmological_Codes/planck/cosmology/PR2/likelihood/plc−2.0/include 
Checking for program 'python'             : /home/koldrakan/anaconda2/bin/python 
Checking for python version               : 2.7.12 
python-config                             : /home/koldrakan/anaconda2/bin/python-config 
Asking python-config for pyembed '–cflags –libs –ldflags' flags : yes 
Getting pyembed flags from python-config                           : yes 
Checking for library dl                                            : yes 
checking for RTLD_DEFAULT in dl                                    : no 
Checking for program 'pmc-config'                                  : not found 
Checking for library pmc                                           : not found 
Check mkl version                                                  : 10.3 
create specific mkl lib                                            : Command 'gcc -shared -Bdynamic  -u dposv_ -u dtrsv_ -u dpotrf_ -u dpotrs_ -u dpotri_ -u dtrtri_ -u dtrmm_ -u dtrmv_ -u dgeqrf_ -u dormqr_ -u dsyev_ -u dgesvd_ -u dsymv_ -u dgemv_ -u dgemm_ -u dsyrk_ -u dsyr2k -u daxpy_ -u dtrsm_ -u dsymm_ -u dsyr_ -u ddot_ -u dsyevd_ -u dlamch_ -u dsyevr_  -Wl,–start-group /opt/intel//lib/intel64/libmkl_intel_lp64.a /opt/intel//lib/intel64/libmkl_intel_thread.a /opt/intel//lib/intel64/libmkl_core.a  -Wl,–end-group -L/home/koldrakan/Cosmological_Codes/planck/cosmology/PR2/likelihood/plc−2.0 -liomp5 -lpthread -lm -o "/home/koldrakan/Cosmological_Codes/planck/cosmology/PR2/likelihood/plc−2.0/lib/libclik_mkl.so" ' returned 1

Shouvik Roychoudhury
Posts: 21
Joined: August 14 2016
Affiliation: Harish-Chandra Research Institute

Installing Planck likelihood

Post by Shouvik Roychoudhury » October 06 2016

configure without the lapack statement and see if it works.try

./waf configure –install_all_deps

Karim Benabed
Posts: 8
Joined: December 18 2008
Affiliation: Institut d'Astrophysique de Paris

Installing Planck likelihood

Post by Karim Benabed » October 07 2016

You wil not be able to compile with mkl if you don't compile with either ifort or icc which provide the intel openmp library (iomp5).

You can try to force the use of the gnu openmp lib, by changing the file
waf_tools/any_lapack.py

replace the line
"linux_10.3_64" :("%s/lib/intel64"," -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm"),

by
"linux_10.3_64" :("%s/lib/intel64"," -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -lgomp -lpthread -lm"),

use at your own risks.

Albin Nilsson
Posts: 13
Joined: October 05 2016
Affiliation: NCBJ, Warsaw

Installing Planck likelihood

Post by Albin Nilsson » October 07 2016

I tried changing the line and it configured and installed without any problems. Thank you!
Will this affect performance and/or accuracy at all?

Karim Benabed
Posts: 8
Joined: December 18 2008
Affiliation: Institut d'Astrophysique de Paris

Installing Planck likelihood

Post by Karim Benabed » October 07 2016

There can be a small performance decrease because of the use of the gnu openmp library.
That being said, performance will be probably more degraded by the use of the gfortran compiler rather than the intel one.

As for accuracy, the use of the gnu openmp lib with mkl has not been tested for the Planck likelihood code. Intel provide dedicated libs for that, so it should be fine. Make sure to check with clik_print that the automatic test of the likelihoods gives correct results, especially the low-ell ones.

Jaykumar Patel
Posts: 4
Joined: September 25 2017
Affiliation: U of A

Installing Planck likelihood

Post by Jaykumar Patel » September 26 2017

I am getting this error when I am trying to install ./waf install


Code: Select allp>

Build failed
 -> task in 'clik' failed (exit status 1): 
	{task 140390064188624: cshlib clik.c.6.o,lklbs.c.6.o,lowly_common.c.6.o,clik_helper.c.6.o,clik_dic.c.6.o,clik_lensing.c.6.
o,cldf.c.6.o,cfrd.c.6.o,errorlist.c.6.o,io.c.6.o,distribution.c.6.o,smica.c.6.o,clik_hfipack.c.6.o,clik_parametric.c.6.o,clik_parametric_addon.c.6.o,clik_plik.c.6.o,fg2015.c.6.o,corrnoise.c.6.o,leakage.c.6.o,clik_lowlike.c.6.o,clik_gibbs.c.6.o,clik_bflike.c.6.o,clik_cmbonly.c.6.o,plenslike_dat_mono.c.6.o,plenslike_dat_quad.c.6.o,plenslike_dat_qecl.c.6.o,plenslike_dat_full.c.6.o,qest.c.6.o,wignerd.c.6.o,clik_CAMspec.c.6.o,Planck_options.F90.1.o,Planck_teeebb_pixlike.F90.1.o,Planck_likelihood.F90.1.o,clik_lowlike.f90.1.o,read_archive_map.f90.13.o,read_fits.f90.13.o,br_mod_dist.f90.13.o,clik_bflike.f90.3.o,bflike.f90.3.o,bflike_QUonly.f90.3.o,long_intrinsic_smw.f90.3.o,fitstools_smw.f90.3.o,bflike_smw.f90.3.o,healpix_types.f90.12.o,CAMspec.f90.5.o,clik_CAMspec.F90.5.o,clik_gibbs.f90.2.o,comm_br_mod.f90.2.o,comm_gauss_br_mod.f90.2.o,comm_lowl_mod_dist.f90.2.o,plik_cmbonly.f90.4.o,clik_cmbonly.f90.4.o -> libclik.so}
['/usr/bin/gcc', '-m64', '-shared', 'src/clik.c.6.o', 'src/lklbs.c.6.o', 'src/lowly_common.c.6.o', 'src/clik_helper.c.6.o', 'src/clik_dic.c.6.o', 'src/clik_lensing.c.6.o', 'src/cldf/cldf.c.6.o', 'src/cldf/cfrd.c.6.o', 'src/minipmc/errorlist.c.6.o', 'src/minipmc/io.c.6.o', 'src/minipmc/distribution.c.6.o', 'src/plik/smica.c.6.o', 'src/plik/clik_hfipack.c.6.o', 'src/plik/clik_parametric.c.6.o', 'src/plik/clik_parametric_addon.c.6.o', 'src/plik/clik_plik.c.6.o', 'src/plik/component_plugin/rel2015/fg2015.c.6.o', 'src/plik/component_plugin/rel2015/corrnoise.c.6.o', 'src/plik/component_plugin/rel2015/leakage.c.6.o', 'src/lowlike/clik_lowlike.c.6.o', 'src/gibbs/clik_gibbs.c.6.o', 'src/bflike/clik_bflike.c.6.o', 'src/cmbonly/clik_cmbonly.c.6.o', 'src/lenslike/plenslike/plenslike_dat_mono.c.6.o', 'src/lenslike/plenslike/plenslike_dat_quad.c.6.o', 'src/lenslike/plenslike/plenslike_dat_qecl.c.6.o', 'src/lenslike/plenslike/plenslike_dat_full.c.6.o', 'src/lenslike/plenslike/qest.c.6.o', 'src/lenslike/plenslike/wignerd.c.6.o', 'src/camspec/clik_CAMspec.c.6.o', 'src/lowlike/Planck_options.F90.1.o', 'src/lowlike/Planck_teeebb_pixlike.F90.1.o', 'src/lowlike/Planck_likelihood.F90.1.o', 'src/lowlike/clik_lowlike.f90.1.o', 'src/lowlike/read_archive_map.f90.13.o', 'src/lowlike/read_fits.f90.13.o', 'src/lowlike/br_mod_dist.f90.13.o', 'src/bflike/clik_bflike.f90.3.o', 'src/bflike/bflike.f90.3.o', 'src/bflike/bflike_QUonly.f90.3.o', 'src/bflike/long_intrinsic_smw.f90.3.o', 'src/bflike/fitstools_smw.f90.3.o', 'src/bflike/bflike_smw.f90.3.o', 'src/bflike/healpix_types.f90.12.o', 'src/camspec/CAMspec.f90.5.o', 'src/camspec/clik_CAMspec.F90.5.o', 'src/gibbs/clik_gibbs.f90.2.o', 'src/gibbs/comm_br_mod.f90.2.o', 'src/gibbs/comm_gauss_br_mod.f90.2.o', 'src/gibbs/comm_lowl_mod_dist.f90.2.o', 'src/cmbonly/plik_cmbonly.f90.4.o', 'src/cmbonly/clik_cmbonly.f90.4.o', '-o', '/home/jay/Desktop/Phys 499/CMB data/Code Likehood/likelihood_cf_code/plc−2.0/build/src/libclik.so', '-Wl,-rpath,/home/jay/Desktop/Phys 499/CMB data/Code Likehood/likelihood_cf_code/plc−2.0/lib', '-Wl,-rpath,/home/jay/Desktop/Phys 499/CMB data/Code Likehood/likelihood_cf_code/plc−2.0', '-Wl,-Bstatic', '-Wl,-Bdynamic', '-L/home/jay/Desktop/Phys 499/CMB data/Code Likehood/likelihood_cf_code/plc−2.0', '-llapack', '-lblas', '-ldl', '-lcfitsio', '-lgfortran', '-lgomp']

Post Reply