WMAP 3yr Likelihood code and Intel Fortran compilers

Use of Healpix, camb, CLASS, cosmomc, compilers, etc.
Post Reply
Tristan L Smith
Posts: 14
Joined: November 14 2005
Affiliation: Swarthmore
Contact:

WMAP 3yr Likelihood code and Intel Fortran compilers

Post by Tristan L Smith » March 20 2006

Hello everybody,

While compiling the new WMAP 3rd year likelihood code I was obtaining a large difference between the 'standard' ouput supplied by the WMAP team and the output of my compiled code. The difference came from a large value for the 'low l TTTT chi2'. I have found that this difference is caused by compiling with the ifort v8.0 compiler-- when I compiled with ifort v8.1 (both Healpix and the WMAP likelihood code) the results were in agreement. I am not sure if this issue would be transparent in the new version of cosmomc.

Cheers,

Tristan

Anze Slosar
Posts: 183
Joined: September 24 2004
Affiliation: Brookhaven National Laboratory
Contact:

Re: WMAP 3yr Likelihood code and Intel Fortran compilers

Post by Anze Slosar » March 20 2006

Tristan L Smith wrote: While compiling the new WMAP 3rd year likelihood code I was obtaining a large difference between the 'standard' ouput supplied by the WMAP team and the output of my compiled code. The difference came from a large value for the 'low l TTTT chi2'. I have found that this difference is caused by compiling with the ifort v8.0 compiler-- when I compiled with ifort v8.1 (both Healpix and the WMAP likelihood code) the results were in agreement. I am not sure if this issue would be transparent in the new version of cosmomc.
I also noticed that ifort9 + MKL segfault on some configurations (opterons?). Going to system supplied lapack+blas seems to work, but it is very sub-optimal, of course...

Ben Gold
Posts: 81
Joined: September 25 2004
Affiliation: University of Minnesota
Contact:

WMAP 3yr Likelihood code and Intel Fortran compilers

Post by Ben Gold » March 21 2006

What systems and compilers exactly have people been successful with? I ask because I haven't so far been able to get the likelihood code working on any system. I have a Linux box (i686, kernel 2.4.21-27.0.4.ELsmp) with Intel Fortran versions 7 (Version 7.1 Build 20030307Z) and 8 (Version 8.0 Build 20031016Z), both of which die quite early in the compilation stage (I'm downloading version 9 to try it as I post).

I've actually gotten further with my PowerBook G4, where I can get through compilation with g95 but have problems at the linking stage. Some of them appear to have to do with my Healpix installation (which I'm also having trouble with), but some appear general. This is G95 (GCC 4.0.1 (g95!) Sep 25 2005) running on Darwin Kernel Version 8.5.0 (powerpc).

I'm getting really frustrated. Can anyone describe what they did to get the code working on a similar platform?

***update: version 9 doesn't work either. It gives the same error I get from version 8

Code: Select all

fortcom: Severe: WMAP_3yr_tt_pixlike.f90, line 61: Please report this error along with the circumstances in which it occurred in a Software Problem Report
ALLOCATE(cl_fiducial(2:512))
^
[ Aborting due to internal error. ]
compilation aborted for WMAP_3yr_tt_pixlike.f90 (code 1)
I don't understand why that statement (or really any of the previous statements) should be a problem...

Ben Gold
Posts: 81
Joined: September 25 2004
Affiliation: University of Minnesota
Contact:

WMAP 3yr Likelihood code and Intel Fortran compilers

Post by Ben Gold » March 21 2006

Okay, I did finally get it working... on my PowerBook. It turns out the link errors weren't about Healpix routines, they were about LAPACK routines, which Apple actually ships with the rest of their developer tools. So here are the relevant lines in my Makefile:

Code: Select all

F90    = g95
INCS   = -I. -I/usr/local/include -I/Users/bgold/packages/Healpix_2.00/include
LIBS   = -L. -L/usr/local/lib -L/Users/bgold/packages/Healpix_2.00/lib -lhealpix -lcfitsio -lSystemStubs -framework Accelerate
You can see that I'm linking to my own installation of Healpix, and my version of cfitsio is in /usr/local (I believe I installed it via Fink, but I don't recall).

Then I can get the likelihood code to compile with g95 (GCC 4.0.1 (g95!) Sep 25 2005) obtained from g95.org and run on a PowerBook G4 (OS X version 10.4.5). I ran the test routine and the results (other than bunch of warnings and debugging info) are the same as the "correct" output out to about seven digits.

I'd still like to get it working on the local Linux boxes with the Intel compiler, though, as this PowerBook is not the fastest thing around. The test program takes about 4.5 minutes on my machine.

Anze Slosar
Posts: 183
Joined: September 24 2004
Affiliation: Brookhaven National Laboratory
Contact:

Re: WMAP 3yr Likelihood code and Intel Fortran compilers

Post by Anze Slosar » March 21 2006

Ben Gold wrote: I'd still like to get it working on the local Linux boxes with the Intel compiler, though, as this PowerBook is not the fastest thing around. The test program takes about 4.5 minutes on my machine.
Well, that is quite surprising... My version takes 30sec on a 2.6GHz pentium 4 (not really the fastest machine around). The version of my ifort is
Intel(R) Fortran Compiler for 32-bit applications, Version 9.0 Build 20050430, and I compiled both WMAP3 and Healpix with just basic "ifort -O3 -openmp" and I use MKL on my home linux machine, but this has nothing to do with compilation... AL says that they have updated the code, though....

anze

Ben Gold
Posts: 81
Joined: September 25 2004
Affiliation: University of Minnesota
Contact:

WMAP 3yr Likelihood code and Intel Fortran compilers

Post by Ben Gold » March 21 2006

I'm not terribly surprised; PowerBooks have a pretty slow system bus and the likelihood code is probably pretty harsh on it. In any case, I've tried both Version 8.0 Build 20031016Z and Version 9.0 Build 20051201Z of ifort and both die with an internal compiler error on the "ALLOCATE(cl_fiducial(2:512))" statement as I described above. All on 3.2 GHz Pentium D cpus.

As far as I can tell, the problem can't be with anything other than ifort, as it's not even getting to the linking stage. Except other people seem to be compiling on nearly identical platforms without a problem. So I'm completely baffled.

Adam Tregre
Posts: 3
Joined: March 23 2006
Affiliation: UW-Madison

Re: WMAP 3yr Likelihood code and Intel Fortran compilers

Post by Adam Tregre » March 23 2006

Ben Gold wrote:
I'd still like to get it working on the local Linux boxes with the Intel compiler, though, as this PowerBook is not the fastest thing around. The test program takes about 4.5 minutes on my machine.
I don't think this has to do with your PowerBook------I am having similar performance (accuracy to 7 digits, runtime of ~5 min.) on a Linux box. Have you had any luck with this problem? I'm still stuck.

Gil Holder
Posts: 27
Joined: September 25 2004
Affiliation: McGill University

WMAP 3yr Likelihood code and Intel Fortran compilers

Post by Gil Holder » March 24 2006

The running time seems to be hugely dependent on your setup. I compiled with g95 and a version of LAPACK that I had compiled and it took about 3 minutes to run the code on a reasonably modern linux box. I figured that was a crazy number, so I recompiled with gfortran and linked with the acml library (the AMD equivalent of the Intel mkl) and it takes 20 seconds. I have not verified that the difference is in the acml (I don't feel like compiling yet another version of LAPACK), but I think it is likely. So if your code is slow, you may want to at least try it with an optimized version of LAPACK if you can find it.

Christophe Ringeval
Posts: 17
Joined: November 07 2004
Affiliation: Theoretical and Mathematical Physics Group, CP3, Louvain University
Contact:

Re: WMAP 3yr Likelihood code and Intel Fortran compilers

Post by Christophe Ringeval » March 24 2006

I also noticed that ifort9 + MKL segfault on some configurations (opterons?). Going to system supplied lapack+blas seems to work, but it is very sub-optimal, of course...
Same story here on ia32. Apparently it comes from -openmp, or rather the fact than in that case all local variables are on the stack (and not initialized to zero). Adding "-save" to force them to static allocation is a workaround, but it is not safe in parallel (threaded).

Chris.

PS: ifort bug of Mr uninitialized?

Adam Tregre
Posts: 3
Joined: March 23 2006
Affiliation: UW-Madison

Re: WMAP 3yr Likelihood code and Intel Fortran compilers

Post by Adam Tregre » March 27 2006

Gil Holder wrote:The running time seems to be hugely dependent on your setup. I compiled with g95 and a version of LAPACK that I had compiled and it took about 3 minutes to run the code on a reasonably modern linux box. I figured that was a crazy number, so I recompiled with gfortran and linked with the acml library (the AMD equivalent of the Intel mkl) and it takes 20 seconds. I have not verified that the difference is in the acml (I don't feel like compiling yet another version of LAPACK), but I think it is likely. So if your code is slow, you may want to at least try it with an optimized version of LAPACK if you can find it.
I tried the above setup (gfortran & acml), and my compilation can't see three subroutines: _gfortran_filename, _gfortran_ioparm, and _gfortran_line. It seems like these three routines just aren't in the gfortran libraries, and I cant find much in the way of documentation. Is there some other place I should be looking?
Thanks,
Adam

Post Reply