CosmoMC -- gfortran / MacOS / MPI

Use of Healpix, camb, CLASS, cosmomc, compilers, etc.
Post Reply
Richard Easther
Posts: 16
Joined: April 14 2005
Affiliation: Yale

CosmoMC -- gfortran / MacOS / MPI

Post by Richard Easther » March 18 2009

Hi,

I am compiling CosmoMC on a new x86 Mac pro (running 10.5.6) with gfortran.

It works fine as single threaded app, but won't compile with the DMPI flag. Would appreciate any suggestions... I am missing the mpi.h file, and think I am giving the appropriate include path, but I am clearly not!

Richard

Jochen Weller
Posts: 45
Joined: September 24 2004
Affiliation: Ludwig-Maximilians-University Munich
Contact:

CosmoMC -- gfortran / MacOS / MPI

Post by Jochen Weller » March 20 2009

Hi Richard

I am a step behind you right now. I am trying to install MPI on my MAC. Which MPI are you using ?

Cheers
Jochen

Richard Easther
Posts: 16
Joined: April 14 2005
Affiliation: Yale

CosmoMC -- gfortran / MacOS / MPI

Post by Richard Easther » March 20 2009

Ah. If you are using 10.5 the MPI stuff is there already -- the question is simply how to invoke it from within the makefile

Anze Slosar
Posts: 183
Joined: September 24 2004
Affiliation: Brookhaven National Laboratory
Contact:

CosmoMC -- gfortran / MacOS / MPI

Post by Anze Slosar » March 21 2009

I don't know much about macs, but I very much doubt that gfortran has MPI builit in... It probably has openmp, which is what parallelizes camb, but for cosmomc parallelization you need an implementation of MPI, such as MPICH or Open-MPI, which then run on top of the compiler and you compile with mpif90 (which then takes care of all the right linking issues, etc and essentially wraps around the underlying compiler) rather than gfortran...

Richard Easther
Posts: 16
Joined: April 14 2005
Affiliation: Yale

Re: CosmoMC -- gfortran / MacOS / MPI

Post by Richard Easther » March 21 2009

Anze Slosar wrote:I don't know much about macs, but I very much doubt that gfortran has MPI builit in... It probably has openmp, which is what parallelizes camb, but for cosmomc parallelization you need an implementation of MPI, such as MPICH or Open-MPI, which then run on top of the compiler and you compile with mpif90 (which then takes care of all the right linking issues, etc and essentially wraps around the underlying compiler) rather than gfortran...
I believe MacOS 10.5 ships with OpenMPI preinstalled -- it does not need to be added.

But you are right that there is no guarantee that this will work with a standard gfortran.

Jonathan Pritchard
Posts: 2
Joined: January 06 2005
Affiliation: Imperial College London
Contact:

CosmoMC -- gfortran / MacOS / MPI

Post by Jonathan Pritchard » April 09 2009

Hi Richard,

I just got this working on a Mac Pro running 10.5 using the ifort compiler. It shouldn't be too different with gfortran. I got a lot of help from the following two webpages

http://software.intel.com/en-us/article ... compilers/

http://www.macresearch.org/getting_star ... _and_xgrid

By default, the Apple installation of OpenMPI doesn't support a fortran compiler so you need to download and compile a new version of OpenMPI with fortran support.

When doing this you need to ensure that your C and FORTRAN compilers compile for the same architecture i.e. 32 vs 64 bit. On my Mac Pro gcc defaults to 32bit, while ifort defaults to 64bit. The following worked for me

Code: Select all

./configure --prefix=/opt/openmpi --with-fortran --enable-shared --disable-static CFLAGS="-arch x86_64" CXXFLAGS="-arch x86_64"
sudo make all install
Then to make sure that by default I used my new installation of OpenMPI rather than the preinstalled one I edited ~/.bashrc with the lines

Code: Select all

export PATH=/opt/openmpi/bin:$PATH
export LD_LIBRARY_PATH=/opt/openmpi/lib:${LD_LIBRARY_PATH}
export DYLD_LIBRARY_PATH=/opt/openmpi/lib:${DYLD_LIBRARY_PATH}
alias mpirun=/opt/openmpi/bin/mpirun
Compile cosmomc with a Makefile that looks something like (although the linking line is probably redundant after editing .bashrc)

Code: Select all

MPILIB=/opt/openmpi/lib
F90C     = mpif90
FFLAGS = -O2 -fpp -openmp -DMPI
LAPACKL = -L$(MPILIB) -L$(MKLLIB) -lmkl_lapack -lmkl_intel_lp64 -lmkl_core -lguide -lpthread
Then

Code: Select all

mpirun -np8 ./cosmomc param.ini 
Seems to run fine (although running 8 chains is probably too many) and takes about 2 1/2 hours on my Mac Pro.

Hope that helps!
Jonathan

P.S. For whatever reason the WMAP5 likelihood code won't run when I set the -DOPTIMIZE flag. It errors with an "illegal instruction" message. If anyone knows how to get around this, I'd love to hear from them.

Post Reply