CosmoMC compiled with -DMPI: why this error

Use of Cobaya. camb, CLASS, cosmomc, compilers, etc.
Post Reply
Jie Ren
Posts: 28
Joined: May 13 2006
Affiliation: Theoretical Physics Division, Chern Institute of Mathematics, Nankai University, Tianjin, China

CosmoMC compiled with -DMPI: why this error

Post by Jie Ren » October 13 2006

I want to run CosmoMC on a cluster. After I edited Makefile as

F90C = ifort
FFLAGS = -O2 -fpp -DMPI
LAPACKL = -L/opt/intel/mkl70/lib/32 -lmkl_lapack -lmkl_ia32 -lguide -lpthread

and run make, an error occurred. Why this error? Think you very much for any helps.

[renphysics@nkstar2 source]$ make all
ifort -O2 -fpp -DMPI -I/nfs/s07r2p1/renphysics/cfitsio/include -I/nfs/s07r2p1/renphysics/Healpix_2.01/include -I/nfs/s07r2p1/renphysics/wmap_likelihood -I../camb -c utils.F90
......
ifort -O2 -fpp -DMPI -I/nfs/s07r2p1/renphysics/cfitsio/include -I/nfs/s07r2p1/renphysics/Healpix_2.01/include -I/nfs/s07r2p1/renphysics/wmap_likelihood -I../camb -c driver.F90
ifort -o ../cosmomc /nfs/s07r2p1/renphysics/wmap_likelihood/read_archive_map.o /nfs/s07r2p1/renphysics/wmap_likelihood/read_fits.o /nfs/s07r2p1/renphysics/wmap_likelihood/WMAP_3yr_options.o /nfs/s07r2p1/renphysics/wmap_likelihood/WMAP_3yr_util.o /nfs/s07r2p1/renphysics/wmap_likelihood/WMAP_3yr_tt_pixlike.o /nfs/s07r2p1/renphysics/wmap_likelihood/WMAP_3yr_tt_beam_and_ptsrc_corr.o /nfs/s07r2p1/renphysics/wmap_likelihood/WMAP_3yr_teeebb_pixlike.o /nfs/s07r2p1/renphysics/wmap_likelihood/WMAP_3yr_likelihood.o utils.o settings.o cmbtypes.o cmbdata.o WeakLen.o mpk.o supernovae.o SDSSLy-a-v3.o CMB_Cls_simple.o paramdef.o propose.o params_CMB.o calclike.o conjgrad_wrapper.o EstCovmat.o postprocess.o MCMC.o driver.o -L/nfs/s07r2p1/renphysics/cfitsio/lib -L/nfs/s07r2p1/renphysics/Healpix_2.01/lib -L/nfs/s07r2p1/renphysics/wmap_likelihood -lhealpix -lcfitsio -L../camb -lcamb -L/opt/intel/mkl70/lib/32 -lmkl_lapack -lmkl_ia32 -lguide -lpthread -O2 -fpp -DMPI -I/nfs/s07r2p1/renphysics/cfitsio/include -I/nfs/s07r2p1/renphysics/Healpix_2.01/include -I/nfs/s07r2p1/renphysics/wmap_likelihood -I../camb
paramdef.o(.text+0x74): In function `paramdef_mp_dostop_':
: undefined reference to `mpi_wtime_'
paramdef.o(.text+0x1e0): In function `paramdef_mp_dostop_':
: undefined reference to `mpi_finalize_'
paramdef.o(.text+0x289): In function `paramdef_mp_doabort_':
: undefined reference to `mpi_abort_'
paramdef.o(.text+0x3500): In function `paramdef_mp_addmpiparams_':
: undefined reference to `mpi_wtime_'
paramdef.o(.text+0x3721): In function `paramdef_mp_addmpiparams_':
: undefined reference to `mpi_isend_'
paramdef.o(.text+0x3778): In function `paramdef_mp_addmpiparams_':
: undefined reference to `mpi_irecv_'
paramdef.o(.text+0x3a13): In function `paramdef_mp_addmpiparams_':
: undefined reference to `mpi_testall_'
paramdef.o(.text+0x3b3a): In function `paramdef_mp_addmpiparams_':
: undefined reference to `mpi_testall_'
paramdef.o(.text+0x3be5): In function `paramdef_mp_addmpiparams_':
: undefined reference to `mpi_issend_'
paramdef.o(.text+0x3c26): In function `paramdef_mp_addmpiparams_':
: undefined reference to `mpi_iprobe_'
paramdef.o(.text+0x4a05): In function `paramdef_mp_addmpiparams_':
: undefined reference to `mpi_bcast_'
paramdef.o(.text+0x4c50): In function `paramdef_mp_addmpiparams_':
: undefined reference to `mpi_bcast_'
paramdef.o(.text+0x679d): In function `paramdef_mp_addmpiparams_':
: undefined reference to `mpi_recv_'
paramdef.o(.text+0x7143): In function `paramdef_mp_checklimitsconverge_':
: undefined reference to `mpi_bcast_'
driver.o(.text+0x32): In function `MAIN__':
: undefined reference to `mpi_init_'
driver.o(.text+0xc2): In function `MAIN__':
: undefined reference to `mpi_comm_rank_'
driver.o(.text+0x13e): In function `MAIN__':
: undefined reference to `mpi_comm_size_'
driver.o(.text+0x176): In function `MAIN__':
: undefined reference to `mpi_bcast_'
driver.o(.text+0x20e): In function `MAIN__':
: undefined reference to `mpi_wtime_'
make: *** [cosmomc] Error 1

Jie Ren
Posts: 28
Joined: May 13 2006
Affiliation: Theoretical Physics Division, Chern Institute of Mathematics, Nankai University, Tianjin, China

CosmoMC compiled with -DMPI: why this error

Post by Jie Ren » October 14 2006

The following link is on the similar problem, but I still do not know how to solve it.

http://www.pgroup.com/userforum/viewtop ... 7573d2476b

By the way, if "F90C = ifort" is changed to "F90C = mpif90",

mpif90 -O2 -fpp -DMPI -I/nfs/s07r2p1/renphysics/cfitsio/include -I/nfs/s07r2p1/renphysics/Healpix_2.01/include -I/nfs/s07r2p1/renphysics/wmap_likelihood -I../camb -c utils.F90
ifort: Command line warning: no action performed for specified file(s)
...

Although there is no error, the program cannot run when I submit it as a parallel job.

The output (if any) follows:

0 - MPI_COMM_RANK : Null communicator
1 - MPI_COMM_RANK : Null communicator
[0] Aborting program !
[1] Aborting program !
[0] Aborting program!
[1] Aborting program!
2 - MPI_COMM_RANK : Null communicator
3 - MPI_COMM_RANK : Null communicator
[3] Aborting program !
[3] Aborting program!
[2] Aborting program !
[2] Aborting program!

Thanks very much for any suggestions!

Susana Landau
Posts: 21
Joined: December 10 2004
Affiliation: Buenos Aires University

CosmoMC compiled with -DMPI: why this error

Post by Susana Landau » October 19 2006

the problem is with the underscores, ifort needs to read with two underscores, I don't know the right options for compilation yet, but you can fix almost all the problems you have replacing for example

call mpi_finalize(ierror)

with

call mpi_finalize_(ierror)

in each call mpi_* (actually you have to modify only driver.f90 and paramsdef.f90

however, making this replacement I still get the following error

paramdef.o(.text+0x152f): In function `paramdef_mp_addmpiparams_.':
: undefined reference to `mpi_wtime_'
paramdef.o(.text+0x8002): In function `paramdef_mp_dostop_':
: undefined reference to `mpi_wtime_'
driver.o(.text+0x228): In function `MAIN__':
: undefined reference to `mpi_wtime_'
make: *** [cosmomc] Error 1

and I cannot find any call mpi_wtime

any ideas????

Jie Ren
Posts: 28
Joined: May 13 2006
Affiliation: Theoretical Physics Division, Chern Institute of Mathematics, Nankai University, Tianjin, China

CosmoMC compiled with -DMPI: why this error

Post by Jie Ren » October 21 2006

If I modify "mpi_abort" to "mpi_abort_", the error becomes "mpi_abort__".

paramdef.o(.text+0x289): In function `paramdef_mp_doabort_':
: undefined reference to `mpi_abort__'

Is there any alternative methods to compile CosmoMC as a parallel program?

Jie Ren
Posts: 28
Joined: May 13 2006
Affiliation: Theoretical Physics Division, Chern Institute of Mathematics, Nankai University, Tianjin, China

CosmoMC compiled with -DMPI: why this error

Post by Jie Ren » November 01 2006

This problem has been solved right now.

-lfmpich -lmpich -lgm (libfmpich.a, libmpich.a, libgm.a)
and the relevant paths should be added

Post Reply