CosmoMC : MPI + OpenMp doesn't work !!

[Pierre Delsart]

Hello,
Since 2 days, I'm trying to run cosmomc using MPI + OpenMP.
My problem is simple :

I installed cosmomc on a cluster and I want to run 8 chains using 8 nodes. Moreover, I want that each chains run on 8 cpu. Note that each node contains 8 cpu. The batch script I submitted is :


#!/bin/bash
#PBS -j eo
#PBS -l select=8:ncpus=8:mpiprocs=1
#PBS -N test

cd \$PBS_O_WORKDIR/cosmomc_p

export LD_LIBRARY_PATH= \$LD_LIBRARY_PATH:$HOME/local/lib

export OMP_NUM_THREADS=8

mpiexec_mpt -np 8 ./cosmomc params_noe.ini

(I had to use the \ here else there is an error in the forum) where select is the number of node I require, ncpus is the number of cpu I use per node et mpiprocs is the number of MPI process per node, in my case it the number of chains by node. Then export OMP_NUM_THREADS because I don't know the value in the params_noe.ini is taken into account. Finally cosmomc is started by mpiexec_mpt where 8 is the number of chains I want.

Once the script run, cosmomc begin to compute but each chain doesn't run on 8 cpu. Each chain run on 1 cpu per node. At the end of my computation I received a report where it's note the resources_used.cpupercent = 800. This means I used 8 cpu. But I want to use 64 cpus (8*8). I have no idea to solve my problem.

Could you help me please??

Thanks in advance.

[Jonathan Sievers]

If you just run CAMB on one node, does it use all 8 cores?

[Pascal Vaudrevange]

Hi Pierre,

maybe add the option

Code: Select all

-npernode 1

to mpiexec to instruct it to explicitly launch just a single mpi chain on each of the 8 nodes.

Pascal