mpirun exceeds requested CPU usage

[Christian Neissner]

I'm trying to run cobaya with

Code: Select all

mpirun -n 8 cobaya-run test.yaml

which I run from inside a script that is sent to an HTCondor node. The submit file looks like this:

Code: Select all

executable = cobaya.sh
request_memory = 32000
request_cpus = 8
queue

The job runs for a while, but constantly execeeds the CPU usage. After some grace period the job is stopped because of it.

The run uses CAMB, is there something I should put as a parameter during the compilation/installation with pip?

[Christian Neissner]

On the mpirun level I tried the mapping parameter:

Code: Select all

mpirun --map-by slot:PE=1 -n 8 cobaya-run test.yaml

but CAMB still exceeds the limits.

[Antony Lewis]

Make sure you set OMP_NUM_THREADS for the number of threads per MPI process

[Christian Neissner]

Hi!

Thanks for the suggestion but it didn't work:

Code: Select all

%CPU  %MEM     TIME+ COMMAND                                                                                                                                  
 355.3   0.0   7:54.72 cobaya-run                                                                                                                               
 315.1   0.0   8:19.71 cobaya-run                                                                                                                               
 274.6   0.0   7:17.33 cobaya-run                                                                                                                               
 266.9   0.0   6:18.57 cobaya-run                                                                                                                               
 254.3   0.0   6:35.39 cobaya-run                                                                                                                               
 253.4   0.0   5:56.99 cobaya-run                                                                                                                               
 219.0   0.0   5:01.43 cobaya-run                                                                                                                               
 211.6   0.0   5:57.86 cobaya-run

Even in a single run:

Code: Select all

cobaya-run test.yaml

Exceeds the 100% CPU usage.

[Antony Lewis]

Looks like OMP_NUM_THREADS=4. May also be other threading parameters, ask the cluster admin for advice in your specific set up for OpenMP+MPI runs.