CosmoMC: Running with LSF queues

Use of Healpix, camb, CLASS, cosmomc, compilers, etc.
Post Reply
Sarah Bridle
Posts: 144
Joined: September 24 2004
Affiliation: University College London (UCL)
Contact:

CosmoMC: Running with LSF queues

Post by Sarah Bridle » October 19 2004

Hi,
Thought I'd send a brief note on what I just learnt about submitting cosmomc jobs to the LSF Cambridge COSMOS supercomputer queues in case useful to anyone. NB. can crash the supercomputer if you get this wrong (oops, sorry about that),
Sarah

eg. if you want to run with MxN processors (ie. M chains each running on N processors) under the aux queue:

Set number of processors to {N} in params.ini

Make a jobscript containing eg.:
cd {workingdir}
mpirun -np {M} cosmomc params.ini > output.txt

where "{M}" is the number of chains eg. "4".

Write a submission script eg. using the aux script writer:
aux jobscript
answer yes to all the default options, including number of processors=4.
But don't submit the job, choose "keep" for later.

Then edit the script eg. pico aux.99999 and change the line
# BSUB -n 4 # No. processors. Remember to set OMP_NUM_THREADS.
to:
# BSUB -n {MxN} # No. processors. Remember to set OMP_NUM_THREADS.
eg.
# BSUB -n 16 # No. processors. Remember to set OMP_NUM_THREADS.

But the important thing is to leave the line
OMP_NUM_THREADS={N}
eg.
OMP_NUM_THREADS=4
Otherwise you can crash cosmos.

Then submit the job using
bsub < aux.99999

Check that the aux queue is appropriate by looking in /var/log/cosmos_jobs and /var/log/cosmos_jobs.prev and checking the last column is less than the second to last column (memory used cf memory requested).

Thanks very much to Stuart Rankin for explaining the above.

Post Reply