CosmoMC: gfortran memory problem

Use of Cobaya. camb, CLASS, cosmomc, compilers, etc.
Post Reply
Graeme Addison
Posts: 34
Joined: July 17 2014
Affiliation: Johns Hopkins University

CosmoMC: gfortran memory problem

Post by Graeme Addison » April 16 2020

I'm having trouble with some kind of memory leak running CosmoMC compiled with gfortran. I tried downloading the updated CAMB/CosmoMC files from GitHub, which I saw mentioned a gfortran memory issue fix, but I still have the same problem.

The sympton is that the memory usage by the mpi processes continuously increases when I run an action=0 chain, until eventually the system kills the job. I am running with gfortran 7.5.

I ran the python test script in the camb/camb/tests folder and it didn't come back with any message or confirmation of the memory leak issue. So maybe my issue is unrelated.

I don't know what actually caused this to start occurring, my jobs with the exact same options were running fine earlier (few weeks ago), and as far as I can tell there haven't been any updates to the computer's compilers etc. on the timescale when I started seeing this problem. Thanks for any help.

Antony Lewis
Posts: 1941
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: CosmoMC: gfortran memory problem

Post by Antony Lewis » April 17 2020

I would check your forutils is actually updated and compiled to the latest version OK. Is this on a Mac? The recent github fixes avoid all gfortran-memory bugs on linux for me, but someone reported they still seeing a (much smaller) leak on Mac.

Your can run the camb_tests/camb.test.py.test_memory test to check for immediate issues in camb.

Graeme Addison
Posts: 34
Joined: July 17 2014
Affiliation: Johns Hopkins University

Re: CosmoMC: gfortran memory problem

Post by Graeme Addison » April 17 2020

My forutils matches the latest version on GitHub (including e.g. the Interpolation.f90 changes from ~3 weeks ago). I ran the camb test I mentioned in the first post and it didn't report anything, is there another test I could do? This is on linux Ubuntu 18.04.3.

Antony Lewis
Posts: 1941
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: CosmoMC: gfortran memory problem

Post by Antony Lewis » April 18 2020

It may be a new issue then. I'd try running with the some memory analysis tool to track down where it is being leaked (valgrind, etc - I'm no export on linux).

Could also try compiling with new gfortran in case it's an issue that has been fixed (e.g. ubuntu-toolchain-test gfortran 9).

Antony Lewis
Posts: 1941
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: CosmoMC: gfortran memory problem

Post by Antony Lewis » May 13 2020

I just pushed another cosmomc workaround for gfortran memory leak https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94361, try that?
However even for just CAMB someone reported this for different gfortran versions:

version memory leak
7.3.0 no
8.2.0 no
7.5.0 yes
8.4.0 yes
9.2.0 yes
9.3.0 yes

Antony Lewis
Posts: 1941
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: CosmoMC: gfortran memory problem

Post by Antony Lewis » June 02 2020

The gcc leak bug is now fixed in gcc heads, but will of course take a while to propagate to next release iterations.

Antony Lewis
Posts: 1941
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: CosmoMC: gfortran memory problem

Post by Antony Lewis » June 05 2020

Unfortunately, this was not the only gfortran bug - also leak (which also affects latest CAMB) in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94109

Post Reply