Hi!
I am using Cobaya as an MCMC sampler for a user-defined likelihood and I am observing a memory-leak.
The likelihood is a native python code and doesn't exhibit any signs of memory leak when run without Cobaya but as soon as I use it within Cobaya I face a memory leak.
The likelihood python code has two functions, one computes the model and the other computes the log likelihood which is then fed into the Cobaya input file as mentioned in the documentation. I have also tried using del keyword and gc.collect() in the likelihood file with no improvements in the memory leak.
Cobaya version: 3.4.1
OpenMPI version: 4.1.5
Chains run: 8
System memory: 32GB
Thanks,
Umang.
Cobaya: Memory Leak when Using External Likelihood
-
- Posts: 6
- Joined: March 15 2023
- Affiliation: Ashoka University
-
- Posts: 1983
- Joined: September 23 2004
- Affiliation: University of Sussex
- Contact:
Re: Cobaya: Memory Leak when Using External Likelihood
You can try tools like tracemalloc to diagnose. Cobaya memory will gradually increase as it stores more points, but not to an extent that it is likely to be a problem.
-
- Posts: 6
- Joined: March 15 2023
- Affiliation: Ashoka University
Re: Cobaya: Memory Leak when Using External Likelihood
I had never faced such issues with Cobaya when using Planck likelihood which prompted me to think that there was a problem in the likelihood function that I had written. The major difference being that, I was using numba jitted code and as soon as I removed the jit commands the memory leak problem was resolved.
So, I think as of now there is a compatibility issue between Cobaya and numba jitted external likelihood.
So, I think as of now there is a compatibility issue between Cobaya and numba jitted external likelihood.