CosmoMC: Strange Memory Leak

Use of Healpix, camb, CLASS, cosmomc, compilers, etc.
Post Reply
Julius Wons
Posts: 2
Joined: June 17 2020
Affiliation: UNSW Sydney

CosmoMC: Strange Memory Leak

Post by Julius Wons » June 24 2020

I want to modify CosmoMC, not using the MCMC module, to analyse models with non-standard primordial power spectrum(oscillations for example). In the process, I have to evaluate the likelihood several times leading to a memory leak, which becomes troublesome if I increase lSampleBoost to sample all [math] without interpolation.

The problem with memory arises when I calculate the likelihood. Calling the likelihood for the first time allocates 1.3 GB, which won't be released later on. The next time the likelihood is called around 0.18 GB memory leaks. The numbers depend on the choice of accuracy parameters, especially with lSampleBoost and IntkAccuracyBoost. For standard settings, it is around 10 MB. When running the MCMC normally with action = 0, this memory leak does not occur. The initial 1.3 GB are allocated and then the used memory stays constant. I don't understand why this leak occurs in my case, but not when running the MCMC.

Am I doing something wrong in the way I call the likelihood function?


I wrote a little test subroutine, which shows the described memory leak. It is a simple loop changing one parameter (A_s in this case) slightly and then calculating the likelihood. After calling the likelihood, the used memory is printed calling another function called Memory. This is the subroutine calling the likelihood:

Code: Select all

subroutine TChainSampler_Tests(this, Params)
class(TChainSampler) :: this
class(ParamSet) :: Params
real(mcp) :: Likelihood
integer :: i, MemGB
integer :: which_param = 17 ! This refers to A_s 

print*, 'Allocation'
call this%Memory(MemGB)
print*, 'Start Sampling'
do i=1, 100
	print*, i
	Params%P(which_param) = Params%P(which_param)+ BaseParams%PWidth(which_param)
	Likelihood = this%LogLike(Params)
	call this%Memory(MemGB)
end do

end subroutine TChainSampler_Tests
The procedure Memory calls a system file displaying the used memory usage and prints this value. See code below:

Code: Select all

	subroutine TSamplingAlgorithm_Memory(this, valueRSS)
!	use ifport !if on intel compiler
	class(TSamplingAlgorithm) :: this


	integer, intent(out) :: valueRSS

	character(len=200):: filename=' '
	character(len=80) :: line
	character(len=8)  :: pid_char=' '
	integer :: pid
	logical :: ifxst

	valueRSS=-1    ! return negative number if not found

	!--- get process ID

	pid=getpid()
	write(pid_char,'(I8)') pid
	filename='/proc/'//trim(adjustl(pid_char))//'/status'

	!--- read system file

	inquire (file=filename,exist=ifxst)
	if (.not.ifxst) then
	  write (*,*) 'system file does not exist'
	  return
	endif

	open(unit=100, file=filename, action='read')
	do
	  read (100,'(a)',end=120) line
	  if (line(1:6).eq.'VmRSS:') then
		 read (line(7:),*) valueRSS
		 exit
	  endif
	enddo
	120 continue
	close(100)
	write(*,*) valueRSS , 'Used memory'
	return
	end subroutine TSamplingAlgorithm_Memory
When setting lSampleBoost = 50 in Calculator_CAMB.f90, the described leak occurs.

Also: I have tested this with gfortran 7.4, as well as different ifort versions and the problem occurs with all compilers/versions. I have also tried different likelihoods with Planck, unbinned Planck and without Planck. I tried this with the recent May 2020 version (CAMB 1.1.2) and the previous July 2019 version(CAMB 1.0.7) of CosmoMC.

Antony Lewis
Posts: 1652
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: Strange Memory Leak in CAMB

Post by Antony Lewis » June 24 2020

You need to free the data that is cached in TCalculationAtParamPoint%Info, in CosmoMC this is usually done by calling AcceptReject.

Julius Wons
Posts: 2
Joined: June 17 2020
Affiliation: UNSW Sydney

Re: CosmoMC: Strange Memory Leak

Post by Julius Wons » June 25 2020

Thank you so much for the quick reply. This solved my problem.

Post Reply