CosmoMC: big spread in log-likes for Planck 2018 found with BOBYQA

Use of Cobaya. camb, CLASS, cosmomc, compilers, etc.
Post Reply
Aviv Padawer-Blatt
Posts: 9
Joined: October 28 2021
Affiliation: University of Waterloo

CosmoMC: big spread in log-likes for Planck 2018 found with BOBYQA

Post by Aviv Padawer-Blatt » October 28 2021

Hi,

I know that the best fit results (ie. parameters and log likelihoods) from an MCMC run in CosmoMC are not necessarily very stable or trustworthy, and that it is ideal to obtain these from Powell's BOBYQA bounded minimization routine with action=2 (instead of action=0). However, I am finding that when I run action=2 on the Planck 2018 CMB data (specifics below) in the base Lambda-CDM cosmology so that the algorithm runs minimizations from multiple random starting points (here 8) to check that they converge, the output gives a warning:

Code: Select all

 synched bestfits:   1386.34328873440        1386.07481135887
   1384.92572090509        1388.61913598510        1390.61331600264
   1391.12270691582        1383.58172831837        1394.93734999466
 WARNING: big spread in log-likes

In particular, I am using the following .ini files from batch3/ with no modifications:

Code: Select all

#Planck 2018, default just include native likelihoods (others require clik)
DEFAULT(batch3/plik_rd12_HM_v22_TTTEEE.ini)
DEFAULT(batch3/lowl.ini)
DEFAULT(batch3/simall_EE.ini)
#DEFAULT(batch3/lensing.ini)

I am finding that when running action=2 multiple separate times on the same data with the same settings, the "best" of these best fit log-likelihoods (ie. the points with the best fit of all points) are very similar (within 0.5 of each other), but still have a difference around 3 with the MCMC best fit log-likelihood. I am not sure if the consistency over multiple runs of action=2 implies that the results are trustworthy, even though there is a "big spread in log-likes".

I also have to note that originally I found running the BOBYQA routine in CosmoMC was resulting in a segmentation fault, and that the fix I found (from another post here) led me to set the parameter minimize_mcmc_refine_num from 20 to 0:

Code: Select all

#if non-zero do some low temperature MCMC steps to check minimum stable
minimize_mcmc_refine_num = 0
I am not sure if this is what affecting the stability of the resultant best fit log-likelihoods.


Lastly, I am wondering if the Planck Collaboration ever released results about the best-fit values they find from running the BOBYQA routine in a base Lambda-CDM cosmology (I cannot seem to find something relating to this online). Knowing their results would help me to verify mine.

Antony Lewis
Posts: 1936
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: CosmoMC: big spread in log-likes for Planck 2018 found with BOBYQA

Post by Antony Lewis » October 28 2021

The best-fit values for Planck are in the parameter tables on the Planck legacy archive. They were run with BOBYQA on multiple chains, though I think before the workaround you mention was neccessary. You could try comparing with Cobaya.

A difference with the best fit sample is expected, since in high dimensions there's negligible parameter volume around the best fit point.

Aviv Padawer-Blatt
Posts: 9
Joined: October 28 2021
Affiliation: University of Waterloo

Re: CosmoMC: big spread in log-likes for Planck 2018 found with BOBYQA

Post by Aviv Padawer-Blatt » November 04 2021

Oh I see, thank you. I did not realize that the best-fit log-likelihoods and parameter values in the Planck Parameter Tables are from the BOBYQA algorithm (I was under the impression that these values are from the MCMC chains, as I can't find anywhere that states they are from BOBYQA).
A difference with the best fit sample is expected, since in high dimensions there's negligible parameter volume around the best fit point.
If I understand, you are saying that the workaround where I set minimize_mcmc_refine_num from 20 to 0 results in a lack of convergence of the best-fits in BOBYQA, but this is to be expected, and I should be able to trust the minimum log-likelihood of the 8 samples I use?

Post Reply