How to calculate the AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion) from CosmoMC chains

Use of Cobaya. camb, CLASS, cosmomc, compilers, etc.
Post Reply
srijita sinha
Posts: 29
Joined: April 12 2019
Affiliation: Indian Institute of Science Education and Research Kolkata

How to calculate the AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion) from CosmoMC chains

Post by srijita sinha » December 08 2020

Hi,

I have 2 models and the corresponding CosmoMC chains. I want to find which model is preferred. For that how can I calculate the AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion)? Other than that how can I say which model is preferred?

Sorry for this trivial question. Any help would be great.

Srijita

Luke Hart
Posts: 64
Joined: July 13 2015
Affiliation: University of Manchester

Re: How to calculate the AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion) from CosmoMC chains

Post by Luke Hart » January 02 2021

If you have the same number of parameters in both models then I believe the AIC becomes a straight forward check of the maximum likelihood. If not, then you can compare the two AIC by using the minimise (mode 2 of CosmoMC) to calculate the Maximum Likelihood value of your model and compare that with the usual formula for the AIC. For the BIC, you will need the number of samples as well although its the same process: I would recommend running the minimise routine independently of your chains to find the ML. Of course this isn't the best way to test the comparisons of models and I would recommend (though obviously this is an external code to CosmoMC) something like MultiNest or PolyChord and then calculating the Bayesian evidence and then subsequently the Bayes factor for the two models.

srijita sinha
Posts: 29
Joined: April 12 2019
Affiliation: Indian Institute of Science Education and Research Kolkata

Re: How to calculate the AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion) from CosmoMC chains

Post by srijita sinha » January 04 2021

Thank you for the reply and PolyChord suggestion. It is very helpful.

Natalie Hogg
Posts: 4
Joined: September 04 2020
Affiliation: ICG Portsmouth

Re: How to calculate the AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion) from CosmoMC chains

Post by Natalie Hogg » January 08 2021

A quick alternative to find the Bayesian evidence without nested sampling is to use the method presented in https://arxiv.org/abs/1704.03472 which is implemented in the MCEvidence code: https://github.com/yabebalFantaye/MCEvidence. However, I'm not sure how reliable it is for a large number of parameters (i.e. a high number of dimensions).

Post Reply