[astro-ph/0702542] Tainted Evidence: Cosmological Model Sele
Posted: February 23 2007
This paper is a strong criticism against model selection, and in favor of parameter fitting.
The paper makes (among other arguments) the point that model selection is difficult to interpret correctly because the result depends on the priors. The authors use an analogy which I didn't quite grasp:
Rather than discussing mathematical niceties, consider searching for, say, the smartest person in the world: applying a uniform prior across the population would lead a model selector to conclude that the evidence favors that person living in England rather than the US, because the US represents a larger parameter volume.
The authors also write, as an example of the adequacy of parameter fitting, that
A standard cosmological fit analysis of actual SNAP + SNF + Planck data will produce a central value in the ([tex]w_0,w_a[/tex]) plane (recall that [tex]w_a[/tex] = −2[tex]w[/tex]′(z = 1)) and a contour around it encompassing some chosen confidence level (CL), typically 68, 90, or 95%. The point corresponding to a cosmological constant (-1,0) may or may not lie inside the contour. If it does not, we may say that we exclude [tex]\Lambda[/tex]CDM at that CL.
More precisely, in a frequentist analysis in which one aims to prove or disprove [tex]\Lambda[/tex]CDM, one would simulate the expected SNAP + SNF + Planck data sample assuming a [tex]\Lambda[/tex]CDM universe, and, analyzing this synthetic data sample as if it were the real data, one would draw a, say, 90% CL contour around the (-1,0) point, much like the one depicted as the inner contour in Fig. 1. That contour tells us that, if [tex]\Lambda[/tex]CDM is true, we expect to get a central value inside that contour in 90% of the observations like SNAP + SNF + Planck that we may perform. Therefore, if our one real SNAP + SNF + Planck observation delivers a central value outside that contour, irrespectively of its associated error ellipse, we will be able to say that we have excluded [tex]\Lambda[/tex]CDM at greater than 90% CL.
It would be interesting to hear comments from people who advocate model selection.
The paper makes (among other arguments) the point that model selection is difficult to interpret correctly because the result depends on the priors. The authors use an analogy which I didn't quite grasp:
Rather than discussing mathematical niceties, consider searching for, say, the smartest person in the world: applying a uniform prior across the population would lead a model selector to conclude that the evidence favors that person living in England rather than the US, because the US represents a larger parameter volume.
The authors also write, as an example of the adequacy of parameter fitting, that
A standard cosmological fit analysis of actual SNAP + SNF + Planck data will produce a central value in the ([tex]w_0,w_a[/tex]) plane (recall that [tex]w_a[/tex] = −2[tex]w[/tex]′(z = 1)) and a contour around it encompassing some chosen confidence level (CL), typically 68, 90, or 95%. The point corresponding to a cosmological constant (-1,0) may or may not lie inside the contour. If it does not, we may say that we exclude [tex]\Lambda[/tex]CDM at that CL.
More precisely, in a frequentist analysis in which one aims to prove or disprove [tex]\Lambda[/tex]CDM, one would simulate the expected SNAP + SNF + Planck data sample assuming a [tex]\Lambda[/tex]CDM universe, and, analyzing this synthetic data sample as if it were the real data, one would draw a, say, 90% CL contour around the (-1,0) point, much like the one depicted as the inner contour in Fig. 1. That contour tells us that, if [tex]\Lambda[/tex]CDM is true, we expect to get a central value inside that contour in 90% of the observations like SNAP + SNF + Planck that we may perform. Therefore, if our one real SNAP + SNF + Planck observation delivers a central value outside that contour, irrespectively of its associated error ellipse, we will be able to say that we have excluded [tex]\Lambda[/tex]CDM at greater than 90% CL.
It would be interesting to hear comments from people who advocate model selection.