### CosmoMC: On the minimun \\Chi^2 and priors etc.

Posted:

**November 02 2005**I agree with Pier-Stefano.

Page **2** of **2**

Posted: **November 02 2005**

I agree with Pier-Stefano.

Posted: **November 07 2005**

Hi Pier-Stefano,

very nice reply!

I'm more than willing to switch back to the 68% counting method (for the 1-dim likelihoods I compute both anyhow).

But I'm still not entirely happy with the "significance" issue: Suppose you put a limit on some parameter (let's call it gamma) which has a distribution that is non-gaussian. Let's say that at 68%, I infer

\gamma = 1 + 0.2 -0.1

would I then have to quote the \Delta^\chi^2 for this 1 \sigma ?

And what about 2-dimensional plots ? The \Delta\chi^2 may vary depending on direction.

So what's your advise ?

Greetings, Michael

very nice reply!

I'm more than willing to switch back to the 68% counting method (for the 1-dim likelihoods I compute both anyhow).

But I'm still not entirely happy with the "significance" issue: Suppose you put a limit on some parameter (let's call it gamma) which has a distribution that is non-gaussian. Let's say that at 68%, I infer

\gamma = 1 + 0.2 -0.1

would I then have to quote the \Delta^\chi^2 for this 1 \sigma ?

And what about 2-dimensional plots ? The \Delta\chi^2 may vary depending on direction.

So what's your advise ?

Greetings, Michael

Posted: **November 08 2005**

Hi Micheal,

I still don't understand what you mean by "significance", as I explained in the previous

post, in the Bayesian framework, once you have the posterior and you use it to infer the

interval where there is 68% probability of finding the true value, that is it.

What more significant than this?? To me this is pretty crystal clear or better still it is

"evident", but perhaps is because I read too many books ;0)..

In the case of your parameter

[tex]

\gamma=1\pm^{0.2}_{0.1}[/tex]

the interaval [0.9,1.2] is the one in which there is 68% probability of finding the true value.

In order to have this meaning this interval has to be

inferred as the intersection of the posterior distribution with the slice which contain 68%

of the total parameter volume. For a closed 1-dimensional posterior there are two

intersections which give the lower and upper value of the confidence interval.

Now if you want to quote also the correspoding \Delta\chi^2, this will be just

the difference between the best fit value and the one corresponding to the slice which

encloses the 68% volume.

I don't understand why this \Delta\chi^2 should depend on the direction in the parameter

space for the 2-dimensional case. The slicing of the posterior determines

iso-\chi^2 contours, no matter whether the posterior is Gaussian or not.

The difference with what you propose is that while Bayesians do the slicing at

values of \Delta\chi^2 for which the volume contained between the posterior

and the slice is 68%, you propose to slice is at fixed values of \Delta\chi^2

corresponding to the 68% of a Gaussian, but as I explained so far,

if the posterior is not a Gaussian this slicing would determine a region of

the parameter space where the probability of finding the true value is not 68%,

but something else depending on the shape of the posterior.

Is this clear?

Of course if the posterior is non-Gaussian in the sense that there are more than one maxima,

you have to chose the highest peak and repeat the procedure above.

So in conclusion do I have an advice for you? No, sorry I don't,

everyone is free to do whatever they want, but if they want to be Bayesian there is a clear

prescription for being one, even in the most extreme situations you can imagine,

and it is not me who has established what is Bayesian and what is not,

rather the past 60 yrs of work in Statistics.

I found a number of statistics textbooks very helpful and if people are curious

to know more about Bayesian reasoning I would suggest the reading of:

D.S. Sivia, Data analysis - A Bayesian Tutorial (Oxford, 1997)

B. De Finetti, Theory of Probability, 1970 (J. Wiley and Sons, 1974) (this is the bible of subjectivist)

H. Jeffreys, Theory of Probabilty (Oxford, 1961)

Stuart & Kendall, Bayesian Inference, Vol 2B (Halsted Press, 1994)

Hopefully I have clarified your point of concern, if not please let me know I am more

than happy to continue this "Bayesian Statistics" exam for as long as you like...;0))

The best,

Pier-Stefano

I still don't understand what you mean by "significance", as I explained in the previous

post, in the Bayesian framework, once you have the posterior and you use it to infer the

interval where there is 68% probability of finding the true value, that is it.

What more significant than this?? To me this is pretty crystal clear or better still it is

"evident", but perhaps is because I read too many books ;0)..

In the case of your parameter

[tex]

\gamma=1\pm^{0.2}_{0.1}[/tex]

the interaval [0.9,1.2] is the one in which there is 68% probability of finding the true value.

In order to have this meaning this interval has to be

inferred as the intersection of the posterior distribution with the slice which contain 68%

of the total parameter volume. For a closed 1-dimensional posterior there are two

intersections which give the lower and upper value of the confidence interval.

Now if you want to quote also the correspoding \Delta\chi^2, this will be just

the difference between the best fit value and the one corresponding to the slice which

encloses the 68% volume.

I don't understand why this \Delta\chi^2 should depend on the direction in the parameter

space for the 2-dimensional case. The slicing of the posterior determines

iso-\chi^2 contours, no matter whether the posterior is Gaussian or not.

The difference with what you propose is that while Bayesians do the slicing at

values of \Delta\chi^2 for which the volume contained between the posterior

and the slice is 68%, you propose to slice is at fixed values of \Delta\chi^2

corresponding to the 68% of a Gaussian, but as I explained so far,

if the posterior is not a Gaussian this slicing would determine a region of

the parameter space where the probability of finding the true value is not 68%,

but something else depending on the shape of the posterior.

Is this clear?

Of course if the posterior is non-Gaussian in the sense that there are more than one maxima,

you have to chose the highest peak and repeat the procedure above.

So in conclusion do I have an advice for you? No, sorry I don't,

everyone is free to do whatever they want, but if they want to be Bayesian there is a clear

prescription for being one, even in the most extreme situations you can imagine,

and it is not me who has established what is Bayesian and what is not,

rather the past 60 yrs of work in Statistics.

I found a number of statistics textbooks very helpful and if people are curious

to know more about Bayesian reasoning I would suggest the reading of:

D.S. Sivia, Data analysis - A Bayesian Tutorial (Oxford, 1997)

B. De Finetti, Theory of Probability, 1970 (J. Wiley and Sons, 1974) (this is the bible of subjectivist)

H. Jeffreys, Theory of Probabilty (Oxford, 1961)

Stuart & Kendall, Bayesian Inference, Vol 2B (Halsted Press, 1994)

Hopefully I have clarified your point of concern, if not please let me know I am more

than happy to continue this "Bayesian Statistics" exam for as long as you like...;0))

The best,

Pier-Stefano

Posted: **November 08 2005**

Hi Pier-Stefano,

my mistake with the directional dependence:

you are of course right: by definition, the

boundary is an iso-\Delta\chi^2.

If Bayes was a priest, you should be one,

too :-) At least a priest for the bayesian

cause :-)

So you have me convinced. I'll go back to the

68% area method for 2-d (as said, for 1-d,

the cmbeasy gui quotes both methods

anyhow). At least until someone else

convinces me of abandoning it again :-]

Which leaves one question: Is this the last

post in this thread ? Or is there anyone out

there with a different point of view. Or even

someone who wishes to convert Pier-Stefano

from the true faith?

my mistake with the directional dependence:

you are of course right: by definition, the

boundary is an iso-\Delta\chi^2.

If Bayes was a priest, you should be one,

too :-) At least a priest for the bayesian

cause :-)

So you have me convinced. I'll go back to the

68% area method for 2-d (as said, for 1-d,

the cmbeasy gui quotes both methods

anyhow). At least until someone else

convinces me of abandoning it again :-]

Which leaves one question: Is this the last

post in this thread ? Or is there anyone out

there with a different point of view. Or even

someone who wishes to convert Pier-Stefano

from the true faith?