CosmoCoffee

Susana Landau · Post by **Susana Landau** » March 23 2009

We are working with a model with a power spectrum different from the
\Lambda CDM model, P(k) = A_s k C(k), where C(k) includes two extra free
parameters. We know from previous analysis that the spectra is not
sensitive to huge values of the free parameters. Therefore, for huge values
of these parameters, there is always good fit to the data
(likelihood/likelihood_max \simeq 1). The
question is: How can we determine the confidence interval for a given
parameter (i.e, the value X for the parameter for which I can say
"The value of the parameter is greater
that X with 68% confidence") when the likelihood does not fall to zero at
infinity (hence it is not normalized)?.

Antonio C. C. Guimaraes · Post by **Antonio C. C. Guimaraes** » March 24 2009

Not sure if I understood you right, but the normalization of the likelihood is primarily defined on the space of the values of the quantity that you are examining, not on the parameter space of the model. If you want to operate on the space of the model parameters, then you have to make a transformation.

For example, if you have a measurement of the power spectrum P_{obs}(k_i) and a model P(k) for it, them your likelihood is going to be something like:
L= [ \exp(P(k_i)-P_{obs}(k_i))^2/2\sigma^2 ] / \sqrt(2 pi)\sigma}
and this is already normalized. If we define \chi=[P(k_i)-P_{obs}(k_i)] / \sigma
then \int{Ld\chi}=1 (\chi can be anything from -infinity to +infinity)

but if \chi^2=\chi^2(p) , where p is a parameter set (the parameters that define the model P(k)). Then, if I want to write the integral of the likelihood in terms of the parameters p then I have to use that d\chi= d\chi /dp dp.
It can happen that the P(k) does not change for a given range of the parameter p, but then dP/dp is 0, so you see that integrating the likelihood in this range of the parameter space is not going to contribute. Maybe that is your case.

Susana Landau · Post by **Susana Landau** » April 06 2009

Thanks very much Antonio for your answer, it helps a lot,

is there any reference where I can check about the likelihood normalization? because I can not find anything about it in the usual cosmomc bibliography, thank you very much in advanced, regards

Susana

Andrew Liddle · Post by **Andrew Liddle** » May 11 2009

Dear Susana,

Antonio is right that in general likelihoods are not normalized with respect to models, as the likelihood is defined with respect to the observations and need not know what model you are then going to compare to the data. Usually its normalization can be ignored because we just write
[tex]{\rm posterior} \propto {\rm prior} \times {\rm likelihood}[/tex]
and confidence intervals can still be extracted without explicitly normalizing the posterior. More generally the normalization factor is the Bayesian evidence, which is used in model selection analyses.

However in your specific problem it is not the likelihood you should be worrying about, but your prior distribution, as the prior does need to be normalized. If the prior probability does not tend to zero as your parameter gets large, then you are effectively putting all of your prior probability at infinitely large parameter value. Then no amount of data will be able to overrule that (which is why you are having trouble setting a confidence level). As almost certainly you don't believe that the parameter starts out with 100% probability of being infinitely large, you have to find a more appropriate prior which does account for your beliefs about the likely value before the data comes along. Then you can apply the likelihood, normalized or not, to find out how the data has changed your initial view.

best regards,

Andrew

Susana Landau · Post by **Susana Landau** » May 14 2009

Thank you Andrew very much for your answer. However, now I have a different problem, because I still can not establish bounds on the parameters. The problem is the following: I am working with COSMOMC, and following Andrew's suggestion I put a prior on the parameter of the model which has a physical motivation. The problem is that now L/L_max being L the likelihood never takes values below 0.6, except on the priors. Let me assume that the parameter name is B and the prior is -10^6 < B < 10^2. From a previous analysis, I know that for B=-100 and for B=-10 the value of the likelihood is very low, however this does not appear in the 1 dimensional likelihood plot. Why? Because COSMOMC calculates the value of the likelihood for B=-8 10^5, -6 10^5, ....-2 10^5 and B=0 and there are no points calculated between the two last values I have mentioned (only those that are very near to the prior and cannot be considered in the analysis). In a grid based analysis, I can calculated B=-1000, B=-100, B=-10, etc, but with COSMOMC this is not possible. Therefore, should I conclude that the data can not give me additional information on the value of the parameter even though I know there are some values of B within the prior, that do not fit the data and others that fit very well,? I have already tried to change the start width and the st.dev estimate in cosmomc, but this did not work

I apprecciate much any help with this problem
regards

Susana Landau

Susana Landau · Post by **Susana Landau** » May 14 2009

Maybe the previous explanation was not clear enough to explain the problem. I know from previous analysis that for very huge values of the parameter B, the model always fits the data, and that for some values like B=-100, B=-10 B=-1, there is no good fit to the data. If I want that COSMOMC calculates some of the points where there is no good fit to the data, I can put a prior which has no physical motivation like -2.4 10^4 < B < 5 10^3, and thus in the 1 dimensional likelihood plot, I find point where L/L_{max} is nearly 0, and also intermediate points (L/L_{max}=0.2, L/L_{max}=0.3, L/L_{max}=0.5, etc). However, from this plot I can not establish the confidence intervals, because the prior has no physical motivation. When I work with a physical motivated prior, COSMOMC does not calculate points where L/L_{max} is below 0.6. I think this is a problem of the method, anyone agrees? any suggestion to improve the analysis?

thanks again for any comment or suggestions

Andrew Liddle · Post by **Andrew Liddle** » May 14 2009

Dear Susana,

If you choose a uniform prior going to -10^6, but are interested in imposing a parameter constraint for values orders of magnitude smaller, you will always tend to run into sampling problems that MCMC does not manage to probe the small region of the prior space that you want it to. You are already saying by the prior that the chance of |B| being less than 10^2 is only about one part in 10^4, so the MCMC regards it as already ruled out at that confidence without needing any data.

The fact that you still want to impose the constraint means that you don't think that those small values are already ruled out without needing any data. So probably your prior is still not quite right. In cases where a parameter varies by orders of magnitude often a log prior is more appropriate (eg if you think in advance that you are as likely to be between 10^4 and 10^5 as between 10^5 and 10^6). However you may then run into problems deciding what to do with zero, so there may be no easy answer.

best,

Andrew

CosmoCoffee

confidence intervals when the likelihood is not normalized

confidence intervals when the likelihood is not normalized

confidence intervals when the likelihood is not normalized

confidence intervals when the likelihood is not normalized

confidence intervals when the likelihood is not normalized

confidence intervals when the likelihood is not normalized

confidence intervals when the likelihood is not normalized

confidence intervals when the likelihood is not normalized