CosmoCoffee

 FAQ   Search  SmartFeed   Memberlist    Register Profile   Log in Arxiv New Filter | Bookmarks & clubs | Arxiv ref/author:

 confidence intervals when the likelihood is not normalized
Author Message
Susana Landau

Joined: 10 Dec 2004
Posts: 18
Affiliation: Buenos Aires University

 Posted: March 23 2009 We are working with a model with a power spectrum different from the ΛCDM model, P(k) = As k C(k), where C(k) includes two extra free parameters. We know from previous analysis that the spectra is not sensitive to huge values of the free parameters. Therefore, for huge values of these parameters, there is always good fit to the data (likelihood/likelihood_max ≈ 1). The question is: How can we determine the confidence interval for a given parameter (i.e, the value X for the parameter for which I can say "The value of the parameter is greater that X with 68% confidence") when the likelihood does not fall to zero at infinity (hence it is not normalized)?.
Antonio C. C. Guimaraes

Joined: 02 Dec 2004
Posts: 9
Affiliation: Astronomy Department, IAG, Universidade de Sao Paulo

 Posted: March 24 2009 Not sure if I understood you right, but the normalization of the likelihood is primarily defined on the space of the values of the quantity that you are examining, not on the parameter space of the model. If you want to operate on the space of the model parameters, then you have to make a transformation. For example, if you have a measurement of the power spectrum Pobs(ki) and a model P(k) for it, them your likelihood is going to be something like: L= [ exp(P(ki)-Pobs(ki))2/2σ2 ] / \sqrt(2 pi)σ} and this is already normalized. If we define χ=[P(ki)-Pobs(ki)] / σ then Ldχ=1 (χ can be anything from -infinity to +infinity) but if χ2=χ2(p) , where p is a parameter set (the parameters that define the model P(k)). Then, if I want to write the integral of the likelihood in terms of the parameters p then I have to use that dχ= dχ /dp dp. It can happen that the P(k) does not change for a given range of the parameter p, but then dP/dp is 0, so you see that integrating the likelihood in this range of the parameter space is not going to contribute. Maybe that is your case.
Susana Landau

Joined: 10 Dec 2004
Posts: 18
Affiliation: Buenos Aires University

 Posted: April 06 2009 Thanks very much Antonio for your answer, it helps a lot, is there any reference where I can check about the likelihood normalization? because I can not find anything about it in the usual cosmomc bibliography, thank you very much in advanced, regards Susana
Andrew Liddle

Joined: 28 Sep 2004
Posts: 21
Affiliation: University of Edinburgh

 Posted: May 11 2009 Dear Susana, Antonio is right that in general likelihoods are not normalized with respect to models, as the likelihood is defined with respect to the observations and need not know what model you are then going to compare to the data. Usually its normalization can be ignored because we just write ${\rm posterior} \propto {\rm prior} \times {\rm likelihood}$ and confidence intervals can still be extracted without explicitly normalizing the posterior. More generally the normalization factor is the Bayesian evidence, which is used in model selection analyses. However in your specific problem it is not the likelihood you should be worrying about, but your prior distribution, as the prior does need to be normalized. If the prior probability does not tend to zero as your parameter gets large, then you are effectively putting all of your prior probability at infinitely large parameter value. Then no amount of data will be able to overrule that (which is why you are having trouble setting a confidence level). As almost certainly you don't believe that the parameter starts out with 100% probability of being infinitely large, you have to find a more appropriate prior which does account for your beliefs about the likely value before the data comes along. Then you can apply the likelihood, normalized or not, to find out how the data has changed your initial view. best regards, Andrew
Susana Landau

Joined: 10 Dec 2004
Posts: 18
Affiliation: Buenos Aires University

 Posted: May 14 2009 Thank you Andrew very much for your answer. However, now I have a different problem, because I still can not establish bounds on the parameters. The problem is the following: I am working with COSMOMC, and following Andrew's suggestion I put a prior on the parameter of the model which has a physical motivation. The problem is that now L/L_max being L the likelihood never takes values below 0.6, except on the priors. Let me assume that the parameter name is B and the prior is −106 < B < 102. From a previous analysis, I know that for B=−100 and for B=−10 the value of the likelihood is very low, however this does not appear in the 1 dimensional likelihood plot. Why? Because COSMOMC calculates the value of the likelihood for B=−8 105, −6 105, ....−2 105 and B=0 and there are no points calculated between the two last values I have mentioned (only those that are very near to the prior and cannot be considered in the analysis). In a grid based analysis, I can calculated B=−1000, B=−100, B=−10, etc, but with COSMOMC this is not possible. Therefore, should I conclude that the data can not give me additional information on the value of the parameter even though I know there are some values of B within the prior, that do not fit the data and others that fit very well,? I have already tried to change the start width and the st.dev estimate in cosmomc, but this did not work I apprecciate much any help with this problem regards Susana Landau
Susana Landau

Joined: 10 Dec 2004
Posts: 18
Affiliation: Buenos Aires University

 Posted: May 14 2009 Maybe the previous explanation was not clear enough to explain the problem. I know from previous analysis that for very huge values of the parameter B, the model always fits the data, and that for some values like B=−100, B=−10 B=−1, there is no good fit to the data. If I want that COSMOMC calculates some of the points where there is no good fit to the data, I can put a prior which has no physical motivation like −2.4 104 < B < 5 103, and thus in the 1 dimensional likelihood plot, I find point where L/Lmax is nearly 0, and also intermediate points (L/Lmax=0.2, L/Lmax=0.3, L/Lmax=0.5, etc). However, from this plot I can not establish the confidence intervals, because the prior has no physical motivation. When I work with a physical motivated prior, COSMOMC does not calculate points where L/Lmax is below 0.6. I think this is a problem of the method, anyone agrees? any suggestion to improve the analysis? thanks again for any comment or suggestions
Andrew Liddle

Joined: 28 Sep 2004
Posts: 21
Affiliation: University of Edinburgh

 Posted: May 14 2009 Dear Susana, If you choose a uniform prior going to −106, but are interested in imposing a parameter constraint for values orders of magnitude smaller, you will always tend to run into sampling problems that MCMC does not manage to probe the small region of the prior space that you want it to. You are already saying by the prior that the chance of |B| being less than 102 is only about one part in 104, so the MCMC regards it as already ruled out at that confidence without needing any data. The fact that you still want to impose the constraint means that you don't think that those small values are already ruled out without needing any data. So probably your prior is still not quite right. In cases where a parameter varies by orders of magnitude often a log prior is more appropriate (eg if you think in advance that you are as likely to be between 104 and 105 as between 105 and 106). However you may then run into problems deciding what to do with zero, so there may be no easy answer. best, Andrew
 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First
 All times are GMT + 5 Hours Page 1 of 1

 Jump to: Select a forum Arxiv paper discussion----------------arXiv papers Topic discussion----------------Early UniverseCosmological ModelCosmological Observations  Projects and Resources----------------Computers and softwareTeaching, Papers and PresentationsResearch projectsiCosmo Coming up----------------Job vacanciesConferences and meetings Management----------------CosmoCoffee
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum