CosmoCoffee Forum Index CosmoCoffee

 
 FAQFAQ   SearchSearch  MemberlistSmartFeed   MemberlistMemberlist    RegisterRegister 
   ProfileProfile   Log inLog in 
Arxiv New Filter | Bookmarks & clubs | Arxiv ref/author:

confidence intervals when the likelihood is not normalized
 
Post new topic   Reply to topic    CosmoCoffee Forum Index -> Cosmological Observations
View previous topic :: View next topic  
Author Message
Susana Landau



Joined: 10 Dec 2004
Posts: 18
Affiliation: Buenos Aires University

PostPosted: March 23 2009  Reply with quote

We are working with a model with a power spectrum different from the
ΛCDM model, P(k) = As k C(k), where C(k) includes two extra free
parameters. We know from previous analysis that the spectra is not
sensitive to huge values of the free parameters. Therefore, for huge values
of these parameters, there is always good fit to the data
(likelihood/likelihood_max ≈ 1). The
question is: How can we determine the confidence interval for a given
parameter (i.e, the value X for the parameter for which I can say
"The value of the parameter is greater
that X with 68% confidence") when the likelihood does not fall to zero at
infinity (hence it is not normalized)?.
Back to top
View user's profile [ Hidden ]
Antonio C. C. Guimaraes



Joined: 02 Dec 2004
Posts: 9
Affiliation: Astronomy Department, IAG, Universidade de Sao Paulo

PostPosted: March 24 2009  Reply with quote

Not sure if I understood you right, but the normalization of the likelihood is primarily defined on the space of the values of the quantity that you are examining, not on the parameter space of the model. If you want to operate on the space of the model parameters, then you have to make a transformation.

For example, if you have a measurement of the power spectrum Pobs(ki) and a model P(k) for it, them your likelihood is going to be something like:
L= [ exp(P(ki)-Pobs(ki))2/2σ2 ] / \sqrt(2 pi)σ}
and this is already normalized. If we define χ=[P(ki)-Pobs(ki)] / σ
then Ldχ=1 (χ can be anything from -infinity to +infinity)

but if χ2=χ2(p) , where p is a parameter set (the parameters that define the model P(k)). Then, if I want to write the integral of the likelihood in terms of the parameters p then I have to use that dχ= dχ /dp dp.
It can happen that the P(k) does not change for a given range of the parameter p, but then dP/dp is 0, so you see that integrating the likelihood in this range of the parameter space is not going to contribute. Maybe that is your case.
Back to top
View user's profile [ Hidden ] Visit poster's website
Susana Landau



Joined: 10 Dec 2004
Posts: 18
Affiliation: Buenos Aires University

PostPosted: April 06 2009  Reply with quote

Thanks very much Antonio for your answer, it helps a lot,

is there any reference where I can check about the likelihood normalization? because I can not find anything about it in the usual cosmomc bibliography, thank you very much in advanced, regards

Susana
Back to top
View user's profile [ Hidden ]
Andrew Liddle



Joined: 28 Sep 2004
Posts: 21
Affiliation: University of Edinburgh

PostPosted: May 11 2009  Reply with quote

Dear Susana,

Antonio is right that in general likelihoods are not normalized with respect to models, as the likelihood is defined with respect to the observations and need not know what model you are then going to compare to the data. Usually its normalization can be ignored because we just write
{\rm posterior} \propto {\rm prior} \times {\rm likelihood}
and confidence intervals can still be extracted without explicitly normalizing the posterior. More generally the normalization factor is the Bayesian evidence, which is used in model selection analyses.

However in your specific problem it is not the likelihood you should be worrying about, but your prior distribution, as the prior does need to be normalized. If the prior probability does not tend to zero as your parameter gets large, then you are effectively putting all of your prior probability at infinitely large parameter value. Then no amount of data will be able to overrule that (which is why you are having trouble setting a confidence level). As almost certainly you don't believe that the parameter starts out with 100% probability of being infinitely large, you have to find a more appropriate prior which does account for your beliefs about the likely value before the data comes along. Then you can apply the likelihood, normalized or not, to find out how the data has changed your initial view.

best regards,

Andrew
Back to top
View user's profile   Visit poster's website
Susana Landau



Joined: 10 Dec 2004
Posts: 18
Affiliation: Buenos Aires University

PostPosted: May 14 2009  Reply with quote

Thank you Andrew very much for your answer. However, now I have a different problem, because I still can not establish bounds on the parameters. The problem is the following: I am working with COSMOMC, and following Andrew's suggestion I put a prior on the parameter of the model which has a physical motivation. The problem is that now L/L_max being L the likelihood never takes values below 0.6, except on the priors. Let me assume that the parameter name is B and the prior is −106 < B < 102. From a previous analysis, I know that for B=−100 and for B=−10 the value of the likelihood is very low, however this does not appear in the 1 dimensional likelihood plot. Why? Because COSMOMC calculates the value of the likelihood for B=−8 105, −6 105, ....−2 105 and B=0 and there are no points calculated between the two last values I have mentioned (only those that are very near to the prior and cannot be considered in the analysis). In a grid based analysis, I can calculated B=−1000, B=−100, B=−10, etc, but with COSMOMC this is not possible. Therefore, should I conclude that the data can not give me additional information on the value of the parameter even though I know there are some values of B within the prior, that do not fit the data and others that fit very well,? I have already tried to change the start width and the st.dev estimate in cosmomc, but this did not work

I apprecciate much any help with this problem
regards

Susana Landau
Back to top
View user's profile [ Hidden ]
Susana Landau



Joined: 10 Dec 2004
Posts: 18
Affiliation: Buenos Aires University

PostPosted: May 14 2009  Reply with quote

Maybe the previous explanation was not clear enough to explain the problem. I know from previous analysis that for very huge values of the parameter B, the model always fits the data, and that for some values like B=−100, B=−10 B=−1, there is no good fit to the data. If I want that COSMOMC calculates some of the points where there is no good fit to the data, I can put a prior which has no physical motivation like −2.4 104 < B < 5 103, and thus in the 1 dimensional likelihood plot, I find point where L/Lmax is nearly 0, and also intermediate points (L/Lmax=0.2, L/Lmax=0.3, L/Lmax=0.5, etc). However, from this plot I can not establish the confidence intervals, because the prior has no physical motivation. When I work with a physical motivated prior, COSMOMC does not calculate points where L/Lmax is below 0.6. I think this is a problem of the method, anyone agrees? any suggestion to improve the analysis?

thanks again for any comment or suggestions
Back to top
View user's profile [ Hidden ]
Andrew Liddle



Joined: 28 Sep 2004
Posts: 21
Affiliation: University of Edinburgh

PostPosted: May 14 2009  Reply with quote

Dear Susana,

If you choose a uniform prior going to −106, but are interested in imposing a parameter constraint for values orders of magnitude smaller, you will always tend to run into sampling problems that MCMC does not manage to probe the small region of the prior space that you want it to. You are already saying by the prior that the chance of |B| being less than 102 is only about one part in 104, so the MCMC regards it as already ruled out at that confidence without needing any data.

The fact that you still want to impose the constraint means that you don't think that those small values are already ruled out without needing any data. So probably your prior is still not quite right. In cases where a parameter varies by orders of magnitude often a log prior is more appropriate (eg if you think in advance that you are as likely to be between 104 and 105 as between 105 and 106). However you may then run into problems deciding what to do with zero, so there may be no easy answer.

best,

Andrew
Back to top
View user's profile   Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    CosmoCoffee Forum Index -> Cosmological Observations All times are GMT + 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group. Sponsored by WordWeb online dictionary and dictionary software.