adjusting priors with getdist

 Posts: 3
 Joined: July 02 2008
 Affiliation: Institute for Space Science
 Contact:
adjusting priors with getdist
Dear all,
I am trying to use the adjust priors option of Getdist to add constraints from a new dataset to the cosmological parameters distributions obtained only for WMAP5. Following the example of the corresponding routine in Getdist, I calculated the chi^2 for the new dataset for the corresponding combinations of the cosmological parameters from .txt files.
In order to calculate the likelihoods for WMAP5 + the new dataset I have to modify the first two columns of the .txt files.
My questions is how is the right way to do that? It seems to be logical to add to the column 2 the entire chi^2/2, but if I multiply the first column by exp(chi^2/2) the weights are too small. Should I use the chi^2 reduced by the number of independent data points or/and a kind of offset for the first column?
Thanks,
Petruta.
I am trying to use the adjust priors option of Getdist to add constraints from a new dataset to the cosmological parameters distributions obtained only for WMAP5. Following the example of the corresponding routine in Getdist, I calculated the chi^2 for the new dataset for the corresponding combinations of the cosmological parameters from .txt files.
In order to calculate the likelihoods for WMAP5 + the new dataset I have to modify the first two columns of the .txt files.
My questions is how is the right way to do that? It seems to be logical to add to the column 2 the entire chi^2/2, but if I multiply the first column by exp(chi^2/2) the weights are too small. Should I use the chi^2 reduced by the number of independent data points or/and a kind of offset for the first column?
Thanks,
Petruta.

 Posts: 1592
 Joined: September 23 2004
 Affiliation: University of Sussex
 Contact:
Re: adjusting priors with getdist
If you are using cosmomc to postprocess there is an input option redo_likeoffset to rescale the log likelihood.
In GetDist you can add any constant you like to the log likelihood. But if the scatter of weights is too large that's telling you that you need to run new chains (importance sampling only works well if the new distribution is subset and similar).
In GetDist you can add any constant you like to the log likelihood. But if the scatter of weights is too large that's telling you that you need to run new chains (importance sampling only works well if the new distribution is subset and similar).

 Posts: 3
 Joined: July 02 2008
 Affiliation: Institute for Space Science
 Contact:
adjusting priors with getdist
Dear Antony,
Thanks for your prompt reply!
Please, could you tell me if I have understood right? Is the outlier
fraction given by getdist a measure of the scatter of weights so that if
it is not too large I can use the distributions obtained by
running getdist on the original chains with the adjust priors option?
From running getdist without and with the adjust prior I have obtained:
case 1) without adjust priors
Number of chains used = 8
var(mean)/mean(var), 1/2 chains, worst evalue: R1 = 0.0182
RL: Thin for Markov: 45
RL: Thin for indep samples: 89
RL: Estimated burn in steps: 480 (137 rows)
mean input multiplicity = 3.502123
using 41914 rows, processing 13 parameters
Approx indep samples: 1649
...
case 2) with adjust priors  using the entire chi^2
Adjusting priors
outlier fraction 2.3858376E05
Number of chains used = 8
var(mean)/mean(var), 1/2 chains, worst evalue: R1 = 0.0290
Prior removed 17313 models
mean input multiplicity = 7.1085290E11
using 24601 rows, processing 13 parameters
effective number of samples (assuming indep): 499
...
case 3) with adjust priors  using the chi^2 divided by the number
of indep. data points
Adjusting priors
Number of chains used = 8
var(mean)/mean(var), 1/2 chains, worst evalue: R1 = 0.0093
mean input multiplicity = 0.8225374
using 41914 rows, processing 13 parameters
effective number of samples (assuming indep): 2053
...
My question is: may be one of the cases 2) and 3) OK from this point of
view or the solution is to use cosmomc to postprocess the chains or to
make new chains using both the datasets?
Thanks,
Petruta.
Thanks for your prompt reply!
Please, could you tell me if I have understood right? Is the outlier
fraction given by getdist a measure of the scatter of weights so that if
it is not too large I can use the distributions obtained by
running getdist on the original chains with the adjust priors option?
From running getdist without and with the adjust prior I have obtained:
case 1) without adjust priors
Number of chains used = 8
var(mean)/mean(var), 1/2 chains, worst evalue: R1 = 0.0182
RL: Thin for Markov: 45
RL: Thin for indep samples: 89
RL: Estimated burn in steps: 480 (137 rows)
mean input multiplicity = 3.502123
using 41914 rows, processing 13 parameters
Approx indep samples: 1649
...
case 2) with adjust priors  using the entire chi^2
Adjusting priors
outlier fraction 2.3858376E05
Number of chains used = 8
var(mean)/mean(var), 1/2 chains, worst evalue: R1 = 0.0290
Prior removed 17313 models
mean input multiplicity = 7.1085290E11
using 24601 rows, processing 13 parameters
effective number of samples (assuming indep): 499
...
case 3) with adjust priors  using the chi^2 divided by the number
of indep. data points
Adjusting priors
Number of chains used = 8
var(mean)/mean(var), 1/2 chains, worst evalue: R1 = 0.0093
mean input multiplicity = 0.8225374
using 41914 rows, processing 13 parameters
effective number of samples (assuming indep): 2053
...
My question is: may be one of the cases 2) and 3) OK from this point of
view or the solution is to use cosmomc to postprocess the chains or to
make new chains using both the datasets?
Thanks,
Petruta.

 Posts: 1592
 Joined: September 23 2004
 Affiliation: University of Sussex
 Contact:
Re: adjusting priors with getdist
chi^2 is log likelihood, so you should only add or subtract a constant, not multiply or divide.
The outlier fraction just gives some idea of large variations. It's usually obvious from looking at the plots whether importance sampling is working well or not.
The outlier fraction just gives some idea of large variations. It's usually obvious from looking at the plots whether importance sampling is working well or not.

 Posts: 3
 Joined: July 02 2008
 Affiliation: Institute for Space Science
 Contact:
adjusting priors with getdist
Thank you very much!
Petruta.
Petruta.