Cosmomc: importance sampling

Use of Healpix, camb, CLASS, cosmomc, compilers, etc.
Shu-Rong Chen
Posts: 7
Joined: August 12 2015
Affiliation: National Taiwan University

Cosmomc: importance sampling

Hello,
I'm using CosmoMC as a generic sampler.
I want to apply the constraint of one parameter derived from another MCMC analysis to further importance sampling the new data.

I adopt the following steps to use functions in GetDist to do importance sampling:
1.) use get1DDensity() and Prob() to get the probability distribution in another MCMC analysis
to do importance sampling

Could you let me know whether it's the correct procedure to use GetDist for importance sampling?
If not, could you tell me which part of the CosmoMC code I should modify to load a .data file and recalculate the weight and likelihood to do importance sampling (action=1)?

I really appreciate your time and help.

Antony Lewis
Posts: 1444
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: Cosmomc: importance sampling

You can do something like this (where "samples" is an MCSamples instance)

Code: Select all


#Example of how to do importance sampling, modifying the samples by re-weighting by a new likelihood
#e.g. to modify samples to be from the original distribution times a Gaussian in x1 &#40;centered on 1, with sigma=1.2&#41;

import copy
new_samples = copy.deepcopy&#40;samples&#41; #make a copy so don't change the original
p=samples.getParams&#40;&#41;
new_loglike = &#40;p.x1-1&#41;**2/1.2**2/2
new_samples.loglikes = np.zeros&#40;samples.numrows&#41; #code currently assumes existing loglikes are set, set to zero here
new_samples.reweightAddingLogLikes&#40;new_loglike&#41; #re-weight cut_samples to account for the new likelihood
You don't normally need to modify CosmoMC to use it with action=1.

Shu-Rong Chen
Posts: 7
Joined: August 12 2015
Affiliation: National Taiwan University

Cosmomc: importance sampling

Since I want to use the constraint from a previous MCMC run,
I was wondering if the following procedure is correct.

("samples_prior" is a MCSamples instance in a previous MCMC run)

Code: Select all

new_loglike = np.zeros&#40; samples.numrows &#41;
prior1 = samples_prior.get1DDensity&#40; 'x1' &#41;
for i in range&#40; 0 , samples1.numrows , 1 &#41;&#58;
new_loglike&#91;i&#93; = -math.log&#40; prior1.Prob&#40;p.mb&#91;i&#93;&#41; &#41;
it's better to thin the chains further in order to keep nearly
as "samples" (please refer to the code above)?

Thanks again for your great help!

Antony Lewis
Posts: 1444
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: Cosmomc: importance sampling

You don't need to thin.

However the importance sampling should be done in the full parameter space, not using one-dimensional marginalized distributions - i.e. your suggestion would be correct only if other likelihood only depends on x1 (and other parameters independent of the current likelihood). Using action=1 cosmomc run to re-weight all points by a another likelihood if needed.

Shu-Rong Chen
Posts: 7
Joined: August 12 2015
Affiliation: National Taiwan University

Cosmomc: importance sampling

Could you tell me which part of the CosmoMC code I should modify in order to use "action=1" to load another likelihood for re-weighting all points?

Thank you so much for the great help!

Antony Lewis
Posts: 1444
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: Cosmomc: importance sampling

Why do you need to modify it?

Shu-Rong Chen
Posts: 7
Joined: August 12 2015
Affiliation: National Taiwan University

Cosmomc: importance sampling

Sorry for the confusion. I'm a bit puzzled because I'm not using any
built-in likelihood installed in the batch2/ directory.

I understand that, for example, if I want to use the built-in BAO
constraint, I can add â€œDEFAULT(batch2/BAO.ini)â€ to the parameter file
"params_generic.ini". Then the corresponding datasets (MCMC chains)
will be loaded to re-weight all data points.

However, since I want to load a likelihood generated by my own
previous CosmoMC run, I was wondering whether I need to add something
to "params_generic.ini" and/or modify any CosmoMC code in order to
re-weight data accordingly.

Again, I really appreciate your time and kind help!

Antony Lewis
Posts: 1444
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: Cosmomc: importance sampling

Generally speaking, you cannot get a reliable likelihood from a previous MC run. But if you really want to do that (e.g. using a low-dimensional KDE fit), you'd have to make your own new likelihood module.