[astro-ph/0407028] Power spectrum estimation from high-resolution maps by Gibbs sampling

Authors:  H. K. Eriksen, I. J. O'Dwyer, J. B. Jewell, B. D. Wandelt, D. L. Larson, K. M. Gorski, S. Levin, A. J. Banday, P. B. Lilje
Abstract:  We revisit a recently introduced power spectrum estimation technique based on Gibbs sampling, with the goal of applying it to the high-resolution WMAP data. In order to facilitate this analysis, a number of sophistications have to be introduced, each of which is discussed in detail. We have implemented two independent versions of the algorithm to cross-check the computer codes, and to verify that a particular solution to any given problem does not affect the scientific results. We then apply these programs to simulated data with known properties at intermediate (N_side = 128) and high (N_side = 512) resolutions, to study effects such as incomplete sky coverage and white vs. correlated noise. From these simulations we also establish the Markov chain correlation length as a function of signal-to-noise ratio, and give a few comments on the properties of the correlation matrices involved. Parallelization issues are also discussed, with emphasis on real-world limitations imposed by current super-computer facilities. The scientific results from the analysis of the first-year WMAP data are presented in a companion letter.
[PDF]  [PS]  [BibTex]  [Bookmark]

Discussion related to specific recent arXiv papers
Post Reply
Antony Lewis
Posts: 1618
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

[astro-ph/0407028] Power spectrum estimation from high-resol

Post by Antony Lewis » March 11 2005

In this paper the authors compare two methods of handling foreground cuts:
  1. by setting the inverse noise to zero in the cut
  2. by including a foreground component with infinite prior variance in the cut and adding a new Gibbs step
The advantage of (2) over (1) is that the noise can be taken to be what you like over the cut, and in particular if it is close to the global value, the solution for the signal samples becomes much easier because the matrices become more diagonal.

Question: what did the authors actually choose for the noise N_cut in the cut?

It could be chosen to be the experimental noise, but to some extent this seems to be an arbitary choice (unless perhaps you are interested in learning about foregrounds). In the limit N_cut = 0, the Gibbs iterations never converge. In the limit N_cut is inifinite, (2) becomes the same as (1). Somewhere in between, there may be an optimal choice. Having N_cut the experimental noise (or average of the out-of-cut noise) probably optimises the matrix inversion, but it's not clear this is the best choice for overall performance if it makes for slow Gibbs convergence. So what should you choose for N_cut?

Ian ODwyer
Posts: 1
Joined: September 24 2004
Affiliation: Jet Propulsion Laboratory

[astro-ph/0407028] Power spectrum estimation from high-resol

Post by Ian ODwyer » March 12 2005

For the MAGIC implementation we used the infinite variance approach. We did not alter the noise matrix, using the same N as for the rest of the equations/sampling steps. The IVF approach was seen by us as an intermediate, crude but self-consistent step towards having full foreground modelling within the sampling process. Once we have more elaborate foreground models included, we are of course interested in the foreground components themselves in addition to just wanting to remove them from the data. For that we need to use the experimental N.

As I think Hans Krsitian says in the paper, we get faster convergence by carrying the IVF component compared to the COMMANDER implementation which uses a mask approach, but we pay for this with higher correlations between samples (e.g. less independent samples). I think COMMANDER now includes more foreground toys, including IVF, but I will leave that for Hans Kristian to comment on.

I guess I would not focus too much on the infinite variance
approach as this was a stepping stone to better things, and once you get to those better things your choice of noise matrix is much more limited. If your experiment has partial sky coverage IVF may have some applicability.

Cheers

Ian

Hans Kristian Eriksen
Posts: 60
Joined: September 25 2004
Affiliation: ITA, University of Oslo
Contact:

[astro-ph/0407028] Power spectrum estimation from high-resol

Post by Hans Kristian Eriksen » March 14 2005

Yes, it's correct that Commander now has the option of using the infinite variance foreground trick. However, in actual analyses I still use the mask approach, since I feel that I understand its correlation properties much better than I do for the IVF approach. I'm not sure how much the difference has to say in practical analyses, but I'm playing it safe for now, so to speak.

About the choice of RMS inside the cut for the IVF approach: I would expect that the optimal choice with respect to matrix inversion is indeed the instrument noise. As far as I understand the issue (but I may very well be wrong here), the crucial point is to avoid structure in the RMS pattern. A smooth extension of the high-latitude pattern seems to me to be the best option.

Suppose we had access to the WMAP maps pixelized in the ecliptic reference frame (and -- this is a hint to the WMAP team -- we should! :-)). Then the instrument noise matrix is (almost) block-diagonal in m, and the Oh et al. preconditioner should work superbly. (Actually, I have a feeling this is how they got convergence in six iterations in the first place.) It is difficult to imagine a simpler structure than this.

But, as Antony suggests, the correlation properties of this approach are less than trivial, and, personally, I prefer to use a "hard" mask for now.

Antony Lewis
Posts: 1618
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

[astro-ph/0407028] Power spectrum estimation from high-resol

Post by Antony Lewis » March 14 2005

Thanks for the replies.

I was thinking more of the incomplete sky case, where the cut correponds to absence of any data. But I certainly take the point that for WMAP/Planck it would potentially be interesting to use real foreground priors.

Good point about the noise being azimuthally symmetric in the ecliptic coordinates, and hence making the full-sky coupling matrix block diagonal in m (and hence easily computable and invertable).

This suggests another alternative: make the largest hard cut you can that is azimuthally symmetric. For example in WMAP you could put in a hard mask around the central part of the galactic plane, then mask out the remaining stuff by using the IVF method. (though of course the noise is unfortunately not aligned with the galactic). For something like Boomerang you could put in a hard circular cut around the observed region, and treat the remaning unobserved pixels inside the cut using IVF. But I don't have much intuition for how the Gibbs correlation scaling performs relative to the gain in having nearly azimuthal symmetry in the hard cut - and hence whether this helps?

Post Reply