CAMB & Twopoint correlation function: weird behaviour

 Posts: 7
 Joined: May 20 2008
 Affiliation: University of Sussex
CAMB & Twopoint correlation function: weird behaviour
Hi all!
I am trying to obtain the twopoint correlation function [tex]\xi (r)[/tex] from CAMB's output power spectrum using the following relation
[tex]\xi (r)=\frac{1}{2\pi ^2}\int_{0}^{\infty}k^2 P(k) \frac{\sin(k\cdot r)}{kr} dk[/tex].
I need to calculate [tex]\xi[/tex] up to [tex]r=10^6[/tex].
I used several methods to solve the integral, including the dftint routine from Numerical Recipes, Romberg quadrature and some simpler algorithms (e.g. trapezoidal method).
The result is always the same: I get an anticorrelation [tex]\xi(r) < 0[/tex] for [tex]r > 120h^{1}Mpc[/tex] and some very noisy behaviour when r becomes bigger than ~[tex]10^4h^{1}Mpc[/tex]. By noisy I mean that the correlation function goes quickly to zero (it has [tex]r^{4}[/tex] strength) but by doing so it oscillates wildly.
I first tought that the oscillation could be due to the finite extent of CAMB's P(k), so I extrapolated the power spectrum in this way:
[tex]P(k) \propto k[/tex] when [tex]k < 10^{6}[/tex]
[tex]P(k) \propto k^{3}[/tex] when [tex]k >500[/tex]
In this way I can do the integration analytically on the tails. The resulting [tex]\xi(r)[/tex] is a little less noisy, but the problem persists.
I will be grateful to anybody who could answer the following questions:
1) Am I doing some terrible mistake I am not aware of?
2) Does the anticorrelation at big r's physically make sense?
2) Most importantly, are you aware of an algorithm specifically designed to obtain the two point correlation function from a model power spectrum? (and hopefully eliminate the noisiness?)
Thank you very much.
Cheers,
Guido
P. S. I noticed that CAMB's power spectrum shows a [tex]k^{2.8\div3.2}[/tex] dependence at large k's. The exponent varies depending on k_max, i.e. I get 3.1 setting k_max=500 and 2.9 with k_max=10000. I realize that going to such scales is a physical nonsense; nonetheless, shouldn't P(k) show a [tex]k^{3}[/tex] dependency at large k's?
I am trying to obtain the twopoint correlation function [tex]\xi (r)[/tex] from CAMB's output power spectrum using the following relation
[tex]\xi (r)=\frac{1}{2\pi ^2}\int_{0}^{\infty}k^2 P(k) \frac{\sin(k\cdot r)}{kr} dk[/tex].
I need to calculate [tex]\xi[/tex] up to [tex]r=10^6[/tex].
I used several methods to solve the integral, including the dftint routine from Numerical Recipes, Romberg quadrature and some simpler algorithms (e.g. trapezoidal method).
The result is always the same: I get an anticorrelation [tex]\xi(r) < 0[/tex] for [tex]r > 120h^{1}Mpc[/tex] and some very noisy behaviour when r becomes bigger than ~[tex]10^4h^{1}Mpc[/tex]. By noisy I mean that the correlation function goes quickly to zero (it has [tex]r^{4}[/tex] strength) but by doing so it oscillates wildly.
I first tought that the oscillation could be due to the finite extent of CAMB's P(k), so I extrapolated the power spectrum in this way:
[tex]P(k) \propto k[/tex] when [tex]k < 10^{6}[/tex]
[tex]P(k) \propto k^{3}[/tex] when [tex]k >500[/tex]
In this way I can do the integration analytically on the tails. The resulting [tex]\xi(r)[/tex] is a little less noisy, but the problem persists.
I will be grateful to anybody who could answer the following questions:
1) Am I doing some terrible mistake I am not aware of?
2) Does the anticorrelation at big r's physically make sense?
2) Most importantly, are you aware of an algorithm specifically designed to obtain the two point correlation function from a model power spectrum? (and hopefully eliminate the noisiness?)
Thank you very much.
Cheers,
Guido
P. S. I noticed that CAMB's power spectrum shows a [tex]k^{2.8\div3.2}[/tex] dependence at large k's. The exponent varies depending on k_max, i.e. I get 3.1 setting k_max=500 and 2.9 with k_max=10000. I realize that going to such scales is a physical nonsense; nonetheless, shouldn't P(k) show a [tex]k^{3}[/tex] dependency at large k's?

 Posts: 16
 Joined: November 06 2004
 Affiliation: CITA
 Contact:
CAMB & Twopoint correlation function: weird behaviour
Andrew Hamilton's FFTLog code is generally good for this kind of thing.

 Posts: 1498
 Joined: September 23 2004
 Affiliation: University of Sussex
 Contact:
Re: CAMB & Twopoint correlation function: weird behavio
CAMB includes baryons, so you see the effect of them on very small scales (though not at high precision).
For direct evaluation (not tried FFTLog, but sounds like a good idea..):
In the tail when the sine is oscillating fast compared to variations in P you can also do a series exapansion of P, then integrate analytically over the oscillations to give a result in terms of the derivatives of P that is much more numerically stable. Other tricks include writing the integral in terms of a sine offset by half a phase, then integrating the sum of that and the original, so the integrand is now something small.
For direct evaluation (not tried FFTLog, but sounds like a good idea..):
In the tail when the sine is oscillating fast compared to variations in P you can also do a series exapansion of P, then integrate analytically over the oscillations to give a result in terms of the derivatives of P that is much more numerically stable. Other tricks include writing the integral in terms of a sine offset by half a phase, then integrating the sum of that and the original, so the integrand is now something small.

 Posts: 7
 Joined: May 20 2008
 Affiliation: University of Sussex
CAMB & Twopoint correlation function: weird behaviour
Hi Patrick, Antony,
thank you very much for the quick answers. I will check FFTLog out and post my results.
I have a question concerning the first trick you suggested me, Antony.
Is it to some extent equivalent to do a spline interpolation of P(k), extracts the coefficients of the piecewise polynomial, and then sum the contributes of the now analitycally integrable function over all the intervals?
Schematically:
CAMB [tex]\longrightarrow[/tex] [tex]P(k_i)[/tex], i=1, 2, ..., N [tex]\longrightarrow[/tex] Spline interpolation
[tex]\longrightarrow[/tex] I have got N1 polynomials it this form: [tex]P_i(k)[/tex] with k belonging to [tex][k_{i}, \, k_{i+1}][/tex] [tex]\longrightarrow[/tex] analytical integration of [tex]\sum_{i=0}^{N1}\int_{k_i}^{k_{i+1}}k^2P(k)\frac{\sin{kr}}{kr}dk[/tex].
The results are the same as when using Romberg quadrature or dftint, but somewhat quicker. However, the "ringing" effect is still there.
Cheers,
Guido
thank you very much for the quick answers. I will check FFTLog out and post my results.
I have a question concerning the first trick you suggested me, Antony.
Is it to some extent equivalent to do a spline interpolation of P(k), extracts the coefficients of the piecewise polynomial, and then sum the contributes of the now analitycally integrable function over all the intervals?
Schematically:
CAMB [tex]\longrightarrow[/tex] [tex]P(k_i)[/tex], i=1, 2, ..., N [tex]\longrightarrow[/tex] Spline interpolation
[tex]\longrightarrow[/tex] I have got N1 polynomials it this form: [tex]P_i(k)[/tex] with k belonging to [tex][k_{i}, \, k_{i+1}][/tex] [tex]\longrightarrow[/tex] analytical integration of [tex]\sum_{i=0}^{N1}\int_{k_i}^{k_{i+1}}k^2P(k)\frac{\sin{kr}}{kr}dk[/tex].
The results are the same as when using Romberg quadrature or dftint, but somewhat quicker. However, the "ringing" effect is still there.
Cheers,
Guido

 Posts: 183
 Joined: September 24 2004
 Affiliation: Brookhaven National Laboratory
 Contact:
CAMB & Twopoint correlation function: weird behaviour
Correlation funtion is subject to integral constraint, so if you want to have positive correlation at small scales, then you MUST have negative correlation at large scales...
The problems you're seeing are connected with oscillating integrand as other people have suggested. One simple solution is to use NAG's D01AUF which can integrate oscillating functions by using magic.
The problems you're seeing are connected with oscillating integrand as other people have suggested. One simple solution is to use NAG's D01AUF which can integrate oscillating functions by using magic.

 Posts: 7
 Joined: May 20 2008
 Affiliation: University of Sussex
CAMB & Twopoint correlation function: weird behaviour
Hi all!
I tried the FFTLog algorithm. It is very fast, but the obtained correlation function shows an even more noisy behaviour for r>10^4 h^1 Mpc. In the attached plots you can see it very clearly. The three curves represent three different methods I adopted: Hamilton's fftlog, Numerical Recipes' dftint and the splines semianalytical approach I explained in my last post in this board. The first graph spans a range from r=10^2 to r=10^4 h^1 Mpc, while the second one is the zoom between 10^3 and 10^4 . I rescaled the curves to make them fit in one graph; being a loglog graph, I had to plot xi * (1) for r>120 h^1 Mpc (the curves change color at that point).
Is this oscillating behaviour at small scales entirely due to the baryons?
@Anze:
Thank you very much for your suggestion. Alas I have no access to the NAG library; instead I used the dftint routine for Fourier sine integrals from Numerical Recipes. By the way, why is the correlation function subject to integral constraint? Is it some sort of normalization?
Thank you all for your attention.
Cheers,
Guido
I tried the FFTLog algorithm. It is very fast, but the obtained correlation function shows an even more noisy behaviour for r>10^4 h^1 Mpc. In the attached plots you can see it very clearly. The three curves represent three different methods I adopted: Hamilton's fftlog, Numerical Recipes' dftint and the splines semianalytical approach I explained in my last post in this board. The first graph spans a range from r=10^2 to r=10^4 h^1 Mpc, while the second one is the zoom between 10^3 and 10^4 . I rescaled the curves to make them fit in one graph; being a loglog graph, I had to plot xi * (1) for r>120 h^1 Mpc (the curves change color at that point).
Is this oscillating behaviour at small scales entirely due to the baryons?
@Anze:
Thank you very much for your suggestion. Alas I have no access to the NAG library; instead I used the dftint routine for Fourier sine integrals from Numerical Recipes. By the way, why is the correlation function subject to integral constraint? Is it some sort of normalization?
Thank you all for your attention.
Cheers,
Guido

 Posts: 2
 Joined: December 11 2008
 Affiliation: The Ohio State University
CAMB & Twopoint correlation function: weird behaviour
Hi Guido,
The noisiness at large r is likely from evaluating the integrand at very high k values where P(k) may not be evaluated perfectly (e.g. k = 50 h /Mpc, well beyond the usual range of interest for cosmologists). The solution is to add an exponential damping factor in your integrand, exp(k*lambda0), where lambda0 is some constant, say 0.1 Mpc/h. See section 42 of Peebles Large Scale Structure to see what I mean. The only consequence of introducing this cutoff is that xi (r) will be wrong on scales smaller than lambda0. And, as I said, this should make the oscillations go away. Also see section 42 for a discussion of how zero crossings are an expected feature in xi (r).
Chris
The noisiness at large r is likely from evaluating the integrand at very high k values where P(k) may not be evaluated perfectly (e.g. k = 50 h /Mpc, well beyond the usual range of interest for cosmologists). The solution is to add an exponential damping factor in your integrand, exp(k*lambda0), where lambda0 is some constant, say 0.1 Mpc/h. See section 42 of Peebles Large Scale Structure to see what I mean. The only consequence of introducing this cutoff is that xi (r) will be wrong on scales smaller than lambda0. And, as I said, this should make the oscillations go away. Also see section 42 for a discussion of how zero crossings are an expected feature in xi (r).
Chris

 Posts: 7
 Joined: May 20 2008
 Affiliation: University of Sussex
CAMB & Twopoint correlation function: weird behaviour
Thank you very much for your suggestion and for the reference, Chris. I had a look at section 42 of Peebles' book and I read the zerocrossing part. I also found out that the [tex] r^{4} [/tex] behaviour is expected.
I implemented the damping exponential function in my code and I obtained a great improvement. The noisiness is still there, but now it starts at [tex] r_n \simeq 10^8 \text{Mpc}/h [/tex] instead of [tex] r_n \simeq 10^4 \text{Mpc}/h [/tex]. As damping scales I used 10, 100, 1000 Mpc/h (my input P(k) file arrives up to 15k) but, apart from a slightly modified low [tex] r [/tex] behaviour, the output does not change much.
However, I noted that for [tex] r > r_n [/tex] the correlation function shows a [tex] r^{2} [/tex] dependence instead of the predicted [tex] r^{4} [/tex]. This is easily seen in the attached figure, where I plotted the fit to the noisy part as well. I am convinced that this behaviour is a numerical artifact, possibly ringing http://en.wikipedia.org/wiki/Ringing_artifact), but I do not know how to remove it. Do you have any idea?
Cheers,
Guido
I implemented the damping exponential function in my code and I obtained a great improvement. The noisiness is still there, but now it starts at [tex] r_n \simeq 10^8 \text{Mpc}/h [/tex] instead of [tex] r_n \simeq 10^4 \text{Mpc}/h [/tex]. As damping scales I used 10, 100, 1000 Mpc/h (my input P(k) file arrives up to 15k) but, apart from a slightly modified low [tex] r [/tex] behaviour, the output does not change much.
However, I noted that for [tex] r > r_n [/tex] the correlation function shows a [tex] r^{2} [/tex] dependence instead of the predicted [tex] r^{4} [/tex]. This is easily seen in the attached figure, where I plotted the fit to the noisy part as well. I am convinced that this behaviour is a numerical artifact, possibly ringing http://en.wikipedia.org/wiki/Ringing_artifact), but I do not know how to remove it. Do you have any idea?
Cheers,
Guido

 Posts: 2
 Joined: December 11 2008
 Affiliation: The Ohio State University
CAMB & Twopoint correlation function: weird behaviour
I need to think about it some more to explain the r^2 behavior in detail. If you really need it to be smooth out there, my only advice is to try different lambda0 values (usually the smaller the better) and to try double precision instead of single precision (i.e. "float" in C programming) if those plots were constructed with single precision. However I'm surprised that you need to calculate xi (r) out that far. That's well beyond the range probed with Large Scale structure surveys and (I think) beyond the range one would need to compare with CMB observations. Do you really need it to be accurate all the way out to 10^5 Gpc?

 Posts: 7
 Joined: May 20 2008
 Affiliation: University of Sussex
CAMB & Twopoint correlation function: weird behaviour
Hi Chris,
you are absolutely right, it does not make much physical sense to go at such high separations. However, I need my correlation function to satisfy the integral constraint [tex] \int_0^\infty r^2 \xi (r) = 0 [/tex] as much as possible, since any residual makes the successive step in my project less precise. This is achieved only by having a very high upper integration limit, i.e. a very large [tex]r_{\text{max}}[/tex]. I could solve my problem by just extrapolating [tex] \xi(r) [/tex] with a r^4 curve, but I would rather prefer avoiding this.
By the way, I am already using double precision. Actually, I implemented the MPFR library in my code, so that I can attain 128 bit precision, but alas with no difference at all.
A last remark: the [tex] r^{2} [/tex] dependence of the correlation function at large r is not peculiar to the exponential damping technique you suggested me since I can reproduce it with a undumped input power spectrum as well. The only difference is that with a plain P(k) the [tex] r^{2} [/tex] behaviour starts at lower r's. Moreover, it seems that the starting point of the [tex] r^{2} [/tex] dependence, [tex] r_n [/tex], is inversely proportional to [tex] k_\text{max} [/tex]: the more extended the power spectrum, the less noisy the correlation function.
Thank you again for your help!
Cheers,
Guido
EDIT: I ran my code with [tex] \lambda_0 = 1 [/tex] and [tex] 0.1 [/tex], thus obtaining the same result: the [tex] r^{2} [/tex] behaviour is still there. In this way the damping is so large that at large k's the damped P(k) is 0 within machine precision.
you are absolutely right, it does not make much physical sense to go at such high separations. However, I need my correlation function to satisfy the integral constraint [tex] \int_0^\infty r^2 \xi (r) = 0 [/tex] as much as possible, since any residual makes the successive step in my project less precise. This is achieved only by having a very high upper integration limit, i.e. a very large [tex]r_{\text{max}}[/tex]. I could solve my problem by just extrapolating [tex] \xi(r) [/tex] with a r^4 curve, but I would rather prefer avoiding this.
By the way, I am already using double precision. Actually, I implemented the MPFR library in my code, so that I can attain 128 bit precision, but alas with no difference at all.
A last remark: the [tex] r^{2} [/tex] dependence of the correlation function at large r is not peculiar to the exponential damping technique you suggested me since I can reproduce it with a undumped input power spectrum as well. The only difference is that with a plain P(k) the [tex] r^{2} [/tex] behaviour starts at lower r's. Moreover, it seems that the starting point of the [tex] r^{2} [/tex] dependence, [tex] r_n [/tex], is inversely proportional to [tex] k_\text{max} [/tex]: the more extended the power spectrum, the less noisy the correlation function.
Thank you again for your help!
Cheers,
Guido
EDIT: I ran my code with [tex] \lambda_0 = 1 [/tex] and [tex] 0.1 [/tex], thus obtaining the same result: the [tex] r^{2} [/tex] behaviour is still there. In this way the damping is so large that at large k's the damped P(k) is 0 within machine precision.