cosmomc: problems using as generic sampler
Posted: February 19 2009
I'm trying to use cosmomc as a generic sampler. I link in my own likelihood function, which calls my own simulator (not CMB-related) and does a simple \chi^2 comparison to several datasets. I've got problems getting it to work on simple test cases.
My first problem may be a stupid question of understanding. In the simplest test with a simulated test dataset (using only 19 data points and six paramters), it returns a model which appears perfectly reasonable in a "chi-by-eye" sense. The \chi^2 is better than that of the true, input model (a reduced \chi^2 of 0.5 compared to 1.2). I think that's to be expected, since it's fitting the noise a little bit. Both models look like they should fit the data well enough. The problem is that the widths of the marginalized distributions returned by getdist imply that three of the six parameters are more than 2 sigma away from their correct values. What does this mean? Am I simply misunderstanding what those widths ought to mean, or does this mean there is a problem in the likelihood my function is returning? It's simply returning 0.5*\chi^2.
There are correlations in the parameters. The largest value in the .corr file is 0.85, and that is between two of the three problematic paramters, and the next most correlated is that with the third problem param. But even the most correlated 2D distributions show nicely isolated confidence regions well away from the correct values. Is this the behaviour you would expect to result from the correlations in the parameters? I thought that would be reflected in the widths, i.e. that the widths would reflect the uncertainty because of the correlation. I still wouldn't have expected the correct value to lie well outside the 68% contour.
The second problem comes when I add another dataset (512 points) which should break those degeneracies. From what I can see of the values in the resulting chain files, things look reasonable. But getdist barfs immediately saying
MKL ERROR : Parameter 5 was incorrect on entry to SSYEV
Error in Matrix_Diagonalize
Debugging fortran is not my favorite exercise in the world. Any suggestions for what to look at in the data?
Any help and advice would be appreciated!
My first problem may be a stupid question of understanding. In the simplest test with a simulated test dataset (using only 19 data points and six paramters), it returns a model which appears perfectly reasonable in a "chi-by-eye" sense. The \chi^2 is better than that of the true, input model (a reduced \chi^2 of 0.5 compared to 1.2). I think that's to be expected, since it's fitting the noise a little bit. Both models look like they should fit the data well enough. The problem is that the widths of the marginalized distributions returned by getdist imply that three of the six parameters are more than 2 sigma away from their correct values. What does this mean? Am I simply misunderstanding what those widths ought to mean, or does this mean there is a problem in the likelihood my function is returning? It's simply returning 0.5*\chi^2.
There are correlations in the parameters. The largest value in the .corr file is 0.85, and that is between two of the three problematic paramters, and the next most correlated is that with the third problem param. But even the most correlated 2D distributions show nicely isolated confidence regions well away from the correct values. Is this the behaviour you would expect to result from the correlations in the parameters? I thought that would be reflected in the widths, i.e. that the widths would reflect the uncertainty because of the correlation. I still wouldn't have expected the correct value to lie well outside the 68% contour.
The second problem comes when I add another dataset (512 points) which should break those degeneracies. From what I can see of the values in the resulting chain files, things look reasonable. But getdist barfs immediately saying
MKL ERROR : Parameter 5 was incorrect on entry to SSYEV
Error in Matrix_Diagonalize
Debugging fortran is not my favorite exercise in the world. Any suggestions for what to look at in the data?
Any help and advice would be appreciated!