Proposal Covariance Matrix not positive definite - generic sampler case

Use of Cobaya. camb, CLASS, cosmomc, compilers, etc.
Post Reply
Teeraparb Chantavat2
Posts: 12
Joined: May 03 2021
Affiliation: Naresuan University

Proposal Covariance Matrix not positive definite - generic sampler case

Post by Teeraparb Chantavat2 » May 16 2022

Dear All,

I am running CosmoMC as a generic MCMC sampler. However, the proposal covariance matrix is not positive definite. I noticed that the metropolis acceptance ratio is rather small so I tried adjusting the min, max and the widths of the parameters but no success thus far. Here is the output from CosmoMC which I also print the Covariance matrix before and after the sampling.

Code: Select all

 Number of MPI processes:           1
 file_root:test
 Random seeds: 16227,  7099 rand_inst:   1
 Fast divided into            1  blocks
  5 parameters
 Printing the Covariance Matrix
   1.0000000000000002E-002   0.0000000000000000        0.0000000000000000        0.0000000000000000        0.0000000000000000     
   0.0000000000000000        1.4400000000000000E-002   0.0000000000000000        0.0000000000000000        0.0000000000000000     
   0.0000000000000000        0.0000000000000000        1.0000000000000000E-004   0.0000000000000000        0.0000000000000000     
   0.0000000000000000        0.0000000000000000        0.0000000000000000        4.0000000000000000        0.0000000000000000     
   0.0000000000000000        0.0000000000000000        0.0000000000000000        0.0000000000000000        2.5000000000000005E-003
 starting Monte-Carlo
           1  metropolis accept. ratio:   1.00000000    
           2  metropolis accept. ratio:   1.00000000    
          11  metropolis accept. ratio:  0.272727281    
          12  metropolis accept. ratio:  0.333333343    
          17  metropolis accept. ratio:  0.294117659    
          19  metropolis accept. ratio:  0.315789461    
          34  metropolis accept. ratio:  0.205882356    
          90  metropolis accept. ratio:   8.88888910E-02
         ....
         ....
         ....
       29852  metropolis accept. ratio:   1.87592127E-03
       29900  metropolis accept. ratio:   1.90635456E-03
       30035  metropolis accept. ratio:   1.93108036E-03
       30693  metropolis accept. ratio:   1.92226237E-03
 Chain1, MPI done 'burn', Samples =30693, like =    3164.91260    
 Time:    29.567820000000001      output lines=          57
 MPI_Min_Sample_Update          70       30693
           1 all_burn done
           1 DoUpdates
 Chain           1  MPI communicating
 updating proposal density
 Printing the Covariance Matrix
   1.1093356479670479E-031  -4.0675640425458421E-031  -5.5466782398352393E-032  -2.2186712959340957E-031  -5.5466782398352393E-032
  -4.0675640425458421E-031   1.4914401489334754E-030   2.0337820212729211E-031   8.1351280850916842E-031   2.0337820212729211E-031
  -5.5466782398352393E-032   2.0337820212729211E-031   2.7733391199176196E-032   1.1093356479670479E-031   2.7733391199176196E-032
  -2.2186712959340957E-031   8.1351280850916842E-031   1.1093356479670479E-031   4.4373425918681914E-031   1.1093356479670479E-031
  -5.5466782398352393E-032   2.0337820212729211E-031   2.7733391199176196E-032   1.1093356479670479E-031   2.7733391199176196E-032
 Matrix_Cholesky: not positive definite 2
 MpiStop:            0
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 128.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------

So my questions are
1) If the problem is from the prior - should I increase/decrease prior range and/or increase/decrease proposal width?
2) If the problem is from something else, please suggest.

Best,
Teera

Antony Lewis
Posts: 1842
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: Proposal Covariance Matrix not positive definite - generic sampler case

Post by Antony Lewis » May 27 2022

I would look at the chain files, or plot the 1D plots, and see which parameters are not moving down the chain. For those parameters you need to decrease the propose width.

Normally you also shouldn't just have fast parameters.

Teeraparb Chantavat2
Posts: 12
Joined: May 03 2021
Affiliation: Naresuan University

Re: Proposal Covariance Matrix not positive definite - generic sampler case

Post by Teeraparb Chantavat2 » May 30 2022

Thanks Anthony,

I am using CosmoMC for a generic sampling not for CAMB. Is it possible to set fast-slow parameters for a generic sampling case?

Antony Lewis
Posts: 1842
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: Proposal Covariance Matrix not positive definite - generic sampler case

Post by Antony Lewis » May 31 2022

Yes, but doesn't make sense unless you have to likelihoods with very different speeds.

Teeraparb Chantavat2
Posts: 12
Joined: May 03 2021
Affiliation: Naresuan University

Re: Proposal Covariance Matrix not positive definite - generic sampler case

Post by Teeraparb Chantavat2 » June 01 2022

Hi Anthony,

A few questions remain:

1) I checked the program even though it said five fast parameters, it actually ran GetProposalSlow routine for all parameters instead. It seems counter-intuitive that all the fast parameters go through this routine. Is there any difference between the routine GetProposalSlow and GetProposalFast?

2) How do I set a set of fast parameters and a set of slow parameters in the initialization file if I want to exploit fast-slow sampling method.

Many Thanks,
Teera

Antony Lewis
Posts: 1842
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: Proposal Covariance Matrix not positive definite - generic sampler case

Post by Antony Lewis » June 01 2022

Actually maybe it automatically converts fast to slow when there are only fast (a while since I wrote this!)

Teeraparb Chantavat2
Posts: 12
Joined: May 03 2021
Affiliation: Naresuan University

Re: Proposal Covariance Matrix not positive definite - generic sampler case

Post by Teeraparb Chantavat2 » June 03 2022

Hi Anthony,

I found out the reason why the covariance matrix is not positive definite. It is because the parameter vectors that are used to calculate the covariance matrix are highly degenerate (a lot of the same parameter points - Of all 63 rows, there are about 3 - 4 independent parameter points.)

I independently check the MCMC chains using my python code with the same data and model. It initially take a lot of samplings to move as well. So there is an independent confirmation that this set of data need a lot more burn-in time than usual.

So I would like to ask

1) Is there any parameter setting in ini file that could help me with this issue? i.e. increase more burn-in time or get rid of duplicated parameter points before calculating the covariance matrix?

2) If not I will have to modify the code a little, would you recommend increasing burn-in time or get rid of duplicated parameters points?

Best,
Teera

Antony Lewis
Posts: 1842
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: Proposal Covariance Matrix not positive definite - generic sampler case

Post by Antony Lewis » June 05 2022

Decrease the propose width/check your starting covmat.

Post Reply