misleading efficiency statistics in -bayesmh-?

[skolenik]

I am reproducing Example 25 of [BAYES] bayesmh:

 
webuse pig
fvset base none id
set seed 14
bayesmh weight i.id i.id#c.week, likelihood(normal({var_0})) noconstant prior({weight:i.id i.id#c.week}, mvnormal(2, {weight:_cons}, {weight:week}, {Sigma,m})) prior({weight:week _cons}, normal(0, 1e2)) prior({var_0}, igamma(0.001,0.001)) prior({Sigma,m}, iwishart(2,3,I(2))) block({var_0}, gibbs) block({Sigma,m}, gibbs) block({weight:_cons}) block({weight:week}) block({weight:i.id}, reffects) block({weight:i.id#c.week}, reffects) noshow({weight:i.id i.id#c.week}) mcmcsize(5000) dots

As a part of output, I get the summary statistics of efficiency as follows:

                                                 Efficiency:  min =     .07112
                                                              avg =      .1423
Log marginal-likelihood = -926.22043                          max =      .2238

Checks out against the manual, which is good. Now, let me turn off the noshow() option to see all parameters at once:

set seed 14  
bayesmh weight i.id i.id#c.week, likelihood(normal({var_0})) noconstant prior({weight:i.id i.id#c.week}, mvnormal(2, {weight:_cons}, {weight:week}, {Sigma,m})) prior({weight:week _cons}, normal(0, 1e2)) prior({var_0}, igamma(0.001,0.001)) prior({Sigma,m}, iwishart(2,3,I(2))) block({var_0}, gibbs) block({Sigma,m}, gibbs) block({weight:_cons}) block({weight:week}) block({weight:i.id}, reffects) block({weight:i.id#c.week}, reffects) mcmcsize(5000) dots

Hm. The average efficiency falls by a factor of about five:

                                                 Efficiency:  min =     .00895
                                                              avg =     .02973
Log marginal-likelihood = -926.22043                          max =      .2238

So I am not very convinced when the manual goes on to say "The average sampling efficiency is about 14% with no indications for convergence problems." Effective sample sizes are worse for the random effects (as expected, of course; it is a complicated statistic to deal with). Maybe a more honest language would be, "For the parameters presented in the table, the average sampling efficiency is about 14%. The omitted parameters (random effects) have lower efficiency. If the interest is in the random effects, e.g. for prediction purposes, more thorough sampling could be recommended."

[Leonardo Guizzetti]

I tend to agree with your proposed language modification. Indeed, the efficiency calculation appears to be based only on the presented parameters in the table. To see the omitted random effect parameters for example, you can run -bayesstats ess _all- after the code of example 25, and that will reproduce the min/avg efficiency stats when explicitly showing the random effects.