[Danielle Laurie]

Hi Clyde,

Thank you so much this works perfectly! Much appreciated!

And fortunately with your help, I was able to detect the error: I was using an additional variable without noticing (I have over 55 regressors, so I accidentally included x22 along with x2). Now it runs perfectly. Thank you and sorry for all the complication. Now both codes work.

I bring two followup questions, hopefully much simpler:

1. Is there any way I can get sig_05 to show more than one decimal place? I've tried command dp(#) but it doesn't seem to work.

2. The average coefficients over the 28 models is, I assume, an equally weighted average. What if I want to weight each regression by its log-likelihood (stored in e(ll) I think). Is there a way to do this?

Thank you so much and sorry for all the trouble.
Danielle

[Clyde Schechter]

1. You can apply whatever display format you like to sig_05 once you are past the -collapse- command. In the code, as I wrote it, I didn't specify anything and Stata defaulted it to %8.0g--which when I ran it with your example data gives sig_05 to 5 or 6 decimal places. It goes down to 0 decimal places when sig_05 is 1, and to a single decimal place when sig_05 is 0.5. If you want two decimal places in all cases, -format sig05 %3.2f- anywhere after -collapse- and before -list- will do that.

2. Um, -regress- does not have a log likelihood. Even if you have modified the code to use with a different regression command, like -probit-, that does have one, I don't think weighting by e(ll) would make sense. I say that because of the way that software programs estimate maximum likelihood. Likelihood functions, correctly formulated, often contain multiplicative constants. Those multiplicative constants do not affect the values of the parameters that maximize the likelihood. So to improve efficiency, such constants are often stripped out of the calculation in the maximization process. That has the effect of removing an additive constant from the log-likelihood. So the "log-likelihood" results you see may not really be the log of the maximized likelihood--they may be that modified by some additive constant. For nearly all purposes, this is not a problem, because when log-likelihoods are used subsequently it is usually to compare the difference in log likelihoods between two models (as, for example, when doing a likelihood ratio test, or a comparing AIC or BIC between two models). Since the same additive constant has been stripped out in each case, the difference is the same as the difference between the real log-likelihoods. But the ratio of two of these "log-likelihoods" is not going to be correct. And when you are using weights, it is the ratios of the various weights that matter.

[Rich Goldstein]

regarding the beginning of Clyde Schechter 's #2, yes, -regress- does have and returns a log likelihood; after estimating -regress-, type -ereturn li- and you will see it in "e(ll)"

[Clyde Schechter]

Interesting! I wasn't aware of that. I knew that least squares estimation is equivalent to maximum likelihood, and a likelihood can be calculated, but I didn't realize Stata actually did that. Thanks for correcting that.

[Rich Goldstein]

you're welcome - and note that within Stata there are 2 easy checks for things like this: in addition to the above check the post-estimation commands and if -estat ic- is available (as it is after -regress-), then the log-likelihood must be available also

[Danielle Laurie]

Thank you both for your answers.

Clyde, your argument is very sensible and makes a lot of sense. Thank you once again. But if I wanted to weight the regressions (with any weight at all, I don't have one yet, I am just experimenting with the data, I'm still a newbie and I think this is a good way to learn), how could I code that?

Also, if I wanted to run this for other models (I'm attempting several regressions, namely probit and logit regressions, lets say I have variable z, binary like you did before), is there a way I could obtain the average marginal effect like I obtain the coefficients? I've attempted what you suggested in 1325756-saving-estimates-from-margins post #6, but I'm probably doing this wrong, I get:

forvalues i = 1/`ntuples' {
        display `"`tuple`i''"'
        quietly probit z x1 x2 x3 `tuple`i'', vce(robust)
        margins, dydx(*)
        matrix B = r(b)
        gen dydx = .
        replace dydx = B[1, `v']
}

(4,806 missing values generated)
invalid syntax
r(198);

I was using this just before the postclose handle line.

Thank you,
Danielle

[Clyde Schechter]

The following code, which is substantially revised from before, even if the overall appearance is quite similar, will also give you the mean of the average marginal effect. Now, for linear regression, that will always be the same as the average coefficient. But for non-linear models it will not.

* Example generated by -dataex-. To install: ssc install dataex
clear
input int date double(roa1 roa2 pp1 pp2 pp3 x1 x3) byte x2 double(pp4 pp5 pp6)
16110     . 16.39 2.2739999999999996 2.02 2.1119999999999997              2.169 4041 0 -.016000000000000014 . 19.666666666666668
16111     . 15.94                  .    .                  .                  . 4041 0  -.01499999999999968 . 19.666666666666668
16112     . 15.39              2.222 2.06              2.055              2.116 4041 0  -.01499999999999968 . 19.666666666666668
16113     . 15.31 2.3930000000000007 1.89              2.229 2.2880000000000003 4041 0  -.01499999999999968 . 19.666666666666668
16114 19.02 15.58              2.244 2.02 2.0829999999999997 2.1430000000000002 4041 0 -.016000000000000014 . 19.666666666666668
16117 18.37     .              2.256 2.02              2.089 2.1510000000000002 4041 0  -.01499999999999968 . 19.666666666666668
16118 18.02  15.4 2.2260000000000004 2.02              2.063 2.1220000000000003 4041 0 -.016000000000000014 . 19.666666666666668
16119 17.94 15.59              2.227 2.02               2.06              2.124 4041 0 -.016000000000000014 . 19.666666666666668
16120 17.13  15.8              2.259 2.02              2.102              2.166 4041 0  -.01499999999999968 . 19.666666666666668
16121 17.96 16.04 2.2560000000000002 2.04 2.0949999999999998 2.1609999999999996 4041 0 -.016000000000000014 . 19.666666666666668
16124 18.47 16.29              2.237 2.05 2.0780000000000003              2.144 4041 0 -.015000000000000568 . 19.666666666666668
16125 19.47  15.9              2.199 2.05 2.0360000000000005 2.1029999999999998 4041 0  -.01499999999999968 . 19.666666666666668
16126 18.81 14.93 2.1900000000000004 2.05              2.032              2.098 4041 0 -.016000000000000014 . 19.666666666666668
16127 18.46 14.83                2.2 2.05 2.0380000000000003 2.1050000000000004 4041 0 -.015000000000000568 . 19.666666666666668
16128 18.06 14.55               2.15 2.06              1.991 2.0580000000000003 4041 0 -.016000000000000014 . 19.666666666666668
16131     . 14.44                  .    .                  .                  . 4008 0 -.015000000000000568 . 19.666666666666668
16132 21.35 14.86 2.2080000000000006 2.03 2.0400000000000005 2.1120000000000005 4008 0 -.015000000000000568 . 19.666666666666668
16133 17.67 14.55              2.293 1.99               2.13              2.199 4008 0  -.01499999999999968 . 19.666666666666668
16134     .  14.4                  .    .                  .                  . 4008 0 -.016000000000000014 . 19.666666666666668
16135     . 14.48              2.104 2.04              1.944              2.011 4008 0 -.016000000000000014 . 19.666666666666668
16138     . 15.79              2.042 2.05 1.8760000000000003 1.9450000000000003 4008 0 -.015000000000000568 . 19.666666666666668
16139     .  16.6              2.064 2.01 1.9000000000000004 1.9700000000000002 4008 0 -.016000000000000014 . 19.666666666666668
16140 18.69 18.67                  .    .                  .                  . 4008 0 -.015000000000000568 . 19.666666666666668
16141 23.22 20.67 1.9480000000000004 2.13              1.779              1.854 4008 0 -.015000000000000568 . 19.666666666666668
16142 24.65  18.3              2.253 1.82 2.0809999999999995               2.16 4008 0  -.01499999999999968 . 19.666666666666668
16145 28.77 21.13 2.0370000000000004 2.02 1.8649999999999998              1.951 4008 0  -.01499999999999968 . 19.666666666666668
16146 27.89 20.34               2.09 2.01 1.9160000000000004 2.0040000000000004 4008 0 -.015000000000000568 . 19.666666666666668
16147 24.41 18.11 2.0520000000000005 2.01 1.8800000000000003 1.9660000000000002 4008 0  -.01499999999999968 . 19.666666666666668
16148 26.51 18.53 2.0540000000000003    2              1.884 1.9689999999999999 4008 0  -.01499999999999968 . 19.666666666666668
16149    26 19.15              2.066    2              1.891 1.9769999999999999 4008 0  -.01499999999999968 . 19.666666666666668
16152 31.74 21.58 2.0330000000000004 2.01 1.8600000000000003 1.9450000000000003 4008 0  -.01499999999999968 . 19.666666666666668
16153 29.19 20.67 2.0290000000000004 2.03              1.858              1.943 4008 0  -.01499999999999968 . 19.666666666666668
16154 28.37 19.81 1.9809999999999999 2.05               1.81 1.8930000000000002 4008 0  -.01499999999999968 . 19.666666666666668
16155 25.92 17.88 1.9699999999999998 2.05 1.7960000000000003              1.879 4008 0  -.01499999999999968 . 19.666666666666668
16156 25.05 17.33                  2 2.05 1.8200000000000003 1.9060000000000001 4008 0  -.01499999999999968 . 19.666666666666668
16159 22.84  16.5 2.0740000000000003 2.06 1.8940000000000001 1.9810000000000003 4008 0 -.016000000000000014 . 19.666666666666668
16160 23.61 16.28 2.0710000000000006 2.05 1.8930000000000002 1.9770000000000003 4008 0  -.01499999999999968 . 19.666666666666668
16161 22.67 16.74 2.0450000000000004 2.06              1.871 1.9529999999999998 4031 0 -.015000000000000568 . 19.666666666666668
16162 21.71 16.65                  .    .                  .                  . 4031 0  -.01499999999999968 . 19.666666666666668
16163 19.04 15.64              2.225 2.03 2.0500000000000003              2.132 4031 0  -.01499999999999968 . 19.666666666666668
16166 18.53 14.97              2.232 2.04              2.061 2.1449999999999996 4031 0  -.01499999999999968 . 19.666666666666668
16167  20.1 15.32              2.128 2.12 1.9610000000000003               2.04 4031 0  -.01499999999999968 . 19.666666666666668
16168 20.76 15.76                  .    .                  .                  . 4031 0  -.01499999999999968 . 19.666666666666668
16169 20.58 16.26 2.1850000000000005 2.05              2.018 2.0970000000000004 4031 0 -.016000000000000014 . 19.666666666666668
16170     .     .                  .    .                  .                  . 4031 0 -.015000000000000568 . 19.666666666666668
16173     . 15.28                  .    .                  .                  . 4031 0 -.016000000000000014 . 19.666666666666668
16174 20.84 17.26 2.2699999999999996 2.04              2.104 2.1849999999999996 4031 0  -.01499999999999968 . 19.666666666666668
16175 22.03 15.62 2.3010000000000006 2.03 2.1380000000000003 2.2160000000000006 4031 0  -.01499999999999968 . 19.666666666666668
16176 22.37 15.74 2.2870000000000004 2.03              2.126              2.203 4031 0 -.016000000000000014 . 19.666666666666668
16177 19.56 14.94              2.243 2.03 2.0830000000000006              2.156 4031 0 -.015000000000000568 . 19.666666666666668
16180 19.87 15.42 2.2550000000000003 2.02              2.092              2.169 4031 0 -.016000000000000014 . 19.666666666666668
16181  19.3 16.67              2.286 2.02              2.131              2.203 4031 0 -.015000000000000568 . 19.666666666666668
16182 19.84  15.6 2.3030000000000004 2.02              2.147 2.2159999999999997 4031 0 -.016000000000000014 . 19.666666666666668
16183    18 14.61 2.2950000000000004 2.02 2.1430000000000002 2.2129999999999996 4031 0 -.016000000000000014 . 19.666666666666668
16184 18.59 14.01              2.313 2.02              2.161              2.233 4031 0 -.016000000000000014 . 19.666666666666668
16187  19.5 14.77              2.305 2.03 2.1570000000000005 2.2230000000000003 4031 0  -.01499999999999968 . 19.666666666666668
16188 18.93 15.07 2.3060000000000005 2.03 2.1600000000000006 2.2230000000000003 4031 0 -.016000000000000014 . 19.666666666666668
16189 21.14 16.29                2.3 2.04 2.1529999999999996 2.2169999999999996 4031 0 -.016000000000000014 . 19.666666666666668
16190 22.28  16.6              2.309 2.04 2.1559999999999997 2.2219999999999995 4031 0 -.016000000000000014 . 19.666666666666668
16191 22.73 17.19 2.2600000000000002 2.06 2.1149999999999998              2.177 4074 0  -.01499999999999968 . 19.666666666666668
16194 22.02 16.62              2.458 1.84 2.3120000000000003               2.37 4074 0 -.016000000000000014 . 19.666666666666668
16195 21.83 16.55              2.043 2.26 1.8970000000000002 1.9610000000000003 4074 0 -.016000000000000014 . 19.666666666666668
16196 19.88 15.77              2.271 2.04              2.123              2.186 4074 0  -.01499999999999968 . 19.666666666666668
16197 22.91 17.05                  .    .                  .                  . 4074 0 -.016000000000000014 . 19.666666666666668
16198 22.99 18.13              2.337 2.11 2.1919999999999997 2.2550000000000003 4074 0 -.016000000000000014 . 19.666666666666668
16201    26 19.77 2.3320000000000003  2.1 2.1910000000000003              2.251 4074 0 -.016000000000000014 . 19.666666666666668
16202 24.04 18.57 2.3120000000000003 2.12 2.1719999999999997 2.2329999999999997 4074 0  -.01499999999999968 . 19.666666666666668
16203 25.99 18.14                  .    .                  .                  . 4074 1 -.016000000000000014 . 19.666666666666668
16204 25.42 18.86 2.4570000000000003 2.03              2.318 2.3750000000000004 4074 1 -.016000000000000014 . 19.666666666666668
16205 25.28 18.47 2.4200000000000004 2.03 2.2790000000000004 2.3350000000000004 4074 1 -.016000000000000014 . 19.666666666666668
16208 26.76 19.96              2.384 2.03              2.243              2.299 4074 1  -.01499999999999968 . 19.666666666666668
16209 25.34 19.33              2.406 2.03 2.2630000000000003 2.3190000000000004 4074 1 -.016000000000000014 . 19.666666666666668
16210 23.29 18.93              2.461 2.02              2.323 2.3770000000000002 4074 1 -.016000000000000014 . 19.666666666666668
16211  23.6 18.67              2.445 2.02 2.3080000000000003 2.3640000000000003 4074 1  -.01499999999999968 . 19.666666666666668
16212 22.38 18.49              2.424 2.03               2.28 2.3380000000000005 4074 1 -.016000000000000014 . 19.666666666666668
16215 24.88 18.08 2.4440000000000004 2.03              2.307              2.363 4074 1 -.016000000000000014 . 19.666666666666668
16216 21.35 15.96 2.4310000000000005 2.03 2.2920000000000003 2.3489999999999998 4074 1 -.016000000000000014 . 19.666666666666668
16217 21.05 15.97 2.4010000000000002 2.03 2.3000000000000003               2.32 4074 1 -.016000000000000014 . 19.666666666666668
16218  20.7 15.28              2.358 2.03              2.254              2.275 4074 1  -.01499999999999968 . 19.666666666666668
16219 20.26  15.5 2.3949999999999996 2.04 2.2880000000000003 2.3120000000000003 4074 1 -.016000000000000014 . 19.666666666666668
16222 21.23     . 2.4190000000000005 2.05              2.324              2.346 4180 1 -.016000000000000014 . 19.666666666666668
16223 22.45  16.3              2.434 2.04 2.3289999999999997 2.3529999999999998 4180 1 -.016000000000000014 . 19.666666666666668
16224 20.74 16.08 2.4680000000000004 2.02              2.363              2.389 4180 1 -.016000000000000014 . 19.666666666666668
16225 20.76 17.03                  .    .                  .                  . 4180 1 -.016000000000000014 . 19.666666666666668
16226 19.33 16.78 1.7489999999999997 2.75 1.6449999999999996 1.6639999999999997 4180 1 -.016000000000000014 . 19.666666666666668
16229 19.43 15.39 1.9570000000000003 2.53 1.8530000000000002 1.8729999999999998 4180 1 -.016000000000000014 . 19.666666666666668
16230 19.07 15.01              2.537 1.93 2.4350000000000005 2.4540000000000006 4180 1 -.016000000000000014 . 19.666666666666668
16231 19.52 15.39 2.3419999999999996 2.16 2.2359999999999998 2.2560000000000002 4180 1 -.016000000000000014 . 19.666666666666668
16232 18.41 15.04              2.378 2.13 2.2729999999999997 2.2910000000000004 4180 1 -.016000000000000014 . 19.666666666666668
16233 18.17     .                  .    .                  .                  . 4180 1 -.016000000000000014 . 19.666666666666668
16236  20.5 16.07 2.4979999999999998 2.03 2.3930000000000002              2.411 4180 1 -.016000000000000014 . 19.666666666666668
16237 18.15 15.05 2.4170000000000003 2.03 2.3160000000000003              2.329 4180 1 -.016000000000000014 . 19.666666666666668
16238 17.67 14.79              2.448 2.03 2.3430000000000004              2.363 4180 1 -.016000000000000014 . 19.666666666666668
16239 18.75 15.15              2.459 2.03              2.355 2.3720000000000003 4180 1 -.016000000000000014 . 19.666666666666668
16240 18.18 14.99              2.429 2.03              2.323 2.3380000000000005 4180 1 -.016000000000000014 . 19.666666666666668
16243 17.72 15.26              2.414 2.03              2.312              2.328 4180 1 -.016000000000000014 . 19.666666666666668
16244  18.8 14.31              2.435 2.03 2.3320000000000003               2.35 4180 1 -.017000000000000348 . 19.666666666666668
16245 17.92 13.98 2.4340000000000006 2.03 2.3330000000000006 2.3510000000000004 4180 1 -.016000000000000014 . 19.666666666666668
16246 17.83 14.81 2.3700000000000006 2.03 2.2690000000000006              2.286 4180 1 -.016000000000000014 . 19.666666666666668
16247 18.11 15.19 2.3869999999999996 2.04              2.285 2.3019999999999996 4180 1 -.016000000000000014 . 19.666666666666668
end
format %tdnn/dd/CCYY date

//  CREATE VARIABLE LISTS
local dvar x1   // DEPENDENT VARIABLE
local always_vars x2 x3 //  INDEPENDENT VARIABLES APPEARING IN ALL REGRESSIONS
local tlist roa1 roa2 pp1 pp2 pp3 pp4 pp5 pp6  // VARIABLES FOR TUPLES

//  SINCE I DON'T USE -tuples- THIS PART OF THE CODE IS A SUBSTITUTE FOR THAT
//  YOU CAN JUST USE -tuples- INSTEAD

local nvars: word count `tlist'

local ntuples = 0

forvalues i = 1/`nvars' {
    forvalues j = `=`i'+1'/`nvars' {
        local v1: word `i' of `tlist'
        local v2: word `j' of `tlist'
        local ++ntuples
        local tuple`ntuples' `v1' `v2'
    }
}



//    THE CODE YOU WANT BEGINS HERE
tempfile results

capture postutil clear

local results_vars
foreach v of varlist `always_vars' `tlist' {
    local results_vars `results_vars' b_`v' se_`v'
}
foreach v of varlist `always_vars' `tlist' {
    local results_vars `results_vars' ame_`v'
}

postfile handle `results_vars' using `results'

    
forvalues i = 1/`ntuples' {
     display `"`tuple`i''"'
     capture noisily regress `dvar' `always_vars' `tuple`i'', vce(robust)
     if c(rc) == 0 {
         local topost `initial'
         foreach v of varlist `always_vars' `tlist' {
             if strpos("`e(cmdline)'", "`v'") {
                 local topost `topost' (`=_b[`v']') (`=_se[`v']')
             }
             else {
                 local topost `topost' (.) (.)
             }
         }
         margins, dydx(*) post
         foreach v of varlist `always_vars' `tlist' {
              if strpos("`e(xvars)'", "`v'") {
                 local topost `topost' (`=_b[`v']')
             }
             else {
                 local topost `topost' (.)
             }
        }
         post handle `topost'
     }
     else if inlist(c(rc), 2000, 2001) { // NO, OR TOO FEW OBSERVATIONS
         continue
     }
     else { //  REGRESSION FAILED DUE UNEXPECTED PROBLEM: ABORT
         display as error "Unexpected error with `tuple`i''"
         exit(c(rc))
     }
             
}

postclose handle

//    LOAD RESULTS AND CALCULATE "SIGNIFICANCE" TEST
use `results', clear


local critical_value_5 = invnormal(0.975)

foreach v in `always_vars' `tlist' {
     gen byte sig05_`v' = abs(b_`v'/se_`v') > `critical_value_5' ///
        if !missing(b_`v', se_`v')
    order sig05_`v', after(se_`v')
}

//    AGGREGATE UP TO MEANS OVER ALL ANALYSES
//    THEN MAKE A TABLE

collapse (mean) _all

gen one = 1

reshape long b_ se_ ame_ sig05_, i(one) j(varname) string

rename *_ *

drop one

list, noobs clean

I don't have any good ideas for the weighting you want to do. You might consider weighting the mean of each coefficient by the inverse of its standard error square. But there is nothing analogous that you could do with the standard errors or significance levels. And while I could imagine, for linear regressions, weighting those standard errors and sig05 by the inverse square of e(rmse), there would be nothing analogous to this for the non-linear models.

Concerning the code you show near the end of #21, the syntax error arises because in your -replace- command you reference a local macro `v' which does not exist at that point in the code. Also, when you fix that, to actually get what you want, it would have to be B[1, "`v'"]. Anyway, the code shown above handles the marginal effects and is a little cleaner.

[Danielle Laurie]

It runs perfectly, thank you very much! I appreciate all your kind advice.

Best,
Danielle

[Danielle Laurie]

Hello again,

I was attempting to do the regression weighting as before, following Clyde Schechter's helpful advice to Jonas Bollinger on post 1577628-e-r2_p-as-a-weight-on-a-loop-of-regressions #4, to try and weight the regressions by the pseudo R^2 of each model and tried to code it as follows:

tempfile results

capture postutil clear

local results_vars
foreach v of varlist `always_vars' `tlist' {
    local results_vars `results_vars' b_`v' se_`v'
}
foreach v of varlist `always_vars' `tlist' {
    local results_vars `results_vars' ame_`v'
}

postfile handle `results_vars' r2p using `results'

    
forvalues i = 1/`ntuples' {
     quietly probit `dvar' `always_vars' `tuple`i'', vce(robust)
     if c(rc) == 0 {
         local topost
         foreach v of varlist `always_vars' `tlist' {
             if strpos("`e(cmdline)'", "`v'") {
                 local topost `topost' (`=_b[`v']') (`=_se[`v']')
             }
             else {
                 local topost `topost' (.) (.) (.)
             }
             local topost `topost' (`e(r2_p)')

         }
         
         quietly margins, dydx(*) post
         foreach v of varlist `always_vars' `tlist' {
              if strpos("`e(xvars)'", "`v'") {
                 local topost `topost' (`=_b[`v']')
             }
             else {
                 local topost `topost' (.)
             }
        }
         post handle `topost'
     }
     else if inlist(c(rc), 2000, 2001) { // NO, OR TOO FEW OBSERVATIONS
         continue
     }
     else { //  REGRESSION FAILED DUE UNEXPECTED PROBLEM: ABORT
         display as error "Unexpected error with `tuple`i''"
         exit(c(rc))
     }
             
}

postclose handle

However, I run into:

postfile: variable r2p already exists

I think I'm probably just placing the -local topost `topost' (`e(r2_p)')- in the wrong place, but I have no idea where to place it to fix this.

Also, after capturing the weight appropriately, in order to present the table with the average, weighted betas and marginal effects, should I generate a new variable? Or does using [aweight = r2p] in front of _b and _ame is enough? I have no idea how to include this new info in the presenting table.


Again, sorry for all the trouble, and forever indebted to all you of who might take the time to answer this.

Best,
Danielle

[Clyde Schechter]

First, a disclaimer. I could not replicate the error you are getting. Your code does not work, and I have identified some errors in it (see below), but I could not get Stata to give me the particular error message you are getting. So I suspect that in addition to what I have found wrong, there is some other problem, perhaps arising out of other parts of the code that you have not shown me.

You do have -local topost `topost' (`e(r2_p)') in the wrong place. Since r2p is the last variable in the results file, this has to be the last thing added to `topost'. So, it has to be the very last command in the -if c(rc) == 0- block. But, there's a slight complication. After the means and standard deviations are tacked onto `topost', we then do -margins, post-, which will cause e(r2_p) to be erased. So before we run -margins, post- we have to save the value of e(r2_p) in a local macro, and then use that in the -post- command when we get to it. Finally, there is one other mistake in your code. The command -local topost `topost' (.) (.) (.)- should only have two missing value expressions, not three.

To test this code, I have put back my proxy code for -tuples- and also set up some fake toy data. With this, the following code runs without error messages and produces sensible looking results:

clear*

//  CREATE A TOY DATA SET
set seed 12345
set obs 100
local dvar y
gen `dvar' = runiformint(0, 1)
local always_vars x1 x2 x3
foreach a of local always_vars {
    gen `a' = rnormal()
}
local tlist w1 w2 w3 w4
foreach t of local tlist {
    gen `t' = rnormal()
}

local nvars: word count `tlist'

//  SUBSTITUTE FOR -tuples.ado-
local ntuples = 0

forvalues i = 1/`nvars' {
    forvalues j = `=`i'+1'/`nvars' {
        local v1: word `i' of `tlist'
        local v2: word `j' of `tlist'
        local ++ntuples
        local tuple`ntuples' `v1' `v2'
    }
}


//  THE MAIN EVENT BEGINS HERE
tempfile results

capture postutil clear

local results_vars
foreach v of varlist `always_vars' `tlist' {
    local results_vars `results_vars' b_`v' se_`v'
}
foreach v of varlist `always_vars' `tlist' {
    local results_vars `results_vars' ame_`v'
}

postfile handle `results_vars' r2p using `results'
    
forvalues i = 1/`ntuples' {
     quietly probit `dvar' `always_vars' `tuple`i'', vce(robust)
     if c(rc) == 0 {
         local topost
         foreach v of varlist `always_vars' `tlist' {
             if strpos("`e(cmdline)'", "`v'") {
                 local topost `topost' (`=_b[`v']') (`=_se[`v']')
             }
             else {
                 local topost `topost' (.) (.) // NOTE ONLY TWO EXPRESSIONS, NOT THREE
             }
         }
         local er2p = e(r2_p) // NOTE HAVE TO SAVE VALUE OF e(r2p) FOR LATER USE
         quietly margins, dydx(*) post // ERASES e(r2p)
         foreach v of varlist `always_vars' `tlist' {
              if strpos("`e(xvars)'", "`v'") {
                 local topost `topost' (`=_b[`v']')
             }
             else {
                 local topost `topost' (.)
             }
        }
             local topost `topost' (`er2p') // NOTE CORRECT PLACEMENT OF THIS COMMAND
        
         post handle `topost'
     }
     else if inlist(c(rc), 2000, 2001) { // NO, OR TOO FEW OBSERVATIONS
         continue
     }
     else { //  REGRESSION FAILED DUE UNEXPECTED PROBLEM: ABORT
         display as error "Unexpected error with `tuple`i''"
         exit(c(rc))
     }
             
}

postclose handle

use `results', clear

list, noobs clean
summ [aweight = r2p]

[Danielle Laurie]

Thanks once again Clyde. You are really saving my work.

I just have one (hopefully final!!) question:

Using -summ [aweight = r2p]- gives me a list of all the b_ se_ and ame_ per variable, where instead of having a column for the weighted betas I have a row for the beta of x1.
Running exactly your code (seed and all), I get:

 Variable |     Obs      Weight        Mean   Std. Dev.       Min        Max
-------------+-----------------------------------------------------------------
        b_x1 |       6  .206150087   -.0111409    .025058   -.033048   .0182027
       se_x1 |       6  .206150087    .1208577   .0030671   .1172353   .1236351
        b_x2 |       6  .206150087    .2182712   .0080128    .205728   .2291215
       se_x2 |       6  .206150087    .1336522   .0020558   .1316489   .1359968
        b_x3 |       6  .206150087   -.0654246   .0102642  -.0745585  -.0512666
-------------+-----------------------------------------------------------------
       se_x3 |       6  .206150087    .1219312   .0018676   .1196898   .1234253
        b_w1 |       3  .103742743   -.1092334   .0027095  -.1125301  -.1075864
       se_w1 |       3  .103742743    .1176135   .0011195   .1167258   .1188956
        b_w2 |       3  .119292043   -.1889954   .0032195  -.1927291  -.1863691
       se_w2 |       3  .119292043    .1430961   .0009217    .142335    .144135
-------------+-----------------------------------------------------------------
        b_w3 |       3  .097040698    .0764421   .0058489   .0726009   .0831995
       se_w3 |       3  .097040698    .1262231   .0015481   .1247661   .1276716
        b_w4 |       3  .092224689    .0254461   .0121822   .0127919   .0366504
       se_w4 |       3  .092224689    .1243392   .0015281   .1227453   .1256964
      ame_x1 |       6  .206150087   -.0041569   .0094533  -.0123836    .006947
-------------+-----------------------------------------------------------------
      ame_x2 |       6  .206150087    .0824262   .0030202   .0779459   .0863474
      ame_x3 |       6  .206150087   -.0246859   .0037242  -.0279382  -.0195656
      ame_w1 |       3  .103742743   -.0412448    .001201  -.0426352  -.0403143
      ame_w2 |       3  .119292043   -.0710259   .0013484  -.0726324   -.070068
      ame_w3 |       3  .097040698    .0289309   .0022734   .0272953   .0315225
-------------+-----------------------------------------------------------------
      ame_w4 |       3  .092224689    .0096307   .0045549    .004882   .0138122
         r2p |       6  .206150087    .0353735   .0063294   .0257299   .0426146

This isn't really concerning issue, its just a technicality of presentation I suppose.
If I wanted to get a table more in the lines of what you did without the weights:

varname        b         ame            se    sig05      
pp1         .7621593     0.5532      .031944    .75
pp2        -.4605175     0.0367       .03946    .75  
pp3         1.045931     0.3758      .026136      1
roa1        .0040047     0.0134      .001926    .25  
roa2       -.0068664     0.4872     .0030597    .25  
x3         -.0000639     0.3848     .0000592     .5

but where the columns are already weighted, this would be ideal for me.

I've tried reshaping the data like you suggest in your original code:

collapse (mean) _all

gen one = 1

reshape long b_ se_ ame_ sig05_, i(one) j(varname) string

rename *_ *

drop one

list, noobs clean
summ [aweight = r2p]

But this makes it even worse because -list, noobs clean- only adds an additional column reporting the r2p and when using -summ [aweight = r2p]- now I lose the variables entirely:

  Variable |     Obs      Weight        Mean   Std. Dev.       Min        Max
-------------+-----------------------------------------------------------------
     varname |       0           0
           b |       7  .240508441   -.0073091   .1325852   -.189077   .2181071
          se |       7  .240508441    .1267443   .0088585   .1177071   .1430928
         ame |       7  .240508441    -.002681   .0500337  -.0710655   .0824555
         r2p |       7  .240508441    .0343583          0   .0343583   .0343583
-------------+-----------------------------------------------------------------
       sig05 |       0           0

Makes me wonder if -summ- is really the correct command in this case, but then again, I am not aware if any other can take the [aweight = r2p] option. I though about generating a variable with the coefficients weighted by the r2p, but I don't know how to do this either, nor do I know if it's a viable option.

I appreciate all the help you've given me so far (seriously, I am beyond grateful). Here's to hoping you'll have some clues on how to solve this small riddle (at least for me) as well.

Best,
Danielle

[Clyde Schechter]

To get the layout you are looking for, take the code in #25 up through (including) -use `results', clear-. Then add:

foreach x in mean min max {
    local `x's
    foreach v of varlist `results_vars' {
        local `x's ``x's' `x'_`v' = `v'
    }
}


collapse (mean) `means' (min) `mins' (max) `maxs' [aweight = r2p]
gen one = 1

reshape long mean_b_ min_b_ max_b_ mean_se_ min_se_ max_se_ mean_ame_ min_ame_ max_ame_, i(one) j(variable) string
drop one
rename *_ *
list, noobs clean

to get the layout I think you are looking for.

[Danielle Laurie]

Thanks Clyde, it is exactly that yes.

One small thing: to get the Significance column like before, I tried:

use `results', clear

. 
. foreach x in mean {
  2.     local `x's
  3.     foreach v of varlist `results_vars' {
  4.         local `x's ``x's' `x'_`v' = `v'
  5.     }
  6. }

. 
. 
. 
. local critical_value_5 = invnormal(0.975)

. 
. foreach v in `always_vars' `tlist' {
  2.      gen byte sig05_`v' = abs(b_`v'/se_`v') > `critical_value_5' ///
>         if !missing(b_`v', se_`v')
  3.      order sig05_`v', after(ame_`v')
  4. }       
(3 missing values generated)
(3 missing values generated)
(3 missing values generated)
(3 missing values generated)

. 
. 
. collapse (mean) `means' [aweight = r2p]

. gen one = 1

. 
. reshape long mean_b_ mean_se_ mean_ame_ sig05_, i(one) j(variable) string
(note: j = w1 w2 w3 w4 x1 x2 x3)
(note: sig05_w1 not found)
(note: sig05_w2 not found)
(note: sig05_w3 not found)
(note: sig05_w4 not found)
(note: sig05_x1 not found)
(note: sig05_x2 not found)
(note: sig05_x3 not found)

Data                               wide   ->   long
-----------------------------------------------------------------------------
Number of obs.                        1   ->       7
Number of variables                  22   ->       6
j variable (7 values)                     ->   variable
xij variables:
      mean_b_w1 mean_b_w2 ... mean_b_x3   ->   mean_b_
   mean_se_w1 mean_se_w2 ... mean_se_x3   ->   mean_se_
mean_ame_w1 mean_ame_w2 ... mean_ame_x3   ->   mean_ame_
         sig05_w1 sig05_w2 ... sig05_x3   ->   sig05_
-----------------------------------------------------------------------------

. drop one

. rename *_ *

. list, noobs clean

    variable      mean_b    mean_se    mean_ame   sig05  
          w1   -.1092334   .1176135   -.0412448       .  
          w2   -.1889954   .1430961   -.0710259       .  
          w3    .0764421   .1262231    .0289309       .  
          w4    .0254461   .1243392    .0096307       .  
          x1   -.0111409   .1208577   -.0041569       .  
          x2    .2182712   .1336522    .0824262       .  
          x3   -.0654246   .1219312   -.0246859       .

As you can see, nothing is recognised so I don't have the significance for anything.

Perhaps I'm placing the significance in the wrong place? I've tried shifting it around, but I haven't had much luck.

Hopefully awaiting your suggestions,

Thank you again,
Danielle

[Clyde Schechter]

Your code has a few problems. First, you have to calculate the sig05_* variables before you do that -foreach x in mean...- loop so that you can calculate the mean values of the sig05_* variables themselves.

Next, you have to include the sig05_* variables in the -foreach v of varlist...- loop inside of the one just mentioned, so that they get appropriate included in the `means' local macro that gets fed into -collapse-.

Finally, the -reshape- statement has to refer to mean_sig05_, not sig05_. So we get:

use `results', clear

local critical_value_5 = invnormal(0.975)
foreach v in `always_vars' `tlist' {
     gen byte sig05_`v' = abs(b_`v'/se_`v') > `critical_value_5' ///
     if !missing(b_`v', se_`v')
     order sig05_`v', after(ame_`v')
} 

foreach x in mean {
    local `x's
    foreach v of varlist `results_vars' sig05_* {
        local `x's ``x's' `x'_`v' = `v'
    }
}

collapse (mean) `means' [aweight = r2p]

gen one = 1

 
reshape long mean_b_ mean_se_ mean_ame_ mean_sig05_, i(one) j(variable) string
drop one

rename *_ *

list, noobs clean

Code that has been moved, but not changed, is in bold face. Code that has been modified is in red.

[Danielle Laurie]

Works perfectly! Thank you so much, you just saved by research project. I appreciate it immensely!