How to find non-existing variable in my code

[Chul-Kyoo Jung]

Hi, I try to replicate the code of estimating firm's markup ratio, but my codes keep generating the following error.
To me, the code has every necessary variable; I was wondering if anyone could help me on how to fix the issue.
Sorry for the lengthy post.

 
. qui {
variable __000003 not found
(31 lines skipped)
r(111);

end of do-file

qui {
clear mata
use "D:\markupest\firmdata.d3.dta", clear
foreach i of numlist 3 10/17 19/33 35 41 42 45/47 49/52 58/64 71 72 75 91{
keep if ksic1c == `i' 
// ksic1c: 2-digit KSIC
*----------------------------------------------------------------------------------------------------*
* OLS REGRESSION FOR STARTING VALUES & First stage
// xi: reg y l k i.year
xi: reg lny_10 lnl lnk_10 i.year                        
// lny_10: log of Real Value Added (denominated by 2010 PPI), lnl: log of Number of Workers, lnk_10: log of Real Average Capital Stock (denominated by 2010 PPI)
gen blols=_b[lnl]
gen bkols=_b[lnk_10]
predict phiS
predict epsilon, res
label var phiS "Expected value added" 
label var epsilon "Measurement error first stage"

bys firmid (year): gen phiS_lag=phiS[_n-1]                // firmid: company ID
bys firmid (year): gen lnl_lag=lnl[_n-1]
bys firmid (year): gen lnk_10_lag=lnk_10[_n-1]
*---COMPUTE CORRECTED SHARES-------------------------------------------------------------------------*
gen va_c=exp(lny_10-epsilon)
gen alpha_l=wb_10/va_c                                // wb: Real Wage Bill (denominated by 2010 PPI)
*---------------------------------------------------------------------------------------------------*
drop _I*
sort firmid year
gen const=1
drop if lnl_lag==.
drop if lnk_10==.
drop if phiS==.
drop if phiS_lag==.
drop if lny_10==.
}

qui {
*-------------------------------------BEGIN MATA PROGRAM--------------------------------------------*
mata:
void GMM_DLW(todo,betas,crit,g,H)
{
    PHI=st_data(.,("phiS"))
    PHI_LAG=st_data(.,("phiS_lag"))
    Z=st_data(.,("const","lnl_lag","lnk_10"))
    X=st_data(.,("const","lnl","lnk_10"))
    X_lag=st_data(.,("const","lnl_lag","lnk_10_lag"))
    Y=st_data(.,("lny_10"))
    C=st_data(.,("const"))

    OMEGA=PHI-X*betas'
    OMEGA_lag=PHI_LAG-X_lag*betas'
    OMEGA_lag_pol=(C,OMEGA_lag)
    g_b = invsym(OMEGA_lag_pol'OMEGA_lag_pol)*OMEGA_lag_pol'OMEGA
    XI=OMEGA-OMEGA_lag_pol*g_b
    crit=(Z'XI)'(Z'XI)
}

void DLW()
    {
S=optimize_init()
optimize_init_evaluator(S, &GMM_DLW())
optimize_init_evaluatortype(S,"d0")
optimize_init_technique(S, "nm")
optimize_init_nmsimplexdeltas(S, 0.1)
optimize_init_which(S,"min")
optimize_init_params(S,(0,0,0))
p=optimize(S)
p
st_matrix("beta_dlw",p)
}
end
*-----------------------------------------END MATA PROGRAM---------------------------------------*
cap program drop dlw
program dlw, rclass
preserve 
sort firmid year
mata DLW()
end

*------------------------------------COMPUTE MARKUPS --------------------------------------------*
// OLS, DLW estimates for variable input (here labor) are used to compute markup distribution

*------------------------------------ACF estimates-----------------------------------------------*
dlw
gen beta_cS=beta_dlw[1,1]
gen beta_lS=beta_dlw[1,2]
gen beta_kS=beta_dlw[1,3]
gen mu_dlwS=beta_lS/alpha_l                            // Markup
gen tfp_dlwS=phiS-beta_lS*lnl-beta_kS*lnk_10                // TFP
*-------------------------------------------------------------------------------------------------*

* collect markup estimates and productivity for analysis:
keep firmid year mu_dlwS tfp_dlwS lbg30 lny lny_10 lnl lnk_10 beta_*S
// nind: KSIC 5-digit, region: province, surv: A dummy for surviving firms 
save "DLW simplified ind `i'.dta"
}
}

* Example generated by -dataex-. To install: ssc install dataex
clear
input str6 firmid float(year lny_10 lnk_10 lnl) byte ksic1c float(wb_10 va_10 lbg30)
"00020" 1991   19.3579 19.059488  6.861712 21   85082680  255287248 0
"00020" 1992 19.584406 19.332605  6.887553 21   98165352  320183616 0
"00020" 1993  19.68283 19.415634  6.884487 21  103217744  353300416 0
"00020" 1994 19.655144 19.597256  6.925595 21  126507592  343653216 0
"00020" 1995 19.829073  19.92676  6.891626 21  143004016  408937472 0
"00020" 1996  19.88891  19.97675  6.877296 21  153120448  434155008 0
"00020" 1997  19.98304 19.594515  6.877296 21  173322752  477007040 0
"00020" 1998  20.03464 20.490465  6.811244 21  165947840  502266688 0
"00020" 1999 20.366825 20.493475  6.684612 21  142220672  700163712 0
"00020" 2000 19.225157 20.417496   6.60665 21  151860352  223552560 0
"00020" 2001 19.880404 20.428104  6.638568 21  174416144  430476832 0
"00020" 2002  19.99138  20.44749  6.719013 21  194770464  481001792 0
"00020" 2003 20.023157  20.45587  6.736967 21  227870304  496531424 0
"00020" 2004 19.964947    20.477  6.694562 21  246957056  468452992 0
"00020" 2005  19.64121  20.49848  6.687109 21  252665280  338897376 0
"00020" 2006 19.760237  20.53007  6.708084 21  278346592  381735008 0
"00020" 2007 19.804644  20.62971  6.719013 21  288193824  399068672 0
"00020" 2008 20.223125 20.654533  6.747587 21  392295040  606445760 0
"00020" 2009  20.41848  20.66776  6.791222 21  407581344  737280192 0
"00020" 2010  20.26437  21.32764  6.700731 21  401954080  631980032 0
"00020" 2011 20.352667  21.57517  6.697034 21  369337152  690320768 0
"00040" 1991  18.90984  19.18728  6.692084 31   59712000  163094480 0
"00040" 1992 18.936691 19.290876  6.704414 31   70852288  167533008 0
"00040" 1993 18.870255 19.263805  6.098074 31   50863244  156764272 0
"00040" 1994  18.27907  19.08454  5.666427 31   35530044   86795832 0
"00040" 1995 16.246649  19.93878  5.236442 31   21839752   11371811 0
"00040" 1996  19.64167    21.173  6.755769 31  156773440  339054592 0
"00040" 1998  18.78638  21.13175  6.309918 31   69435184  144152352 0
"00040" 2001  18.61337  20.25646  6.161207 31   84917904  121250816 0
"00040" 2002  20.62024 20.285717  6.033086 31   95591840  902104448 0
"00040" 2003 19.485666  20.32173  5.993961 31   96553104  290079680 0
"00040" 2005 18.427166  20.33254  6.228511 31  124734488  100650720 0
"00040" 2006 17.768404  20.39086  6.204558 31  157852784   52085856 0
"00040" 2007  19.41714  20.18016  5.955837 31  154134000  270867488 0
"00040" 2008 17.036263  20.63604  5.894403 31  115710312   25046948 0
"00040" 2010 18.649565   20.6711  5.616771 31   85225560  125719608 0
"00040" 2011 18.322151  20.61899  5.575949 31   88346952   90616872 0
"00050" 1991 19.736704    20.792  7.665753 47  126578504  372856256 0
"00050" 1992  19.67181  20.79624  7.597898 47  125399344  349429568 0
"00050" 1993 19.616495   20.8451  7.561642 47  154960480  330625120 0
"00050" 1994 19.910204 21.351084  7.580189 47  182598496  443497888 0
"00050" 1995 20.051735  21.36872  7.523481 47  186261568  510925568 0
"00050" 1996  19.93037  21.42523  7.441907 47  176302560  452532896 0
"00050" 1997  20.05103  21.37677  7.403061 47  216485712  510566528 0
"00050" 1998  20.18749 21.377886  7.428333 47  190811808  585215296 0
"00050" 1999  20.30957  21.38439  7.428333 47  240185504  661203968 0
"00050" 2000 20.252745 22.239214  7.342779 47  218779152  624676352 0
"00050" 2001  19.97776 22.238256  7.265429 47  196103776  474493216 0
"00050" 2002 20.216713 22.236393  7.290293 47  196124688  602568640 0
"00050" 2003 20.050127  22.23799  7.034388 47  201969504  510105376 0
"00050" 2004 18.607327  22.23487  7.013016 47  210122416  120520000 0
"00050" 2005  19.65082  22.21079   6.96319 47  212625936  342169920 0
"00050" 2007  20.72574 22.197374  6.790097 47  218275488 1002479808 0
"00050" 2009 19.742474  23.34659   6.36647 47  147584128  375013920 0
"00050" 2010  20.37641 23.348055  6.284134 47  141175312  706905984 0
"00050" 2011  19.83197  22.02695  6.385194 47  157011584  410124576 0
"00070" 1991  21.41153  22.21817  8.295548 71  590223936 1990259200 0
"00070" 1992  21.34508    22.321  8.250359 71  616592512 1862300416 0
"00070" 1993  21.28655 22.473345  8.230044 71  626537472 1756437120 0
"00070" 1994 21.225464  22.70506  8.209309 71  656553344 1652350720 0
"00070" 1995 21.628273  22.84865  8.167352 71  723180992 2471951616 0
"00070" 1996  21.42681  22.85015 8.1847925 71  834200576 2020904704 0
"00070" 1997   21.6401 22.901455  7.988882 71  824515136 2501367040 0
"00070" 1998 21.643507  22.97195  7.935945 71  731285312 2509895680 0
"00070" 1999 21.683695 22.984415  7.806289 71  666699776 2612815360 1
"00070" 2000  21.70328  23.00116  7.767264 71  746026752 2664485376 0
"00070" 2001 21.639383 22.095545  7.082549 71  440844640 2499567616 0
"00070" 2002  21.40102  22.11936  7.102499 71  461538464 1969449216 0
"00070" 2003  21.16278 22.200516  7.082549 71  478893504 1551956608 0
"00070" 2004 21.123363  22.23138  7.118826 71  531922688 1491969024 0
"00070" 2005 21.185413 22.263657  7.154615 71  611091136 1587479296 0
"00070" 2006   20.9378  22.31887  7.132498 71  626800640 1239280896 0
"00070" 2007  20.71781 22.350697  7.081708 71  631066496  994560512 0
"00070" 2008 18.385065  22.38079  7.096721 71  541220480   96501040 0
"00070" 2009  21.32536  22.40013  7.074963 71  530485760 1825938560 0
"00070" 2010  21.49709 22.430126   7.06732 71  562743616 2168036352 0
"00070" 2011  20.77043 19.854544  5.135798 71   93672864 1048294400 0
"00080" 1991  20.48774 20.537426  7.467942 11  193956928  790152896 1
"00080" 1992 20.180016 20.849087  7.511525 11  208696608  580856768 1
"00080" 1993 20.530087  20.99852  7.501082 11  223775296  824334656 1
"00080" 1994 20.798023  21.44802  7.627544 11  270271008 1077623424 1
"00080" 1995 20.923046 21.501064   7.64874 11  313184288 1221134720 1
"00080" 1996  20.98694 21.768847  7.699389 11  351874080 1301701760 1
"00080" 1997 21.107384 21.838957  7.596894 11  367156320 1468318976 1
"00080" 1998  21.20994  21.87965  7.504392 11  312635232 1626893824 1
"00080" 1999 21.313227 21.845306  7.418781 11  318669984 1803916928 1
"00080" 2000 22.042416 21.941887  7.567863 11  422749408 3740237312 1
"00080" 2001  21.23661  21.92831  7.648263 11  527411616 1670870784 0
"00080" 2002  20.98096  21.98829   7.60589 11  584142912 1293941760 0
"00080" 2009 21.646545 22.670763  7.373374 11  868317632 2517532672 0
"00080" 2010 21.596853  22.69542  7.366445 11  865121600 2395489024 0
"00080" 2011  21.71331 23.947563  8.131236 11 1982124928 2691360256 0
"00100" 1991 19.600487   19.5321  7.337588 21  157803760  325374496 0
"00100" 1992 19.683735  19.64753  7.338888 21  182938608  353620256 0
"00100" 1993 19.729046  19.90186  7.312553 21  200397008  370012032 0
"00100" 1994 19.766304  19.94266  7.273093 21  203282176  384058080 0
"00100" 1995 19.926914 21.002775  7.257003 21  226232000  450971520 0
"00100" 1996 19.851276 21.022654   7.26333 21  217063888  418118720 0
"00100" 1997 20.008125  21.28038  7.161622 21  217988720  489123264 0
"00100" 1998 20.290564  21.28397  7.028202 21  216982624  648753088 0
end

[Matthew Alexander]

Have you run the code with - set trace on -? This will show the specific line that is throwing an error.
Also, I note that in the section after "computing corrected shares", you use

drop _I*

Try adding an extra underscore

drop __I*

Best,
Matt

[Chul-Kyoo Jung]

Matthew Alexander Thanks so much, but I still got the following error.

. 
. qui {
variable __I* not found
(31 lines skipped)
r(111);

And when I just remove the line, I get the same error as before.

. qui {
variable __000003 not found
(31 lines skipped)

[Nick Cox]

When you're debugging the prefix quietly is not a good idea. You want output to be as noisy as possible. Knowing where the code fails is a first step to knowing why it fails.

That kind of variable name indicates a temporary variable created somewhere. My only certainty about it is that __000003 is nothing at all to do with _I* or __I*

A very old debugging technique that hasn't lost its usefulness is to pepper your Stata code with display statements such as

di "#1" 
di "#2" 

and your Mata code with equivalent statements such as

"#3" 
"#4"

Then -- so the idea runs -- you can home in on where the code is failing.

[Chul-Kyoo Jung]

Nick Cox , Thanks, I got the following, but I really don't know where variable __000003 not found is coming from.

      Source |       SS           df       MS      Number of obs   =        68
-------------+----------------------------------   F(22, 45)       =      4.55
       Model |  37.0207303        22  1.68276047   Prob > F        =    0.0000
    Residual |  16.6380701        45  .369734891   R-squared       =    0.6899
-------------+----------------------------------   Adj R-squared   =    0.5383
       Total |  53.6588004        67  .800877618   Root MSE        =    .60806

------------------------------------------------------------------------------
      lny_10 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         lnl |  -.0479143   .2477348    -0.19   0.848    -.5468777    .4510491
      lnk_10 |   .9256837   .2207048     4.19   0.000     .4811614    1.370206
 _Iyear_1992 |  -.2816993   .4992041    -0.56   0.575    -1.287148    .7237493
 _Iyear_1993 |   -.480232   .5443627    -0.88   0.382    -1.576635    .6161708
 _Iyear_1994 |  -.3960255   .5582117    -0.71   0.482    -1.520322    .7282705
 _Iyear_1995 |  -.3611287    .558335    -0.65   0.521    -1.485673    .7634158
 _Iyear_1996 |  -.7623236   .5533112    -1.38   0.175     -1.87675    .3521024
 _Iyear_1997 |  -.8949632   .5155147    -1.74   0.089    -1.933263    .1433366
 _Iyear_1998 |   -.159586   .5322584    -0.30   0.766     -1.23161    .9124374
 _Iyear_1999 |  -.4112009   .5340328    -0.77   0.445    -1.486798    .6643964
 _Iyear_2000 |  -.4796987   .5237685    -0.92   0.365    -1.534623    .5752253
 _Iyear_2001 |  -.6928212   .5462582    -1.27   0.211    -1.793042    .4073994
 _Iyear_2002 |  -.5053029   .5578476    -0.91   0.370    -1.628866    .6182598
 _Iyear_2003 |  -.4826727   .5376696    -0.90   0.374    -1.565595    .6002495
 _Iyear_2004 |  -1.149434   .5992446    -1.92   0.061    -2.356375    .0575062
 _Iyear_2005 |  -1.638771   .5993359    -2.73   0.009    -2.845896   -.4316468
 _Iyear_2006 |  -1.387564   .6957307    -1.99   0.052    -2.788838    .0137097
 _Iyear_2007 |  -1.163709   .7123103    -1.63   0.109    -2.598375    .2709579
 _Iyear_2008 |  -1.270216   .6708944    -1.89   0.065    -2.621467    .0810345
 _Iyear_2009 |  -1.066203   .6924299    -1.54   0.131    -2.460828    .3284227
 _Iyear_2010 |   -.848531    .680521    -1.25   0.219    -2.219171    .5221086
 _Iyear_2011 |  -1.080469   .6816236    -1.59   0.120     -2.45333    .2923911
       _cons |   2.126182   2.888059     0.74   0.465    -3.690666    7.943031
------------------------------------------------------------------------------
(option xb assumed; fitted values)
(4 missing values generated)
(4 missing values generated)
(4 missing values generated)
(4 observations deleted)
(0 observations deleted)
(0 observations deleted)
(0 observations deleted)
(0 observations deleted)
(64 observations deleted)
variable __000003 not found

[Nick Cox]

Well, you didn't follow my suggestion about displaying flag text, but you should be able to see that your code fails after 6 commands to drop data.

I am left wondering whether you have any, or enough, data in memory to carry out the GMM estimation. which appears to come next.

I would insert a list command just after

 
 drop if lny_10==.

so that you can see what data the estimation has to play with. I'm afraid that your code tails at the point where I least understand it. But -- another possibility -- you read in your data then loop over subsets of data, You first work on the data for which ksic1c == 3 How does the code ever to get to look at the other subsets? I guess you may need to read in the dataset each time round the loop, but it is hard to follow your code and I may be missing something.

[Matthew Alexander]

Matthew Alexander Thanks so much, but I still got the following error.

.
. qui {
variable __I* not found
(31 lines skipped)
r(111);

And when I just remove the line, I get the same error as before.

. qui {
variable __000003 not found
(31 lines skipped)

My mistake - I meant

drop __*

Though, having considered Nick's more detailed observations, I definetly misdiagnosed the issue.

As Nick says, the problem relates to some kind of temporary variable. Perhaps you can try witing a Stata program to emulate what you are trying to do in mata? Or at least some of what you are trying to do, dropping down to mata where needed?

[Bjarte Aagnes]

did you try trace?

set trace on
set tracedepth 4
set tracesep on 
set traceindent on

capt log close trace
log using tracelog, replace name(trace)

YOUR CODE HERE

log close trace

Then search the log for error:

view tracelog.smcl

see --help trace- including the pdf help

[Chul-Kyoo Jung]

Bjarte Aagnes thanks, I tried to get all the logs, in which I still have no idea what's going on.
The following is the part of the last end of the result log: can you find any clue?

        - sort `touse' `varlist'
        = sort __000004 year
        - quietly by `touse' `varlist': gen `type' `g'=1 if _n==1 & `touse'
        = quietly by __000004 year: gen float __000003=1 if _n==1 & __000004
        - replace `g'=sum(`g')
        = replace __000003=sum(__000003)
variable __000003 not found
          replace `g'=. if `touse'!=1
          if "`label'"!="" {
          if "`truncate'" == "" {
          local truncate `c(namelenchar)'
          }
          if "`lname'" == "" {
          local lname $EGEN_Varname
          unab lab_name_list : _all
          foreach var of local lab_name_list {
          local lab : value label `var'
          local lab_list `lab_list' `lab'
          }
          local dup_check : list lname in lab_list
          if `dup_check' {
          di as err "you must specify the lname() option"
          exit 198
          }
          }
          local dfltfmt : set dp
          local dfltfmt = cond("`dfltfmt'"=="period","%9.0g","%9,0g")
          count if !`touse'
          local j = 1 + r(N)
          sum `g', meanonly
          local max `r(max)'
          local i 1
          while `i' <= 0`max' {
          tokenize `varlist'
          local vtmp ""
          local x 0
          while "`1'"!="" {
          local vallab : value label `1'
          local val = `1'[`j']
          if "`vallab'" != "" {
          local vtmp2 : label `vallab' `val' `truncate'
          }
          else {
          cap confirm numeric var `1'
          if _rc==0 {
          local vtmp2 = string(`1'[`j'],"`dfltfmt'")
          }
          else {
          if "`truncate'"=="" {
          local vtmp2 : display trim(`1'[`j'])
          }
          else {
          local vtmp2 : display trim(usubstr(trim(`1'[`j']),1,`truncate'))
          }
          }
          }
          if ("`qlabel'" != "") {
          if (`"`vtmp'"' == "") {
          local x = length(`"`vtmp2'"') + 4
          local vtmp `"`"`vtmp2'"'"'
          }
          else {
          local x = `x' + length(`"`vtmp2'"') + 5
          local vtmp `"`vtmp' `"`vtmp2'"'"'
          }
          }
          else {
          if (`"`vtmp'"' == "") {
          local x = length(`"`vtmp2'"')
          local vtmp `"`vtmp2'"'
          }
          else {
          local x = `x' + length(`"`vtmp2'"') + 1
          local vtmp `"`vtmp' `vtmp2'"'
          }
          }
          mac shift
          }
          local val `"`vtmp'"'
          label def `lname' `i' `"`val'"', modify
          count if `g' == `i'
          local j = `j' + r(N)
          local i = `i' + 1
          }
          label val `g' `lname'
          }
          }
        -----------------------------------------------------------------------
> ------ end _ggroup ----------------------------------------------------------
      - global EGEN_SVarname
      - global EGEN_Varname
      - if _rc { exit _rc }
      -------------------------------------------------------------------------
> --------- end egen ----------------------------------------------------------
    ---------------------------------------------------------------------------
> ----- end xi.xi_ei ----------------------------------------------------------
    if `"$S_1"' != "." {
    local orig : subinstr local orig `"`t'"' `"$S_1"'
    }
    else {
    local orig : subinstr local orig `"`t'"' ""
    }
    if "`c(os)'"=="MacOSX" {
    local ++update
    }
    }
    gettoken t 0 : 0, parse(`" :,"()[]="') quotes
    }
  -----------------------------------------------------------------------------
> ----------- end xi ----------------------------------------------------------
  gen blols=_b[lnl]
  gen bkols=_b[lnk_10]
  predict phiS
  predict epsilon, res
  label var phiS "Expected value added"
  label var epsilon "Measurement error first stage"
  bys firmid (year): gen phiS_lag=phiS[_n-1]
  bys firmid (year): gen lnl_lag=lnl[_n-1]
  bys firmid (year): gen lnk_10_lag=lnk_10[_n-1]
  gen va_c=exp(lny_10-epsilon)
  gen alpha_l=wb_10/va_c
  drop _I*
  sort firmid year
  gen const=1
  drop if lnl_lag==.
  drop if lnk_10==.
  drop if phiS==.
  drop if phiS_lag==.
  drop if lny_10==.
  }
(31 lines skipped)
r(111);

end of do-file

[Chul-Kyoo Jung]

Nick Cox , Nick I think I don't quite understand your suggestion.
Could you be a bit more specific about your suggestions in #4 and #6? Thank you.

Code:
di "#1" di "#2"
and your Mata code with equivalent statements such as

Code:
"#3" "#4"
Then -- so the idea runs -- you can home in on where the code is failing.
I would insert a list command just after

Code:
drop if lny_10==.

[William Lisowski]

This code

        - sort `touse' `varlist'
        = sort __000004 year
        - quietly by `touse' `varlist': gen `type' `g'=1 if _n==1 & `touse'
        = quietly by __000004 year: gen float __000003=1 if _n==1 & __000004
        - replace `g'=sum(`g')
        = replace __000003=sum(__000003)
variable __000003 not found

came from within the depths of the xi code. We see that the code that was supposed to generate the temporary variable __000003 did not generate any observations.

This is almost certainly for the reason described by Nick in post #6. The first time through the loop you kept only the observations for ksic1c == 3. Then the next time through the loop you kept only the observations for ksic1c == 10. But because you had not read your data back in, there were no observations with ksic1c==10 to be kept.

Change this

use "D:\markupest\firmdata.d3.dta", clear
foreach i of numlist 3 10/17 19/33 35 41 42 45/47 49/52 58/64 71 72 75 91{
keep if ksic1c == `i'
// ksic1c: 2-digit KSIC

to this

foreach i of numlist 3 10/17 19/33 35 41 42 45/47 49/52 58/64 71 72 75 91{
use "D:\markupest\firmdata.d3.dta", clear
keep if ksic1c == `i'
noisily display "KSIC `i' `c(N)' observations"
// ksic1c: 2-digit KSIC

or better yet

foreach i of numlist 3 10/17 19/33 35 41 42 45/47 49/52 58/64 71 72 75 91{
use if ksic1c == `i'  using "D:\markupest\firmdata.d3.dta", clear
noisily display "KSIC `i' `c(N)' observations"
// ksic1c: 2-digit KSIC

[Chul-Kyoo Jung]

William Lisowski , thank you so much. It is working and wondering if I could ask just one last question:

The codes are supposed to generate each DLW simplified ind `i'.dta.
And my eventual goal is to append all those files as the following last codes of my original codes indicate.

* collect markup estimates and productivity for analysis:
keep firmid year mu_dlwS tfp_dlwS lbg30 lny lny_10 lnl lnk_10 beta_*S
// nind: KSIC 5-digit, region: province, surv: A dummy for surviving firms
save "DLW simplified ind `i'.dta"

*** Appending the data sets
use "DLW simplified ind 3.dta", clear
save "DLW simplified total.dta", replace

foreach I of numlist 10/17 19/33 35 41 42 45/47 49/52 58/64 71 72 75 91{
    use "DLW simplified total.dta", clear
    append using "DLW simplified ind `I'.dta"
    save "DLW simplified total.dta", replace
}

But I cannot find those files in my working directory; instead only I could see the following error message.
I was wondering why I can't really produce those files separately and any way to fix the problem so that I may append them.
Thanks again for your help.

. *** Appending the data sets
. use "DLW simplified ind 3.dta", clear
file DLW simplified ind 3.dta not found
r(601);

end of do-file

[William Lisowski]

The forvalues i loop ends long before you reach this code that creates the datasets you want to append.

* collect markup estimates and productivity for analysis:
keep firmid year mu_dlwS tfp_dlwS lbg30 lny lny_10 lnl lnk_10 beta_*S
// nind: KSIC 5-digit, region: province, surv: A dummy for surviving firms
save "DLW simplified ind `i'.dta"

[Chul-Kyoo Jung]

William Lisowski , Thank you.
I am not quite sure that I understand your comments.
So do you suggest that I should include all the codes of appending before the loop ends.
For example,

* collect markup estimates and productivity for analysis:
keep firmid year ksic1c mu_dlwS tfp_dlwS lbg30 lny lny_10 lnl lnk_10 beta_*S

save "DLW simplified ind `i'.dta"

*** Appending the data sets
use "DLW simplified ind 3.dta", clear
save "DLW simplified total.dta", replace

foreach I of numlist 10/17 19/33 35 41 42 45/47 49/52 58/64 71 72 75 91{
    use "DLW simplified total.dta", clear
    append using "DLW simplified ind `I'.dta"
    save "DLW simplified total.dta", replace
}

Otherwise, could you specify the desirable location of the codes? Thank you.

[William Lisowski]

I do not understand your complicated code well enough to feel confident rearranging it, and I have attempted to do so several times now, but withdrew each attempt because I haven't any idea what the goal of the code is, starting with the fact that I do not work in finance. Further explanation of the code will not help me.

All I can tell you is that what follows is the complete foreach loop in post #1

foreach i of numlist 3 10/17 19/33 35 41 42 45/47 49/52 58/64 71 72 75 91{
use "D:\markupest\firmdata.d3.dta", clear
keep if ksic1c == `i'
// ksic1c: 2-digit KSIC
*----------------------------------------------------------------------------------------------------*
* OLS REGRESSION FOR STARTING VALUES & First stage
// xi: reg y l k i.year
xi: reg lny_10 lnl lnk_10 i.year                        
// lny_10: log of Real Value Added (denominated by 2010 PPI), lnl: log of Number of Workers, lnk_10: log of Real Average Capital Stock (denominated by 2010 PPI)
gen blols=_b[lnl]
gen bkols=_b[lnk_10]
predict phiS
predict epsilon, res
label var phiS "Expected value added"
label var epsilon "Measurement error first stage"

bys firmid (year): gen phiS_lag=phiS[_n-1]                // firmid: company ID
bys firmid (year): gen lnl_lag=lnl[_n-1]
bys firmid (year): gen lnk_10_lag=lnk_10[_n-1]
*---COMPUTE CORRECTED SHARES-------------------------------------------------------------------------*
gen va_c=exp(lny_10-epsilon)
gen alpha_l=wb_10/va_c                                // wb: Real Wage Bill (denominated by 2010 PPI)
*---------------------------------------------------------------------------------------------------*
drop _I*
sort firmid year
gen const=1
drop if lnl_lag==.
drop if lnk_10==.
drop if phiS==.
drop if phiS_lag==.
drop if lny_10==.
}

After that loop is done, the remaining code is executed only one time, using the data that remains in memory at the end of the loop, and the local macro i is no longer defined.