Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • imputing panel data - number of imputations to be added

    hello sirs

    I'm imputing panel data and forced stata to impute, because the data on independent variables are missing as well.

    I did something like: <mi impute regress a b c x y z add(1000) rseed(1234) force>

    Now stata is running for more that an hour, and not finished calculating.

    did I make any mistake in specifying the imputation?

    Please advise.

  • #2
    Probably not a problem. Multiple imputation is computationally intensive. For 1,000 imputations, an hour is not a long time at all. Figure more like a day for that many. Anyway, in the future, when running -mi impute-, specify the -dots- option so you'll be able to see progress along the way and not have to wonder if it is still calculating.

    Why so many, though? It's rarely necessary to do that many imputations.

    Also, if this is panel data, then you should have in in wide layout for the imputation, and then when you're done, use -mi reshape- to put it back to long for analysis. The values of any variable in other time periods are among the most useful sources of information for multiple imputation of that same variable in panel data. This is one of those uncommon Stata situations where wide layout is better. (But you must -mi reshape- it back to long for analysis afterward.)

    Comment


    • #3
      Thanks Clyde for the guidance. Much appreciated.

      Comment


      • #4
        Now Stata has imputed missing values (after four hours).

        But I encountered another problem:

        why there is no dynamic panel data (DPD) regression option under the "MI Estimates by Fitting Data"?

        I want to use Dynamic Panel Data (DPD) - but this option is missing in the "MI Estimates by fitting Model" panel under the MI.

        Comment


        • #5
          I take it you are using the menu interface. I never use it, so I'm not familiar with it. Suffice it to say that -help mi estimate-, then click on the -estimation commands- link, shows you all the commands that are supported by -mi estimate-. All of the basic panel data commands are there. If by dynamic panel data you mean things like Arellano-Bond, that is not supported. You can still run it, if you want, with the -cmdok- option specified for -mi estimate-. But you do that at your own risk: as a general rule if a Stata command doesn't support something, it's because that something is not appropriate to use in that context. I'm not enough of an expert on either these more "exotic" panel data commands, nor on multiple imputation, to know if the two are statistically compatible. Either way, Stata will give you results, but they may or may not be meaningful.

          There are other Forum members who know more about both of these things than I do, and perhaps one of them will chime in.
          Last edited by Clyde Schechter; 16 Aug 2020, 22:39.

          Comment


          • #6
            Originally posted by kedar badu View Post
            I'm imputing panel data and forced stata to impute, because the data on independent variables are missing as well.
            You (almost) never want force. If your so-called independent variables have missing values, they are not independent variables in mi's sense. Instead of adding the force option, drop the equals sign, and impute all missing values.

            Comment


            • #7
              Thank you very much Daniel and Clyde.

              Much appreciated.

              Comment


              • #8
                Originally posted by Clyde Schechter View Post
                Probably not a problem. Multiple imputation is computationally intensive. For 1,000 imputations, an hour is not a long time at all. Figure more like a day for that many. Anyway, in the future, when running -mi impute-, specify the -dots- option so you'll be able to see progress along the way and not have to wonder if it is still calculating.

                Why so many, though? It's rarely necessary to do that many imputations.

                Also, if this is panel data, then you should have in in wide layout for the imputation, and then when you're done, use -mi reshape- to put it back to long for analysis. The values of any variable in other time periods are among the most useful sources of information for multiple imputation of that same variable in panel data. This is one of those uncommon Stata situations where wide layout is better. (But you must -mi reshape- it back to long for analysis afterward.)
                Hi Clyde,

                Why is it better to use a wide set up when using panel data for mi? I already have my current data set up in long format. Is this a problem when wanting to do mi estimates. Beside, I wish to do imputation which is based on an IDs own earlier observations, and not on the global average. Is this possible. My data consists of 195 countries over 31 years. For some countries, due to their statistical prowess (usually LDEs, least developed economies) have a lack of observations and do not want advanced economies influencing the imputation of the data of the LDEs.

                Comment


                • #9
                  I think you are misunderstanding long and wide layouts.

                  If your data are in the standard panel data layout, you have multiple observations per panel, one for each time period. While it could be done another way, the usual wide layout for panel data has one observation per panel, and multiple versions of the long layout's variables, one version for each time period. The reason you need a wide layout for mi is that you want to be able to impute values from any time period using values from all time periods. The way -mi impute- works, all of the values that you use for imputation must be in the same observation as the value being imputed. So in a long layout, you would be restricted to imputing values using other variables in the same time period. The wide layout will enable to use information from all time periods to impute values because those values, in the wide layout, are just different variables in the same observation.

                  Once you have done the multiple imputations, you would most likely then -mi reshape- the resulting data set back to long layout for carrying out the analyses, as Stata's panel data commands usually (actually, always as far as I know) require long data.

                  Comment


                  • #10
                    Hi Clyde, Thanks for the response. This is the first time I have read this, and as such I am a bit confused. I will share my current data and dofile. If you perform the dofile until row 449 you can see what I am working with. Hopefully you can give me some guidance! I will add in a link to where the data can be downloaded as it exceeds the file size allowed. https://datacatalog.worldbank.org/se...ataset/0037651. To make it easier for you, here I have a screenshot of my data:

                    Click image for larger version

Name:	Knipsel.PNG
Views:	1
Size:	234.4 KB
ID:	1756726
                    Attached Files

                    Comment


                    • #11
                      The Forum FAQ, which all members are supposed to read before they post, explain the most effective ways to share code and data here. Attachments are strongly discouraged. Some members, including me, will not download attachments from people we don't know. Similarly, I don't click on links provided by strangers. As they say, only the paranoid survive on the internet! FAQ #12 gives detailed information about the best ways to show code and data.

                      Although screenshots are not usually helpful, in this case I think I can explain based on your screenshot. You appear to have a standard panel data set in long layout, where the panel identifier is country (or iso3 or panel id) and the time variable is year. Let's look at the first observation. It has missing values for many variables. Some of the variables have missing values in every year, and some have missing values in some years but not others.

                      If you do -mi impute- on this data set as it stands, to impute values for, say, ABW in year 2001, you will only be able to use the values of variables in year 2001 for the purpose. But that's a very severe restriction that eviscerates the strength of multiple imputation. It is almost certainly true that many, if not all, of the variables are time series that have trends or other patterns in them. The values in other years are informative about the values in 2001, and you should not discard that information.

                      The solution is to first put the data set into wide layout. Something like this:
                      Code:
                      ds country iso3 panelid, not
                      reshape wide `r(varlist)', i(iso3) j(year)
                      The resulting data set will now have just one observation per country. It will have a large number of variables, variables with names like AG_LND_1900, AG_LND_1901, ...,
                      AG_LND_2021, ..., EG_USE_1900, EG_USE_1901, ..., EG_USE_2021. Now, because every year's values are all in one observation for the country, if you run -mi impute- with this version of the data, you will be able to draw on all years' values of the variables to impute any missing values, regardless of which year(s) the missing values come from. This leverages the information provided by any trends that exist in the time series of your variables. It is a much better imputation. Once you have done that, -mi reshape long- will bring you back to long layout for analysis with -mi estimate:-.

                      Comment


                      • #12
                        Thanks for the elaborate explanation Clyde. Some countries dont have any observations for certain variables, and as such I would like to skip the imputation process for those countries. At the same time, which format of imputation should I then use? f or mlong wide?

                        Comment


                        • #13
                          Hi Clyde, unfortunately your code doesn't work as it told me that factor-variable and time-series operators are not allowed when doing the reshide wide command. I will add in my dofile as code. However, it is in extensive piece.

                          Code:
                          reshape long y, i(ind iso3) j(year)
                          rename y y_
                          replace ind=subinstr(ind,".","_",.)
                          reshape wide y_, i(iso3 year) j(ind) string
                          rename y_* *
                          
                          
                          * Now all is formatted into how I want it to. Now I will label all the variables to know what they measure When doubting what a variable means, look it up on: https://datacatalog.worldbank.org/search/dataset/0037651 *
                          
                          label var AG_LND_AGRI_ZS "Agricultural land as a percentage of land area"
                          label var AG_LND_FRST_ZS "Forest area as a percentage of land area"
                          label var EN_ATM_CO2E_PC "CO2 emissions in metric tons per capita"
                          label var EN_ATM_CO2E_KT "CO2 emissions in kilotonnes"
                          label var EN_CLC_GHGR_MT_CE "GHG net emissions/removals by LUCF in Mt of CO2 equivalent"
                          label var EN_ATM_METH_PC "Methane emissions in metric tons of CO2 equivalent per capita"
                          label var EN_ATM_NOXE_PC "Nitrous oxide emissions in metric tons of CO2 equivalent per capita"
                          label var EN_ATM_PM25_MC_M3 "PM2.5 air pollution, mean annual exposure in micrograms per cubic meter"
                          label var EG_ELC_COAL_ZS "Electricity production from coal sources as a percentage of total"
                          label var EG_IMP_CONS_ZS "Net energy imports as a percentage of energy use"
                          label var EG_EGY_PRIM_PP_KD "Energy intensity level of primary energy (MJ/$2017 PPP GDP)"
                          label var EG_USE_PCAP_KG_OE "Energy use in kg of oil equivalent per capita"
                          label var EG_USE_COMM_FO_ZS "Fossil fuel energy consumption as a percentage of total"
                          label var EG_ELC_RNEW_ZS "Renewable electricity output as a percentage of total electricity output"
                          label var EG_FEC_RNEW_ZS "Renewable energy consumption as a percentage of total final energy consumption"
                          label var EN_POP_DNST "Population density (people per square mile of land area)"
                          label var EN_CLC_CDDY_XD "Cooling degree days"
                          label var EN_CLC_HDDY_XD "Heating degree days"
                          label var EN_CLC_HEAT_XD "Heat index 35"
                          label var EN_CLC_SPEI_XD "Standardised precipitation-evapotranspiration index"
                          label var EN_CLC_LTMP_XD "Land surface temperature"
                          label var EN_CLC_CSTP_XD "Coastal protection"
                          label var EN_CLC_WTR_STRS "Level of water stress: freshwater withdrawal as a proportion of available freshwater resources"
                          label var EN_CLC_WTR_BDYS "Proportion of bodies of water with good ambient water quality"
                          label var NV_AGR_TOTL_ZS "Agriculture, forestry, and fishing, vallue added as a percentage of GDP"
                          label var AG_PRD_FOOD_XD "Food production index (2014-2016 = 100)"
                          label var NY_ADJ_DRES_GN_ZS "Adjusted savings: natural resources depletion as a percentage of GNI"
                          label var NY_ADJ_DFOR_GN_ZS "Adjusted savings: net forest depletion as a percentage of GNI"
                          label var ER_H2O_FWTL_ZS "Annual freshwater withdrawals, total (% of internal resources)"
                          label var EN_MAM_THRD_NO "Mammal species, threatened"
                          label var ER_PTD_TOTL_ZS "Terrestrial and marine protected ares (% of total territorial area)"
                          label var AG_LND_FRLS_XD "Forest cover loss"
                          label var EG_CFT_ACCS_ZS "Access to clean fuels and technologies for cooking (% of population)"
                          label var EG_ELC_ACCS_ZS "Access to electricity (% of population)"
                          label var SH_H2O_SMDW_ZS "People using safely managed drinking water services (% of population)"
                          label var SH_STA_SMSS_ZS "People using safely managed sanitation services (% of population)"
                          label var SP_DYN_TFRT_IN "Fertility rate, total (births per woman)"
                          label var SP_DYN_LE00_IN "Life expectancy at birth, total (years)"
                          label var SP_POP_65UP_TO_ZS "Population ages 65 and above (% of total population)"
                          label var SE_XPD_TOTL_GB_ZS "Government expenditure on education, total (% of government expenditure)"
                          label var SE_ADT_LITR_ZS "Literacy rate, adult total (% of people ages 15 and above)"
                          label var SE_PRM_ENRR "School enrollment, primary (% gross)"
                          label var SL_TLF_0714_ZS "Children in employment, total (% of children ages 7-14)"
                          label var SL_TLF_ACTI_ZS "Labour force participation rate, total (% of total population ages 15-64) (Modeled ILO estimate)"
                          label var SL_UEM_TOTL_ZS "Unemployment, total (% of labour force) (Modeled ILO estimate)"
                          label var SH_DTH_COMM_ZS "Cause of death, by communicable diseases and maternal, prenatal and nutrition conditions (% of total)"
                          label var SH_MED_BEDS_ZS "Hospital beds (per 1,000 people)"
                          label var SH_DYN_MORT "Mortality reate, under-5 (per 1,000 live births)"
                          label var SH_STA_OWAD_ZS "Prevalence of overwight (% of adults)"
                          label var SN_ITK_DEFC_ZS "Prevalence of undernourishment (% of population)"
                          label var SI_SPR_PCAP_ZG "Annualized average growth rate in per capita real survey mean consumption or income, total population (%)"
                          label var SI_POV_GINI "Gini index"
                          label var SI_DST_FRST_20 "Income share held by lowest 20%"
                          label var SI_POV_NAHC "Poverty headcount ratio at national poverty lines (% of population)"
                          label var NY_GDP_MKTP_KD_ZG "GDP growth (annual %)"
                          label var IT_NET_USER_ZS "Individuals using he internet (% of population)"
                          label var SG_GEN_PARL_ZS "Proportion of seats held by women in national parliaments (%)"
                          label var SL_TLF_CACT_FM_ZS "Ratio of female to male labour force participation rate (%) (Modeled ILO estimate)"
                          label var SE_ENR_PRSC_FM_ZS "School enrollment, primary and secondary (gross), gender parity index (GPI)"
                          label var SP_UWT_TFRT "Unmet need for contraception (% of married women ages 15-49)"
                          label var GE_EST "Government effectiveness: estimate"
                          label var RQ_EST "Regulatory quality: estimate"
                          label var IC_LGL_CRED_XQ "Strength of legal rights index (0 = weak to 12 = strong)"
                          label var VA_EST "Voice and accountability: estimate"
                          label var SG_ESR_XD "Economic and social rights performance score"
                          label var IP_PAT_RESD "Patent applications, residents"
                          label var GB_XPD_RSDV_GD_ZS "Research and development expenditure (% of GDP)"
                          label var IP_JRN_ARTC_SC "Scientific and technical journal articles"
                          label var CC_EST "Control of corruption: estimate"
                          label var SM_POP_NETM "Net migration"
                          label var PV_EST "Political stability and absence of violence/terrorism: estimate"
                          label var RL_EST "Rule of law: estimate"
                          label var EN_CLC_MDAT_ZS "Droughts, floods, extreme temperature (% of population, average 1990 - 2009)"
                          label var IC_BUS_EASE_XQ "Ease of doing business index (1 = most business-friendly regulations)"
                          label var EN_CLC_PRCP_XD "Maximum 5-day rainfall, 25-year return level (projected change in mm)"
                          
                          * All variables have been dropped for the years until 1990. This due to the fact that most variables have no recorded data for <1990 *
                          
                          drop if year<1990
                          
                          * one weird thing: I have 289 unique iso3 identifiers. Some which should not exist. *
                          
                          gen numbcountry = 1 if iso3[_n] != iso3[_n-1]
                          sum numbcountry
                          sort numbcountry (iso3)
                          
                          * for own clarity I create a var that gives the name of every country according to the iso3 code *
                          gen country="."
                          order country
                          replace country = "Afghanistan" if iso3 == "AFG"
                          replace country = "Albania" if iso3 == "ALB"
                          replace country = "Algeria" if iso3 == "DZA"
                          replace country = "Andorra" if iso3 == "AND"
                          replace country = "Angola" if iso3 == "AGO"
                          replace country = "Antigua and Barbuda" if iso3 == "ATG"
                          replace country = "Argentina" if iso3 == "ARG"
                          replace country = "Armenia" if iso3 == "ARM"
                          replace country = "Aruba" if iso3 == "ABW"
                          replace country = "Australia" if iso3 == "AUS"
                          replace country = "Austria" if iso3 == "AUT"
                          replace country = "Azerbaijan" if iso3 == "AZE"
                          replace country = "Bahamas" if iso3 == "BHS"
                          replace country = "Bahrain" if iso3 == "BHR"
                          replace country = "Bangladesh" if iso3 == "BGD"
                          replace country = "Barbados" if iso3 == "BRB"
                          replace country = "Belarus" if iso3 == "BLR"
                          replace country = "Belgium" if iso3 == "BEL"
                          replace country = "Belize" if iso3 == "BLZ"
                          replace country = "Benin" if iso3 == "BEN"
                          replace country = "Bhutan" if iso3 == "BTN"
                          replace country = "Bolivia" if iso3 == "BOL"
                          replace country = "Bosnia and Herzegovina" if iso3 == "BIH"
                          replace country = "Botswana" if iso3 == "BWA"
                          replace country = "Brazil" if iso3 == "BRA"
                          replace country = "Brunei" if iso3 == "BRN"
                          replace country = "Bulgaria" if iso3 == "BGR"
                          replace country = "Burkina Faso" if iso3 == "BFA"
                          replace country = "Burundi" if iso3 == "BDI"
                          replace country = "Cabo Verde" if iso3 == "CPV"
                          replace country = "Cambodia" if iso3 == "KHM"
                          replace country = "Cameroon" if iso3 == "CMR"
                          replace country = "Canada" if iso3 == "CAN"
                          replace country = "Central African Republic" if iso3 == "CAF"
                          replace country = "Chad" if iso3 == "TCD"
                          replace country = "Chile" if iso3 == "CHL"
                          replace country = "China" if iso3 == "CHN"
                          replace country = "Colombia" if iso3 == "COL"
                          replace country = "Comoros" if iso3 == "COM"
                          replace country = "Congo" if iso3 == "COG"
                          replace country = "Costa Rica" if iso3 == "CRI"
                          replace country = "Croatia" if iso3 == "HRV"
                          replace country = "Cuba" if iso3 == "CUB"
                          replace country = "Cyprus" if iso3 == "CYP"
                          replace country = "Czechia" if iso3 == "CZE"
                          replace country = "Democratic Republic of the Congo" if iso3 == "COD"
                          replace country = "Denmark" if iso3 == "DNK"
                          replace country = "Djibouti" if iso3 == "DJI"
                          replace country = "Dominica" if iso3 == "DMA"
                          replace country = "Dominican Republic" if iso3 == "DOM"
                          replace country = "East Timor" if iso3 == "TLS"
                          replace country = "Ecuador" if iso3 == "ECU"
                          replace country = "Egypt" if iso3 == "EGY"
                          replace country = "El Salvador" if iso3 == "SLV"
                          replace country = "Equatorial Guinea" if iso3 == "GNQ"
                          replace country = "Eritrea" if iso3 == "ERI"
                          replace country = "Estonia" if iso3 == "EST"
                          replace country = "Eswatini" if iso3 == "SWZ"
                          replace country = "Ethiopia" if iso3 == "ETH"
                          replace country = "Fiji" if iso3 == "FJI"
                          replace country = "Finland" if iso3 == "FIN"
                          replace country = "France" if iso3 == "FRA"
                          replace country = "Gabon" if iso3 == "GAB"
                          replace country = "Gambia" if iso3 == "GMB"
                          replace country = "Georgia" if iso3 == "GEO"
                          replace country = "Germany" if iso3 == "DEU"
                          replace country = "Ghana" if iso3 == "GHA"
                          replace country = "Greece" if iso3 == "GRC"
                          replace country = "Grenada" if iso3 == "GRD"
                          replace country = "Guatemala" if iso3 == "GTM"
                          replace country = "Guinea" if iso3 == "GIN"
                          replace country = "Guinea-Bissau" if iso3 == "GNB"
                          replace country = "Guyana" if iso3 == "GUY"
                          replace country = "Haiti" if iso3 == "HTI"
                          replace country = "Honduras" if iso3 == "HND"
                          replace country = "Hungary" if iso3 == "HUN"
                          replace country = "Iceland" if iso3 == "ISL"
                          replace country = "India" if iso3 == "IND"
                          replace country = "Indonesia" if iso3 == "IDN"
                          replace country = "Iran" if iso3 == "IRN"
                          replace country = "Iraq" if iso3 == "IRQ"
                          replace country = "Ireland" if iso3 == "IRL"
                          replace country = "Israel" if iso3 == "ISR"
                          replace country = "Italy" if iso3 == "ITA"
                          replace country = "Ivory Coast" if iso3 == "CIV"
                          replace country = "Jamaica" if iso3 == "JAM"
                          replace country = "Japan" if iso3 == "JPN"
                          replace country = "Jordan" if iso3 == "JOR"
                          replace country = "Kazakhstan" if iso3 == "KAZ"
                          replace country = "Kenya" if iso3 == "KEN"
                          replace country = "Kiribati" if iso3 == "KIR"
                          replace country = "North Korea" if iso3 == "PRK"
                          replace country = "South Korea" if iso3 == "KOR"
                          replace country = "Kuwait" if iso3 == "KWT"
                          replace country = "Kyrgyzstan" if iso3 == "KGZ"
                          replace country = "Laos" if iso3 == "LAO"
                          replace country = "Latvia" if iso3 == "LVA"
                          replace country = "Lebanon" if iso3 == "LBN"
                          replace country = "Lesotho" if iso3 == "LSO"
                          replace country = "Liberia" if iso3 == "LBR"
                          replace country = "Libya" if iso3 == "LBY"
                          replace country = "Liechtenstein" if iso3 == "LIE"
                          replace country = "Lithuania" if iso3 == "LTU"
                          replace country = "Luxembourg" if iso3 == "LUX"
                          replace country = "Madagascar" if iso3 == "MDG"
                          replace country = "Malawi" if iso3 == "MWI"
                          replace country = "Malaysia" if iso3 == "MYS"
                          replace country = "Maldives" if iso3 == "MDV"
                          replace country = "Mali" if iso3 == "MLI"
                          replace country = "Malta" if iso3 == "MLT"
                          replace country = "Marshall Islands" if iso3 == "MHL"
                          replace country = "Mauritania" if iso3 == "MRT"
                          replace country = "Mauritius" if iso3 == "MUS"
                          replace country = "Mexico" if iso3 == "MEX"
                          replace country = "Micronesia" if iso3 == "FSM"
                          replace country = "Moldova" if iso3 == "MDA"
                          replace country = "Monaco" if iso3 == "MCO"
                          replace country = "Mongolia" if iso3 == "MNG"
                          replace country = "Montenegro" if iso3 == "MNE"
                          replace country = "Morocco" if iso3 == "MAR"
                          replace country = "Mozambique" if iso3 == "MOZ"
                          replace country = "Myanmar" if iso3 == "MMR"
                          replace country = "Namibia" if iso3 == "NAM"
                          replace country = "Nauru" if iso3 == "NRU"
                          replace country = "Nepal" if iso3 == "NPL"
                          replace country = "Netherlands" if iso3 == "NLD"
                          replace country = "New Zealand" if iso3 == "NZL"
                          replace country = "Nicaragua" if iso3 == "NIC"
                          replace country = "Niger" if iso3 == "NER"
                          replace country = "Nigeria" if iso3 == "NGA"
                          replace country = "North Macedonia" if iso3 == "MKD"
                          replace country = "Norway" if iso3 == "NOR"
                          replace country = "Oman" if iso3 == "OMN"
                          replace country = "Pakistan" if iso3 == "PAK"
                          replace country = "Palau" if iso3 == "PLW"
                          replace country = "Panama" if iso3 == "PAN"
                          replace country = "Papua New Guinea" if iso3 == "PNG"
                          replace country = "Paraguay" if iso3 == "PRY"
                          replace country = "Peru" if iso3 == "PER"
                          replace country = "Philippines" if iso3 == "PHL"
                          replace country = "Poland" if iso3 == "POL"
                          replace country = "Portugal" if iso3 == "PRT"
                          replace country = "Qatar" if iso3 == "QAT"
                          replace country = "Romania" if iso3 == "ROU"
                          replace country = "Russia" if iso3 == "RUS"
                          replace country = "Rwanda" if iso3 == "RWA"
                          replace country = "Saint Kitts and Nevis" if iso3 == "KNA"
                          replace country = "Saint Lucia" if iso3 == "LCA"
                          replace country = "Saint Vincent and the Grenadines" if iso3 == "VCT"
                          replace country = "Samoa" if iso3 == "WSM"
                          replace country = "San Marino" if iso3 == "SMR"
                          replace country = "Sao Tome and Principe" if iso3 == "STP"
                          replace country = "Saudi Arabia" if iso3 == "SAU"
                          replace country = "Senegal" if iso3 == "SEN"
                          replace country = "Serbia" if iso3 == "SRB"
                          replace country = "Seychelles" if iso3 == "SYC"
                          replace country = "Sierra Leone" if iso3 == "SLE"
                          replace country = "Singapore" if iso3 == "SGP"
                          replace country = "Slovakia" if iso3 == "SVK"
                          replace country = "Slovenia" if iso3 == "SVN"
                          replace country = "Solomon Islands" if iso3 == "SLB"
                          replace country = "Somalia" if iso3 == "SOM"
                          replace country = "South Africa" if iso3 == "ZAF"
                          replace country = "South Sudan" if iso3 == "SSD"
                          replace country = "Spain" if iso3 == "ESP"
                          replace country = "Sri Lanka" if iso3 == "LKA"
                          replace country = "Sudan" if iso3 == "SDN"
                          replace country = "Suriname" if iso3 == "SUR"
                          replace country = "Sweden" if iso3 == "SWE"
                          replace country = "Switzerland" if iso3 == "CHE"
                          replace country = "Syria" if iso3 == "SYR"
                          replace country = "Tajikistan" if iso3 == "TJK"
                          replace country = "Tanzania" if iso3 == "TZA"
                          replace country = "Thailand" if iso3 == "THA"
                          
                          * Current problem: there are a lot of small or dependent territories like Greenland and Reunion, which do not add anything to our analysis and may even dilute them as they cannot be seen as independent countries *
                          * I have also found out certain economies have been listed together. Like ISO3 code ARB which codifies the arab world economies *
                          replace country = "Arab World" if iso3 == "ARB"
                          replace country = "Central Europe and the Baltics" if iso3 == "CEB"
                          replace country = "Caribean Small States" if iso3 == "CSS"
                          replace country = "East Asia and Pacific (Excluding high income)" if iso3 == "EAP"
                          replace country = "East Asia and Pacific" if iso3 == "EAS"
                          replace country = "Europe and Central Asia (excluding high income)" if iso3 == "ECA"
                          replace country = "Europe and Central Asia" if iso3 == "ECS"
                          replace country = "Euro Area" if iso3 == "EMU"
                          replace country = "European Union" if iso3 == "EUU"
                          replace country = "Fragile and conflict affected situations" if iso3 == "FCS"
                          replace country = "High income" if iso3 == "HIC"
                          replace country = "Heavily indebted poor countries" if iso3 == "HPC"
                          replace country = "IBRD only" if iso3 == "IBD"
                          replace country = "IDA and IBRD total" if iso3 == "IBT"
                          replace country = "IDA total" if iso3 == "IDA"
                          replace country = "IDA blend" if iso3 == "IDB"
                          replace country = "IDA only" if iso3 == "IDX"
                          replace country = "Latin America and Caribbean (excluding high income)" if iso3 == "LAC"
                          replace country = "Latin America and Caribbean" if iso3 == "LCN"
                          replace country = "Least developed countries (UN classification)" if iso3 == "LDC"
                          replace country = "Low income" if iso3 == "LIC"
                          replace country = "Lower middle income" if iso3 == "LMC"
                          replace country = "Low and middle income" if iso3 == "LMY"
                          replace country = "Middle income" if iso3 == "MIC"
                          replace country = "Middle East and North Africa" if iso3 == "MEA"
                          replace country = "Middle East and North Africa (excluding high income)" if iso3 == "MNA"
                          replace country = "North America" if iso3 == "NAC"
                          replace country = "OECD members" if iso3 == "OED"
                          replace country = "Other small states" if iso3 == "OSS"
                          replace country = "Pacific island small states" if iso3 == "PSS"
                          replace country = "South Asia" if iso3 == "SAS"
                          replace country = "Sub-Saharan Africa (excluding high income)" if iso3 == "SSA"
                          replace country = "Sub-Saharan Africa" if iso3 == "SSF"
                          replace country = "Upper middle income" if iso3 == "UMC"
                          replace country = "World" if iso3 == "WLD"
                          replace country = "Africa Eastern and Southern" if iso3 == "AFE"
                          replace country = "Africa Western and Central" if iso3 == "AFW"
                          
                          * Kosova has, due to some political disputes, 2 iso3 codes *
                          replace country = "Kosovo2" if iso3 == "XXK"
                          
                          * After incoporating all these regions, there are a few that are not interesting for our analysis, like the borrowing class per region. I have dropped these 12 iso3 codes *
                          drop if country == "."
                          drop numbcountry
                          sort iso3 (year)
                          
                          * Now we have our overview correct. Now we can properly follow the paper*
                          * However, we have way more countries than we are going to use for our analysis. In the end, we need to have the countries which have data in the productivity dataset. I will drop all other observations, as those just make life more difficult whilst dealing with missing values. *
                          
                          drop if iso3 != "AFG" & iso3 != "ALB" & iso3 != "DZA" & iso3 != "AND" & iso3 != "AGO" & iso3 != "ATG" & iso3 != "ARG" & iso3 != "ARM" & iso3 != "ABW" & iso3 != "AUS" & iso3 != "AUT" & iso3 != "AZE" & iso3 != "BHS" & iso3 != "BHR" & iso3 != "BGD" & iso3 != "BRB" & iso3 != "BLR" & iso3 != "BEL" & iso3 != "BLZ" & iso3 != "BEN" & iso3 != "BTN" & iso3 != "BOL" & iso3 != "BIH" & iso3 != "BWA" & iso3 != "BRA" & iso3 != "BRN" & iso3 != "BGR" & iso3 != "BFA" & iso3 != "BDI" & iso3 != "CPV" & iso3 != "KHM" & iso3 != "CMR" & iso3 != "CAN" & iso3 != "CAF" & iso3 != "TCD" & iso3 != "CHL" & iso3 != "CHN" & iso3 != "COL" & iso3 != "COM" & iso3 != "COG" & iso3 != "CRI" & iso3 != "HRV" & iso3 != "CUB" & iso3 != "CYP" & iso3 != "CZE" & iso3 != "COD" & iso3 != "DNK" & iso3 != "DJI" & iso3 != "DMA" & iso3 != "DOM" & iso3 != "TLS" & iso3 != "ECU" & iso3 != "EGY" & iso3 != "SLV" & iso3 != "GNQ" & iso3 != "ERI" & iso3 != "EST" & iso3 != "SWZ" & iso3 != "ETH" & iso3 != "FJI" & iso3 != "FIN" & iso3 != "FRA" & iso3 != "GAB" & iso3 != "GMB" & iso3 != "GEO" & iso3 != "DEU" & iso3 != "GHA" & iso3 != "GRC" & iso3 != "GRD" & iso3 != "GTM" & iso3 != "GIN" & iso3 != "GNB" & iso3 != "GUY" & iso3 != "HTI" & iso3 != "HND" & iso3 != "HUN" & iso3 != "ISL" & iso3 != "IND" & iso3 != "IDN" & iso3 != "IRN" & iso3 != "IRQ" & iso3 != "IRL" & iso3 != "ISR" & iso3 != "ITA" & iso3 != "CIV" & iso3 != "JAM" & iso3 != "JPN" & iso3 != "JOR" & iso3 != "KAZ" & iso3 != "KEN" & iso3 != "KIR" & iso3 != "PRK" & iso3 != "KOR" & iso3 != "KWT" & iso3 != "KGZ" & iso3 != "LAO" & iso3 != "LVA" & iso3 != "LBN" & iso3 != "LSO" & iso3 != "LBR" & iso3 != "LBY" & iso3 != "LIE" & iso3 != "LTU" & iso3 != "LUX" & iso3 != "MDG" & iso3 != "MWI" & iso3 != "MYS" & iso3 != "MDV" & iso3 != "MLI" & iso3 != "MLT" & iso3 != "MHL" & iso3 != "MRT" & iso3 != "MUS" & iso3 != "MEX" & iso3 != "FSM" & iso3 != "MDA" & iso3 != "MCO" & iso3 != "MNG" & iso3 != "MNE" & iso3 != "MAR" & iso3 != "MOZ" & iso3 != "MMR" & iso3 != "NAM" & iso3 != "NRU" & iso3 != "NPL" & iso3 != "NLD" & iso3 != "NZL" & iso3 != "NIC" & iso3 != "NER" & iso3 != "NGA" & iso3 != "MKD" & iso3 != "NOR" & iso3 != "OMN" & iso3 != "PAK" & iso3 != "PLW" & iso3 != "PAN" & iso3 != "PNG" & iso3 != "PRY" & iso3 != "PER" & iso3 != "PHL" & iso3 != "POL" & iso3 != "PRT" & iso3 != "QAT" & iso3 != "ROU" & iso3 != "RUS" & iso3 != "RWA" & iso3 != "KNA" & iso3 != "LCA" & iso3 != "VCT" & iso3 != "WSM" & iso3 != "SMR" & iso3 != "STP" & iso3 != "SAU" & iso3 != "SEN" & iso3 != "SRB" & iso3 != "SYC" & iso3 != "SLE" & iso3 != "SGP" & iso3 != "SVK" & iso3 != "SVN" & iso3 != "SLB" & iso3 != "SOM" & iso3 != "ZAF" & iso3 != "SSD" & iso3 != "ESP" & iso3 != "LKA" & iso3 != "SDN" & iso3 != "SUR" & iso3 != "SWE" & iso3 != "CHE" & iso3 != "SYR" & iso3 != "TJK" & iso3 != "TZA" & iso3 != "THA" & iso3 != "TGO" & iso3 != "TON" & iso3 != "TTO" & iso3 != "TUN" & iso3 != "TUR" & iso3 != "TKM" & iso3 != "TUV" & iso3 != "UGA" & iso3 != "UKR" & iso3 != "ARE" & iso3 != "GBR" & iso3 != "USA" & iso3 != "URY" & iso3 != "UZB" & iso3 != "VUT" & iso3 != "VAT" & iso3 != "VEN" & iso3 != "VNM" & iso3 != "YEM" & iso3 != "ZMB" & iso3 != "ZWE"
                          drop EN_CLC_PRCP_XD
                          
                          * Only downside is: this code also drops regions and economic classifications like EU and OECD-countries. However, I will make indicators for that
                          *Also, for 1 variable we miss all data: EN_CLC_PRCP_XD. This variable has 100% missing observations, and as such, we drop it
                          
                          *setting our data as panel data, as now it is somewhat cleaned up.
                          
                          by iso3 (year), sort : gen byte first = _n==1
                          gen panelid = sum(first)
                          drop first
                          order panelid, before(year)
                          
                          global id panelid
                          global t year
                          local varlist AG_LND_AGRI_ZS AG_LND_FRLS_XD AG_LND_FRST_ZS AG_PRD_FOOD_XD AG_SRF_TOTL_K2 CC_EST EG_CFT_ACCS_ZS EG_EGY_PRIM_PP_KD EG_ELC_ACCS_RU_ZS EG_ELC_ACCS_ZS EG_ELC_COAL_ZS EG_ELC_RNEW_ZS EG_FEC_RNEW_ZS EG_IMP_CONS_ZS EG_USE_COMM_FO_ZS EG_USE_PCAP_KG_OE EN_ATM_CO2E_KT EN_ATM_CO2E_PC EN_ATM_METH_PC EN_ATM_NOXE_PC EN_ATM_PM25_MC_M3 EN_CLC_CDDY_XD EN_CLC_CSTP_XD EN_CLC_GHGR_MT_CE EN_CLC_HDDY_XD EN_CLC_HEAT_XD EN_CLC_LTMP_XD EN_CLC_MDAT_ZS EN_CLC_SPEI_XD EN_CLC_WTR_BDYS EN_CLC_WTR_STRS EN_MAM_THRD_NO EN_POP_DNST ER_H2O_FWTL_ZS ER_PTD_TOTL_ZS FP_CPI_TOTL_ZG GB_XPD_RSDV_GD_ZS GE_EST HD_HCI_OVRL IC_BUS_EASE_XQ IC_LGL_CRED_XQ IP_JRN_ARTC_SC IP_PAT_RESD IT_NET_USER_ZS NV_AGR_TOTL_ZS NW_HCA_FEMA_PC NW_HCA_FEMA_TO NW_HCA_FEMP_PC NW_HCA_FEMP_TO NW_HCA_FSEM_PC NW_HCA_FSEM_TO NW_HCA_MALE_PC NW_HCA_MALE_TO NW_HCA_MEMP_PC NW_HCA_MEMP_TO NW_HCA_MSEM_PC NW_HCA_MSEM_TO NW_HCA_PC NW_HCA_TO NW_NCA_AGRI_PC NW_NCA_AGRI_TO NW_NCA_CROL_PC NW_NCA_CROL_TO NW_NCA_FECO_PC NW_NCA_FECO_TO NW_NCA_FISH_PC NW_NCA_FISH_TO NW_NCA_FOSL_PC NW_NCA_FOSL_TO NW_NCA_FTIM_PC NW_NCA_FTIM_TO NW_NCA_MANG_PC NW_NCA_MANG_TO NW_NCA_MINR_PC NW_NCA_MINR_TO NW_NCA_PASL_PC NW_NCA_PASL_TO NW_NCA_PC NW_NCA_PRAR_PC NW_NCA_PRAR_TO NW_NCA_RNEW_PC NW_NCA_RNEW_TO NW_NCA_SACO_PC NW_NCA_SACO_TO NW_NCA_SAGA_PC NW_NCA_SAGA_TO NW_NCA_SAOI_PC NW_NCA_SAOI_TO NW_NCA_SSOI_PC NW_NCA_SSOI_TO NW_NCA_TO NW_NFA_PC NW_NFA_TO NW_PCA_PC NW_PCA_TO NW_TOW_PC NW_TOW_TO NY_ADJ_DFOR_GN_ZS NY_ADJ_DRES_GN_ZS NY_GDP_MKTP_CD NY_GDP_MKTP_KD_ZG NY_GDP_PCAP_CD PV_EST RL_EST RQ_EST SE_ADT_LITR_ZS SE_ENR_PRSC_FM_ZS SE_PRM_CMPT_ZS SE_PRM_ENRR SE_XPD_TOTL_GB_ZS SG_ESR_XD SG_GEN_PARL_ZS SH_DTH_COMM_ZS SH_DYN_MORT SH_H2O_SMDW_ZS SH_MED_BEDS_ZS SH_STA_OWAD_ZS SH_STA_SMSS_ZS SI_DST_FRST_20 SI_POV_GINI SI_POV_NAHC SI_SPR_PCAP_ZG SL_TLF_0714_ZS SL_TLF_ACTI_ZS SL_TLF_CACT_FM_ZS SL_UEM_TOTL_ZS SM_POP_NETM SN_ITK_DEFC_ZS SP_DYN_LE00_IN SP_DYN_TFRT_IN SP_POP_65UP_TO_ZS SP_POP_GROW SP_POP_TOTL SP_UWT_TFRT VA_EST
                          
                          *this excludes country, iso3, panelid and year
                          
                          local allvar country iso3 panelid year AG_LND_AGRI_ZS AG_LND_FRLS_XD AG_LND_FRST_ZS AG_PRD_FOOD_XD AG_SRF_TOTL_K2 CC_EST EG_CFT_ACCS_ZS EG_EGY_PRIM_PP_KD EG_ELC_ACCS_RU_ZS EG_ELC_ACCS_ZS EG_ELC_COAL_ZS EG_ELC_RNEW_ZS EG_FEC_RNEW_ZS EG_IMP_CONS_ZS EG_USE_COMM_FO_ZS EG_USE_PCAP_KG_OE EN_ATM_CO2E_KT EN_ATM_CO2E_PC EN_ATM_METH_PC EN_ATM_NOXE_PC EN_ATM_PM25_MC_M3 EN_CLC_CDDY_XD EN_CLC_CSTP_XD EN_CLC_GHGR_MT_CE EN_CLC_HDDY_XD EN_CLC_HEAT_XD EN_CLC_LTMP_XD EN_CLC_MDAT_ZS EN_CLC_SPEI_XD EN_CLC_WTR_BDYS EN_CLC_WTR_STRS EN_MAM_THRD_NO EN_POP_DNST ER_H2O_FWTL_ZS ER_PTD_TOTL_ZS FP_CPI_TOTL_ZG GB_XPD_RSDV_GD_ZS GE_EST HD_HCI_OVRL IC_BUS_EASE_XQ IC_LGL_CRED_XQ IP_JRN_ARTC_SC IP_PAT_RESD IT_NET_USER_ZS NV_AGR_TOTL_ZS NW_HCA_FEMA_PC NW_HCA_FEMA_TO NW_HCA_FEMP_PC NW_HCA_FEMP_TO NW_HCA_FSEM_PC NW_HCA_FSEM_TO NW_HCA_MALE_PC NW_HCA_MALE_TO NW_HCA_MEMP_PC NW_HCA_MEMP_TO NW_HCA_MSEM_PC NW_HCA_MSEM_TO NW_HCA_PC NW_HCA_TO NW_NCA_AGRI_PC NW_NCA_AGRI_TO NW_NCA_CROL_PC NW_NCA_CROL_TO NW_NCA_FECO_PC NW_NCA_FECO_TO NW_NCA_FISH_PC NW_NCA_FISH_TO NW_NCA_FOSL_PC NW_NCA_FOSL_TO NW_NCA_FTIM_PC NW_NCA_FTIM_TO NW_NCA_MANG_PC NW_NCA_MANG_TO NW_NCA_MINR_PC NW_NCA_MINR_TO NW_NCA_PASL_PC NW_NCA_PASL_TO NW_NCA_PC NW_NCA_PRAR_PC NW_NCA_PRAR_TO NW_NCA_RNEW_PC NW_NCA_RNEW_TO NW_NCA_SACO_PC NW_NCA_SACO_TO NW_NCA_SAGA_PC NW_NCA_SAGA_TO NW_NCA_SAOI_PC NW_NCA_SAOI_TO NW_NCA_SSOI_PC NW_NCA_SSOI_TO NW_NCA_TO NW_NFA_PC NW_NFA_TO NW_PCA_PC NW_PCA_TO NW_TOW_PC NW_TOW_TO NY_ADJ_DFOR_GN_ZS NY_ADJ_DRES_GN_ZS NY_GDP_MKTP_CD NY_GDP_MKTP_KD_ZG NY_GDP_PCAP_CD PV_EST RL_EST RQ_EST SE_ADT_LITR_ZS SE_ENR_PRSC_FM_ZS SE_PRM_CMPT_ZS SE_PRM_ENRR SE_XPD_TOTL_GB_ZS SG_ESR_XD SG_GEN_PARL_ZS SH_DTH_COMM_ZS SH_DYN_MORT SH_H2O_SMDW_ZS SH_MED_BEDS_ZS SH_STA_OWAD_ZS SH_STA_SMSS_ZS SI_DST_FRST_20 SI_POV_GINI SI_POV_NAHC SI_SPR_PCAP_ZG SL_TLF_0714_ZS SL_TLF_ACTI_ZS SL_TLF_CACT_FM_ZS SL_UEM_TOTL_ZS SM_POP_NETM SN_ITK_DEFC_ZS SP_DYN_LE00_IN SP_DYN_TFRT_IN SP_POP_65UP_TO_ZS SP_POP_GROW SP_POP_TOTL SP_UWT_TFRT VA_EST
                          
                          xtset $id $t
                          
                          * first step of the paper was fixing missing observations, the authors use a mean filling technique, however, the question is if this method is deemed to be correct and reliable. Simply using an indicators' mean results in less variability in the data. By just imputing the mean we are creating extra observations which are in the middle of the distribution.  
                          
                          ssc install mdesc
                          mdesc
                          
                          * We can now also see that of certain vars we have little to no observations. Above all, multiple imputation requires data to be at least missing at random, whilst we cannot say that all of our data is scertainly missing at random. Some data is not available due to statstical prowess of a country, some variables just miss some years as they have not been updated yet. We first need to find out when we can impute, as some vars miss 90% of their data. Above all, I would like to impute based on only a countries trend and not on the global trend. If that is not possible, I would like to work based on their economic development.
                          
                          ds country iso3 panelid, not
                          reshape wide r(allvar), i(iso3) j(year)
                          
                          * Creating indicator var for each var in the list)
                          foreach var of local varlist {
                              bysort panelid: egen has_obs_`var' = total(!missing(`var'))
                              gen impute_`var' = has_obs_`var' > 0
                          }
                          
                          * Setting mi structure, Question: is flong or mlong correct? mlong removes all observations for which the things are complete. Also: how does this now exactly work?
                          mi set mlong
                          foreach var of local varlist {
                              mi register imputed `var'
                          }
                          mi register regular panelid year
                          
                          * imputing missing vars
                          foreach var of local varlist {
                              mi impute regress `var' = EN_POP_DNST NY_GDP_PCAP_CD if impute_`var', add(5) rseed(54321) force
                          }
                          
                          * deleting indicator var
                          foreach var of local varlist {
                              drop has_obs_`var' impute_`var'
                          }
                          Last edited by Maarten Loomans; 21 Jun 2024, 08:43.

                          Comment


                          • #14
                            I will also add in an example of my data when the dofile is executed until ssc install mdesc
                            Code:
                            * Example generated by -dataex-. For more info, type help dataex
                            clear
                            input str3 iso3 int year double(AG_PRD_FOOD_XD CC_EST SE_PRM_ENRR)
                            "ABW" 1990           .         .           .
                            "ABW" 1991           .         .           .
                            "ABW" 1992           .         .           .
                            "ABW" 1993           .         .           .
                            "ABW" 1994           .         .           .
                            "ABW" 1995           .         .           .
                            "ABW" 1996           .         .           .
                            "ABW" 1997           .         .           .
                            "ABW" 1998           .         .           .
                            "ABW" 1999           .         .           .
                            "ABW" 2000           .         .           .
                            "ABW" 2001           .         .           .
                            "ABW" 2002           .         .           .
                            "ABW" 2003           .         .           .
                            "ABW" 2004           .  1.184472           .
                            "ABW" 2005           .  1.278153           .
                            "ABW" 2006           .  1.259531           .
                            "ABW" 2007           .  1.269965           .
                            "ABW" 2008           .  1.288655           .
                            "ABW" 2009           .  1.116361           .
                            "ABW" 2010           .  1.127541           .
                            "ABW" 2011           .  1.110036           .
                            "ABW" 2012           .  1.102628           .
                            "ABW" 2013           .   1.12658           .
                            "ABW" 2014           .  1.019108           .
                            "ABW" 2015           .   1.29756           .
                            "ABW" 2016           .  1.285871           .
                            "ABW" 2017           .  1.294589           .
                            "ABW" 2018           .  1.252447           .
                            "ABW" 2019           .  1.217546           .
                            "ABW" 2020           .  1.235405           .
                            "ABW" 2021           .         .           .
                            "AFG" 1990 52.97000122         . 29.04141998
                            "AFG" 1991 60.79999924         . 27.41422081
                            "AFG" 1992 58.93000031         .           .
                            "AFG" 1993 62.18999863         . 29.10314941
                            "AFG" 1994 65.76000214         . 40.00014114
                            "AFG" 1995 68.68000031         . 42.75143051
                            "AFG" 1996 72.29000092 -1.291705           .
                            "AFG" 1997 77.38999939         .           .
                            "AFG" 1998 81.62000275 -1.180848 31.05335045
                            "AFG" 1999 80.69000244         . 25.30797005
                            "AFG" 2000 68.62999725  -1.29538 20.95758057
                            "AFG" 2001 62.36000061         . 20.88290977
                            "AFG" 2002 75.66999817 -1.263366 68.98684692
                            "AFG" 2003       77.75 -1.351042 93.36218262
                            "AFG" 2004 75.69999695 -1.345281 104.6638184
                            "AFG" 2005 84.59999847 -1.443609 98.09989166
                            "AFG" 2006 78.34999847  -1.43289 101.6978531
                            "AFG" 2007 85.77999878 -1.587331 98.93350983
                            "AFG" 2008 80.65000153 -1.638287 100.7797318
                            "AFG" 2009 93.48000336 -1.534796 96.89640045
                            "AFG" 2010 92.91000366 -1.636177 100.0717087
                            "AFG" 2011 88.56999969 -1.579174 97.64163971
                            "AFG" 2012 97.12000275 -1.419741 103.4923935
                            "AFG" 2013 94.19999695  -1.43651 104.4984283
                            "AFG" 2014 101.6100006 -1.354829 105.9197083
                            "AFG" 2015 95.98999786 -1.342216  103.534523
                            "AFG" 2016 102.4000015 -1.526172 102.4864731
                            "AFG" 2017 99.16999817 -1.515626 102.1757889
                            "AFG" 2018 95.91000366 -1.487624 103.9961624
                            "AFG" 2019 110.6600037 -1.400733 106.7010803
                            "AFG" 2020           . -1.475405           .
                            "AFG" 2021           .         .           .
                            "AGO" 1990 19.92000008         . 73.03958893
                            "AGO" 1991 20.80999947         . 66.80436707
                            "AGO" 1992 22.26000023         . 64.24714661
                            "AGO" 1993 22.09000015         .           .
                            "AGO" 1994 24.87999916         .           .
                            "AGO" 1995 24.03000069         .           .
                            "AGO" 1996 25.82999992 -1.167702           .
                            "AGO" 1997 25.76000023         .           .
                            "AGO" 1998 29.93000031 -1.405836  71.5319519
                            "AGO" 1999 28.76000023         .           .
                            "AGO" 2000 33.59000015 -1.522685           .
                            "AGO" 2001 38.63999939         .           .
                            "AGO" 2002 43.77999878 -1.183629           .
                            "AGO" 2003 47.20999908 -1.320665           .
                            "AGO" 2004 50.97999954 -1.313559           .
                            "AGO" 2005 55.58000183 -1.296139           .
                            "AGO" 2006 56.54999924 -1.220061           .
                            "AGO" 2007 64.12999725 -1.289693 98.97357941
                            "AGO" 2008 68.26999664 -1.284621 105.1802063
                            "AGO" 2009 85.33999634 -1.404655 104.1305695
                            "AGO" 2010 90.27999878 -1.326095 105.7810364
                            "AGO" 2011 98.59999847 -1.344277 119.5301437
                            "AGO" 2012 81.11000061  -1.26847           .
                            "AGO" 2013 110.6999969 -1.309463           .
                            "AGO" 2014 97.37999725 -1.443944           .
                            "AGO" 2015 100.3000031 -1.395014 113.4779587
                            "AGO" 2016 102.3099976 -1.439528           .
                            "AGO" 2017 103.4199982 -1.410953           .
                            "AGO" 2018 107.3799973 -1.139192           .
                            "AGO" 2019 110.1600037 -1.054451           .
                            "AGO" 2020           . -.9270931           .
                            "AGO" 2021           .         .           .
                            "ALB" 1990 48.66999817         . 99.45175171
                            "ALB" 1991 41.34000015         . 101.2331467
                            "ALB" 1992 45.59000015         . 99.55587006
                            "ALB" 1993 51.52000046         . 98.52026367
                            end

                            Comment


                            • #15
                              I'm don't know why my code would have produced an error about factor variables with the -reshape- command, as it does not involve any factor variables.

                              When I take your example data and run an equivalent command (omitting mention of the country and panelid variables, which are not in the example data, but also are not really needed here) it runs just fine:
                              Code:
                              . * Example generated by -dataex-. For more info, type help dataex
                              . clear
                              
                              . input str3 iso3 int year double(AG_PRD_FOOD_XD CC_EST SE_PRM_ENRR)
                              
                                        iso3      year  AG_PRD_F~D      CC_EST  SE_PRM_E~R
                                1. "ABW" 1990           .         .           .
                                2. "ABW" 1991           .         .           .
                                3. "ABW" 1992           .         .           .
                                4. "ABW" 1993           .         .           .
                                5. "ABW" 1994           .         .           .
                                6. "ABW" 1995           .         .           .
                                7. "ABW" 1996           .         .           .
                                8. "ABW" 1997           .         .           .
                                9. "ABW" 1998           .         .           .
                               10. "ABW" 1999           .         .           .
                               11. "ABW" 2000           .         .           .
                               12. "ABW" 2001           .         .           .
                               13. "ABW" 2002           .         .           .
                               14. "ABW" 2003           .         .           .
                               15. "ABW" 2004           .  1.184472           .
                               16. "ABW" 2005           .  1.278153           .
                               17. "ABW" 2006           .  1.259531           .
                               18. "ABW" 2007           .  1.269965           .
                               19. "ABW" 2008           .  1.288655           .
                               20. "ABW" 2009           .  1.116361           .
                               21. "ABW" 2010           .  1.127541           .
                               22. "ABW" 2011           .  1.110036           .
                               23. "ABW" 2012           .  1.102628           .
                               24. "ABW" 2013           .   1.12658           .
                               25. "ABW" 2014           .  1.019108           .
                               26. "ABW" 2015           .   1.29756           .
                               27. "ABW" 2016           .  1.285871           .
                               28. "ABW" 2017           .  1.294589           .
                               29. "ABW" 2018           .  1.252447           .
                               30. "ABW" 2019           .  1.217546           .
                               31. "ABW" 2020           .  1.235405           .
                               32. "ABW" 2021           .         .           .
                               33. "AFG" 1990 52.97000122         . 29.04141998
                               34. "AFG" 1991 60.79999924         . 27.41422081
                               35. "AFG" 1992 58.93000031         .           .
                               36. "AFG" 1993 62.18999863         . 29.10314941
                               37. "AFG" 1994 65.76000214         . 40.00014114
                               38. "AFG" 1995 68.68000031         . 42.75143051
                               39. "AFG" 1996 72.29000092 -1.291705           .
                               40. "AFG" 1997 77.38999939         .           .
                               41. "AFG" 1998 81.62000275 -1.180848 31.05335045
                               42. "AFG" 1999 80.69000244         . 25.30797005
                               43. "AFG" 2000 68.62999725  -1.29538 20.95758057
                               44. "AFG" 2001 62.36000061         . 20.88290977
                               45. "AFG" 2002 75.66999817 -1.263366 68.98684692
                               46. "AFG" 2003       77.75 -1.351042 93.36218262
                               47. "AFG" 2004 75.69999695 -1.345281 104.6638184
                               48. "AFG" 2005 84.59999847 -1.443609 98.09989166
                               49. "AFG" 2006 78.34999847  -1.43289 101.6978531
                               50. "AFG" 2007 85.77999878 -1.587331 98.93350983
                               51. "AFG" 2008 80.65000153 -1.638287 100.7797318
                               52. "AFG" 2009 93.48000336 -1.534796 96.89640045
                               53. "AFG" 2010 92.91000366 -1.636177 100.0717087
                               54. "AFG" 2011 88.56999969 -1.579174 97.64163971
                               55. "AFG" 2012 97.12000275 -1.419741 103.4923935
                               56. "AFG" 2013 94.19999695  -1.43651 104.4984283
                               57. "AFG" 2014 101.6100006 -1.354829 105.9197083
                               58. "AFG" 2015 95.98999786 -1.342216  103.534523
                               59. "AFG" 2016 102.4000015 -1.526172 102.4864731
                               60. "AFG" 2017 99.16999817 -1.515626 102.1757889
                               61. "AFG" 2018 95.91000366 -1.487624 103.9961624
                               62. "AFG" 2019 110.6600037 -1.400733 106.7010803
                               63. "AFG" 2020           . -1.475405           .
                               64. "AFG" 2021           .         .           .
                               65. "AGO" 1990 19.92000008         . 73.03958893
                               66. "AGO" 1991 20.80999947         . 66.80436707
                               67. "AGO" 1992 22.26000023         . 64.24714661
                               68. "AGO" 1993 22.09000015         .           .
                               69. "AGO" 1994 24.87999916         .           .
                               70. "AGO" 1995 24.03000069         .           .
                               71. "AGO" 1996 25.82999992 -1.167702           .
                               72. "AGO" 1997 25.76000023         .           .
                               73. "AGO" 1998 29.93000031 -1.405836  71.5319519
                               74. "AGO" 1999 28.76000023         .           .
                               75. "AGO" 2000 33.59000015 -1.522685           .
                               76. "AGO" 2001 38.63999939         .           .
                               77. "AGO" 2002 43.77999878 -1.183629           .
                               78. "AGO" 2003 47.20999908 -1.320665           .
                               79. "AGO" 2004 50.97999954 -1.313559           .
                               80. "AGO" 2005 55.58000183 -1.296139           .
                               81. "AGO" 2006 56.54999924 -1.220061           .
                               82. "AGO" 2007 64.12999725 -1.289693 98.97357941
                               83. "AGO" 2008 68.26999664 -1.284621 105.1802063
                               84. "AGO" 2009 85.33999634 -1.404655 104.1305695
                               85. "AGO" 2010 90.27999878 -1.326095 105.7810364
                               86. "AGO" 2011 98.59999847 -1.344277 119.5301437
                               87. "AGO" 2012 81.11000061  -1.26847           .
                               88. "AGO" 2013 110.6999969 -1.309463           .
                               89. "AGO" 2014 97.37999725 -1.443944           .
                               90. "AGO" 2015 100.3000031 -1.395014 113.4779587
                               91. "AGO" 2016 102.3099976 -1.439528           .
                               92. "AGO" 2017 103.4199982 -1.410953           .
                               93. "AGO" 2018 107.3799973 -1.139192           .
                               94. "AGO" 2019 110.1600037 -1.054451           .
                               95. "AGO" 2020           . -.9270931           .
                               96. "AGO" 2021           .         .           .
                               97. "ALB" 1990 48.66999817         . 99.45175171
                               98. "ALB" 1991 41.34000015         . 101.2331467
                               99. "ALB" 1992 45.59000015         . 99.55587006
                              100. "ALB" 1993 51.52000046         . 98.52026367
                              101. end
                              
                              .
                              . ds iso3 year, not
                              AG_PRD_FOO~D  CC_EST        SE_PRM_ENRR
                              
                              . reshape wide `r(varlist)', i(iso3) j(year)
                              (j = 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 201
                              > 5 2016 2017 2018 2019 2020 2021)
                              
                              Data                               Long   ->   Wide
                              -----------------------------------------------------------------------------
                              Number of observations              100   ->   4           
                              Number of variables                   5   ->   97          
                              j variable (32 values)             year   ->   (dropped)
                              xij variables:
                                                       AG_PRD_FOOD_XD   ->   AG_PRD_FOOD_XD1990 AG_PRD_FOOD_XD1991 ... AG_PRD_FOOD_XD2021
                                                               CC_EST   ->   CC_EST1990 CC_EST1991 ... CC_EST2021
                                                          SE_PRM_ENRR   ->   SE_PRM_ENRR1990 SE_PRM_ENRR1991 ... SE_PRM_ENRR2021
                              -----------------------------------------------------------------------------
                              and it produces the results you need:
                              Code:
                              . des, fullnames
                              
                              Contains data
                               Observations:             4                  
                                  Variables:            97                  
                              -------------------------------------------------------------------------------------------------------------------------------------
                              Variable      Storage   Display    Value
                                  name         type    format    label      Variable label
                              -------------------------------------------------------------------------------------------------------------------------------------
                              iso3            str3    %9s                   
                              AG_PRD_FOOD_XD1990
                                              double  %10.0g                1990 AG_PRD_FOOD_XD
                              CC_EST1990      double  %10.0g                1990 CC_EST
                              SE_PRM_ENRR1990 double  %10.0g                1990 SE_PRM_ENRR
                              AG_PRD_FOOD_XD1991
                                              double  %10.0g                1991 AG_PRD_FOOD_XD
                              CC_EST1991      double  %10.0g                1991 CC_EST
                              SE_PRM_ENRR1991 double  %10.0g                1991 SE_PRM_ENRR
                              AG_PRD_FOOD_XD1992
                                              double  %10.0g                1992 AG_PRD_FOOD_XD
                              CC_EST1992      double  %10.0g                1992 CC_EST
                              SE_PRM_ENRR1992 double  %10.0g                1992 SE_PRM_ENRR
                              AG_PRD_FOOD_XD1993
                                              double  %10.0g                1993 AG_PRD_FOOD_XD
                              CC_EST1993      double  %10.0g                1993 CC_EST
                              SE_PRM_ENRR1993 double  %10.0g                1993 SE_PRM_ENRR
                              AG_PRD_FOOD_XD1994
                                              double  %10.0g                1994 AG_PRD_FOOD_XD
                              CC_EST1994      double  %10.0g                1994 CC_EST
                              SE_PRM_ENRR1994 double  %10.0g                1994 SE_PRM_ENRR
                              AG_PRD_FOOD_XD1995
                                              double  %10.0g                1995 AG_PRD_FOOD_XD
                              CC_EST1995      double  %10.0g                1995 CC_EST
                              SE_PRM_ENRR1995 double  %10.0g                1995 SE_PRM_ENRR
                              AG_PRD_FOOD_XD1996
                                              double  %10.0g                1996 AG_PRD_FOOD_XD
                              CC_EST1996      double  %10.0g                1996 CC_EST
                              SE_PRM_ENRR1996 double  %10.0g                1996 SE_PRM_ENRR
                              AG_PRD_FOOD_XD1997
                                              double  %10.0g                1997 AG_PRD_FOOD_XD
                              CC_EST1997      double  %10.0g                1997 CC_EST
                              SE_PRM_ENRR1997 double  %10.0g                1997 SE_PRM_ENRR
                              AG_PRD_FOOD_XD1998
                                              double  %10.0g                1998 AG_PRD_FOOD_XD
                              CC_EST1998      double  %10.0g                1998 CC_EST
                              SE_PRM_ENRR1998 double  %10.0g                1998 SE_PRM_ENRR
                              AG_PRD_FOOD_XD1999
                                              double  %10.0g                1999 AG_PRD_FOOD_XD
                              CC_EST1999      double  %10.0g                1999 CC_EST
                              SE_PRM_ENRR1999 double  %10.0g                1999 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2000
                                              double  %10.0g                2000 AG_PRD_FOOD_XD
                              CC_EST2000      double  %10.0g                2000 CC_EST
                              SE_PRM_ENRR2000 double  %10.0g                2000 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2001
                                              double  %10.0g                2001 AG_PRD_FOOD_XD
                              CC_EST2001      double  %10.0g                2001 CC_EST
                              SE_PRM_ENRR2001 double  %10.0g                2001 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2002
                                              double  %10.0g                2002 AG_PRD_FOOD_XD
                              CC_EST2002      double  %10.0g                2002 CC_EST
                              SE_PRM_ENRR2002 double  %10.0g                2002 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2003
                                              double  %10.0g                2003 AG_PRD_FOOD_XD
                              CC_EST2003      double  %10.0g                2003 CC_EST
                              SE_PRM_ENRR2003 double  %10.0g                2003 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2004
                                              double  %10.0g                2004 AG_PRD_FOOD_XD
                              CC_EST2004      double  %10.0g                2004 CC_EST
                              SE_PRM_ENRR2004 double  %10.0g                2004 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2005
                                              double  %10.0g                2005 AG_PRD_FOOD_XD
                              CC_EST2005      double  %10.0g                2005 CC_EST
                              SE_PRM_ENRR2005 double  %10.0g                2005 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2006
                                              double  %10.0g                2006 AG_PRD_FOOD_XD
                              CC_EST2006      double  %10.0g                2006 CC_EST
                              SE_PRM_ENRR2006 double  %10.0g                2006 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2007
                                              double  %10.0g                2007 AG_PRD_FOOD_XD
                              CC_EST2007      double  %10.0g                2007 CC_EST
                              SE_PRM_ENRR2007 double  %10.0g                2007 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2008
                                              double  %10.0g                2008 AG_PRD_FOOD_XD
                              CC_EST2008      double  %10.0g                2008 CC_EST
                              SE_PRM_ENRR2008 double  %10.0g                2008 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2009
                                              double  %10.0g                2009 AG_PRD_FOOD_XD
                              CC_EST2009      double  %10.0g                2009 CC_EST
                              SE_PRM_ENRR2009 double  %10.0g                2009 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2010
                                              double  %10.0g                2010 AG_PRD_FOOD_XD
                              CC_EST2010      double  %10.0g                2010 CC_EST
                              SE_PRM_ENRR2010 double  %10.0g                2010 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2011
                                              double  %10.0g                2011 AG_PRD_FOOD_XD
                              CC_EST2011      double  %10.0g                2011 CC_EST
                              SE_PRM_ENRR2011 double  %10.0g                2011 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2012
                                              double  %10.0g                2012 AG_PRD_FOOD_XD
                              CC_EST2012      double  %10.0g                2012 CC_EST
                              SE_PRM_ENRR2012 double  %10.0g                2012 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2013
                                              double  %10.0g                2013 AG_PRD_FOOD_XD
                              CC_EST2013      double  %10.0g                2013 CC_EST
                              SE_PRM_ENRR2013 double  %10.0g                2013 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2014
                                              double  %10.0g                2014 AG_PRD_FOOD_XD
                              CC_EST2014      double  %10.0g                2014 CC_EST
                              SE_PRM_ENRR2014 double  %10.0g                2014 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2015
                                              double  %10.0g                2015 AG_PRD_FOOD_XD
                              CC_EST2015      double  %10.0g                2015 CC_EST
                              SE_PRM_ENRR2015 double  %10.0g                2015 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2016
                                              double  %10.0g                2016 AG_PRD_FOOD_XD
                              CC_EST2016      double  %10.0g                2016 CC_EST
                              SE_PRM_ENRR2016 double  %10.0g                2016 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2017
                                              double  %10.0g                2017 AG_PRD_FOOD_XD
                              CC_EST2017      double  %10.0g                2017 CC_EST
                              SE_PRM_ENRR2017 double  %10.0g                2017 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2018
                                              double  %10.0g                2018 AG_PRD_FOOD_XD
                              CC_EST2018      double  %10.0g                2018 CC_EST
                              SE_PRM_ENRR2018 double  %10.0g                2018 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2019
                                              double  %10.0g                2019 AG_PRD_FOOD_XD
                              CC_EST2019      double  %10.0g                2019 CC_EST
                              SE_PRM_ENRR2019 double  %10.0g                2019 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2020
                                              double  %10.0g                2020 AG_PRD_FOOD_XD
                              CC_EST2020      double  %10.0g                2020 CC_EST
                              SE_PRM_ENRR2020 double  %10.0g                2020 SE_PRM_ENRR
                              AG_PRD_FOOD_XD2021
                                              double  %10.0g                2021 AG_PRD_FOOD_XD
                              CC_EST2021      double  %10.0g                2021 CC_EST
                              SE_PRM_ENRR2021 double  %10.0g                2021 SE_PRM_ENRR
                              -------------------------------------------------------------------------------------------------------------------------------------
                              Sorted by: iso3
                              I have skimmed your do-file. I do see one error: you should not use the -rseed()- option in your -mi impute regress- command, because you have it in a loop. That means that with each iteration of the loop, the random number generator gets reset to the same initial condition, so you are not going to get independent random imputations of each of the variables you impute. You should instead use the -set seed- command, once, before the loop, to set the random number seed and then let it run from there.



                              Comment

                              Working...
                              X