Tobit regression

[Anya hewertson]

Hi i am running a tobit regresison for data across 2 years 2007-2008 ( they were 2 individual datasets which i appended to make one overall dataset):
My variables include 10 log price categories for alcohol types on trade and off trade : l_p_wine_on l_p_wine_off etc
I also have a log income variable : log_income
My dependent variables are the expenditure shares of the alcohol type expenditure divided by total expenditure : e.g exp_share_wine_on expshare_wine_off
I am looking at the price elasticities of demand and the cross price elasticities of demand vary across each alcohol type and vary across socio-economic groups, government regions and gender

My prices for alcohols are constant throughout the year (i am using the average year price) however they vary between years

here is a data-ex for some of my variables

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(l_p_wine_on l_p_beer_on l_p_spirits_on l_p_wine_off l_p_spirits_off l_p_beer_off expshare_wine_on expshare_beer_off logincome) byte(socio_group gor) int year byte sexhrp
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  4.433789 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0   .01142119  5.898746 3 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  4.898213 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584    .0550356  .015000853  6.399842 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0 .0016348386  5.584012 3 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .015073973  7.020905 2 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.225338 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  4.911331 3 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.219934 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.533279 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0 4.2492094 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0   .00609936  6.168564 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  4.835587 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.940566 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .006249688  5.331317 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.786775 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .003858888  7.201894 2 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.476967 2 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.009435 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .010382757  6.377679 3 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.982862 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0   6.11283 2 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0 .0023888294  6.279646 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .001813489  6.294915 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .005435922  6.704463 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.747566 3 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .005957043   6.11456 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584  .014408222  .016718158  6.605068 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584  .018981254           0  6.019785 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.088818 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  4.779476 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .008590408  6.514719 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584  .005628793  .018012136  6.960443 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .005657709  6.424075 3 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.920457 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584  .008473212           0  6.898255 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0   .02177079  5.623837 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  4.812526 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.182973 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  4.514611 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.109314 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.362559 3 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0   5.30903 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0   5.26414 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0 4.3593974 3 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584  .017695729           0   4.77104 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.069847 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0   .01336186  6.690271 2 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0   5.80408 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.628306 5 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .016276948  6.522627 5 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.519619 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .035966147  6.422951 3 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.557673 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.602438 2 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.402017 3 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584 .0029820926           0  7.401286 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .015186014  7.176426 2 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.746554 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.474176 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0   .01607261  6.874416 3 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0  .010095213  6.662046 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584  .024986824    .0423164  6.069906 3 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584  .001250104           0  7.438652 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584   .03693495           0  7.021414 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0   .02268917    6.5658 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.958667 2 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.192117 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584 .0016070686           0  5.815264 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0   5.34921 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0     6.279 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.516609 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.554516 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.347932 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584   .01880577           0   5.93925 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0   .07133046  6.985651 2 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584  .005728897  .005415315   7.12227 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584   .01053234  .021376746    6.8088 2 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584 .0019776237           0  6.519822 3 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.490757 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.787439 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.457868 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.921752 2 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  7.098411 2 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  4.400603 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584  .013181653           0  6.857086 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584  .003744323           0  6.710182 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.136498 6 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584   .02200635           0  7.438652 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584 .0019496685           0  6.911319 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0 .0006786454  6.854755 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  7.438652 2 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  6.609726 1 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584  .004789272           0  6.868133 3 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.182907 4 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.823194 1 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  4.812526 6 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0   .04808098  5.530222 4 2 2007 2
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0           0  5.793585 3 2 2007 1
.60158 -.010050327 .662688 -.9416085 -.967584 -.967584           0   .11235794  6.436151 1 2 2007 2
end
label values gor gor
label def gor 2 "north west", modify
label values sexhrp sexhrp
label def sexhrp 1 "male", modify
label def sexhrp 2 "female", modify

I am then running a tobit regression as follows:

tobit expshare_wine_on l_p_wine_on l_p_beer_on l_p_cider_on l_p_spirits_on l_p_alcopops_on l_p_wine_off l_p_beer_off l_p_spirits_off l_p_cider_off l_p_alcopops_off logincome i.socio_group i.gor i.year i.sexhrp , ll(0)

I have censored the data at zero since some households report no consumption of alcohol

However my results are as follows:

tobit expshare_wine_on l_p_wine_on l_p_beer_on l_p_cider_on l_p_spirits_on l_p_alcopops_on l_p_wine_off
>  l_p_beer_off l_p_spirits_off l_p_cider_off l_p_alcopops_off logincome i.socio_group i.gor i.year i.sex
> hrp , ll(0)
note: l_p_beer_on omitted because of collinearity
note: l_p_cider_on omitted because of collinearity
note: l_p_spirits_on omitted because of collinearity
note: l_p_alcopops_on omitted because of collinearity
note: l_p_wine_off omitted because of collinearity
note: l_p_beer_off omitted because of collinearity
note: l_p_spirits_off omitted because of collinearity
note: l_p_cider_off omitted because of collinearity
note: l_p_alcopops_off omitted because of collinearity
note: 2008.year omitted because of collinearity

Refining starting values:

Grid node 0:   log likelihood = -5976.9775

Fitting full model:

Iteration 0:   log likelihood = -5976.9775  
Iteration 1:   log likelihood = -640.92644  
Iteration 2:   log likelihood =   1103.185  
Iteration 3:   log likelihood =  1808.8673  
Iteration 4:   log likelihood =  1909.2432  
Iteration 5:   log likelihood =   1910.562  
Iteration 6:   log likelihood =  1910.5625  
Iteration 7:   log likelihood =  1910.5625  

Tobit regression                                Number of obs     =     11,962
                                                   Uncensored     =      2,312
Limits: lower = 0                                  Left-censored  =      9,650
        upper = +inf                               Right-censored =          0

                                                LR chi2(14)       =     927.97
                                                Prob > chi2       =     0.0000
Log likelihood =  1910.5625                     Pseudo R2         =    -0.3207

-------------------------------------------------------------------------------------------
         expshare_wine_on |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
--------------------------+----------------------------------------------------------------
              l_p_wine_on |   -.028118   .0162136    -1.73   0.083    -.0598992    .0036632
              l_p_beer_on |          0  (omitted)
             l_p_cider_on |          0  (omitted)
           l_p_spirits_on |          0  (omitted)
          l_p_alcopops_on |          0  (omitted)
             l_p_wine_off |          0  (omitted)
             l_p_beer_off |          0  (omitted)
          l_p_spirits_off |          0  (omitted)
            l_p_cider_off |          0  (omitted)
         l_p_alcopops_off |          0  (omitted)
                logincome |   .0125922   .0006706    18.78   0.000     .0112778    .0139066
                          |
              socio_group |
                       2  |   .0014811   .0010997     1.35   0.178    -.0006745    .0036368
                       3  |  -.0078991   .0012672    -6.23   0.000    -.0103829   -.0054152
                       4  |  -.0098159    .003836    -2.56   0.011    -.0173351   -.0022968
                       5  |   .0065436   .0035439     1.85   0.065    -.0004031    .0134903
                       6  |  -.0027114   .0010429    -2.60   0.009    -.0047556   -.0006672
                          |
                      gor |
              north west  |  -.0004291   .0014892    -0.29   0.773    -.0033481      .00249
              merseyside  |  -.0009579   .0014654    -0.65   0.513    -.0038303    .0019145
yorkshire and the humber  |   .0017352   .0014609     1.19   0.235    -.0011284    .0045987
           east midlands  |  -.0012567   .0021903    -0.57   0.566      -.00555    .0030366
           west midlands  |  -.0024181   .0018331    -1.32   0.187    -.0060112    .0011751
                 eastern  |  -.0016493   .0017609    -0.94   0.349     -.005101    .0018023
                          |
                     year |
                    2008  |          0  (omitted)
                          |
                   sexhrp |
                  female  |   .0011694    .000803     1.46   0.145    -.0004046    .0027435
                    _cons |  -.0853166   .0110156    -7.75   0.000     -.106909   -.0637242
--------------------------+----------------------------------------------------------------
   var(e.expshare_wine_on)|   .0007865   .0000268                      .0007357    .0008408
-------------------------------------------------------------------------------------------

.

Q1. Why are my price variables other than the price of the same dependent variable omitted (as i am trying to work out cross price elasticity of demand) i understand its due to collinearity but what is causing this and how do i overcome it?
Q2. Why is the year dummy variable omitted?
Q3. Originally i had 2 separate datasets for 2007 and 2008 but they contained results for the same variables so i appended them together to get one overall dataset- was this correct, has this caused a problem?

I am following a model which has done close to the same thing and they didn't have this problem

Thanks so much in advance

[Carlo Lazzaro]

Anya:
1) the main issue seems to be tha lack of variation in many of your predictors;
2) 2007 is the reference category,and as such, unreported; 2008 is omitted due to collinearity with some other predictors;
3) if -append-ing enede up with no variations, this maybe the issue.

[Anya hewertson]

Hi thanks for your reply Carlo, throughout each year there isn't any variation in prices but the prices do change between years. However there is variation in all the other predictors such as income, gor, socio_group and sexhrp . Is this a problem?
And sorry do you mean ' if -append-ing ended up with no variations' - if so i believe it did this 2007 and 2008 is only a snapshot to try and make it easier for someone to help me but the full set of data I'm using is from 2007-2012 so 6 different years are used overall meaning that there are 6 different average prices for each alcohol category

What shall i do to solve this problem?
Thanks so much

[Carlo Lazzaro]

Anya:
assuming that you have good methodological reason to go -tobit-, the usual recipe is to add one predictor at time and see if a more parsimonious model can avoid multicollinearity nuisance.

[Anya hewertson]

Hi my reason for using tobit is because the expenditure share variable e.g expshare_wine_on contains many zero values as only some households report consumption of alcohol. Also sorry to ask what might seem a silly question, but why are my price variables experiencing collinearity since every price is different for every alcohol type and for every year?

Thank you

[Joao Santos Silva]

Dear Anya hewertson,

If you are modelling shares, you should use fractional regression rather than the Tobit.

If your prices do not vary across units, they will be collinear with the time dummies, right?

Best wishes,

Joao

[Anya hewertson]

Joao Santos Silva Thank you for your reply, the reason i chose to use tobit is because many households report zero consumption, would the use of fractional regression only consider the positive values or would it account for the zero values also?
And ok thank you i understand, however when i was doing the regression with all the years 2007-2012 only 3 of the price variables were omitted because of collinearity whereas the rest were not even though each price variable is collinear with the time dummies?
Also i understand the prices are collinear with the time dummies however i believed that the year dummies would account for different price elasticities of demand for different years and although the prices are the same throughout the years the expenditure shares (my dependent variable) are not, am i wrong in thinking this?

[Joao Santos Silva]

Dear Anya hewertson,

Yes, zeros are not a problem with fractional regression.

Best wishes,

Joao