HP Filter on panel data

[Michal Mares]

Dear Stata users,

I'm making a composite index (early warning indicator) for my diploma thesis. I use STATA 13. I have a problem with hp filtering certain time series. I use panel data for 26 countries (1970-2010, quarterly database) and 25 variables. Unfortunately there are missing values (missing all data for few countries 1-5) in 8 variables. When I run HP filter for variable witch missing observations I get following outcome:

. tsfilter hp c_inflation = inflation, smooth(1600) trend(t_inflation)
J(): 3300 argument out of range
_TSFILTER_getQ(): - function returned error
_TSFILTER_hpFilter(): - function returned error
<istmt>: - function returned error
r(3300);

I was trying to run HP filter with command "if countrunum!=5" to exclude country with missing observations but it doesn't work. Is there any possibility to run HP filter and exclude certain panels (few countries from my dataset)?

My aim is to get cyclical part of time variable. Thank you for helping me!

Michal M.

[Nick Cox]

In principle yes. You can exclude panels.

webuse turksales, clear
expand 2
gen panel = _n > 40
tsset panel t
tsfilter hp hpsales = sales , smooth(1600)
tsfilter hp hpsales0 = sales if panel == 0 , smooth(1600)

You may want to wave your data and syntax at StataCorp Technical Support, If your call is not valid, the code should throw a better error message.

But see e.g the distinguished econometrician James Hamilton http://econweb.ucsd.edu/~jhamilto/hp.pdf

It's a pity James has not been persuaded by his brother Larry (Statistics with Stata, the book that got me hooked) to use Stata as his main vehicle.

[Michal Mares]

Thank you for your reply. I have tried it but when I modify your code just to test it for different panel (1) it doesn't work. Here is what I got:

. tsfilter hp hpsales1 = sales if panel == 1 , smooth(1600) 
                     J():  3300  argument out of range
        _TSFILTER_getQ():     -  function returned error
    _TSFILTER_hpFilter():     -  function returned error
                 <istmt>:     -  function returned error
r(3300);

Do you recommend any better econometric technique to decompose time rows on cyclical and trend part? I will read J. Hamilton's paper.

In principle yes. You can exclude panels.

webuse turksales, clear
expand 2
gen panel = _n > 40
tsset panel t
tsfilter hp hpsales = sales , smooth(1600)
tsfilter hp hpsales0 = sales if panel == 0 , smooth(1600)

You may want to wave your data and syntax at StataCorp Technical Support, If your call is not valid, the code should throw a better error message.

But see e.g the distinguished econometrician James Hamilton http://econweb.ucsd.edu/~jhamilto/hp.pdf

It's a pity James has not been persuaded by his brother Larry (Statistics with Stata, the book that got me hooked) to use Stata as his main vehicle.

[Nick Cox]

We can't easily say more about your data problem without seeing more of your data. But I didn't experiment with missing data. I'd be surprised if missing data make H-P filters fall over.

Why specify econometric? Smoothing series cuts across statistical science.

[Michal Mares]

I apologize. Of course it is statistical science... Here you can see how my dataset looks like:

As you can see, there are missing data for m2. But for other countries there are no missing values. When I run HP filter for m2 it ends up with that error 3300 because of missing data...

Thank you for your help.

We can't easily say more about your data problem without seeing more of your data. But I didn't experiment with missing data. I'd be surprised if missing data make H-P filters fall over.

Why specify econometric? Smoothing series cuts across statistical science.

[Nick Cox]

Please review the FAQ Advice #12 on (not) posting screenshots.

All I see is that m2 is missing for all the observations we can see. If that were true of all observations, tsfilter would throw you out with error 2000. The problem, I guess, lies elsewhere.

[Eric de Souza]

I have not read Hamilton's paper as yet, but I see that he proposes an alternative to HP filtering. There are many problems associated with HP filtering.

[Michal Mares]

I'm sorry. I should have used "dataex". As you can see here:

* Example generated by -dataex-. To install: ssc install dataex
clear
input double(reer m2) float countrynum str6 B
101.33           . 1 "2000Q2"
100.88           . 1 "2000Q3"
101.27           . 1 "2000Q4"
101.61           . 1 "2001Q1"
 101.1           . 1 "2001Q2"
100.91           . 1 "2001Q3"
101.17           . 1 "2001Q4"
100.77           . 1 "2002Q1"
100.74           . 1 "2002Q2"
100.72           . 1 "2002Q3"
100.71           . 1 "2002Q4"
100.46           . 1 "2003Q1"
100.04           . 1 "2003Q2"
 99.94           . 1 "2003Q3"
100.08           . 1 "2003Q4"
100.31           . 1 "2004Q1"
 99.81           . 1 "2004Q2"
 99.79           . 1 "2004Q3"
100.18           . 1 "2004Q4"
100.68           . 1 "2005Q1"
 99.96           . 1 "2005Q2"
 99.72           . 1 "2005Q3"
 99.64           . 1 "2005Q4"
 99.87           . 1 "2006Q1"
 99.59           . 1 "2006Q2"
 99.43           . 1 "2006Q3"
 99.33           . 1 "2006Q4"
 99.67           . 1 "2007Q1"
 99.37           . 1 "2007Q2"
 99.25           . 1 "2007Q3"
 99.53           . 1 "2007Q4"
 99.54           . 1 "2008Q1"
 99.43           . 1 "2008Q2"
    99           . 1 "2008Q3"
 99.32           . 1 "2008Q4"
 99.33           . 1 "2009Q1"
 99.12           . 1 "2009Q2"
 99.23           . 1 "2009Q3"
 99.49           . 1 "2009Q4"
 99.71           . 1 "2010Q1"
 99.61           . 1 "2010Q2"
 99.43           . 1 "2010Q3"
 99.68           . 1 "2010Q4"
101.15           . 2 "2000Q2"
101.17           . 2 "2000Q3"
101.42           . 2 "2000Q4"
100.53           . 2 "2001Q1"
101.13           . 2 "2001Q2"
100.92           . 2 "2001Q3"
101.06           . 2 "2001Q4"
100.36           . 2 "2002Q1"
100.25           . 2 "2002Q2"
100.03           . 2 "2002Q3"
   100           . 2 "2002Q4"
 99.67           . 2 "2003Q1"
 99.73           . 2 "2003Q2"
 99.68           . 2 "2003Q3"
 99.72           . 2 "2003Q4"
  99.2           . 2 "2004Q1"
 99.54           . 2 "2004Q2"
  99.5           . 2 "2004Q3"
 99.81           . 2 "2004Q4"
 99.62           . 2 "2005Q1"
100.15           . 2 "2005Q2"
100.17           . 2 "2005Q3"
100.06           . 2 "2005Q4"
100.03           . 2 "2006Q1"
 100.4           . 2 "2006Q2"
100.34           . 2 "2006Q3"
 100.3           . 2 "2006Q4"
 99.97           . 2 "2007Q1"
 99.93           . 2 "2007Q2"
 99.71           . 2 "2007Q3"
100.22           . 2 "2007Q4"
100.58           . 2 "2008Q1"
101.41           . 2 "2008Q2"
 101.5           . 2 "2008Q3"
101.53           . 2 "2008Q4"
101.02           . 2 "2009Q1"
100.87           . 2 "2009Q2"
100.65           . 2 "2009Q3"
 100.9           . 2 "2009Q4"
101.22           . 2 "2010Q1"
101.98           . 2 "2010Q2"
101.77           . 2 "2010Q3"
102.28           . 2 "2010Q4"
 83.09  7629171000 3 "2000Q2"
 85.71  8850909000 3 "2000Q3"
 89.18  9856616000 3 "2000Q4"
 90.14 10224787000 3 "2001Q1"
 88.94 10463745000 3 "2001Q2"
 89.31 11102152000 3 "2001Q3"
 91.61 12400512000 3 "2001Q4"
 94.98 12329707000 3 "2002Q1"
 93.04 12101467000 3 "2002Q2"
 91.46 12812890000 3 "2002Q3"
  92.4 13854251000 3 "2002Q4"
 93.22 13541546000 3 "2003Q1"
 92.33 14197238000 3 "2003Q2"
 92.32 15073173000 3 "2003Q3"
end

Data are missing only for few countries - for country 1,2. There are observations for country 3 and the others too. My idea was to set HP filter only for certain panels => panel 3,4,5,6,10, etc...

Please review the FAQ Advice #12 on (not) posting screenshots.

All I see is that m2 is missing for all the observations we can see. If that were true of all observations, tsfilter would throw you out with error 2000. The problem, I guess, lies elsewhere.

[Nick Cox]

With your data example and this code I get results:

gen qdate = quarterly(B, "YQ") 
tsset countrynum qdate 
tsfilter hp result=m2 if countrynum > 2, smooth(1600)

A reproducible problem would be

1. explicit data

2. entire code that produces the problem.

As mentioned earlier, you might need to contact StataCorp.