NLS estimation and GMM estimation equivalence

[Juan del Pozo]

Dear all
I have a very punctual question about the relationship between GMM estimation and Non-linear least squares. For the latter, let's take the particular case of an exponential conditional expectation

E(y|x)=exp(x'b)

where x is a (column) vector of covariates (let's assume orthogonal to the error) and b is the vector of coefficients of interest. Written in an error-term-form, by the law of iterated expectations, this conditional expectation leads to

y=exp(x'b) + u

So say that I have a random sample, a simple cross-section. Let's take the doctor-visits dataset as in Cameron and Trivedi (2005) Micreconometrics methods and applications

use https://www.stata-press.com/data/r16/docvisits, clear

where for comparability, we take as dependent docvis and as independent variables private, chronic, female and income. I know I can fit this model directly via nls (with Robust standard errors)

gen one=1
nl (docvis = exp({xb: private chronic female income one})), variables(docvis private chronic female income) robust

Now, I am aware that to fit the same model using gmm interactively I need to specify the generalized residual. In this additive model (and using the impliaction of the zero conditional mean assumption on the error term) the moment conditions are

E[x(docvis - exp( b1*private + b2*chronic + b3*female + b4*income + cons ))] = 0 (1)

which leads to the following gmm syntax

gmm (docvis - exp({xb:private chronic female income _cons})), instruments(private chronic female income) onestep

Yet this differs from results using nls and in turn, it provides the same results as if I fit the model via poisson

poisson docvis private chronic female income, robust

which, I am aware, occurs because in expression (1) we have the implicit moments from the score of the log-likelihood function using the Poisson distribution (an equivalence that does not hold, for instance, for the probit model, as show in Stata's GMM documentation manual).

Hence, in the additive model y=exp(x'b) + u, how can I replicate the results from nls for an exponential conditional expectation using the gmm command?
Any insight on this will be greatly appreciated
JM

[Joao Santos Silva]

Dear Juan del Pozo,

The GMM condition that gives you the NLS result is (1) times exp(xb). This is because when you take the derivative of the square of the error with respect to b, you get this weight. This implies that NLS gives more weight to observations where the mean has more curvature because there observations have more information about the parameters; in Poisson these weights disappear because the observations with more curvature have larger variance and the two weights exactly offset each other.

Best wishes,

Joao

[Juan del Pozo]

Dear Joao
Thank you so much for your prompt reply and explanation. I agree with what you say: the NLS estimator in this exponential CEF case has as first-order conditions

E{ x * [(docvis - exp(xb)) * exp(xb)] } = 0

which is the expression also shown in Cameron and Trivedi (2005), section 5.8.5. (Exponential Regression Example, p. 155). Hence, I ran the gmm command changing accordingly the expression for the generalized residual

use https://www.stata-press.com/data/r16/docvisits, clear
gmm ( (docvis - exp({xb:private chronic female income _cons})) * exp({xb:}) ), instruments(private chronic female income) onestep

but now the results are completely different from those using nl above: all the signs are negative and of a completely different magnitude. I presume that I am committing a mistake in this new declaration of gmm but I cannot see where. Sorry if it is quite obvious though!

[Joao Santos Silva]

Dear Juan del Pozo,

That is because gmm is setting the moment conditions to zero not by minimizing the sum of squares but by setting the weights to zero. If you use different starting values, e.g., obtained by Poisson regression, the problem is solved. This example illustrates another issue with NLS when implemented as GMM.

Best wishes,

Joao

[Juan del Pozo]

Dear Joao
You are perfectly right: the following code generates the equivalence (although the SEs have a small discrepancy)

use https://www.stata-press.com/data/r16/docvisits, clear
gen one=1
nl (docvis = exp({xb: private chronic female income one})), variables(docvis private chronic female income) robust

quietly: poisson docvis private chronic female income, robust
matrix b=e(b)
matrix define initval=(_b[private], _b[chronic], _b[female], _b[income], _b[_cons])
gmm ((docvis - exp({xb:private chronic female income _cons})) * exp({xb:})), instruments(private chronic female income) onestep from(initval)

It is clearer now the difference between what GMM minimizes and what NLS minimizes. Yet, it strikes me that taking different starting values leads to completely different answers, as I thought that the algorithm would reach the same stationary point regardless of where the search begins. Maybe this is generated because the function does not have a global optimum?

I would appreciate it if you can suggest any material that covers/explain this important caveat in more detail as, to the best of my knowledge, it is rarely discussed and it can have important implications.

Kind regards and thanks again!

[Joao Santos Silva]

Dear Juan del Pozo

I am not sure about appropriate references, but it is well known that estimation results for non-linear models often depend on the starting values because the objective function has multiple maxima/minima. However, there are other cases like this one where there are "pathological" solutions to the first order conditions that do not correspond to an optimal value of the objective function. In short, one should always think carefully about the model we are estimating...

Best wishes,

Joao