"Failed to compute (r198)" collapse in bigger sample only

[Peter Meier]

Hi all,

a

collapse (sum) var1 var2 (max) var3 (first) stringvar, by(id1 id2 year)

generates the error "

failed to compute (r198)

" when run on my full sample.
It did not when I ran it on a 1% sample, making me believe that I made no coding error, nor do I seem to have insufficient memory as other e.g. estimation commands do run through on the full sample.
Yet Google or a search of this forum did not allow me to find what this error means or how to address it.
Would anyone know?

Thank you!

[William Lisowski]

Congratulations! You've apparently discovered a problem in the innards of the collapse command, specifically in the Mata compute_collapse() function which is called by collapse_17.ado which is called by collapse.ado.

The code in collapse_17.ado that led to this error message is

        cap {
                mata: compute_collapse()
        }

so whatever help Mata might have supplied is suppressed, and a 9901 return code from Mata leads collapse_17 to display the unhelpful error message you received.

I recommend that you contact the helpful members of Stata Technical Services at

https://www.stata.com/support/tech-support/

to seek their assistance with this problem.

[Clyde Schechter]

Just to add to William Lisowski's excellent advice in #2, in order to expedite the handling of this at StataCorp, when you contact them you should provide:

1. A data set that produces the problem you have encountered. If there are confidentiality concerns, try renaming the file itself and all the variables to something bland and uninformative, like file.dta with var1 var2, etc. (but verify that with those changes you still get the same problem), and the command that triggers the problem (with variable names similarly changed if that has been done).

2. The output of running -about- on your Stata installation so they will have the information they need about both your Stata installation and your computer.

[Hua Peng (StataCorp)]

Peter Meier , please follow Clyde's advices, send Stata techsupport the command used, a dataset which reproduces the issue and also the output of -about- and -query compilenumber-. We will look into the issue. Thanks.

[Peter Meier]

Hi, just to conclude the post with an upshot informative to those reading it later; In my case -collapse_16- solved the problem. Thanks for the helpful advice! PM

[Sergiy Radyakin]

Dear All,

Would it be possible to get any insights into the nature of this problem? My colleague has just ran into a similar problem, also working with a large dataset, but not under deficit of memory (~25% of the physical memory consumed before running collapse).

Does version control has anything to do with this issue? (Is the problem that the user is using a newer version of Stata?, while the previous versions were free of this problem? or is the problem present in any version, and is data-dependent?)

Hua Peng (StataCorp) If this was a new problem in 2021, has the fix for it been incorporated into Stata 18? If this can't be resolved with updating the code, can the message be a bit more descriptive about the nature of the issue, or any guidance be given as to what to do with the data to make it compliant with collapse's expectations (such as drop irrelevant variables prior to collapsing, avoid negative values, etc..)

If the recommendation is to use the -collapse_16- command, can this be documented? (both this advice and the command's syntax).

Thank you, Sergiy Radyakin

[Hua Peng (StataCorp)]

collapse in Stata 17 is rewritten and most part is internalized.The old collapse is preserved in collapse_16. The new collapse can uses more memory than collapse_16 (also can use less memory). I would recommend try -collapse_16- firt to see if it works, simply replace collapse with collapse_16, there is no syntax difference between collapse and collapse_16.

If collapse_16 does work, please send techsupport a do-file and a dataset which exibits the problem.

By the way, there is no difference in collapse between fully updated Stata 17 and Stata 18.

[Sergiy Radyakin]

Dear Hua Peng (StataCorp) ,

thank you very much for this additional information. Very helpful.
I will pass the recommendation to the colleague who can try it with the actual dataset.

Best, Sergiy