Hey there, I am working with a large dataset with over 10,000,000 observations (and 23 variables) after I kept only the working variables of interest - file size 928 MB.
My computer is an Intel Core i7-8750H CPU @ 2.20GHz, 32 GB of RAM, 1 TB hard drive, 64-bit OS, running Stata/MP with 6 core capability.
Unfortunately, due to the limitations of the svy command (which does not allow use of the option tvc in stcox in svy), as I'm using complex survey data, I've had to stsplit my data (at failures - 30 failure times) to perform a time varying covariate analysis (TVC) with Cox regression, - which has then resulted in >290,000,000 observations - file >20 GB in size now.
I had tried running the svy: stcox command using the method illustrated after stspitting (https://www.stata.com/manuals/ststcox.pdf and https://www.stata.com/manuals/ststsplit.pdf#ststsplit). My computer then appears to go on overdrive, for a very long time, only to end in this error message:
(running stcox on estimation sample)
op. sys. refuses to provide memory
an error occurred while attempting to compute scores
r(322);
end of do-file
Are there any other suggestions or work arounds? Buy a new computer?
Thanks so much
E
My computer is an Intel Core i7-8750H CPU @ 2.20GHz, 32 GB of RAM, 1 TB hard drive, 64-bit OS, running Stata/MP with 6 core capability.
Unfortunately, due to the limitations of the svy command (which does not allow use of the option tvc in stcox in svy), as I'm using complex survey data, I've had to stsplit my data (at failures - 30 failure times) to perform a time varying covariate analysis (TVC) with Cox regression, - which has then resulted in >290,000,000 observations - file >20 GB in size now.
I had tried running the svy: stcox command using the method illustrated after stspitting (https://www.stata.com/manuals/ststcox.pdf and https://www.stata.com/manuals/ststsplit.pdf#ststsplit). My computer then appears to go on overdrive, for a very long time, only to end in this error message:
(running stcox on estimation sample)
op. sys. refuses to provide memory
an error occurred while attempting to compute scores
r(322);
end of do-file
Are there any other suggestions or work arounds? Buy a new computer?
Thanks so much
E
