No room to add more variables when there is only 1 variable

[Heather Dolphin]

Greetings,
I am trying to generate spaghetti plots representing earnings of young people who participated in a youth employment program using this command: spagplot daily_earnings_dol survey in 1/30, id( id)
but I get the error
no room to add more variables
Up to 2,048 variables are allowed with this version of Stata. Versions are available
that allow up to 120,000 variables.
Since it is a single variable I have requested in the command, I have been perplexed by this error. Based on my reading of other posts, i suspect it is because the observations for the id variable are each coded mixtures of numbers and letters like Hhr-005e9e8-13b. I had already encoded the variable so that it is stored as a numeric "long" storage type rather than string but that hasn't resolved this issue. Does that diagnosis sound like it would cause that error message? Assuming that is the case, I need to create a new variable that gives the observations a new name but it's not so simple as just renaming them numbers from 1 to 6244 because it is a dataset of repeated measures so the same number needs to apply to between 1-6 observations depending on how many data points that person represents in the combined dataset. Do you agree that this is what is causing the error message and if so, can you give me instructions on what code will create a new variable that will assign a new numeric code for the "id" variable even though each one appears varying numbers of times within the dataset? Or do you think that would generate the same error? Of the 6244 observations, there are 2,315 unique values (so an approx. average of 3 data points per person).
Thank you very much for any guidance,
Heather

[Jared Greathouse]

I'm not understanding the issue. I see what's written, but I don't understand. Can you please post your example dataset using dataex?

[Heather Dolphin]

Thank you for your response. I hope this helps:
The code I used for the spaghetti plot is:

spagplot daily_earnings_dol survey in 1/30, id( id)

The response i got is stata is
no room to add more variables
Up to 2,048 variables are allowed with this version of Stata. Versions are available
that allow up to 120,000 variables.

Here is 100 of 10000 observations of the id and daily earnings variables from the data set

* Example generated by -dataex-. To install: ssc install dataex
clear
input float daily_earnings_dol long id
        .  1
        .  1
        .  1
        .  2
        .  3
        .  4
        .  4
        .  4
        .  5
        .  5
        .  5
        .  5
        .  6
        .  6
        .  7
        .  7
        .  8
        .  8
        .  8
        .  9
        .  9
        .  9
        . 10
        . 10
        . 10
        . 11
        . 12
   8.0634 12
  9.67608 12
   8.0634 12
 18.14265 12
        . 13
        . 13
        . 13
   4.0317 13
        . 13
        . 13
        . 14
        . 14
        . 15
        . 15
        . 15
        . 16
   2.6878 16
 17.73948 16
 18.14265 16
 15.32046 16
        . 17
        4 17
        . 18
        . 18
        . 18
        . 19
        . 19
       15 19
        . 20
        . 21
        . 22
        . 22
        . 22
2.3333333 22
        . 23
        . 23
        . 23
        . 24
 2.351825 24
 10.07925 24
        . 24
        . 25
        . 25
        . 25
        . 25
        . 25
        . 25
 2.666667 26
     12.5 26
     12.5 26
15.555555 26
     12.5 26
       14 26
       15 27
        . 27
        . 28
        . 29
        . 29
        . 30
        . 30
        . 30
        . 31
        . 31
        8 32
        5 32
        . 33
        . 34
        . 34
        . 34
        . 34
17.050299 35
        . 35
 14.51412 35
end
label values id id
label def id 1 "Hhr-005e9e8-13b", modify
label def id 2 "Hhr-00b69bf-667", modify
label def id 3 "Hhr-00d90fa-b13", modify
label def id 4 "Hhr-00d90fa-e7a", modify
label def id 5 "Hhr-00da329-1ae", modify
label def id 6 "Hhr-0115e72-290", modify
label def id 7 "Hhr-013440f-718", modify
label def id 8 "Hhr-0147fdd-fc0", modify
label def id 9 "Hhr-0149bcf-f70", modify
label def id 10 "Hhr-015c441-dd8", modify
label def id 11 "Hhr-015f105-a04", modify
label def id 12 "Hhr-016cbf9-e7d", modify
label def id 13 "Hhr-0174723-be8", modify
label def id 14 "Hhr-0185bb7-25a", modify
label def id 15 "Hhr-018adb7-d39", modify
label def id 16 "Hhr-019567a-34c", modify
label def id 17 "Hhr-01b1860-5f2", modify
label def id 18 "Hhr-01c8d31-536", modify
label def id 19 "Hhr-01d086d-fe6", modify
label def id 20 "Hhr-01d7414-105", modify
label def id 21 "Hhr-01d7414-c8b", modify
label def id 22 "Hhr-01eea82-c2f", modify
label def id 23 "Hhr-01f2381-e29", modify
label def id 24 "Hhr-01f3f37-4ac", modify
label def id 25 "Hhr-0222c8a-3b9", modify
label def id 26 "Hhr-022b147-545", modify
label def id 27 "Hhr-0245620-343", modify
label def id 28 "Hhr-025ab3f-902", modify
label def id 29 "Hhr-0269522-737", modify
label def id 30 "Hhr-027de3c-f85", modify
label def id 31 "Hhr-02a5e62-7fd", modify
label def id 32 "Hhr-032443b-c24", modify
label def id 33 "Hhr-034ef97-e16", modify
label def id 34 "Hhr-035dee5-592", modify
label def id 35 "Hhr-03633ec-cc3", modify

[Heather Dolphin]

Sorry I omitted the time variable-- let me repost

[Heather Dolphin]

Here is the command again:

 
 spagplot daily_earnings_dol survey in 1/30, id( id)

These are 100 of 1000 observations pulled by dataex.

* Example generated by -dataex-. To install: ssc install dataex
clear
input long id float(survey daily_earnings_dol)
 1 0         .
 1 1         .
 1 2         .
 2 0         .
 3 0         .
 4 0         .
 4 1         .
 4 6         .
 5 0         .
 5 2         .
 5 3         .
 5 6         .
 6 0         .
 6 1         .
 7 0         .
 7 1         .
 8 0         .
 8 1         .
 8 4         .
 9 0         .
 9 1         .
 9 3         .
10 0         .
10 1         .
10 3         .
11 0         .
12 0         .
12 1    8.0634
12 2   9.67608
12 3    8.0634
12 5  18.14265
13 0         .
13 1         .
13 2         .
13 3    4.0317
13 4         .
13 6         .
14 0         .
14 1         .
15 0         .
15 1         .
15 2         .
16 0         .
16 1    2.6878
16 3  17.73948
16 5  18.14265
16 6  15.32046
17 0         .
17 2         4
18 0         .
18 1         .
18 2         .
19 0         .
19 1         .
19 6        15
20 0         .
21 0         .
22 0         .
22 1         .
22 2         .
22 3 2.3333333
23 0         .
23 1         .
23 6         .
24 0         .
24 1  2.351825
24 2  10.07925
24 3         .
25 0         .
25 1         .
25 2         .
25 3         .
25 4         .
25 5         .
26 0  2.666667
26 1      12.5
26 2      12.5
26 3 15.555555
26 4      12.5
26 5        14
27 0        15
27 1         .
28 0         .
29 0         .
29 1         .
30 0         .
30 1         .
30 2         .
31 0         .
31 1         .
32 0         8
32 3         5
33 0         .
34 0         .
34 1         .
34 2         .
34 6         .
35 0 17.050299
35 1         .
35 2  14.51412
end
label values id id
label def id 1 "Hhr-005e9e8-13b", modify
label def id 2 "Hhr-00b69bf-667", modify
label def id 3 "Hhr-00d90fa-b13", modify
label def id 4 "Hhr-00d90fa-e7a", modify
label def id 5 "Hhr-00da329-1ae", modify
label def id 6 "Hhr-0115e72-290", modify
label def id 7 "Hhr-013440f-718", modify
label def id 8 "Hhr-0147fdd-fc0", modify
label def id 9 "Hhr-0149bcf-f70", modify
label def id 10 "Hhr-015c441-dd8", modify
label def id 11 "Hhr-015f105-a04", modify
label def id 12 "Hhr-016cbf9-e7d", modify
label def id 13 "Hhr-0174723-be8", modify
label def id 14 "Hhr-0185bb7-25a", modify
label def id 15 "Hhr-018adb7-d39", modify
label def id 16 "Hhr-019567a-34c", modify
label def id 17 "Hhr-01b1860-5f2", modify
label def id 18 "Hhr-01c8d31-536", modify
label def id 19 "Hhr-01d086d-fe6", modify
label def id 20 "Hhr-01d7414-105", modify
label def id 21 "Hhr-01d7414-c8b", modify
label def id 22 "Hhr-01eea82-c2f", modify
label def id 23 "Hhr-01f2381-e29", modify
label def id 24 "Hhr-01f3f37-4ac", modify
label def id 25 "Hhr-0222c8a-3b9", modify
label def id 26 "Hhr-022b147-545", modify
label def id 27 "Hhr-0245620-343", modify
label def id 28 "Hhr-025ab3f-902", modify
label def id 29 "Hhr-0269522-737", modify
label def id 30 "Hhr-027de3c-f85", modify
label def id 31 "Hhr-02a5e62-7fd", modify
label def id 32 "Hhr-032443b-c24", modify
label def id 33 "Hhr-034ef97-e16", modify
label def id 34 "Hhr-035dee5-592", modify
label def id 35 "Hhr-03633ec-cc3", modify
label values survey survey1
label def survey1 0 "BL", modify
label def survey1 1 "exit", modify
label def survey1 2 "mo6", modify
label def survey1 3 "mo12", modify
label def survey1 4 "mo18", modify
label def survey1 5 "mo24", modify
label def survey1 6 "final", modify

Thank you very much!

[Nick Cox]

spagplot is a community-contributed command, so you are asked to explain where it comes from (FAQ Advice #12). I recall that the program was written by UCLA people. I can't now find a download location for the code, but I found this file from about 2004 on a computer of mine.

program define spagplot
  version 8
  syntax varlist(min=2 max=2) [if] [in] , id(varlist) [nofit] *

  tokenize `varlist'
  if "`fit'" == "" {
    preserve
    tokenize `varlist'
    quietly xi: regress `1' i.`id'*`2' `if' `in'
    tempvar yhat
    quietly predict `yhat' if e(sample)
    graph twoway scatter `yhat' `2', connect(L) msymbol(i) sort(`id' `2') note("Spaghetti plot of `varlist' id `id'") `options'
  }
  else {
    graph twoway scatter `1' `2', connect(L) msymbol(i) sort(`id' `2') note("Spaghetti plot of `varlist' id `id'") `options'
  }
end

Contrary to #1 your call requires 3 variables. not 1, yet as you say that isn't the problem.

The problem is that this code was written about 20 years ago and the way it processes data requires, on the fly, thousands of extra variables in your set-up.

I'd clone the code under a new name and edit it to avoid using xi:

I think you'd need to edit the regress line to use factor variable notation to define the fit you want by default. Perhaps you'd need to change the version number too.


I was never sure what this program did that was an improvement over xtline (possibly after fitting a regression model of choice).

[Heather Dolphin]

Im happy to use xtline! Thanks-- we had used spagplot for class but for whatever reason it's too complicated for this dataset--xtline works THANKS!!

[Nick Cox]

On spaghetti, or avoiding it, see not just the first here but also the others.

FAQ . . . . . . . . . . . . . . . How can I make spaghetti plots in Stata?
. . . . . . . . . . . . . . . . . . UCLA Academic Technology Services
7/08 https://stats.idre.ucla.edu/stata/faq/how-can-i-make-
spaghetti-plots-in-stata/

SJ-21-2 gr0087 . . Front-and-back plots to ease spaghetti and paella problems
(help fabplot if installed) . . . . . . . . . . . . . . . . N. J. Cox
Q2/21 SJ 21(2):539--554
explores front-and-back plots, in which each subset of data
is shown separately with the other subsets as backdrop

SJ-19-4 gr0080 . . . . . . Some simple devices to ease the spaghetti problem
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . N. J. Cox
Q4/19 SJ 19(4):989--1008 (no commands)
gives guidance on avoiding the spaghetti problem in graphics
(where multiple time series or other functional traces show
mostly a tangled mess)

Sometime between 2019 and 2021 British universities lobbied journal publishers so that someone like me can now publish under an open licence. So, the 2021 paper should be accessible at https://journals.sagepub.com/doi/pdf...6867X211025838 while -- if you or your workplace don't subscribe to the SJ -- the 2019 paper will remain behind a paywall until Q1/2023.