Syntax for ICC for test-retest situation

[Gabriel Fernandez]

Hello,
I'm trying to calculate an icc in a test retest situation.
My variables are:
id (string, 28 different values)
test-number (string, 2 different values)
test-score (numeric, no missing values)

Each subject answered twice (one for a first test, one for a second test). I have 56 observations, 28 for each subject with test 1, and 28 for test 2.
The syntax should be

icc depvar target rater

so the code I tried is

icc test-score id test-number

I always end up with the following error
"no observations
r(2000);"

Am I doing a syntax error? Or should I format my data differently?

Thank you for your help.

[Clyde Schechter]

Well, you don't show any example data, so it's anybody's guess whether you need to reorganize the data in some way.

In any case, what you show is certainly illegal syntax in Stata. You cannot have a variable named test-score or test-number, because hyphens are not permitted in variable names. If you happen to have a variable named test, and another named score, and another named number, then -icc test-score id test-number- will expand into an -icc- command with a list of all the variables located between test and number (including test and number themselves), followed by id, followed by all the variables located between test and score, again inclusive. This will be far more variables than -icc- accepts. So you would get a syntax error, not a no observations error. So my guess is that you are "simplifying" your question by not providing the real names for your variables. The problem with your simplification is that it creates confusion and makes it harder, not easier, to respond to your question. When asking for help, ask the real question!

There are two common reasons people get "no observations" errors when they don't expect them. One is that one or more of the variables used in the command is a string variable, whereas Stata requires a numeric variable. The other arises because in all Stata estimation commands (and -icc- is an estimation command), any observation that has a missing value for any of the variables mentioned in the command is omitted from the estimation sample. It can happen that missing values scattered through the data can result in every observation containing a missing value in one of the variables, so there are no observations left over for estimation.

If you need more concrete assistance, when posting back, be sure to show example data, and be sure to use the -dataex- command to do so. If you are running version 18, 17, 16 or a fully updated version 15.1 or 14.2, -dataex- is already part of your official Stata installation. If not, run -ssc install dataex- to get it. Either way, run -help dataex- to read the simple instructions for using it. -dataex- will save you time; it is easier and quicker than typing out tables. It includes complete information about aspects of the data that are often critical to answering your question but cannot be seen from tabular displays or screenshots. It also makes it possible for those who want to help you to create a faithful representation of your example to try out their code, which in turn makes it more likely that their answer will actually work in your data.

[Gabriel Fernandez]

Thank you for your answer and sorry for the mistakes in my first post.
Yes, I "simplified" my question which led to confusion.

Since I don't have too many observations, I'll post below the full list obtained from dataex:
Concerning the 2 main reasons for "no observations" you mentionned:
1) I don't have any missing value.
2) My depvar is numeric, my "target" and "rater" variables are string.

* Example generated by -dataex-. To install: ssc install dataex
clear
input str4 id str22 testnumber byte Scoreglobal
"AL18" "Second" 13
"AL18" "First"  13
"AP25" "First"  14
"AP25" "Second"  9
"BA37" "First"   5
"BA37" "Second"  6
"BG46" "First"   0
"BG46" "Second"  0
"CR25" "First"  24
"CR25" "Second" 21
"DO37" "First"   4
"DO37" "Second"  5
"ED21" "First"  11
"ED21" "Second"  6
"EG23" "First"   5
"EG23" "Second" 12
"EL29" "First"  10
"EL29" "Second" 10
"ES25" "First"  15
"ES25" "Second"  7
"FR54" "First"   0
"FR54" "Second"  0
"JB29" "First"   4
"JB29" "Second"  2
"JR56" "First"   4
"JR56" "Second"  1
"KR51" "First"   4
"KR51" "Second"  6
"LH24" "First"   8
"LH24" "Second"  3
"LL26" "First"  26
"LL26" "Second" 20
"MD20" "First"   7
"MD20" "Second"  5
"MH28" "First"   3
"MH28" "Second"  6
"MR67" "First"   0
"MR67" "Second"  0
"MR21" "First"   4
"MR21" "Second"  3
"MS60" "First"   1
"MS60" "Second"  2
"NC36" "First"   2
"NC36" "Second"  2
"PB57" "First"   7
"PB57" "Second"  2
"SL55" "First"  12
"SL55" "Second" 15
"SM25" "First"  18
"SM25" "Second" 16
"TD23" "First"  14
"TD23" "Second" 20
"VL56" "First"   6
"VL56" "Second"  4
"YO22" "First"   0
"YO22" "Second"  0
end

[Joseph Coveney]

icc Scoreglobal id testnumber

works for me using that dataset.

Do-file and log file attached.

[Gabriel Fernandez]

And it strangely does not work for me, even starting from a clear database with only those 3 variables and copy pasting your Do-file (minus version and log).
May it be linked to the fact I'm using an older version of STATA (SE 13.0) that I got as a student and never upgraded? Could the icc command have changed?

Well, thanks to you I have the wanted result (well, almost, I should probably used a mixed effect for a test-retest situation) in your log file, but it is still strange. I shoud get a grant dedicated to upgrading my Stata software in the following months, hope it will correct it.

. clear *

.
. input str4 id str22 testnumber byte Scoreglobal

            id              testnumber  Scoreg~l
  1. "AL18" "Second" 13
  2. "AL18" "First"  13
  3. "AP25" "First"  14
  4. "AP25" "Second"  9
  5. "BA37" "First"   5
  6. "BA37" "Second"  6
  7. "BG46" "First"   0
  8. "BG46" "Second"  0
  9. "CR25" "First"  24
 10. "CR25" "Second" 21
 11. "DO37" "First"   4
 12. "DO37" "Second"  5
 13. "ED21" "First"  11
 14. "ED21" "Second"  6
 15. "EG23" "First"   5
 16. "EG23" "Second" 12
 17. "EL29" "First"  10
 18. "EL29" "Second" 10
 19. "ES25" "First"  15
 20. "ES25" "Second"  7
 21. "FR54" "First"   0
 22. "FR54" "Second"  0
 23. "JB29" "First"   4
 24. "JB29" "Second"  2
 25. "JR56" "First"   4
 26. "JR56" "Second"  1
 27. "KR51" "First"   4
 28. "KR51" "Second"  6
 29. "LH24" "First"   8
 30. "LH24" "Second"  3
 31. "LL26" "First"  26
 32. "LL26" "Second" 20
 33. "MD20" "First"   7
 34. "MD20" "Second"  5
 35. "MH28" "First"   3
 36. "MH28" "Second"  6
 37. "MR67" "First"   0
 38. "MR67" "Second"  0
 39. "MR21" "First"   4
 40. "MR21" "Second"  3
 41. "MS60" "First"   1
 42. "MS60" "Second"  2
 43. "NC36" "First"   2
 44. "NC36" "Second"  2
 45. "PB57" "First"   7
 46. "PB57" "Second"  2
 47. "SL55" "First"  12
 48. "SL55" "Second" 15
 49. "SM25" "First"  18
 50. "SM25" "Second" 16
 51. "TD23" "First"  14
 52. "TD23" "Second" 20
 53. "VL56" "First"   6
 54. "VL56" "Second"  4
 55. "YO22" "First"   0
 56. "YO22" "Second"  0
 57. end

.
. icc Scoreglobal id testnumber
no observations
r(2000);

end of do-file

[Clyde Schechter]

I cannot replicate your problem. Your code runs properly with your data on my setup:

. * Example generated by -dataex-. To install: ssc install dataex
. clear

. input str4 id str22 testnumber byte Scoreglobal

            id              testnumber  Scoreg~l
  1. "AL18" "Second" 13
  2. "AL18" "First"  13
  3. "AP25" "First"  14
  4. "AP25" "Second"  9
  5. "BA37" "First"   5
  6. "BA37" "Second"  6
  7. "BG46" "First"   0
  8. "BG46" "Second"  0
  9. "CR25" "First"  24
 10. "CR25" "Second" 21
 11. "DO37" "First"   4
 12. "DO37" "Second"  5
 13. "ED21" "First"  11
 14. "ED21" "Second"  6
 15. "EG23" "First"   5
 16. "EG23" "Second" 12
 17. "EL29" "First"  10
 18. "EL29" "Second" 10
 19. "ES25" "First"  15
 20. "ES25" "Second"  7
 21. "FR54" "First"   0
 22. "FR54" "Second"  0
 23. "JB29" "First"   4
 24. "JB29" "Second"  2
 25. "JR56" "First"   4
 26. "JR56" "Second"  1
 27. "KR51" "First"   4
 28. "KR51" "Second"  6
 29. "LH24" "First"   8
 30. "LH24" "Second"  3
 31. "LL26" "First"  26
 32. "LL26" "Second" 20
 33. "MD20" "First"   7
 34. "MD20" "Second"  5
 35. "MH28" "First"   3
 36. "MH28" "Second"  6
 37. "MR67" "First"   0
 38. "MR67" "Second"  0
 39. "MR21" "First"   4
 40. "MR21" "Second"  3
 41. "MS60" "First"   1
 42. "MS60" "Second"  2
 43. "NC36" "First"   2
 44. "NC36" "Second"  2
 45. "PB57" "First"   7
 46. "PB57" "Second"  2
 47. "SL55" "First"  12
 48. "SL55" "Second" 15
 49. "SM25" "First"  18
 50. "SM25" "Second" 16
 51. "TD23" "First"  14
 52. "TD23" "Second" 20
 53. "VL56" "First"   6
 54. "VL56" "Second"  4
 55. "YO22" "First"   0
 56. "YO22" "Second"  0
 57. end

.
. icc Scoreglobal id testnumber

Intraclass correlations
Two-way random-effects model
Absolute agreement

Random effects: id               Number of targets =        28
Random effects: testnumber       Number of raters  =         2

--------------------------------------------------------------
           Scoreglobal |        ICC       [95% conf. interval]
-----------------------+--------------------------------------
            Individual |   .8654582       .7321211    .9351818
               Average |   .9278774       .8453463    .9665054
--------------------------------------------------------------
F test that
  ICC=0.00: F(27.0, 27.0) = 14.26             Prob > F = 0.000

Note: ICCs estimate correlations between individual measurements
      and between average measurements made on the same target.

.

I wonder what version of Stata you are using. Perhaps in an older veresion, id and testnumber had to be numeric. Try converting them:

encode id, gen(n_id)
encode testnumber, gen(n_testnumber)
icc Scoreglobal n_id n_testnumber

If that doesn't work, then I don't know what to tell you. We would probably be looking at things like updating your Stata installation, or doing a fresh re-install or something like that.

[Gabriel Fernandez]

And it works. So the older version of icc required numeric variables.
Thank you very much.
And I'll update asap to avoid being stuck for problems like that.