[Anders Alexandersson]

Anders did you use st.var?

No, I never used st.var. In post #12, I linked to code which used the compare.dedup() function in the RecordLinkage package. Here is the simplified example with the code and output.

Example of Rcall in interactive mode using \$. There is no problem:
Stata code:

R: library(RecordLinkage)
R: data(RLdata500)
R: rpairs <- compare.dedup(RLdata500)
R: rpairs\$pairs[1:5, ]

The R output of the last code line with the \$ is displayed. The output may seem cryptic at first but is as expected:

 
  id1 id2 fname_c1 fname_c2 lname_c1 lname_c2 by bm bd is_match
1   1   2        0       NA        0       NA  0  1  0       NA
2   1   3        0       NA        0       NA  0  0  0       NA
3   1   4        0       NA        0       NA  0  0  0       NA
4   1   5        0       NA        0       NA  0  0  0       NA
5   1   6        0       NA        0       NA  0  1  0       NA

Example of problem of Rcall in vanilla mode using \$. The problem is an unexpected error which also unexpectedly disappears quickly:
Stata code:

R vanilla: source("test2_error.r")

R code in file test2_error.r:

library(RecordLinkage)
data(RLdata500)
rpairs <- compare.dedup(RLdata500)
rpairs\$pairs[1:5, ]

The error from what I can reconstruct after trying to read the quickly disappearing error message maybe 30 times:

Error in source("test2_error.r") : test2_error.r : 5:7 unexpected input 
4: rpairs <- compare.dedup(RLdata500)
5: rpairs\
         ^
Execution halted

I am guessing that the error message means that the backslash character in line 5 position 7 is not allowed. The backslash character should be allowed, as in interactive mode. Or at least the R code with a $ character should be callable somehow.

[Marc Thevenin]

The solution in #13 seems to work for RecordLinkage.
$pairs is in the second position (=> str(rpairs))

capture program drop rlinkage
program define rlinkage
syntax

Rcall vanilla:                                                  ///
library(RecordLinkage);                                 ///
data(RLdata500);                                        ///   
rpairs = compare.dedup(RLdata500);           ///
invisible(typeof(rpairs[[2]])); rpairs[[2]][1:5,]; ///

end

rlinkage

Output

  id1 id2 fname_c1 fname_c2 lname_c1 lname_c2 by bm bd is_match
1   1   2        0       NA        0       NA  0  1  0       NA
2   1   3        0       NA        0       NA  0  0  0       NA
3   1   4        0       NA        0       NA  0  0  0       NA
4   1   5        0       NA        0       NA  0  0  0       NA
5   1   6        0       NA        0       NA  0  1  0       NA

[haghish]

So there are 2 problems here that you are referring to.

1. Seeing the errors returned that are occurred in R
2. dealing with the $ sign

The first problem is "solely" a Windows problem. In Mac and Linux the error is displayed in Stata results windows. But I updated the package to version 1.3.0 now, and made it much more defensive. Now Rcall returns the error that had occurred in R and breaks the process, which should make it easier to implement in Stata programs. It returns r(rc) scalar, which equals 1 if an error had occurred, and 0 otherwise. So uninstall Rcall and reinstall it from GitHub.

About the second problem, it makes no sense to discuss difference between generic R functions vs R packages. The dollar sign is an indicator of global macro in Stata and Stata attempts to evaluate it before passing it to R. Therefore if there is an error, it's certainly happening before the code is sent to R.

But I personally have no error. I hope you have realized by now that when you use the st.var function, you only get a vector. It's not hard to test that...

. sysuse auto, clear
(1978 Automobile Data)

. R: a = st.var(price)

. R: class(a)
[1] "numeric"

. R: a
 [1]  4099  4749  3799  4816  7827  5788  4453  5189 10372  4082 11385 14500
[13] 15906  3299  5705  4504  5104  3667  3955  3984  4010  5886  6342  4389
[25]  4187 11497 13594 13466  3829  5379  6165  4516  6303  3291  8814  5172
[37]  4733  4890  4181  4195 10371  4647  4425  4482  6486  4060  5798  4934
[49]  5222  4723  4424  4172  9690  6295  9735  6229  4589  5079  8129  4296
[61]  5799  4499  3995 12990  3895  3798  5899  3748  5719  7140  5397  4697
[73]  6850 11995

Marc Thevenin If you repeat your analysis in R, I expect you also to get a NULL.


---

Rcall has an undocumented subcommand called debug. If you use this subcommand, you can see what is transferred from Stata to R, and what is returned from R, as well as all the communications between Stata and R.

PS. please provide a reproducible do-file showing difference between Vanilla and interactive mode regarding the dollar sign, so that I can test it...

[haghish]

Anders Alexandersson

A few points that I wish to add:

• st.var function is slow. It's much faster to temporarily make a subset of the data that you want to pass with R and then use the st.data function to pass it to R.
• The functions that begins with st. are all Stata functions. You cannot include them in your R code. For example, you cannot write an R script that includes st.data function and then source that script file. These functions can only work when Stata interprets them.
• The same thing is true about the backslash. When you source an R script, you don't have to put a backslash before the dollar sign. Simply because Stata does not care about that file. It only commands R to read that file! So your example #16 for Rcall Vanilla is wrong and it has nothing to do with Rcall. You get the same error if you source that file directly in R!

So to conclude, it seems to me that actually, there is no dollar sign problem so far, only bad practice of Rcall.

[Marc Thevenin]

(...)About the second problem, it makes no sense to discuss difference between generic R functions vs R packages. (...)

Basically it's what I wrote yesterday (#15) after testing with other written functions, like glm2 or survival package's functions. No problem with \$ (& with st.var)
For me the problem occurs only with cmprsk package (cuminc & crr functions).

st.var function is slow. It's much faster to temporarily make a subset of the data that you want to pass with R and then use the st.data function to pass it to R.

This is clearly the best way to do

[Anders Alexandersson]

When you source an R script, you don't have to put a backslash before the dollar sign. Simply because Stata does not care about that file. It only commands R to read that file! So your example #16 for Rcall Vanilla is wrong and it has nothing to do with Rcall. [...] So to conclude, it seems to me that actually, there is no dollar sign problem so far, only bad practice of Rcall.

Thanks, instead of source() I will use a program instead like Marc did. I could reproduce Marc's solution in #17. Then, I updated Rcall to 1.3.0 and was able to print "Hello World".

. update query
(contacting http://www.stata.com)

Update status
    Last check for updates:  01 Sep 2016
    New update available:    none         (as of 01 Sep 2016)
    Current update level:    20 Jul 2016  (what's new)

Possible actions

    Do nothing; all files are up to date.

. adoupdate Rcall
(note: adoupdate updates user-written files; type -update- to check for updates to
       official Stata)

Checking status of specified packages...

   [23] Rcall at https://raw.githubusercontent.com/haghish/Rcall/master:
        installed package is up to date

(no packages require updating)

. ado uninstall Rcall

package Rcall from https://raw.githubusercontent.com/haghish/Rcall/master
      'Rcall': Seamless interactive R in Stata. The command also return

(package uninstalled)

. net install Rcall, replace from("https://raw.githubusercontent.com/haghish/Rcall/master/
> ")
checking Rcall consistency and verifying not already installed...
installing into c:\ado\plus\...
installation complete.

. R: print("Hello World")
[1] "Hello World"

But with the updated version of Rcall I can no longer reproduce Marc's solution in #17. The error is "file Rcall.ado not found"

. capture program drop rlinkage

. program define rlinkage
  1. syntax
  2.
. Rcall vanilla:                                                 ///
> library(RecordLinkage);                                 ///
> data(RLdata500);                                        ///  
> rpairs = compare.dedup(RLdata500);           ///
> invisible(typeof(rpairs[[2]])); rpairs[[2]][1:5,]; ///
>
  3. end

.
. rlinkage
file Rcall.ado not found
(error occurred while loading Rcall.ado)
r(601);

Because of the new error, I suspect that I have a setup path issue. This is the content of my profile.do file:

global Rterm_path `"C:\Program Files\R\R-3.3.0\bin\x64\Rterm.exe"'
global Rterm_options `"--vanilla"'

How should I use -Rcall setpath- given the location of my R installation in Windows 7? I tried these eight ways which all failed with the same 601 error.

Rcall setpath `"C:\Program Files\R\R-3.3.0\bin\x64\Rterm.exe"'
Rcall setpath "C:\Program Files\R\R-3.3.0\bin\x64\Rterm.exe"
Rcall setpath "C:/Program Files/R/R-3.3.0/bin/x64/Rterm.exe"
Rcall setpath `"C:/Program Files/R/R-3.3.0/bin/x64/Rterm.exe"'
Rcall setpath `"C:\Program Files\R\R-3.3.0\bin\x64\R.exe"'
Rcall setpath "C:\Program Files\R\R-3.3.0\bin\x64\R.exe"
Rcall setpath "C:/Program Files/R/R-3.3.0/bin/x64/R.exe"
Rcall setpath `"C:/Program Files/R/R-3.3.0/bin/x64/R.exe"'

Also, if Marc's solution in #17 for the backslash is not required, then should this code work?

capture program drop rlinkage
program define rlinkage
syntax

Rcall vanilla:                  ///
library(RecordLinkage);         ///
data(RLdata500);                             ///  
rpairs = compare.dedup(RLdata500);           ///
rpairs\$pairs[c(1:5), ] ///

end

rlinkage

Marc, does the above code work for you or was your solution in #17 required?
(Edited: I forgot /// in the backslash line like Marc did)

[Marc Thevenin]

The above code works well with me

program define rlinkage
  1. syntax
  2.
. Rcall vanilla:                  ///
> library(RecordLinkage);         ///
> data(RLdata500);                             ///  
> rpairs = compare.dedup(RLdata500);           ///
> rpairs\$pairs[c(1:5), ] ///
>
  3. end
r; t=0.00 16:56:05

.
. rlinkage
  id1 id2 fname_c1 fname_c2 lname_c1 lname_c2 by bm bd is_match
1   1   2        0       NA        0       NA  0  1  0       NA
2   1   3        0       NA        0       NA  0  0  0       NA
3   1   4        0       NA        0       NA  0  0  0       NA
4   1   5        0       NA        0       NA  0  0  0       NA
5   1   6        0       NA        0       NA  0  1  0       NA
r; t=2.75 16:56:08

[Anders Alexandersson]

In "Stata tip 65: Beware the backstabbing backslash" at http://www.stata-journal.com/sjpdf.h...iclenum=pr0042, Nick Cox suggested to use forward slashes consistently. When I set trace on, the error shows up in a call to Stata's help file

. set trace on

. set traced 2

. Rcall setpath "C:/Program Files/R/R-3.3.0/bin/x64/`Rterm.exe'"
    -------------------------------------------------------------------- begin markdoc ---
[...]

    - sthlp `smclfile', export("`export'") template("`template'") `replace' `date' title("
> `title'") summary("`summary'") author("`author'") affiliation("`affiliation'") address("
> `address'") `asciitable'
    = sthlp Rcall.ado, export("sthlp") template("") replace  title("") summary("") author(
> "") affiliation("") address("")
file Rcall.ado not found
      }
    ---------------------------------------------------------------------- end markdoc ---
(error occurred while loading Rcall.ado)

[haghish]

So version 1.3.0 has several tiny problems. Got them fixes in the new release (1.3.1). But now it seems it's running smoothly. Give it another try.

Anders Alexandersson using the source function is actually ideal! if you can pass your code directly to R, do that! the source code is much easier to update/review when it's an actual R file than a mix of R and Stata. If you are writing a package for Stata, you can still include an R script file in the package and source it in the R code...

[Patrick Fang]

So version 1.3.0 has several tiny problems. Got them fixes in the new release (1.3.1). But now it seems it's running smoothly. Give it another try.

Anders Alexandersson using the source function is actually ideal! if you can pass your code directly to R, do that! the source code is much easier to update/review when it's an actual R file than a mix of R and Stata. If you are writing a package for Stata, you can still include an R script file in the package and source it in the R code...

The latest version 1.3.1 still has a tiny problem:

(1) input "Rcall: " will return error message
"WARNING
file not found
(error occurred while loading Rcall.ado)
r(601);
"
The same error message also appears when in the interactive mode by entering "R:"

[haghish]

I see. The error is caused by markdoc command (the last line of Rcall.ado), which generates the help file. I forgot to turn that into a comment... I updated the package, so it should work now...

[Anders Alexandersson]

Thank you. I still have an error but further down than before.

* Update Rcall (at 8:00 am EST) to current version 1.3.1 
update query
adoupdate Rcall
ado uninstall Rcall
net install Rcall, replace from("https://raw.githubusercontent.com/haghish/Rcall/master/")
R: print("Hello World")

* Rcall setpath now works. There is no error message. 
Rcall setpath "C:/Program Files/R/R-3.3.0/bin/x64/Rterm.exe"

* set trace on for Marc's tested code of RecordLinkage. (See post #22.)
set trace on
set traced 2

capture program drop rlinkage
program define rlinkage
syntax

Rcall vanilla:                  ///
library(RecordLinkage);         ///
data(RLdata500);                             ///  
rpairs = compare.dedup(RLdata500);           ///
rpairs\$pairs[c(1:5), ] ///

end

rlinkage

rlinkage output with error r(1) "unused argument ()" in "display as error":

  --------------------------------------------------------------------------------------- begin rlinkage ---
  - syntax
  - Rcall vanilla: library(RecordLinkage); data(RLdata500); rpairs = compare.dedup(RLdata500); rpairs\$pairs
> [c(1:5), ]
  = Rcall vanilla: library(RecordLinkage); data(RLdata500); rpairs = compare.dedup(RLdata500); rpairs$pairs[
> c(1:5), ]
    ---------------------------------------------------------------------------------------- begin Rcall ---
    - capture prog drop Rpath
    - capture Rpath

    
    [...]

    
    - macro drop Rpath
    - macro drop Rcall_synchronize_mode
    - if "`Rerror'" == "1" {
    = if "1" == "1" {
    - display as error `"{p}`macval(errorMessage)'"'
    = display as error `"{p}Error in `[.RecLinkData`(rpairs, c(1:5), ) : unused argument ()"'
Error in `[.RecLinkData`(rpairs, c(1:5), ) : unused argument () - if "$Rcall_interactive_mode" != "on" error
1 = if "" != "on" error 1

      }
    ------------------------------------------------------------------------------------------ end Rcall ---
  ----------------------------------------------------------------------------------------- end rlinkage ---
--Break--
r(1);

end of do-file

[haghish]

Anders Alexandersson

Your example helped to find the bug and fix it. Thanks.

PS. When you write an ado program and use Rcall vanilla, especially when you load an R package which needs to be installed by the user, you must program defensively and return proper errors.

Here is a tutorial

[Anders Alexandersson]

Thanks!! With version 1.3.2 I now get the correct results too as in post #22. I will do more testing next week, starting on Tuesday.

[Anders Alexandersson]

I am using version 1.3.3, and I still get the correct results as in post #22. I now try to load an R dataframe in Stata but Rcall does not find the load.data() function.

. clear

. R: library(RecordLinkage)

. R: mydata <- data(RLdata500)

. R: load.data(mydata)
Error: could not find function "load.data"

--Break--
r(1);

Below is the relevant traced output.

. set trace on

. set traced 2

.
. R: load.data(mydata)
  ---------------------------------------------------------------------------------------------- begin R ---
  - capture prog drop Rpath
  - capture Rpath
    ---------------------------------------------------------------------------------------- begin Rpath ---
    - global Rpath "C:/Program Files/R/R-3.3.0/bin/x64/Rterm.exe"
    ------------------------------------------------------------------------------------------ end Rpath ---
  - if missing("$Rpath") {
  = if missing("C:/Program Files/R/R-3.3.0/bin/x64/Rterm.exe") {

[...]

  - macro drop Rpath
  - macro drop Rcall_synchronize_mode
  - if "`Rerror'" == "1" {
  = if "1" == "1" {
  - display as error `"{p}`macval(errorMessage)'"'
  = display as error `"{p}Error: could not find function "load.data""'
Error: could not find function "load.data" - if "$Rcall_interactive_mode" != "on" error 1 = if "" != "on"
error 1

    }
  ------------------------------------------------------------------------------------------------ end R ---
--Break--
r(1);