[Windows only] RProvider doesn't work with R > 4.0.2

[AndrewIOM]

Thank you for the tips @dsyme. I enabled logging and get the same log output as you. Tracing the code, I think this is an upstream problem caused by an interaction between newer versions of R and a Windows security feature "Control Flow Guard" as mentioned in this RDotNet issue rdotnet/rdotnet#139 (comment). That would also explain why the failure is only happening on windows.

It looks like we'll have to wait for that fix to resolve this windows build problem. But I'll also keep poking at it to see if we can get it to build either on on R <= 4.0.2 or with the workaround noted in that RDotNet issue.

More details:
The s.GetPackages() code seems to be failing after the first call to eval("1+4") because the log doesn't print the 5 after eval("1+4").Value:

    let getPackages() : string[] =
        Logging.logf "Communicating with R to get packages"
        Logging.logf "Test: %O" (eval("1+4"))
        Logging.logf "Test: %O" (eval("1+4").Value)
        eval(".packages(all.available=T)").GetValue()

The above referenced RDotNet issue says calling Evaluate on x64 for R >= 4.0.3 causes crashes. I installed R 4.0.2 and set that as R_HOME to make RProvider use it and the logs a) confirm use of 4.02 and b) get farther. Here the System.Double output should be the results of eval("1+4").

[9/17/2021 11:09:02 AM] [Pid:1484, Tid:1, Apid:1] eval(1+4)
[9/17/2021 11:09:02 AM] [Pid:1484, Tid:1, Apid:1] Output: 
[9/17/2021 11:09:02 AM] [Pid:1484, Tid:1, Apid:1] Converting value from R...
[9/17/2021 11:09:02 AM] [Pid:1484, Tid:1, Apid:1] [DEBUG] MEF Container
[9/17/2021 11:09:02 AM] [Pid:1484, Tid:1, Apid:1] [DEBUG] MEF Container 2
[9/17/2021 11:09:02 AM] [Pid:1484, Tid:1, Apid:1] [DEBUG] MEF Container 3
[9/17/2021 11:09:02 AM] [Pid:1484, Tid:1, Apid:1] Test: System.Double[]

The build tests still hang with the log stopping after trying to do something with Test.RProvider\data/sample.rdata. But the fact that we get farther with 4.0.2 than 4.0.3 is consistent with the problem being caused by the R/RDotNet/Control Guard issue.

Originally posted by @nhirschey in #216 (comment)

[siavash-babaei]

It doesn't work with R >= 4.0.2

[nhirschey]

R 4.2.0 is now released with the fix. Below shows R-4.2.0 and RProvider working on windows (albeit with some encoding errors in the SymbolicExpression printer):

#r "nuget: RProvider"

open RDotNet
open RProvider
open RProvider.utils

fsi.AddPrinter FSIPrinters.rValue
R.sessionInfo()

(* Evaluates to
val it: RDotNet.SymbolicExpression =
  R version 4.2.0 (2022-04-22 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22000)

Matrix products: default

locale:
[1] ÿþLC_COLLATE=English_United States.1252ÿþ
[2] ÿþLC_CTYPE=English_United States.1252ÿþ
[3] ÿþLC_MONETARY=English_United States.1252ÿþ
[4] ÿþLC_NUMERIC=Cÿþ
[5] ÿþLC_TIME=English_United States.1252ÿþ

attached base packages:
[1] ÿþgraphicsÿþ  ÿþgrDevicesÿþ ÿþutilsÿþ     ÿþdatasetsÿþ  ÿþmethodsÿþ   ÿþbaseÿþ

loaded via a namespace (and not attached):
[1] ÿþcompiler_4.2.0ÿþ


*)

[matthewcrews]

Is this working now? I'm looking to use the Type Provider but I've been watching this issue.

[nhirschey]

Sort of. It’s still not entirely working out of the box because of some issues with environment variable discoverability (likely the same issues for why R_HOME is t found be default) that prevent it from finding stats.dll

see workarounds here rdotnet/rdotnet#151

[robreno]

Is this still an issue? I cannot get R to work with F# at all. It is 2023 and this is still not fixed?

[nhirschey]

Hi @robreno, I appreciate the frustration but the maintainers are donating their spare time to help here. Nobody is paying them. I'd love to have it fixed, but I myself only have so much time to contribute fixes to this project. Tradeoffs 🤷

This might work for you, assuming you set the R_HOME environment variable. Run the code line by line up until the engine and path are set.

But I'll caveat, there are remaining path and encoding issues, so don't expect things to be smooth.

#r "nuget: RProvider"

open RProvider
open RProvider.stats
open RProvider.utils
open RProvider.Operators

open RDotNet

REngine.SetEnvironmentVariables("c:/program files/r/r-4.2.0/bin/x64", "c:/program files/r/r-4.2.0")

let engine = REngine.GetInstance()
engine.Evaluate("Sys.setenv(PATH = paste(\"C:/Program Files/R/R-4.2.0/bin/x64\", Sys.getenv(\"PATH\"), sep=\";\"))")

R.eval("library(stats)")

fsi.AddPrinter FSIPrinters.rValue

R.mean([1.0; 2.0; 3.0; 4.0; 5.0; 6.0; 7.0; 8.0; 9.0; 10.0])

R.mean([1.0; 2.0; 3.0; 4.0; 5.0; 6.0; 7.0; 8.0; 9.0; 10.0]);;
[1] 5.5
val it: SymbolicExpression =

[nhirschey]

@AndrewIOM as you're thinking about vNext, just FYI there is a Windows UTF encoding bug related to UTF byte order mark (BOM) as shown with the in my prior comment to this thread. #224 (comment) See the "ÿp" characters at the start and end of string entries: [1] ÿþLC_COLLATE=English_United States.1252ÿþ

I have not been able to track down where the bug is introducted. I gave up after awhile but the next things I was going to try were:

1. Hack of explicitly using regex to remove the "ÿp" BOM before printing.
2. Perhaps the encoding bug doesn't happen on mac because you sink the output to a file before printing in fsi

   RProvider/src/RProvider.Runtime/RInterop.fs

   Line 699 in 008c084

   let capturer = if Configuration.isUnixOrMac () then printUsingTempFile else printUsingDevice

   Maybe sinking to a file would fix Windows too.