Todo.xml

<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml-stylesheet type="text/xsl" href="Admin/XSL/Todo.xsl" ?>

<topics xmlns:r="http://www.r-project.org" xmlns:c="http://www.c.org">


<topic>
<title>Tools</title>
<items>

<item>
See tests/forVector.R
Should be able to determine that ctr = 0 is a double since we return ctr and we know the return type. 
Should do this in Type Inference.
</item>

<item status="verify">
Check we haven't broken anything else. Fixed getMathOpType to use the @ref pointers for comparing the types.
<br/>
Comparing double to Int32Type
See tests/compareDiffTypes.R
It seems like we end up with two elements in convert.i and so the subsetting  vars[[convert.i]] is hierarchical.
<br/>
isIntType is false.
Importantly, targetType is NULL.  So neither  operand types will match.
So
<r:code>
getMathOpType(types, call[-1])
</r:code>
is responsible.
<br/>
Does this have anything to do with the Int1Type for the return type?
</item>

<item>
Put better names for variables in multiSubset.R
</item>

<item status="done">
Compilation of code in Rllvm/tests/walkblocks.R  (the part in the FALSE section) fails.
Seems to have wrong types in the fcmp.
<br/>
Same issue with the enums overloaded for Add and Sub (in this case Sub).
</item>

<item status="done">
The addition instruction isn't being added.
<r:code>
library(RLLVMCompile)
g = function(x, y) x[1] + y[1]
i.ptr = pointerType(Int32Type)#, 0L)
#debug(RLLVMCompile:::mathHandler)
gc = compileFunction(g, Int32Type, list(i.ptr, i.ptr), optimize = FALSE)
showModule(gc)
<r:output><![CDATA[
define i32 @g(i32* readonly %x, i32* readonly %y) {
entry:
  %0 = getelementptr i32, i32* %x, i64 0
  %1 = load i32, i32* %0
  %2 = getelementptr i32, i32* %y, i64 0
  %3 = load i32, i32* %2
  %"x[1] + y[1]" = ret i32 %1, i32 %3
  ret i32 %"x[1] + y[1]"
}
]]></r:output>
</r:code>
<br/>
Enums are overloaded for BinOps and BinaryOps and Add was 1 rather than 11.
</item>


<item>
See getBlockGraph()
<br/>Visualize a module, i.e., the relationships between its blocks, etc.
Allow the user to see the flow and then to click on a block and see the instructions.
</item>
</items>
</topic>

<topic>
<title>Compilation Strategies</title>
<items>

<item>
If code refers to a non-local variable, either capture that variable now
and convert it to a value in the module or insert a call to a routine
to get the variable. We can then generate a routine in the module that
retrieves the current value of the R object from the R interpreter.
<br/>
Add a parameter to compileFunction that controls whether we do this
aggressive capture or lazy lookup at call time.
</item>


<item>
Memory management
</item>

<item>
If a function doesn't modify an input, pass by reference.
</item>

<item>
Deal with closures within a function. Create global variables in the module ?
</item>

<item>
Compile closures with shared state. See R/compileClosures.R
Use RCIndex collectors as examples, or SAX handlers.
First compile  the functions, ignoring the global variables and get
them to work.
Use clone() to copy the module for a new instance. In other words,
the call to the generator function is just a call to clone of the
routines.
</item>

<item>
Explore non-standard evaluation.
</item>

</items>
</topic>

<topic>
<title>Code Analysis</title>
<items>
<item status="note">
Started <r:pkg>CodeAnalysis</r:pkg> package. On github too.
</item>

<item status="note">
<r:code>
library(CodeDepends)
library(RLLVMCompile)
sc = new("Script", as.list(body(compileFunction)))
inputs = getInputs(sc)
</r:code>
There are 8 expressions in the function and inputs has 8 elements.
The if() statements, etc. are dealt with as one input and unraveled. So should this be a recursive data structure that gets returned? It is processed recursively, but
the question is whether it should be returned as a recursive data structure.
<br/>
[Check] Need to explore and fix CodeDepends for functions
and in-memory expressions/calls rather than from scripts.
<br/>
Need to turn the expressions into a Script object with new("Script", body(fun)).
That's done.
</item>

<item status="extend">
See constInputs in CodeAnalysis.
<br/>
Identify which inputs/parameters a function doesn't modify.
Can CodeDepends do this for us?
Yes. See explorations/deadVariables.R
<r:code>
sc = readScript("explorations/script.R")
inp = getInputs(sc)
any(sapply(inp, function(node) "x" %in% node@updates)) # true
any(sapply(inp, function(node) "z" %in% node@updates)) # false
</r:code>
</item>

<item status="extend">
See  findLoopConcat in CodeAnalysis.
Recognize/identify idioms of the form
<r:code>
 x = c()
 for(i in z) 
    x = c(x, f(i))
</r:code>
The key is the declaration of an empty vector and then
concatenating it with a single value.
</item>

</items>
</topic>

<topic>
<title>Callbacks to R</title>
<items>

<!--
<item>
Call R functions from C code.
<br/>
Can construct the expression in the compiled code, if we have access to the routines in R.
Alternatively, compile the R function and just call that.
</item>
-->
<item>
When we can determine that we cannot create the R call object
using create_foo_expression, don't compile that routine.
Also, allow caller to turn this off with an option.
</item>

<item>
What if we have 
<r:code>
function(x)
   .R(foo(a + x), ...)
</r:code>
Do we substitute the value of x into the expression before calling back to R?
<br/>
Global variables are simply bad. So do we want to support them at all?
It is complicated to walk this expression generally at run-time, i.e. in our compiled code.
(This example is reasonably simple, of course, but generally.)
</item>

<item status="test">
Handle .typeInfo() in getGlobals and ignore.
This also means using getGlobals rather than findGlobals().
And this involves getting the names of the R callable functions, updating them, and so on.
</item>

<item status="test">
Add a .R() to indicate a callback. Add extra information.
See explorations/dotRCallback.R
</item>

<item>
Rather than copying the result back from Rf_eval(),
we could implicitly protect it (i.e. the compiled code would add this) and then unprotect it.
We could identify when the variable is no longer needed
and add code to release it then.
</item>


<item>
Figure out why explorations/conCall.R doesn't give much speed.
</item>

<item status="done">
Fix while loop in conCall.R and then it might work.
</item>

<item status="verify">
When we use R API routines, add their declarations to the module, e.g. R_CHAR, strdup.
</item>

<item>
If we enable the execution engine in compileFunction implicitly, i.e. during the compilation,
can we extract/query it from the Function/module.
i.e. can we do <r:expr>as(Module, "ExecutionEngine")</r:expr>
<br/>
Doesn't seem to be accessible.
</item>

<item status="done">
Variables that we want to find in R are getting promoted to  global variables.
See callbacks2.R and pi. Really want to identify that this is a call to an R 
function and so variables that are not already declared/defined in the
local function being compiled should be ignored
<br/>
Is this really the case? or is it just that we have pi as a constant.
Yes, this is just pi. But problems with globals.
</item>

<item status="verify">
[callbacks2.R] Named arguments in the call corresponding to local
variables, i.e foo(1, bob, x = a) and we substitute the value of a,
but skip bob.  Unfortunately, we use codetools::findGlobals() to find
the globals before we start compiling the code.  It finds these in the
R callbacks as these are regular R expressions.  So we have to be more
specific than findGlobals and basically find global variables within
each call and then ignore those in R callback functions.  We can also
have the caller specify which are R variables and exclude those.  This
is not a bad thing to do as it ensures that the caller knows these
must be present in the R session. But it would be nicer to determine
these.  So we can adapt codetools or write our own walker.  See
R/getGlobals.R.
<item>
Don't include code that is a callback to R in the analysis of global variables and functions.
</item>
<br/>

</item>

<item status="done">
Handle expressions in the call to the R function, e.g. foo(a + 10)
for local variable a.
<br/>
Done for arguments we are filling in, but what about those
we do not provide.
This goes back to needing to be able to get the original call
at compile time and setting that using setCall_foo_expression.
But need it at run-time.
We can collect these as we compile and then when we are finished
compiling all the code, includeing setCall_*_expression, we can
invoke those if we have an ExecutionEngine.
</item>

<item status="done">
Pass SEXP back to R. See callbacks.R
</item>

<item status="done">
callbacks.R not working. Is now.
</item>

<item status="done">
explorations/simpleCallback.R and explorations/conCall.R (FIX this - see above)
</item>

<item status="done">
Get the value of a global variable, specifically the SEXP for the call expression in a callback.
<r:code>
m[["foo_expression", value = TRUE, ee = ee]]
</r:code>
</item>


<item status="done">
Problem with creating the call. Reneable and debug. simpleCallback2.R 
<br/>
Needed to convert the elements of the call to their R types by compiling calls to
Rf_ScalarReal, Rf_install(), etc.
</item>

<item status="complete">
Before evaluating the call at run time, insert the local variables in the routine 
into to the corresponding positions in the call.
</item>

<item status="done">
Call create_call to create the expression.
<br/>
Need to define it the function first before we can reference it.
Not really, but have to be careful with our compiler when we
reference a function.
</item>

<item status="done">
Protect the call? No need as we have protected it with R_PreserveObject().
If we do, then we have to ensure it is unprotected later. 
If called from R, then R will take care of this.
However, if we are doing complicated things in the native code, we may need to catch any error and unprotect.
</item>

<item>
Release the memory for the arguments we added to the particular call.
<br/>
We can add code to set these to R_NilValue.
We can either loop over the call again. Alternatively, we can store the pointers
to set when we walk the first time.
</item>

<item>
[CHECK]
When compiling R expressions in compileCreateCall,
we need to convert literals into calls to ScalarReal, etc.,
i.e. their R objects, not native versions.
Do this when inserting the values into the call, both at run-time and when creating the call.
</item>

<item status="done">
Find way to capture the call at compile time and use it at run-time.
<br/>
Multiple ways.
<ol>
<li>We serialize the call using RDS, store it as a local variable in a routine we compile
    and can deserialize by calling that routine.</li>
<li>One can explicitly set the global variable for the expression from within R
either by calling the set_foo_expression routine in the module or by setting the global variable in the module.
To set this, we need the ExecutionEngine. If this is available in the call to <r:func>compileFunction</r:func>,
we set the expression before returning. This is probably the fastest way to do things as we don't have to deserialize.
But that is negligible.
</li>
<li>There is routine that creates the call. This only works in simple cases, i.e.
where there are no arguments in the R call that a) we don't supply, and b) that are complex calls.
In these cases, use either of the other two approaches</li>
</ol>
<br/>
Necessary for complex values that are not local variables.
(Of course, caller could use a closure and call the target function with local variables.)
<br/>
Could embed it in the R wrapper function that then calls set_call.
Alternatively, serialize it into the module and then have an initialization
call or have create_foo_expression deserialize it.
</item>

</items>
</topic>

<topic>
<title>Compiler Structure</title>

<items>


<item>
Consolidate the code in compile.= and call.R for numeric(), etc.
and the isPrimitiveConstructor() and compilePrimitiveConstructor()
and how we rewrite to do the Rf_allocVector in call.R
</item>


<item status="high">
Create a class and constructor to represent the compiler.
(Maybe a reference class).
Then allow this to be reused to compile different functions.
</item>


<item status="done">
Function for resetting the compiler to start new Function, new block,
clear local variables.
<br/>
compilerStartFunction
</item>

<item status="high">
Rather than having the createLoopCode() check to see if 
compile.if (or any other compile method) has made the incrBlock
the current insertion block. If it has, then createLoopCode  needs to avoid 
adding the branch instruction. But really, the other methods should try to 
aoid changing the insertion block except into blocks that they created.
<br/>
Same for while that has a break.
</item>

<item>
Check if statement with a continue/next/break inside a loop  works correctly.
See inst/examples/repeat.R
</item>

<item>
Really want to be able to distinguish between REALSXPType and INTSXPType, etc.
i.e. not all just SEXPType and same pointer.
<br/>
Sort out a mechanism for dealing with more specific SEXP types
and not matching the types when calling functions with more
specific types.
</item>

<item>
Consolidate the 2 getLimits functions.
</item>



<item status="low">
Think about introducing a run-time package, e.g. RLLVMCompileRT
with C routines to support the generated code, i.e. that the generated code can call.
Since the idea is to use the code immediately in the session
in which we compile it, this code can be in RLLVMCompile itself.
</item>

<item>
Add structured errors to the compiler to identify user errors, compiler errors.
<br/>
See call.R@86 and WrongNumArgs error.
</item>

</items>
</topic>


<topic>
<title>General</title>
<items>

<item>
Deal with calls on LHS of an assignment:
<c:code>
INTEGER(ans)[i] = 2
</c:code>
Arises in Rllvm/explorations/sapply.R
</item>

<item>
In sapply(), deal with strings potentially being passed as CHARSXP to the function being applied
to each element. If it is expecting a char * (i.e., StringType), then we need to access that pointer
before the call.
<br/>
Do we do this in rewriteSApply() or when we compile the resulting expression? Probably the latter.
</item>

<item>
Add Block graph diagrams to show control flow.
And allow the text to flow around these if they are small.
</item>

<item>
<r:code>
fc = compileFunction(function(x, y) x^y, Int32Type, list(Int32Type, Int32Type))
</r:code>
fails because pow is declared for double types.
This works
<r:code>
fc = compileFunction(function(x, y) x^y, DoubleType, list(DoubleType, DoubleType))
</r:code>
</item>

<item status="done">
tests/multiCondIf.R doesn't work. Can't find <![CDATA[&&]]>.
This is missing from <r:func>getCompilerHandlers</r:func>,
as is || and | and &amp;.
<br/>
This is compiled. But the problem is something else.
We end up in the <r:func>createConditionCode</r:func> code.
<br/>
The <r:func>insertReturn</r:func> doesn't seem to be putting the return in the correct place 
and that is the problem for the code below:
<r:code><![CDATA[
f = function(x, y)  { if(x < y && x > y/2)  3 else 4}
gc = compileFunction(f, Int32Type, list(DoubleType, DoubleType))
]]></r:code>

The following works
<r:code><![CDATA[
gc = compileFunction(function(x, y)  { if(x < y && x > y/2)  return(3) else return(4)}, Int32Type, list(DoubleType, DoubleType), .insertReturn = FALSE)
]]></r:code>

</item>

<item>
Does the compile.for have to move the counter incrementing code to a separate Block if
there is just a break in the body of the loop, but not a next?
Seems like we could branch to the nextBlock on a break, but still increment the code
at the end of the loop otherwise.
</item>

<item status="no">
Add comments to the code to make it easier to understand.
Can't do it. Use metadata in the IR. But we can't add comments  in the IR.
</item>


<item status="finish">
Write functions to visualize the blocks and the jumps.
Create a graph and then use Rgraphviz or igraph to render it.
<br/>
See  showBlocks.R and getBlockGraph()
</item>

<item status="high">
Add calls to Rf_protect and Rf_unprotect appropriately in the compiler.
Possibly need to add an .on.exit to the compiler which will generate code
in the final block.
Also, add a counter for the number of protects for cases where it is deterministic at compile time.
</item>

<item status="done">
tests/convolve.R doesn't do the looping correctly.
It goes into an infinite loop. Are we testing the wrong llen variables?
Do we introduce these erroneously since we already have na and nb.
Or are we merging the setup for the two loops (the initialization one for the result, and the actual one that does  that the convolution).
<br/>
Problem is that the branch for the <xml:tag>i in  1:na</xml:tag> loop goes to the body or
back to the after <xml:tag>for(k in 1:n)</xml:tag> block which starts things all over again.
So setting the next block is incorrect.
We don't seem to be processing the return expression.
<br/>
We needed to set the afterBlock to NULL after we pop it.
<br/>
Seems like we have the after k block is left on the stack when we enter compiling the i loop.
<br/>
Is it that we initialize i = 1 for the second loop and then branch in that block ("after k....")
<br/>
[No] Is it that we use i in both loops?
</item>

<item status="done">
Get the compilation of a vectorization function of a scalar working.
See vec.R and makeVectorized.R
</item>

<item>
Use CXCursor example (or XML parsing) example to use a customized subsetting 
operation, i.e. <r:expr>cursor[[1]]</r:expr> to mean <r:expr>children(x)[[1]]</r:expr>
and in fact to map that to a more explicit function that calculates this.
Currently it does this in RCIndex by visiting all the nodes.
</item>

<item status="done">
For asFunction = TRUE in compileFunction, add ... to the formals of the 
function.
</item>


<item status="check">
Fix while loops the way we did for the  for() loops,
i.e. nextBlock, nextIterBlock and so on.
</item>

<item status="done">
for() loop with a  break doesn't break.
Related to the nextBlock changes we made.
Jump to the incr. block since that is nextBlock.
Need a breakBlock.
<br/>
See tests/forBreak.R
</item>


<item status="done">
Simple if statement is broken!!!
<br/>
See tests/realSimpleIf.R
Problem is that there is no return value in the function and so no
expression following the if statement. Now we get an error.
</item>

<item>
See if we can recover in tests/realSimpleIf.R when return type is VoidType.
See if.R@100
<item>
Handle cases where R doesn't return anything explicitly, i.e. just NULL implicitly, but we need
a particular type, e.g. an integer in 
<r:code><![CDATA[
foo = function(x) if(x < 10) 100L
]]></r:code>
At least identify these for the person calling compile and ask for a default
value. <r:na/> can work for certain types.
<br/>
See insertReturn in utils.R.
This will work for simple cases. But not for  cases with if, while,
for. Here we will have to jump out of those local blocks and then return
from the routine.  So we will need to change the code that would return to assign to a
fixed variable (e.g. .return) and then jump to the end block
</item>

<item>
Check for return - lacking return causes Rllvm to hardcrash, i.e.
<r:code>
function(x) {
  if (!x)
    return(3L)
}
</r:code>
</item>

</item>

<item status="done">
Make getGlobals() try to understand lazy evaluation, i.e. not evaluating the
default values for formal parameters until they are fist used.
We could do this by rewriting to add the <r:expr>if(missing(x)) x = expr</r:expr>.
<br/>
See substituteDefaultValues() and tests/getGlobals.R
</item>

<item status="done">
Add .debug(), .assert() to indicate conditional code.
</item>

<item>
support testthat and stopifnot and so on for assertions.
</item>

<item status="done">
supress the warning message in callHandler when we invoke mapply()
but the number of targetTypes is less than the number of elements in the call.
This happens, for example, in a varargs routine, e.g. 
<r:expr>printf("x = %d, y = %d\n", x, y)</r:expr>
where we have x and y w/o type information.
</item>


<item status="done">
For .assert() and friends, allow a class = c("a", "b")
and then call R_raiseStructuredError().
Need to figure out how to create an array of strings as a global variable.
See err.cpp and the C++ code LLVM generates to do this.
For now, use va_arg.
<br/>
See tests/structuredAssert.R.
</item>

<item status="done">
Raise richer exceptions in the run-time code. Provide a mechanism to specify the classes to raise.
Use stop() or Stop() if not compatible.
See tests/structuredStop.R
</item>


<item status="done">
How to make the global native variable in R R_GlobaEnv
available to the module.
AddSymbol? declare it locally?
<br/>
Seem to need $address rather than just getNativeSymbolInfo().
</item>

<item status="finish">
(Need to pass .targetType down the call stack.)
<br/>
Fix calls of the form foo(1) where 1 is supposed to be Int32Type.
When we know the expected type and have a literal, make it so!
<br/>
tests/checkIntLiteral.R
<br/>
See callHandler and <r:expr>lapply(as.list(call[-1]), compile, .......)</r:expr>
We should put the type information here.
Then we'd have to add, say, a .targetType parameter to each.
</item>

<item status="low">
Generate error or warning when use a wrong variable.
This is not possible, in general.
<br/>
If we change r_ans to rans in callRFunction.R on the LHS of the Rf_eval,
then we just get the wrong answer.
It is legitimate as we created the local variable (why did we need it?)
</item>

<item status="done">
Extend globalCallSEXP.R to process the return value.
<br/>
Now we print it as an R object, and convert it to an int and then print it via LLVM.
</item>

<item status="done">
Get the globalCallSEXP.R sorted out.
Problem with Rcall variable not being resolved.
But the code in globalArray.R  does work.
The Rcall declaration has an "external" qualifier.
Is this because it is a pointer? because it has no initializer?
The latter, so use ConstantPointerNull.
</item>

<item status="done">
Compile <r:expr>if(x != NULL)</r:expr>
Type information for NULL.
tests/nullComparison.R
<br/>
See logicOpHandler.R
</item>


<item status="done"> 
Element subsetting with a data frame.
See dataFrameSubset.R
</item>


<item>
Matrix subsetting for VECSXType. Less common.
</item>

<item status="done">
matrix subsetting for STRSXPType, 
Need to add the same sort of code in SEXP.R to multiSubset.R
<br/>
tests/matrixSubset.R
</item>

<item status="done">
Recognize x[i] for a SEXPType and change this to be work on x.els and 
compute the pointer first as a local variable.
</item>


<item>
Recognize to compile <r:expr>sapply(v, `[[`, 1)</r:expr>,
i.e. to rewrite this  either as a loop over v and insert  
<r:expr>el[[i]]</r:expr> or to rewrite this as
<r:expr>sapply(v, function(x) x[[1]])</r:expr>.
The former is nicer as it is essentially inlining the body of the function.
</item>



<item status="done">
Doug Bate's Rgibbs example is giving NaNs
</item>

<item status="done">
[Done but see above]Compile Doug Bate's Rgibbs example
<br/>
It involves matrix subset assignment, and combining values on the RHS of the assignment.
<br/>
Assignment seems to work and looping fine. But the values we get are  NaN.
First few are not, but then they go nuts.
</item>

<item>
Add optional bounds checking on <r/> objects when subsetting.
Allow the caller to add hints to the compiler to say check these variables, but not these.
Also, develop static analysis to identify which ones go over bounds, if possible.

<item>
Add an option to perform/eliminate bounds checking on arrays, nrow, ncol, etc.
</item>

<item>inbounds for accessing elements of an array.
Rllvm now has isInBounds methods to query and set.
</item>
</item>

<item status="done">
Test subsetting with a list(). Simple accessing an element. Then assign to an element.
Need to  know the type of the element unless we use run-time information.
<br/>
See list2.R
<br/>
Try
<r:code>
x = replicate(N, list(a = 2, b = 3))
sapply(x, function(x) x[[1]])
</r:code>
Don't use names ("a"), but position.
</item>

<item status="done">
Need a SEXP type for lists.
<br/>
Use VECSXPType. Same as getSEXPType("VEC").
</item>


<item status="verify">
[Works now] Error in insertReturn when expecting a return value but none
<r:code>
f = function(x, i) { el = x[[i]]}
fc = compileFunction(f, Int32Type, list(SEXPType, Int32Type))
</r:code>
</item>

<item status="verify">
Constant folding.
Arises in matrix subsetting in SEXP.R around line 60 when we create 2L - 1L.
We know this is 1L.
For 1L -1L convert this to 0 but recognize that we don't have to do the multiplication.
Also, When one operand in multiplication is 1, don't do the multiplication.
When adding, if one of the operands is 0, don't do the addition.
</item>

<item>
When doing constant folding elimination for arithmetic, i.e. remove a mult/dv or an add/sub since it is the same value,
we  need to eliminate/remove the previously compiled instructions.
<br/>
We may want to compile the operands sequentially and check after the first one to see if it or the second one is necessary,
in contrast to lapply() to compile them both and then examine.
</item>


<item status="done">
Eliminate dead code. Does LLVM do this for us?
See explorations/deadCodeElim.R.
<br/>
Yes, LLVM can clean up for us with a pass.
<br/>
Is this a case where a compiler pass would be useful.
</item>


<item status="done">
Access a matrix cell.
See tests/checkToIntSubset.R
And check explorations/matrixElAssign.Rdb
</item>


<item status="done">
When doing arithmetic such as 2 - 1L, recognize that the 2 is an integer value
(not type). Add parameter/flag to compile for arithmetic to say make the result an integer.
When computing the index for subsetting, we'd activate this.
So in <r:expr>x[2, i]</r:expr> we want 2 - 1L to be an integer.
<br/>
See tests/checkToIntSubset.R
</item>


<item status="high">
Compute the REAL(x) pointer just once for a matrix.
See matrixElAssign.Rdb.
Similarly, compute Rf_nrow(), etc. just once.
But detect if the value could change.
<br/>
When we have a REALSXPType and access its elements,
create a local variable and initialize it with a call to REAL().
Then use that in place of the REALSXPType variable.
</item>

<item status="high">
Handle the <r:expr>pragma(IfAssign, var, updatedExpressions)</r:expr>
</item>

<item status="check">
Handle assignments from RHS being if-else
<r:code>
z = if(cond) val1 else val2
</r:code>
See 2DRandomWalk.Rdb.
<br/>
Either have  the assignment determine that the RHS is complicated 
or have the RHS's compile method return information about the two branches.
In either case, this <r:expr>assignHandler = `compile.=` = </r:expr>
method for the compiler needs to understand that this is special and requires
modifying the code. (Same for ifelse).
Need to ensure the types from the two branches are the same.
<br/>
IMPORTANT: we have the ifAssign.R file for rewriting if statements.
[Done] FIX THIS:  see inst/examples/ifAssign.R and specifically bar.
And export these functions and use them in compile.
Also, we may want to "declare" variable before the if. But don't know its type there.
But we can go back and set it
</item>

<item status="high">
getBlocks() seg faults.
2DRandomWalk.Rdb in the if handler.
</item>

<item status="news">
Do we have functions in Rllvm to get the current block, i.e. the block with the insertion point?
Yes  getInsertBlock()
</item>

<item>
Why are we exiting out of R when a module is not valid? Would like to get an error in R.
See 2DRandomWalk.Rdb.
</item>



<item status="done">byval for parameters when passing a struct object directly to a routine. explorations/clangHandler2.R
<br/>
See call.R::callHandler.  Need to determine which argument has a byval attribute.
How do we query this?
<r:code>
arg = mod$clang_CXCursor_getName[[1]]
hasByValAttr(arg)
</r:code>
But it appears we don't need to set this if the declaration of the Function
has this information.
</item>

<item status="verify">
In tests/structByVal.R, if we don't have the return(2L), we get an error about no terminator.
<br/>
This seems to work again. But I saw it in globalArrayAccessors.R once after this.
So perhaps some memory problem in Rllvm.
</item>

<item>
What is .vectorize  doing? Does it actually generate vectorized code.
</item>

<item>
tests/ with a XXX or a FIX
vectorize.R  character3.R
<br/>
Fixed: structByVal.R  globalArrayAccessors.R
</item>

<item>
Add a mechanism to differentiate between integer() 
meaning INTSXPType or Int32Type in R code.
<br/>
See explorations/list2.R for one example.
</item>

<item>
Test the other code now that we switched load = TRUE for subsetHandler
</item>

<item status="done">
Compile rw2d
<br/>
See tests/2DRandomWalk.Rdb
<br/>
Didn't we do this for the StatSci paper? Yes, but manually.
</item>

<item>
Compile vectorized fibonacci (Rllvm/explorationts/fib.R)
and compare with closed form solution approach.
</item>

<item>
handle case of serial assignment
<r:code>
xpos = ypos = numeric(n)
</r:code>
to ensure create a new vector.
Do we do this for the SEXP types, i.e. call Rf_duplicate() when assigning? No.
See tests/serialAssign.R.
So need to recognize that assigning a SEXP to a SEXP needs to duplicate to
respect the R semantics.
</item>

<item>
Get native version of nrow(), ncol() that don't
use a SEXP.
Can implement directly ourselves as C routines.
Nice to compile them however.
</item>

<item>
When we call any of the constructor functions, i.e. numeric(), etc.
protect the result and unprotect them at the end.
</item>

<item status="done">
Allocate the variables within a loop outside of the loop just once.
<br/>
Could be smarter about this. Currently doing it for ever variable.
But only needs to be done for a  loop.
</item>


<item>
Problem with  looping from 1:n and subsetting using i - 1L.
When we have nested loops,  i + j doesn't correspond to the correct element
See tests/convolve.R
We can work with two variables - i and i.index/i.offset and use them for different purposes,
i.e i.offset for subsetting.
</item>

<item>
When compiling expr = expr, pass the 
type of the LHS to compile.call, and generically to compile
as, say, targetType.  Alternatively, we can just  cast the result
when assigning to the LHS.
</item>

<item status="done">
Get the numeric() example in tests/numeric.R  working.
</item>




<item status="done">
map functions to other names in a call, e.g. length to Rf_length and then compile.
May want to check we are dealing with a SEXPType.
</item>

<item>
Check in the loops that we are getting the limits correct.
In explorations/sapply.R in Rllvm/ we are not doing the last element correctly.
Interaction issue here as subsetting subtracts off the 1 which may be necessary to do 
since not always in a loop.
</item>

<item status="check">
recognize function names in calls to sapply.
It would be okay if we rewrote the sapply() calls before determining
the global variables.
<br/>
Does getGlobals() get this right?
This note was from before getGlobals() was in use, but instead codetools::findGlobals().
But I think that got it right as being a global function.
</item>

<item>
There is some inelegancy in handling return(sapply(...)) 
when we do the rewrite of sapply(). 
It would be nice to do the rewrite explicitly and so remove the
return(sapply(....)).
In fact, we actually add it and then remove it.
But there is knowledge in the rewrite about the name of the variable
and actually returning it.
</item>

<item>
Separate storing the type for a variable and creating
it. In rewriteSApply(), we set the type for r_ans, but we haven't 
created it.  So keep track of which variables we have created.
</item>

<item>
In the code we generate for rewriteSApply(), the generic compile() function
doesn't dispatch to the methods for <![CDATA[compile.<-]]> and compile.for.
Instead, they go to call. Yet the class seems to be correct. However
they are created programmatically with quote(), etc.
I have put in a mechanism to do this dispatch explicitly in compile.call.
</item>

<item status="done">
When calling a function, perform a cast for the arguments if they are not of the correct
type.
e.g. STRING_ELT(a, 0) and STRING_ELT(a, 0L). The latter works, but the former doesn't.
<br/>
Works. See tests/argCast.R.
</item>

<item>
[*] Implement assignment to accessor for a STRSXP. assignToSEXPElement
See tests/SEXPStringSubset.R
</item>

<item status="done">
getGlobals() needs to handle variable in for.
</item>

<item>
SEXPSubset.R - 2nd example returns a list with four elements 20, 2, 3 and c(20, 2, 3)
</item>


<item status="done">
Can't add x[1] + 20L when passing in an INTSXPType.
Need to implement the getTypeOfElement for SEXPType types.
tests/SEXPSubset.R.
<br/>
Fix the casting of the x[i] = 20 in SEXPSubset example.
Currently need 20L, but should be able to handle this.
<br/>
Problem was that x[i] wasn't understanding SEXPs as special and working with the pointer to the sxpinfo_struct.
</item>

<item>
We need to know what the target/output type of an expression
such as x[i] is when we x is a SEXP type.
Is it the raw element or is it a SEXP subset, i.e. an integer() vector.
We know this from the context, potentially.
<br/>
Added a SEXPToPrimitive, but not using it yet.
</item>

<item status="done">
Problems with casting when returning an element of a REALSXP as a DoubleType.
tests/SEXPSubset.R
Are we treating the x as a pointer to an array of structs, i.e SEXP*
and so getting it all wrong.
So we need to know that these are SEXPTypes and work differently with these.
Same as for assignment. See getElementAssignmentContainerType
</item>




<item>
[Added some support to subtract 1 via subractOne()]
One- and zero-based indexing.
Could do a rewrite of the expression.
Also could just subtract 1 from the expression.
</item>

<item>
Handle subsetting generally, non-integer values and strings and vectors.
</item>

<item status="done">
Subsetting with x[2] (any literal) fails. Need to make the literal into an integer.
See R/subset.R
</item>

<item status="done">
Allow fun[["n"]] retrieve the parameter named n in fun.
Do we store the names?
</item>

<item status="check">
Recognize a = if() x else y and map to a select/ternary operation.
<br/>
This is done in fixIfAssign.
</item>

<item>
Map switch() in R to a switch in LLVM when the conditions are integers.
Also recognize a series of if() statements.
<br/>
Can use LLVM's SwitchInst class, but not for cases we have a string.

</item>

<item>
introduce declarations as R functions, e.g. strPointer(tmp)
which then creates a local variable tmp of type pointerType(StringType).
Also concepts such as positiveInt(), nonNegInt(), unsignedInt().
These functions would do nothing and so can stay in code that is interpreted.
</item>

<item status="done">
See tests/character*.R
compile references to character() to be the NULL string, i.e. 0.
<r:code>
library(RLLVMCompile)
f = function() { x = character() ; 1L}
fc = compileFunction(f, Int32Type)
</r:code>
</item>

<item id="multiCond" status="done">
Multiple conditions in a test, i.e. if and while condition
<r:code><![CDATA[
f = function(x, y)
{
  ans = 0L
  if(x < 0 || y < 10 || y > 100)  #x + y == 0)
      ans = 3L
  else
      ans = 7L
 
  return(ans)
}
fc = compileFunction(f, Int32Type, list(x = Int32Type, y = Int32Type))
]]></r:code>
How do we represent these - test each separately and branch for each or combine the operations and test the overall result?
For || the second approach won't work as we don't want to evaluate the RHS.
Checking with  how llvm does it on a .c file, it is the multi branch/block setup.
</item>

<item status="done">
When comparing to a literal and the literal has an integer value,
don't use a double/floating point comparison when the other value is an integer, e.g.
<r:code>
if(i &lt; 2) ...
</r:code>
then the compiler currently treats 2 as a numeric, not an integer and promotes i.
<br/>
See tests/intComparison.R
</item>

<item>
Is the code generated for the tests/simpleFor.R correct?
It works, but the incrementing of the loop counter is in the body. That is okay. Did we want a separate block? Unnecessary!
<br/>
Also, the condition block has an extra instruction that seems irrelevant.
<pre>
%3 = load i32* %ctr
</pre>
But it is necessary. It loads the current value of ctr so that the body block can add 1 to the %3 variable.
Why isn't this in the body?  It looks like the insertion point is added back to the condition block
when the body is being processed.
</item>



<item status="done">
Explore why the fuseLoop example with fc for f2 segfaults when we have a vector of length 1e6 rather than 1e5
See tests/loop.R, I believe.
<br/>
My guess is that this was before we handled the local variable  in the loop
by defining it once outside of the loop.
</item>




<item>
Allow caller of compile to customize the code
to a particular data frame or set of inputs, e.g. to
map names to positions just once, e.g.
<r:code>
data$var1[1:n]
data[1:n, "var1"]
</r:code>
would look up the index of var1 and replace "var1" with its index.
(Or use a hash table to represent the data structure.)
</item>

<item>
Support compiling for code with NAs or without, i.e.
allow the caller of compileFunction to specify whether
to include code for NAs.
<br/>
There is a parameter for this in compileFunction.
Is it used? No, at this moment. April 17th.
Basically, we ignore NAs entirely.
</item>

<item>
Support creating code for vectors or scalars,
i.e. different code that takes a vector and iterates over that or
 a function/routine that takes a scalar.
These are qualitatively different routines.
</item>

<item>
Should we use a fixed variable name for the return value.
Then all code could store that and jump  to final block.
</item>

<item status="done">
Compile ! 
<r:code>
library(RLLVMCompile)
h = function(l) !l
hun = compileFunction(h, Int32Type, Int32Type)
.llvm(hun, 1)
</r:code>
This creates an xor.
<br/>
[Fixed] Now fails but in better place. Tries to cast Int1Type to Int32Type.
Write the code for this in utils.R (in RLLVMCompile) to handle this.
<br/>
Works but <r:expr>.llvm(hun, 0)</r:expr> gives -1. 
What should it give us.  See tests/not.R
</item>




<item>
Compute types for the intrinsics and put in a table
so that the R user doesn't have to specify them.
Does this make sense, i.e. for the overloaded ones?
</item>

<item>
Rewrite expressions. 
Do we use a separate pass?
Or do we make the changes as we try to compile a call, e.g. replace pi with the constant,
replace  expr^2 with pow(expr, 2). (Ideally, we'd create a temporary variable for expr and
then multiply.)
<br/>
Both - see CodeAnalysis and its code reweriting functions.
Also rewriteSApply
</item>

<item>
Handle x[1:10], i.e. x[vector] and x[vector] = value(s).
</item>

<item status="low">
[This is probably no longer relevant]
Compile the not() function to test the ! operator.
<br/>
Which not() function. See below for a function(x) !x.
</item>

<item status="check">
Logical data type - Int1Type.
</item>

<item status="done">
Does run() in Rllvm return the mutable parameters?
<br/>
Use .all = TRUE
</item>

<item status="note">
The OPS table/list is now called CompilerHandlers
and it can be customized for a call to 
compileFunction by copying that object and
inserting new elements or replacing existing ones.
</item>

<item status="note">
One can control which functions are compiled
by specifying <r:arg>.globals</r:arg>.
</item>

<item status="high">
Handle default values for parameters in R functions.
<br/>
See substituteDefaultValues() in  CodeAnalysis package.
<br/>
We can compile the code just before the first 
expression at which that parameter is referenced.
<br/>
Need to know about missing. This is a feature of the call.
We can add a parameter to the target function .missing as a vector
and then add this in each call.  Then we can test this in the code
and conditionally evaluate the default value.
</item>

<item>
Handle named arguments and match them during compilation
and reorder. Don't worry about
any assignments  in the expressions which might
make the order important
e.g.
<r:code><![CDATA[
  foo(b = (num <- sum(x)), a = integer(num))
]]></r:code>
This is suicidal code anyhow and 
there is no guarantee b will be evaluated before a
due to lazy evaluation.
</item>

<item status="done">
Allow the caller of compileFunction to specify their own
version of .compilerHandlers to customize how the
code generation operates.
</item>

<item status="done">
Implement the for loop of the form for(i in x) 
by introducing a loop counter, say, .i
and add a new instruction to load x[i].
<br/>
See tests/forVector.R
</item>

<item>
Implement R's switch().
</item>

<item status="done">
Have the supportFunctionInfo split for external routines
and one that we compile ourselves.
<br/>
If the called functions have type information, then we can compile
them. But allow this to be specified by the caller of compileFunction.
<br/>
Or do type inference.
</item>

<item status="check">
Compiling other R functions we reference.
Compile function tries to do this.
</item>


<item status="done">
Process the .supportFunctionInfo in compileFunction.
</item>

<item status="partial">
Calls to other functions, e.g. log or printf.
<br/>
Use codetools to find these before we start compiling.
Get their types. Allow as parameter in compileFunction.
<br/>
See test.R and the example loop.next and printi
</item>

<item>
An R object (function probably) that
can arrange to load the external symbols before calling run,
i.e. the functions we call that we don't compile.
<br/>
asFunction in compileFunction causes the return of a function.
We can add the names of the external routines to the environment
and arrange to load those when the function is invoked, even
just the first time.
<br/>
We really want the name of the DLL for each symbol
so we can use getNativeSymbolInfo() to get  it correctly.
We probably even need the path to the DLL so we can load it.
See setMetdata and getMetadata
<r:code>
a = setMetadata(m, "REAL.dll", "xyz")
a = setMetadata(m, "REAL.packageName", "xyz")
</r:code>
</item>


<item>
Returning R objects.
<br/>
We strip away the SEXP and pass the data to our new routines.
How do we return these as SEXPs, e.g. put them back into
a (new) SEXP.  Rllvm does this for us. But we may want to
take this over as we can.
<br/>
We might want to allocate the space before the call and have
our new routine write into that. Need to avoid duplicating
in the call to run.
<br/>
See how we can do this with a special object with  class
that we return from compileFunction.
</item>


<item status="done">
Split the incrementing in a loop into its own condition so that
we can jump to that block of code for a continue.
<br/>
Done, but giving wrong result for continue.
See loop.next
</item>

<item status="done">
Unary logical operator !. A good way to do this would be use LLVM's
XOR op with 1.
<br/>
tests/unaryLogical.R
</item>

<item status="done">
We just return the compiled function or the R function now and not the module
as we can get the module from the function with 
<r:code>as(fun, "Module")</r:code>
Also, <r:expr>showModule(fun) </r:expr> works without any
need to explicitly coerce the function to a module.
</item>

<item status="done">
The type of the logical in the branch is wrong.
See the rep function
<r:code>a = compileFunction(rep, Int32Type)</r:code>
</item>

<item status="check">
Finish off next and break handlers.
Specifically, have the while and for loops 
register then .nextBlock and .continueBlock.
Put these on a stack.
</item>

<item status="check">
Implement repeat
</item>


<item status="check">
Create temporary variable for if statements 
when we assign the value to a variable in R, e.g.

<r:code><![CDATA[
x = if(a < 1) "abc" else "def"
]]></r:code>
Need to store the value from the expressions
in the bodies of the if statement.
<br/>
Code is in ifAssign.R to rewrite these expressions.
Now done automatically in 
</item>



<item>
[ operator for simple vectors.
<br/>
Have to be aware of factor, Date and POSIXt types.
</item>
<item>
[ operator for matrices
</item>

<item>
[ for data frame.
<br/>
Get the variable then [] on the vector.
We can hope the LLVM optimizer will 

</item>


<item status="done">
May want to simplify compile() and not call getArgs() and pass them to
the handler function - call.op. Many of the handlers will need the entire call.
</item>

<item status="done">
The for loop, like the if, needs to know where to branch to.  So it
needs the next.  It can create a new Block and refer to it.  Then the
next expression being compiled can just write to that without having
to even worry.  So we don't need to create the blocks for each
expression and pass the nextBlock argument.
</item>

<item status="verify">
Fix the if() mechanism. See explorations/BML.Rdb.
The if inside the if (for assigning nextRow) in moveCars
is jumping to the wrong place.
</item>

<item status="done">
fixIfAssign broken again. BML.Rdb
<br/>
For literals, we didn't do the assignment.
</item>

<item status="high">
Make constParams process functions that are called from the original and
see which ones modify their parameters.
</item>

<item status="done">
Make math convert literal values to integers if they are actually integers.
Different from R but more natural.
<br/>
It happens elsewhere already.
</item>

<item status="done">
<r:code>
f =
function(x, y)
{
  a = 1
  b = 2
  c = 1 + x # XXX  putting x here causes problems!!!!
  x
}

fc = compileFunction(f, Int32Type, list(Int32Type, Int32Type), optimize = FALSE)
</r:code>
</item>

<item>
Use SelectInst as a ternary operator.
</item>

<item>
Compile ifelse() to use a SelectInst when the types are such that this makes sense,
i.e. scalars.
This also relates to fixIfAssign(). If that generates a pragma, we can do the conversion
in the compiler depending on the types.
</item>

<item>
Insert calls into the generated code to check for user interrupts.
</item>

<item>
Identify non-const inputs and duplicate them. Allow overriding.
We can generate code to make the copies.
</item>
<item>
Compile the expression <r:expr>list(a, b, c)</r:expr>.
</item>

<item>
Allow .debug to be an integer value indicating a level.
So <r:expr>.debug(foo(x), level = 2)</r:expr>
would then evaluate the expression <r:expr>foo(x)</r:expr> if a global .debug 
variable was &gt;= 2.
</item>

<item status="low">
Need a function to give the name of an available temporary variable
for our function we are compiling.
<br/>
Don't really need this.
If we do, we are maintaining the names of the local variables in the 'env'
object. So we can easily use this to comput a new name.
</item>

<item status="done">
Handle no explicit return() in the code
i.e. 
<r:code>
function() { 
  x = 1
  y = 2
  x + y
}
</r:code>
where the last expression is the return value.
See the <r:pkg>TypeInfo</r:pkg> package for how we do this.
</item>

<item status="done">
Add connections to the different blocks, i.e.
 when generate code for one block and then another
 add a branch from the first to the next.

    getBlocks() gives us the blocks for a function.

</item>

<item> 
Handle vectorized operations of ==, +, !, etc.
</item>

<item status="done"> 
Handle unary "-".
<br/>
See tests/unary.R.
</item>

<item status="done">
Deal with unary constant, e.g. -1 or +2.
These should just be constants.
See tests/unary.R
</item>

<item status="done">
Add returning correct Null in R_Module_getGlobalVariable in Rllvm so
assignHandler gets to second condition.
</item>





</items>
</topic>
<topic>
<title>Type Inference</title>
<items>

<item>
Type inference system needed.
See RTypeInference on github.
</item>

<item>
For callbacks to R, we have to specify the signature of the R function.
We can determine this at compile time and add it dynamically.
We just need to know the name of the function so that we don't assume it is compiled.
Also, need to check the signature is the same across different calls in our code
(at compile time, not run-time).
</item>

<item>
Identify const variables.
<item>
Identify const variables.
This is for our purpose rather than LLVM. We want to be able to identify
these to determine whether we can pass them by reference since they don't change.
</item>

<item status="done">
Identify constants  such as pi in dnorm calculation.
If the current value is in a sealed package and is a numeric value,
then could treat this as constant, heuristically.
Replace in the code.
</item>
</item>

<item>
When compiling a function and we are given its types, follow those
through the expressions to determine the types of local variables and
also then follow calls to other functions and determine the types of
those calls and so recursively follow the expressions in those other
functions.
</item>

<item>
Figure out the output from a function based on its inputs
and the functions it calls.
</item>

<item status="ok">
Handle variable reassignment; possibly by using a strategy of renaming
variables. Are arguments immutable? Should this be treated exactly as
it is in R?
<r:code><![CDATA[
t2 <- function(x) {
  y <- 4L
  x <- y
  return(x)
}
]]></r:code>
<r:code>
ff = compileFunction(t2, DoubleType, DoubleType)
.llvm(ff, 3)
</r:code>
</item>


<item status="done">
If the  return type is VoidType, don't bother with the .insertReturn
See Dnorm_v
</item>

<item status="done">
When we find the free variables, check if they exist before
generating any code and report errors immediately.
</item>
<item status="done" note="works for fib example">
Handle case when an expression is an if() and both branches return.
Need to kill off the "next" block since there isn't one.

See fib
<r:code>
library(RLLVMCompile)
fib =
function(n)
{
  if(n == 0L || n == 1L)
     n
  else
     fib(n - 1L) + fib(n - 2L)
}
fc = compileFunction(fib, Int32Type, list(n = Int32Type), name = "fib")
</r:code>
</item>

<item status="done">
insertReturn for fib function above.
Adding .ret and not return() for the else expression.
<br/>
Check we haven't broken anything else related to insertReturn.
</item>

<item status="done">
Issues with createNeg. 
<r:code>
library(RLLVMCompile)
f = function(n)
{
  return( - n)
}
fc = compileFunction(f, Int32Type, list(n = Int32Type))
</r:code>
Broken module.
Do we need to load n first?
</item>

<item status="done">
The initial assignment to ans doesn't compile
<r:code>
f = function()
{
  ans = -1L 
  return(ans)
}
</r:code>
We can handle this as a literal value and assign the value directly
after evaluating it in R. Done.
<br/>
<r:error>cannot createNeg for this type yet.</r:error>
Now need a method for createNeg which is in Rllvm,
but we need a version .
Compiles now, but get a bad module.
</item>

</items>
</topic>
</topics>