This page describes the basics of the `cgraph`

package. The `cgraph`

package allows to construct, evaluate, and differentiate computational graphs in R. The package supports many operations including arithmetic, trigonometry operations, and linear algebra operations, and can differentiate these operations by reverse automatic differentiation.

The stable version of the `cgraph`

package is available under the Apache 2.0 open source license on CRAN. The package can be installed by running the following command in the R command-line:

` install.packages("cgraph")`

The package can be loaded in the current R session by:

` library(cgraph)`

Once the package has been installed and loaded, a computational graph is created by:

` graph <- cg_graph()`

which yields a new `cg_graph`

object. A `cg_graph`

objects holds all data associated with a computational graph. This includes a list of nodes which have been added to the graph. These can be retrieved by data member `nodes`

. Internally, a node is modeled as a `cg_node`

object which has several data members that describe its properties. These data members include:

Data Member | Type | Description |
---|---|---|

`$id` |
Integer scalar | Id by which the node is identified |

`$name` |
Character scalar | Name of the node |

`$type` |
Integer scalar | Node type |

`$value` |
R object | Current value of the node |

`$grad` |
Numeric vector or array | Current gradient of the node |

There are four types of nodes that can be added to a graph:

A constant node is a node that always evaluates to a fixed value. A constant node is added to a graph by:

` v <- cg_constant(value = 1, name = "v")`

where argument `value`

expects any R object as the value of the constant and argument `name`

expects a character scalar as the name of the constant. In case argument `name`

equals `NULL`

, the constant is added to a graph under an automatically generated name.

A parameter is a node that is subject to an optimization process and whose value likely changes over time. A parameter is added to a graph by:

` v <- cg_parameter(value = 1, name = "v")`

where argument `value`

expects any R object as the value of the parameter and argument `name`

expects a character scalar as the name of the parameter. In case argument `name`

equals `NULL`

, the parameter is added to a graph under an automatically generated name.

An input is a node that has no associated value upon creation. Instead, an input acts as a placeholder for which the user must provide a value later on. An input is added to a graph by:

` v <- cg_input(name = "v")`

where argument `name`

expects a character scalar as the name of the input. In case argument `name`

equals `NULL`

, the input is added to a graph under an automatically generated name.

Finally, an operator is a node whose value is calculated at run-time based on the value of other nodes in a graph. An operator is added to a graph by:

` v3 <- cg_operator(fun = some_function, inputs = list(v1, v2), name = "v3")`

where, argument `fun`

expects a `cg_function`

object as the function that is evaluated by the operator, argument `inputs`

expects a list of `cg_node`

objects as the nodes that are consumed by the operator, and argument `name`

expects a character scalar as the name of the operator. In case argument `name`

equals `NULL`

, the operator is added to a graph under an automatically generated name. Operator `v3`

in the above code block evaluates to `some_function(v1$value, v2$value)`

where `v1$value`

and `v2$value`

are the values of node `v1`

and `v2`

respectively.

The `cgraph`

package provides wrapper functions for many base R functions. For example, to add an operator to a graph that adds node `v1`

and `v2`

together, we can call:

` v3 <- cg_add(x = v1, y = v2, name = "v3")`

where argument `x`

and `y`

expect `cg_node`

objects as the nodes that are added together and argument `name`

expects a character scalar as the name of the addition operator.

Any node that is either implicitly or explicitly created is added to the graph that is currently active. The active graph is stored in a global `cg_session`

object which is created when the `cgraph`

package is loaded in a R session. The active graph can be retrieved by:

` graph <- cg_session_graph()`

Notice that only one graph can be active at a time. A graph is automatically set to be the active graph upon creation. At any time, the active graph can be changed by:

` cg_session_set_graph(graph = some_graph)`

where argument `graph`

expects a `cg_graph`

object as the new active graph.

The `cgraph`

package provides several features to improve the code readability when constructing complex computational graphs. There are S3 overloaded methods available for inflix functions like addition and subtraction. These overloaded inflix functions allow to write more compact statements such as:

` v3 <- v1 + v2`

Note, however, that inflix functions expect only two arguments and we cannot specify a name for the above operation. Instead, the operation is added to the active graph under an automatically generated name.

Furthermore, when an object is provided to an operation that is not a `cg_node`

object, then the object is implicitly coerced to a constant with the original object as value. Consider the following operation:

` v <- cg_abs(cg_constant(-2))`

This is equivalent to:

` v <- cg_abs(-2)`

Here, a constant with value `-2`

is implicitly added to the active graph. Keep in mind that this may not work for overloaded inflix functions. At least one of the arguments provided to an inflix function must be a `cg_node`

object in order for the inflix function to dispatch to the correct operation.

We can perform a forward pass to evaluate a given node in a graph. A forward pass is performed by:

` cg_graph_forward(graph = some_graph, target = v)`

where argument `graph`

expects a `cg_graph`

object as the graph on which the forward pass is performed and argument `target`

expects a `cg_node`

object as the node that is evaluated. If the target node directly or indirectly depends on one or more input nodes, then the user must specify a value for these input nodes by setting their `value`

data member. A forward pass evaluates all nodes that are needed to evaluate the target node (including the target node itself).

Consider the following example:

```
# Initialize a computational graph
graph <- cg_graph()
# Add an input
a <- cg_input(name = "a")
# Square the input (i.e. b = a^2)
b <- cg_pow(a, 2, name = "b")
# Set a equal to 2
a$value <- 2
# Perform forward pass
cg_graph_forward(graph, b)
# Retrieve the value of b
b$value
```

`## [1] 4`

One of the reasons to construct a computational graph is that this enables us to evaluate the partial derivatives of any node in the graph with respect to its parameters by reverse automatic differentiation. We do this by performing a backward pass. A backward pass is performed by:

` cg_graph_backward(graph = some_graph, target = v, index = NULL)`

where, argument `graph`

expects a `cg_graph`

object as the graph on which a backward pass is performed, argument `target`

expects a `cg_node`

object as the target node that is differentiated, and argument `index`

expects a scalar integer as the index of the target node that is differentiated. By default, `index`

equals `NULL`

and all elements of the target node are differentiated element-wise. The values of all nodes on which the target node directly or indirectly depends must have been evaluated. This can be done by first calling `cg_graph_forward`

to evaluate the target node. During a backward pass, only the derivatives of parameters and operators are evaluated. The backward pass only differentiates the target node with respect to all nodes on which it depends. The derivatives have the same dimensions as the values of the nodes. They can be retrieved via the `grad`

data member of the nodes.

Consider the following example:

```
# Initialize a computational graph
graph <- cg_graph()
# Add an input
a <- cg_input(name = "a")
# Add a parameter
b <- cg_parameter(4, name = "b")
# Perform some operations
c <- cg_sin(a) + cg_cos(b) - cg_tan(a)
# Set a equal to 2
a$value <- 2
# Perform forward pass
cg_graph_forward(graph, c)
# Perform backward pass
cg_graph_backward(graph, c)
# Retrieve the derivative of c with respect to b
b$grad
```

`## [1] 0.7568025`

All operations are evaluated eagerly. That is, when an operator is added to a graph, the value of the operation node is evaluated immediately (if possible). This makes it easier to debug code as the user will immediately receive an error if an operation cannot be evaluated.

Consider the following example:

```
# Initialize a computational graph
graph <- cg_graph()
# Add a parameter
a <- cg_parameter(2, name = "a")
# Square the input (i.e. b = a^2)
b <- cg_pow(a, 2, name = "b")
b$value
```

`## [1] 4`

Moreover, eager evaluation allows the user to change the control flow of a graph at run-time. For example:

```
# Initialize a computational graph
graph <- cg_graph()
# Add a parameter
a <- cg_parameter(2, name = "a")
# Square the parameter
b <- a^2
# Keep squaring the parameter
while(b$value < 256)
{
# Square the parameter
b <- b^2
# Print value of b
print(b$value)
}
```

```
## [1] 16
## [1] 256
```

Here, the number of times we iterate through the while loop depends on the value of node `b`

at run-time.

It is possible to extend the standard function library of the `cgraph`

package by custom functions that can be used by operators in a computational graph. Operators require a special function, i.e. a `cg_function`

object, which is an extension of a regular R function. In contrast to a R function, a `cg_function`

also defines how a function is differentiated. A `cg_function`

object is created by:

```
custom_function <- cg_function(
def = function(...) {},
grads = list(function(..., value, grad) {}, ...)
)
```

where argument `def`

expects a R function as the function definition and argument `grads`

expects a list of R functions as the partial derivatives of `def`

with respect to its inputs. The derivatives in `grads`

are matched positionally. That is, the first function supplied to `grads`

is expected to be the partial derivative of `def`

with respect to its first input, the second function supplied to `grads`

is expected to be the partial derivative of `def`

with respect to its second input, and so on. Each derivative must take the same arguments as `def`

plus two additional arguments `value`

and `grad`

. These latter two arguments evaluate to the value and derivative of the corresponding input respectively at run-time.

Suppose we want to extend the standard function library by a custom function that calculates logarithms in base 4. To do this, we create a `cg_function`

object for the logarithm function:

```
log4 <- cg_function(
def = function(x) { log(x, base = 4) },
grads = list(function(x, value, grad) { grad / (x * log(4)) })
)
```

Notice that the logarithm function has one input, i.e. `x`

. We need to provide the partial derivative of the logarithm function with respect to this input to argument `grads`

. We derive this derivative by manually differentiating the logarithm function by the rules of differential calculus. Accordingly, we can create a wrapper function that adds an operator for the logarithm function to the active graph:

```
cg_log4 <- function(x, name = NULL){
cg_operator(log4, list(x), name)
}
```

We can now use `cg_log4`

similar to existing logarithm operations in the standard function library (like `cg_log2`

, `cg_log10`

, and `cg_ln`

).