This page describes the basics of the cgraph package. The cgraph package allows to construct, evaluate, and differentiate computational graphs in R. The package supports many operations including arithmetic, trigonometry operations, and linear algebra operations, and can differentiate these operations by reverse automatic differentiation.

Initialization

The stable version of the cgraph package is available under the Apache 2.0 open source license on CRAN. The package can be installed by running the following command in the R command-line:

  install.packages("cgraph")

The package can be loaded in the current R session by:

  library(cgraph)

Once the package has been installed and loaded, a computational graph is created by:

  graph <- cg_graph()

which yields a new cg_graph object. A cg_graph objects holds all data associated with a computational graph. This includes a list of nodes which have been added to the graph. These can be retrieved by data member nodes. Internally, a node is modeled as a cg_node object which has several data members that describe its properties. These data members include:

Data Member Type Description
$id Integer scalar Id by which the node is identified
$name Character scalar Name of the node
$type Integer scalar Node type
$value R object Current value of the node
$grad Numeric vector or array Current gradient of the node

Node Types

There are four types of nodes that can be added to a graph:

Constant

A constant node is a node that always evaluates to a fixed value. A constant node is added to a graph by:

  v <- cg_constant(value = 1, name = "v")

where argument value expects any R object as the value of the constant and argument name expects a character scalar as the name of the constant. In case argument name equals NULL, the constant is added to a graph under an automatically generated name.

Parameter

A parameter is a node that is subject to an optimization process and whose value likely changes over time. A parameter is added to a graph by:

  v <- cg_parameter(value = 1, name = "v")

where argument value expects any R object as the value of the parameter and argument name expects a character scalar as the name of the parameter. In case argument name equals NULL, the parameter is added to a graph under an automatically generated name.

Input

An input is a node that has no associated value upon creation. Instead, an input acts as a placeholder for which the user must provide a value later on. An input is added to a graph by:

  v <- cg_input(name = "v")

where argument name expects a character scalar as the name of the input. In case argument name equals NULL, the input is added to a graph under an automatically generated name.

Operator

Finally, an operator is a node whose value is calculated at run-time based on the value of other nodes in a graph. An operator is added to a graph by:

  v3 <- cg_operator(fun = some_function, inputs = list(v1, v2), name = "v3")

where, argument fun expects a cg_function object as the function that is evaluated by the operator, argument inputs expects a list of cg_node objects as the nodes that are consumed by the operator, and argument name expects a character scalar as the name of the operator. In case argument name equals NULL, the operator is added to a graph under an automatically generated name. Operator v3 in the above code block evaluates to some_function(v1$value, v2$value) where v1$value and v2$value are the values of node v1 and v2 respectively.

The cgraph package provides wrapper functions for many base R functions. For example, to add an operator to a graph that adds node v1 and v2 together, we can call:

  v3 <- cg_add(x = v1, y = v2, name = "v3")

where argument x and y expect cg_node objects as the nodes that are added together and argument name expects a character scalar as the name of the addition operator.

Active Graph

Any node that is either implicitly or explicitly created is added to the graph that is currently active. The active graph is stored in a global cg_session object which is created when the cgraph package is loaded in a R session. The active graph can be retrieved by:

  graph <- cg_session_graph()

Notice that only one graph can be active at a time. A graph is automatically set to be the active graph upon creation. At any time, the active graph can be changed by:

  cg_session_set_graph(graph = some_graph)

where argument graph expects a cg_graph object as the new active graph.

Code Readability

The cgraph package provides several features to improve the code readability when constructing complex computational graphs. There are S3 overloaded methods available for inflix functions like addition and subtraction. These overloaded inflix functions allow to write more compact statements such as:

  v3 <- v1 + v2

Note, however, that inflix functions expect only two arguments and we cannot specify a name for the above operation. Instead, the operation is added to the active graph under an automatically generated name.

Furthermore, when an object is provided to an operation that is not a cg_node object, then the object is implicitly coerced to a constant with the original object as value. Consider the following operation:

  v <- cg_abs(cg_constant(-2))

This is equivalent to:

  v <- cg_abs(-2)

Here, a constant with value -2 is implicitly added to the active graph. Keep in mind that this may not work for overloaded inflix functions. At least one of the arguments provided to an inflix function must be a cg_node object in order for the inflix function to dispatch to the correct operation.

Forward Pass

We can perform a forward pass to evaluate a given node in a graph. A forward pass is performed by:

  cg_graph_forward(graph = some_graph, target = v)

where argument graph expects a cg_graph object as the graph on which the forward pass is performed and argument target expects a cg_node object as the node that is evaluated. If the target node directly or indirectly depends on one or more input nodes, then the user must specify a value for these input nodes by setting their value data member. A forward pass evaluates all nodes that are needed to evaluate the target node (including the target node itself).

Consider the following example:

  # Initialize a computational graph
  graph <- cg_graph()
  
  # Add an input
  a <- cg_input(name = "a")
  
  # Square the input (i.e. b = a^2)
  b <- cg_pow(a, 2, name = "b")
  
  # Set a equal to 2
  a$value <- 2
  
  # Perform forward pass
  cg_graph_forward(graph, b)
  
  # Retrieve the value of b
  b$value
## [1] 4

Backward Pass

One of the reasons to construct a computational graph is that this enables us to evaluate the partial derivatives of any node in the graph with respect to its parameters by reverse automatic differentiation. We do this by performing a backward pass. A backward pass is performed by:

  cg_graph_backward(graph = some_graph, target = v, index = NULL)

where, argument graph expects a cg_graph object as the graph on which a backward pass is performed, argument target expects a cg_node object as the target node that is differentiated, and argument index expects a scalar integer as the index of the target node that is differentiated. By default, index equals NULL and all elements of the target node are differentiated element-wise. The values of all nodes on which the target node directly or indirectly depends must have been evaluated. This can be done by first calling cg_graph_forward to evaluate the target node. During a backward pass, only the derivatives of parameters and operators are evaluated. The backward pass only differentiates the target node with respect to all nodes on which it depends. The derivatives have the same dimensions as the values of the nodes. They can be retrieved via the grad data member of the nodes.

Consider the following example:

  # Initialize a computational graph
  graph <- cg_graph()
  
  # Add an input
  a <- cg_input(name = "a")
  
  # Add a parameter
  b <- cg_parameter(4, name = "b")
  
  # Perform some operations
  c <- cg_sin(a) + cg_cos(b) - cg_tan(a)
  
  # Set a equal to 2
  a$value <- 2
  
  # Perform forward pass
  cg_graph_forward(graph, c)
  
  # Perform backward pass
  cg_graph_backward(graph, c)
  
  # Retrieve the derivative of c with respect to b
  b$grad
## [1] 0.7568025

Eager Evaluation

All operations are evaluated eagerly. That is, when an operator is added to a graph, the value of the operation node is evaluated immediately (if possible). This makes it easier to debug code as the user will immediately receive an error if an operation cannot be evaluated.

Consider the following example:

  # Initialize a computational graph
  graph <- cg_graph()
  
  # Add a parameter
  a <- cg_parameter(2, name = "a")
  
  # Square the input (i.e. b = a^2)
  b <- cg_pow(a, 2, name = "b")
  
  b$value
## [1] 4

Moreover, eager evaluation allows the user to change the control flow of a graph at run-time. For example:

  # Initialize a computational graph
  graph <- cg_graph()

  # Add a parameter
  a <- cg_parameter(2, name = "a")
  
  # Square the parameter 
  b <- a^2

  # Keep squaring the parameter
  while(b$value < 256)
  {
    # Square the parameter
    b <- b^2
    
    # Print value of b
    print(b$value)
  }
## [1] 16
## [1] 256

Here, the number of times we iterate through the while loop depends on the value of node b at run-time.

Custom Operators

It is possible to extend the standard function library of the cgraph package by custom functions that can be used by operators in a computational graph. Operators require a special function, i.e. a cg_function object, which is an extension of a regular R function. In contrast to a R function, a cg_function also defines how a function is differentiated. A cg_function object is created by:

  custom_function <- cg_function(
    def = function(...) {},
    grads = list(function(..., value, grad) {}, ...)
  )

where argument def expects a R function as the function definition and argument grads expects a list of R functions as the partial derivatives of def with respect to its inputs. The derivatives in grads are matched positionally. That is, the first function supplied to grads is expected to be the partial derivative of def with respect to its first input, the second function supplied to grads is expected to be the partial derivative of def with respect to its second input, and so on. Each derivative must take the same arguments as def plus two additional arguments value and grad. These latter two arguments evaluate to the value and derivative of the corresponding input respectively at run-time.

Suppose we want to extend the standard function library by a custom function that calculates logarithms in base 4. To do this, we create a cg_function object for the logarithm function:

  log4 <- cg_function(
    def = function(x) { log(x, base = 4) },
    grads = list(function(x, value, grad) { grad / (x * log(4)) })
  )

Notice that the logarithm function has one input, i.e. x. We need to provide the partial derivative of the logarithm function with respect to this input to argument grads. We derive this derivative by manually differentiating the logarithm function by the rules of differential calculus. Accordingly, we can create a wrapper function that adds an operator for the logarithm function to the active graph:

  cg_log4 <- function(x, name = NULL){
    cg_operator(log4, list(x), name)
  }

We can now use cg_log4 similar to existing logarithm operations in the standard function library (like cg_log2, cg_log10, and cg_ln).