# Initialization

Make sure that the cgraph package is loaded. The package is avaiable on CRAN.

library(cgraph)

A computational graph can be initialized by:

x <- cg_graph()

This yields a new cg_graph object which holds all the data associated with the graph (i.e. its nodes and their values).

# Nodes

A computational graph consists of a set of nodes. There are four types of nodes: constants, inputs, parameters, and operations. All nodes are cg_node objects and have an assoticated id and name by which they are identified in the graph. The main difference between these types of nodes is whether or not they are assigned a fixed value upon creation and whether or not they are differenatiated when differentiating a graph.

Type Fixed Value? Differentiated?
Constant Yes No
Input No No
Parameter Yes Yes
Operation No Yes

## Constants

Constant nodes are added to a computational graph by function cg_constant(). The result of this function is an object of type cg_constant.

node <- cg_constant(10, name = "node")

Constants represent some fixed value that can be consumed by other nodes in the graph.

## Inputs

Input nodes are added to a computational graph by function cg_input(). The result of this function is an object of type cg_input.

node <- cg_input(name = "node")

Input nodes behave similarly as constant nodes but are not given a fixed value but instead function as placeholders. Their value needs to be provided once a graph is evaluated of differentiated.

## Parameters

Parameter nodes are added to a computational graph by function cg_parameter(). The result of this function is an object of type cg_parameter.

node <- cg_parameter(10, name = "node")

Parameter nodes are given a value upon creation. This value is subject to an optimization process and might change over time.

NOTE

When no name is provided to cg_constant(), cg_input(), or cg_parameter(), the resulting node is added to the graph under an automatically generated name.

## Operations

Operation nodes are added to a computational graph by calling operators on a set of existing nodes. For example, to subtract node node2 from node1, we call:

node3 <- cg_sub(node1, node2, name = "node3")

This adds an operation node to the graph that is currently active. Operation nodes behave as placeholders. Their value is calculated by performing an operation on existing nodes in the graph.

The cgraph package provides operators for many base R functions. These operators are named cg_<name> where <name> is the name of the base R function. For example, to apply the base mean function to a node, you can call operator cg_mean(). The cgraph package also provides overloaded operators for many base inflix operations like addition or subtraction. For example, to subtract node2 from node1, we can simply call:

node3 <- node1 - node2

However, due to the nature of inflix functions, we are not able to supply a name for this operation. Instead, the operation node is added to the active graph under an automatically generated name.

NOTE

It is possible to define custom operators by function cg_operator(). See the documentation of the function for more information on how this is done.

When performing an operation on an object that is not a cg_node object, the object is implicitly converted to a cg_constant object holding the object. This allows to write much cleaner code. Consider the following operation:

node <- cg_neg(cg_constant(2))

This is equivalent to:

node <- cg_neg(2)

Keep in mind that this may not work for overloaded inflix operators. At least one of the arguments provided to an inflix operator should be a cg_node object in order for the operator to dispatch to the corresponding graph operator (if available). Hence, the following code does not produce an operator node:

node <- 3 + 4

Instead, you must explicitly create a node (i.e. a constant, input or parameter) for at least one of the arguments:

node <- 3 + cg_constant(4)

# Active Graph

All nodes are added to the computational graph that is currently active. This also applies to operator nodes that are created by operators like cg_mean or overloaded inflix operators like + or -. A cg_graph object is set to be the active graph upon creation. Function cg_session_graph can be callled to retrieve the graph that is currently active.

x <- cg_graph()

identical(x, cg_session_graph())
## [1] TRUE

Function cg_session_set_graph can be called on an existing cg_graph object to change the active graph.

x <- cg_graph()
y <- cg_graph()

# this node is added to y
a <- cg_parameter(10, "a")

cg_session_set_graph(x)

# this node is added to x
b <- cg_parameter(20, "b")

NOTE

Only one computational graph can be active at a time.

# Run a Graph

In some cases, it is desireable to evaluate a node and retrieve the value of all its ancestors in a computational graph. This procedure is called a forward-pass.

Function cg_graph_run() can be used to perform a forward-pass.

x <- cg_graph()

a <- cg_input("a")

b <- cg_pow(a, 4, "b")

values <- cg_graph_run(x, b, list(a = 2))

as.list(values)
## $b ## [1] 16 ## ##$x2
## [1] 4
##
## $a ## [1] 2 In order to perform a forward-pass, all ancestors of the target node must be available or they must be able to be computed at run-time. Argument values of cg_graph_run() can be used to substitute values for input nodes that do not yet have a value. The output of cg_graph_run() is an environment containing the value of the target node and the values of all its ancestors that are evaluated in the forward-pass. # Differentiate a Graph A nice feature of a computational graph is that its nodes can be differentiated by reservse automatic differenation. This is done by traversing the ancestors of a given target node in the reverse direction and applying elementary rules of calculus to differentiate the nodes. The partial derivatives of the target node with respect to all its ancestors can be efficiently evaluated in a single backward-pass through the graph. Function cg_graph_gradients() can be used to perform a backward-pass. x <- cg_graph() a <- cg_parameter(2, "a") b <- cg_pow(a, 2, "b") values <- cg_graph_run(x, b) grads <- cg_graph_gradients(x, b, values) as.list(grads) ##$a
## [1] 4
##
## \$b
## [1] 1

In order to perform a backward-pass, all ancestors of the target node must have a value. These values can be collected by first performing a forward-pass for the node using function cg_graph_run(). The environent obtained by cg_graph_run() can be supplied to the values argument of cg_graph_gradients().

Currently, the cgraph package can only differentiate scalar target nodes. In case the target node is a vector or array, argument index of cg_graph_gradients() can be used to specific which element of the vector or array is differentiated. The output of cg_graph_gradients() is an environment containing the partial derivative of the target node with respect to each ancestor of the node (including the target node itself) that is an operator node or parameter node. These derivatives have the same shape as the nodes.