plumbing.graph
A Graph is a simple, declarative way to define a composition of functions that is
easy to define, modify, execute, test, and monitor.
This blog post provides a high-level overview of Graph and its benefits:
http://blog.getprismatic.com/blog/2012/10/1/prismatics-graph-at-strange-loop.html
Concretely, a Graph specification is just a Clojure (nested) map with keyword keys
and keyword functions at the leaves.
A Graph is defined recursively as either:
1. a keyword function (i.e., fn satisfying PFnk), or
2. a Clojure map from keywords to (sub)graphs.
A Graph is a declarative specification of a single keyword function that
produces a map output, where each value in the output is produced by executing
the corresponding keyword function in the Graph. The inputs to the keyword
function are given by the outputs of other nodes in the graph with matching
keywords (mimicking lexical scope in the case of nested maps), or failing that,
from keywords in the input map.
For more details and examples of Graphs, see test/plumbing/graph_examples_test.cljx.
->graph
(->graph graph-nodes)
Convert a graph specification into a canonical well-formed 'graph', which
is an array-map with nodes in a correct topological order that will respond
to 'io-schemata' with a specification of the graph inputs and outputs.
The graph specification can be a Clojure map, in which case the topological
order will be computed (an error will be thrown for cyclic specifications),
or a sequence of key-value pairs that are already in a valid topological order
(an error will be thrown if the order is not valid). Values in the input
sequence are also converted to canonical graphs via recursive calls to ->graph.
check-comp-partial!
(check-comp-partial! g instance-fn)
Check that instance-fn is a valid fn to comp-partial with graph g.
comp-partial
(comp-partial g instance-fn)
Experimental.
An extension of pfnk/comp-partial that supplies new parameters to a subgraph,
useful in composing hierarchical graphs.
g is a graph, and instance-fn is a fnk that takes arguments from the surrounding
context and produces new parameters that are fed into g. Works by comp-partialing
all leafs that expects any parameter produced by instance-fn with instance-fn,
so beware of expensive instance-fns, or those that expect caching of some sort
(i.e., attempt to generate shared state).
Throws an error if any parameter supplied by instance-fn is not used by at least
one node in g.
comp-partial-fn
(comp-partial-fn f other)
Return a new pfnk representing the composition #(f (merge % (other %)))
compile
(compile g)
Compile graph specification g to a corresponding fnk using the a default
compile strategy for host.
Clojure: eager-compile
ClojureScript: interpreted-eager-compile
eager-compile
(eager-compile g)
Compile graph specification g to a corresponding fnk that is optimized for
speed. Wherever possible, fnks are called positionally, to reduce the
overhead of creating and destructuring maps, and the return value is a
record, which is much faster to create and access than a map. Compilation
is relatively slow, however, due to internal calls to 'eval'.
graph
(graph & nodes)
An ordered constructor for graphs, which enforces that the Graph is provided
in a valid topological ordering. This is a sanity check, and also enforces
defining graphs in a readable way. Most explicit graphs should be created
with this constructor.
(graph
:x-plus-1 (fnk [x] (inc x))
:2-x-plus-2 (fnk [x-plus-1] (* 2 x-plus-1)))
in addition, an 'inline' graph can be provided in place of a key-value
sequence, which will be merged into the graph at this position.
instance
macro
(instance g m)(instance g bind m)
Experimental.
Convenience macro for comp-partial, used to supply inline parameters to a
subgraph (or fnk).
Example:
(= {:x 21}
(run (instance {:x (fnk [a] (inc a))} [z] {:a (* z 2)})
{:z 10}))interpreted-eager-compile
(interpreted-eager-compile g)
Compile graph specification g to a corresponding fnk that returns an
ordinary Clojure map of the node result fns on a given input. The
compilation is much faster than 'eager-compile', but the compiled fn
will typically be much slower.
io-schemata*
(io-schemata* g)
lazy-compile
(lazy-compile g)
Compile graph specification g to a corresponding fnk that returns a
lazymap of the node result fns on a given input. This fnk returns
the lazymap immediately, and node values are computed and cached as needed
as values are extracted from the lazymap. Besides this lazy behavior,
the lazymap can be used interchangeably with an ordinary Clojure map.
Required inputs to the graph are checked lazily, so you can omit input
keys not required by unneeded output keys.
par-compile
(par-compile g)
Experimental. Launches one future per node at startup; we probably woudln't
use this in production, and will release more sophisticated parallel
compilations later.
Compile graph specification g to a corresponding fnk that returns a
lazymap of the node result fns on a given input. This fnk returns
the lazymap immediately, and node values are computed and cached in parallel
starting immediately (and attempts to extract values from the lazymap will
block until each value is computed). Besides this lazy behavior,
the lazymap can be used interchangeably with an ordinary Clojure map.
positional-eager-compile
(positional-eager-compile g arg-ks)
Like eager-compile, but produce a non-keyword function that can be called
with args in the order provided by arg-ks, avoiding the overhead of creating
and destructuring a top-level map. This can yield a substantially faster
fn for Graphs with very computationally inexpensive node fnks.
profiled
(profiled profile-key g)
Modify graph spec g, producing a new graph spec with a new top-level key
'profile-key'. After each node value is computed, the number of milliseconds
taken to compute its value will be stored under an atom at 'profile-key'.
restricted-call
(restricted-call f in-m)
Call fnk f on the subset of keys its input schema explicitly asks for
run
(run g input)
Eagerly run a graph on an input by compiling and then executing on this input.
simple-flat-compile
(simple-flat-compile g check-input? make-map assoc-f)
Helper method for simple (non-nested) graph compilations that convert a graph
specification to a fnk that returns a Clojure map of the graph node values.
(make-map m) converts an initial Clojure map m to the return type of the fnk,
and (assoc-f m k f) associates the value given by (f) under key k to map m.
simple-hierarchical-compile
(simple-hierarchical-compile g check-input? make-map assoc-f)
Hierarchical extension of simple-nonhierarchical-compile.
working-array-map
(working-array-map & args)
array-map in cljs no longer preserves ordering, replicate the old functionality.