Quick Start

Installation

Let’s first install node-graph via pip:

$ pip install node-graph

First workflow

Suppose we want to calculate (x + y) * z in two steps.

  • step 1: add x and y

  • step 2: then multiply the result with z.

Create task

Task is the basic building block of a workflow. One can create a task from a Python function using the decorator:

from node_graph.decorator import task


@task()
def add(x, y):
    return x + y


@task()
def multiply(x, y):
    return x * y


@task.graph()
def AddMultiply(x, y, z):
    the_sum = add(x=x, y=y).result
    return multiply(x=the_sum, y=z).result


ng = AddMultiply.build(x=1, y=2, z=3)
ng


Engines and provenance Run graphs directly in Python:

from node_graph.engine.local import LocalEngine

graph = AddMultiply.build(x=1, y=2, z=3)

engine = LocalEngine()
results = engine.run(graph)
print("results:", results)
results: {'result': 9}

Provenance for visualization

In interactive notebooks you can display the provenance graph inline

engine.recorder
provenance proc:3da8f46959b64f3d90aa54f4ca840c83 AddMultiply (3da8f46...) state=FINISHED proc:a374abd48d074ee7ae74ef27dc5da48e add (a374abd...) state=FINISHED proc:3da8f46959b64f3d90aa54f4ca840c83->proc:a374abd48d074ee7ae74ef27dc5da48e CALL proc:e07e7690f1274d45b85e829afbd89724 multiply (e07e769...) state=FINISHED proc:3da8f46959b64f3d90aa54f4ca840c83->proc:e07e7690f1274d45b85e829afbd89724 CALL data:3cb6d37a-4477-4f9c-b670-8134483c8ac3 int (3cb6d37...) proc:3da8f46959b64f3d90aa54f4ca840c83->data:3cb6d37a-4477-4f9c-b670-8134483c8ac3 RETURN result data:ad907fc3-1e7b-4422-98ea-7c3d832734d2 int (ad907fc...) proc:a374abd48d074ee7ae74ef27dc5da48e->data:ad907fc3-1e7b-4422-98ea-7c3d832734d2 CREATE result proc:e07e7690f1274d45b85e829afbd89724->data:3cb6d37a-4477-4f9c-b670-8134483c8ac3 CREATE result data:a7ef344c-47af-42a3-995d-ea270978ef2d int (a7ef344...) data:a7ef344c-47af-42a3-995d-ea270978ef2d->proc:3da8f46959b64f3d90aa54f4ca840c83 INPUT x data:a7ef344c-47af-42a3-995d-ea270978ef2d->proc:a374abd48d074ee7ae74ef27dc5da48e INPUT x data:bccfb1c4-b33c-45a1-a504-4c575a8c019f int (bccfb1c...) data:bccfb1c4-b33c-45a1-a504-4c575a8c019f->proc:3da8f46959b64f3d90aa54f4ca840c83 INPUT y data:bccfb1c4-b33c-45a1-a504-4c575a8c019f->proc:a374abd48d074ee7ae74ef27dc5da48e INPUT y data:df86f8ad-93e0-426b-94e0-09dc52512963 int (df86f8a...) data:df86f8ad-93e0-426b-94e0-09dc52512963->proc:3da8f46959b64f3d90aa54f4ca840c83 INPUT z data:df86f8ad-93e0-426b-94e0-09dc52512963->proc:e07e7690f1274d45b85e829afbd89724 INPUT y data:ad907fc3-1e7b-4422-98ea-7c3d832734d2->proc:e07e7690f1274d45b85e829afbd89724 INPUT x


Node graph programming

You can also create a graph programmatically. Three steps:

  • create a empty Graph

  • add nodes: add and multiply.

  • link the output of the add task to one of the x input of the multiply task.

from node_graph import Graph

ng = Graph("first_workflow")
ng.add_task(add, name="add", x=2, y=3)
ng.add_task(multiply, name="multiply", y=4)
ng.add_link(ng.tasks.add.outputs.result, ng.tasks.multiply.inputs.x)
ng


What’s Next

Total running time of the script: (0 minutes 0.478 seconds)

Gallery generated by Sphinx-Gallery