Skip to Content
This documentation is provided with the HEAT environment and is relevant for this HEAT instance only.
Core ConceptsNode Templates

Node Templates

Node Templates define the type of operation a node can perform within HEAT’s Directed Acyclic Graph (DAG) workflows. They act as reusable blueprints for different kinds of data-processing tasks, such as input, transform, processing, or aggregation. The actual configuration, connections, and scheduling of these operations are determined when you place them in a Session Template, where the Session Template Node references a Node Template.

What Is a Node Template?

A Node Template describes:

  • Operation Type: For instance, whether it’s an input node that receives data from an Ingest, a transform node that reshapes data, a processing node running algorithms, or an aggregation node combining multiple data streams.
  • Expected Inputs/Outputs: Defines what kind of data (e.g., JSON, CSV, or custom object) the node expects and what it will produce.
  • Configuration Schema: Declares any parameters or settings required by the node. This could include a destination Data Source, runtime options, or file format preferences.
  • Multi-Input Support: Indicates if the node can handle multiple upstream inputs at once (e.g., merging two CSV outputs) or requires a single input only.
  • Feature/Capability: References the underlying feature or runner (e.g., a custom transformation runner or machine-learning algorithm) that will execute this operation.

You can think of Node Templates as “kinds of operations.” Each node in a Session Template points to a Node Template, along with specifics like how it connects to other nodes and any parameter overrides.

Relationship to Runners

While a Node Template defines what the node is supposed to do conceptually, a runner is what actually executes the logic:

  • Runners are containerized processes deployed by HEAT to handle one or more Node Template types.
  • A Node Template typically references a runner name or ID, but it does not itself dictate how the task is performed internally.
  • This separation of concerns allows you to update or replace the underlying runner without changing every Session Template node.

Defining Node Templates

Like other core HEAT entities (e.g., Projects, Session Templates), Node Templates can be defined or updated:

  • Via the HEAT API: Programmatically create or modify Node Templates for integration into your CI/CD pipelines.

Using Node Templates in Session Templates

When building a Session Template:

  1. Choose a Node Template: For each step in your data pipeline, select the Node Template that aligns with the operation needed (e.g., an “XML-to-JSON Transform Node”).
  2. Set Configuration: Override or fill in any parameters required by that node (e.g., specifying the Data Source, file formats, or API endpoints).
  3. Link to Dependencies: Indicate which nodes feed data into this node, and which node or nodes it outputs to.

Example Use Cases

  • Input Node
    An input node is typically processed by our Ingests and the payload gets stored in a Data Source for downstream processing.
  • Transform Node
    A Node Template called “transform-csv-biometrics” could read raw CSV input, extract particular columns, and convert to a consistent format for another downstream node.
  • Processing Node
    A Node Template called “process-biometrics” might involve a runner that takes JSON input and applies a machine-learning model to generate predictions or additional metadata.
  • Aggregation Node
    “Combine Metrics” could accept outputs from multiple transform or processing nodes, merging them into a single dataset or summary.

Best Practices

  • Keep Node Templates Small & Focused
    Each template should handle a single logical step. This approach reduces complexity and makes troubleshooting easier.
  • Document Expected Inputs/Outputs
    Make sure the Node Template’s schema is well-described, so other team members or external integrators know how to pass data between nodes.
  • Ensure Idempotence
    Node Templates that rely on external state or have side effects can lead to unpredictable results. Strive for operations that produce the same output given the same inputs.

Summary

  • Node Templates define what an operation is supposed to do (type, expected inputs/outputs, config schema).
  • Runners handle how that operation is executed at runtime.
  • By combining Node Templates in a Session Template, you build out an end-to-end data workflow.
  • They can be created or updated programmatically via the HEAT API and stored for reuse across multiple workflows.

Next Steps

Node InstancesProjects