Building Node Templates: Developing Runners

This chapter shows how to turn business logic into an executable Runner. We will build a minimal example called generic-sum-csv which receives a CSV file, sums every numeric column, and returns a JSON object of the form:


{
  "speed": 120.7,
  "altitude": 3412,
  "temperature": null
}

Columns that cannot be coerced into numbers are reported as null.

Runtime contract

Every runner is a container that:

Installs the heat-runtime Python package.
Subclasses BaseProcessor and implements two async hooks: validate() and process().
Registers itself with HeatRuntime.register() and starts the event loop with HeatRuntime().init().
Remains idempotent—process a single task using only the provided inputs + config.

The typical event loop for the HEAT runtime consists of immediately polling for tasks from HEAT Core, if a task is available it is claimed which means no other runner can claim the task. At this point your logic begins to execute, and the results are posted back to the Core API. You always interact with HEAT Core via the runtime and do not need to make any REST requests.

Implementation

src/sum_processor.py


from io import BytesIO
from typing import Dict, Any
 
import pandas as pd
from heat_runtime import BaseProcessor, HeatRuntime
 
class SumProcessor(BaseProcessor):
    """Aggregates all numeric columns of a CSV (supplied in‑memory) into a single JSON row."""
 
    TEMPLATE_NAME = "generic-sum-csv"  # must match the node template
 
    async def validate(self, inputs: Dict[str, Any], config: Dict[str, Any]):
        await super().validate(inputs, config)
        if "file" not in inputs:
            raise ValueError("expected 'file' artefact in inputs")
 
    async def process(self, inputs: Dict[str, Any], config: Dict[str, Any]):
        """Sum every numeric column; non‑numeric become null."""
        raw = inputs["file"]  # bytes or file‑like object provided by HEAT
 
        # normalise to a file‑like buffer that pandas can read
        if isinstance(raw, (bytes, bytearray)):
            buffer = BytesIO(raw)
        else:
            buffer = raw  # assume already file‑like (e.g. StreamingBody)
 
        df = pd.read_csv(buffer)
 
        result = {}
        for col in df.columns:
            if pd.api.types.is_numeric_dtype(df[col]):
                result[col] = float(df[col].sum())
            else:
                result[col] = None
 
        return result
 
if __name__ == "__main__":
    HeatRuntime.register(SumProcessor)
    HeatRuntime().init()

Why async? heat-runtime handles polling and heartbeat in the event loop; your CPU‑bound work can stay synchronous inside process().

Dockerfile

runner/Dockerfile


FROM python:3.12-slim
 
WORKDIR /app
 
# Install runtime and dependencies
RUN pip install heat-runtime pandas==2.*
 
# Copy source
COPY src/ ./src/
 
# Entrypoint
CMD ["python", "src/sum_processor.py"]

Push this image to your registry and reference it in runnerTypes.containerImage.

Updating the manifest

Add a new runner type and node template:

my-feature.heat


{
  "runnerTypes": [
    {
      "name": "sum-csv",
      "containerImage": "registry.example.com/sum-csv:1.0.0",
      "cpuLimit": "1",
      "memoryLimit": "1Gi",
      "description": "Sums numeric columns of a CSV.",
      "enabled": true,
      "featureInternalName": "myFeature"
    }
  ],
  "nodeTemplates": [
    {
      "name": "generic-sum-csv",
      "type": "Processing",
      "acceptsMultipleInputs": false,
      "requiresConfiguration": false,
      "configurationSchema": {},
      "featureInternalName": "myFeature",
      "supportedRunners": ["sum-csv"],
      "hasNonJsonArtifacts": false
    }
  ]
}

Upload the updated manifest via the Cluster Manager ▸ Features screen or POST /api/system/upload_manifest. You can then reference it in Session Templates and the scheduler will provision runner pods matching generic-sum-csv tasks.

Test locally (optional)

You can exercise the processor outside HEAT for quick feedback:


python src/sum_processor.py

TBD.

When no environment flags exist that indicate that the runtime is running inside the HEAT cluster, the runtime will run in local dev mode.

Next steps

Building a Manifest – field‑level deep‑dive and JSON‑Schema validation.
Deploying Changes – CI/CD tips for pushing images and manifests.
Using in a Session Template – wiring generic‑sum‑csv into a DAG.