Sessions

A Session in HEAT represents a single instance of the data workflow defined by a Session Template. While often used to capture and process data from a simulation run, any source or event requiring end-to-end processing can drive a new Session. Each Session spawns its own Directed Acyclic Graph (DAG) of nodes, mirroring the template’s structure, so multiple Sessions can run in parallel without interfering with one another.

How Sessions Are Created

• Via Ingests: Most commonly, data is posted to an Ingest, which triggers a new Session if none is currently active for that upload.
• Via API: In some workflows, you may directly create a Session through the HEAT API and then upload or associate data with it. tracks

Relationship to Templates

Every Session must reference a single Session Template. Once the Session is instantiated:

• Fresh DAG: The nodes defined in the template are copied into the Session. This ensures each Session operates independently, even if you modify the underlying template later.
• Execution Flow: As data moves through the DAG, each node performs its designated task (e.g., parsing, analytics, or transformation). The Session node completion, errors, and outputs in real time.

Parallel Processing

HEAT can manage multiple Sessions concurrently:

• Independent Pipelines: One Session’s node failures or delays will not block or affect another Session.
• Scalable Architecture: Under the hood, HEAT’s Kubernetes-based infrastructure allows you to provision resources that match your concurrency needs, whether you’re running a few Sessions or hundreds at once.

Monitoring & Status

• Session Lifecycle: A Session typically progresses from a pending or “created” state to active (while processing), then completes or fails once all nodes have finished.
• Dashboard & Logs: You can view overall progress, errors, and outputs in the HEAT dashboard or gather deeper insights through the Cluster Manager logs.
• Metadata & Tagging: Sessions may include metadata—like date/time, environment tags, or user-defined fields - helpful for search or filtering later.

Common Use Cases

1. Simulation Runs
   Each training scenario or simulator run can spawn a Session, capturing control inputs, sensor data, and instructor feedback in a structured pipeline.
2. Batch Data Processing
   For scenarios outside simulation, a Session might process large CSV or JSON files, run ML models, and deliver outputs for dashboard visualization.
3. Incremental Data Ingestion
   A Session can be augmented over time if multiple Ingests are configured to add partial data logs or results to the same workflow.

Best Practices

• Align Sessions with Logical Events
  Create a Session when you have a cohesive block of data (e.g., a single simulation run or a complete dataset). This keeps your DAG tasks relevant and manageable.
• Version Your Templates
  If your workflow changes, keep historical sessions and use the updated template for new sessions. This ensures consistency and comparability over time.
• Monitor Parallelism
  While HEAT supports running many Sessions concurrently, be aware of resource constraints. Plan or scale your cluster if you expect large volumes of simultaneous data processing.

Next Steps

• Read about Session Templates to understand the blueprint each Session relies on.
• Explore Ingests to see how new Sessions are triggered automatically via data uploads.
• Check the Cluster Manager for techniques to monitor and troubleshoot ongoing Sessions.