Skip to content

Causify DataFlow: A Framework For High-performance Machine Learning Stream Computing

TL;DR DataFlow is a computational framework for simulating causal models on time series data using a directed acyclic graph architecture enhanced with knowledge time semantics for temporal causal reasoning.

Paper Overview#

Title: Causify DataFlow: A Framework For High-performance Machine Learning Stream Computing

Authors: G.P. Saggese, P. Smith

Publication: arXiv preprint arXiv:2512.23977, 2025

Links: arXiv:2512.23977

  • DAG-based architecture: Computational framework for simulating causal models with time-series data using directed acyclic graph architecture enhanced with knowledge-time semantics for temporal causal reasoning
  • Temporal correctness: Guarantees proper temporal ordering through "tileability" property, ensuring causal models respect information availability constraints and avoid look-ahead bias in streaming contexts
  • Unified execution: Enables seamless transition from research to production through unified batch and streaming execution model, eliminating need for separate implementation paths

Abstract#

We present DataFlow, a computational framework for building, testing, and deploying high-performance machine learning systems on unbounded time-series data. Traditional data science workflows assume finite datasets and require substantial reimplementation when moving from batch prototypes to streaming production systems. This gap introduces causality violations, batch boundary artifacts, and poor reproducibility of real-time failures.

DataFlow resolves these issues through a unified execution model based on directed acyclic graphs (DAGs) with point-in-time idempotency: outputs at any time t depend only on a fixed-length context window preceding t. This guarantee ensures that models developed in batch mode execute identically in streaming production without code changes. The framework enforces strict causality by automatically tracking knowledge time across all transformations, eliminating future-peeking bugs.

DataFlow supports flexible tiling across temporal and feature dimensions, allowing the same model to operate at different frequencies and memory profiles via configuration alone. It integrates natively with the Python data science stack and provides fit/predict semantics for online learning, caching and incremental computation, and automatic parallelization through DAG-based scheduling. We demonstrate its effectiveness across domains including financial trading, IoT, fraud detection, and real-time analytics.