The BI Palace
Use Streaming Analytics to Identify and Visualise Fraudulent ATM Transactions in Real-Time
Storing and analysing large amounts of data is no longer the differentiator for successful companies. Today's world is about how fast decision makers receive the right information to make the right decision before it's too late. Streaming analytics is correctly referred to as Perishable Insights — insights that lose value if not acted upon immediately.
With the right technology set, a streaming analytics platform can be assembled quickly. This case study walks through detecting fraudulent ATM transactions in real-time using Confluent (Apache Kafka and KSQL) and MemSQL, then visualising results as they stream in.
Step 1: Build and Analyse Streams of Data
The foundation is a streaming platform built on Apache Kafka. Confluent provides the distribution and tooling. KSQL — the streaming SQL engine for Kafka — is used to define and analyse streams of ATM transaction data. Fraud patterns are expressed as KSQL queries that run continuously against live data, flagging suspicious transactions the moment they occur.
Confluent's blog post on ATM fraud detection with Kafka and KSQL details the stream-building process, and the open-source GESS (GitHub: rmoff/gess) project provides a ready-made transaction generator for testing.
Streaming analytics turns raw, fast-moving events into decisions that matter right now. Traditional batch pipelines store data first and analyse it later, which is fine for historical reporting but useless when a payment is already clearing or a sensor is already drifting. Streaming shifts the emphasis from how much data you hold to how quickly you can act on it. Records arrive from applications, devices and microservices, get processed in near real time, and trigger responses while the moment is still relevant. When insight is perishable, the value of acting early is far greater than the value of a perfectly complete report delivered too late.
Building a streaming platform is less about one tool and more about a coherent pipeline. Event sources publish messages to distributed log topics, where each record carries a key, a timestamp and a payload in a structured format. From there, stream processing jobs filter, join and aggregate, writing results into tables, data lakes or serving stores. SQL-like abstractions, user-defined functions and schema handling make this approachable without deep custom code. Because every step is event-driven, the same architecture can scale from a handful of sensors to thousands of concurrent producers without redesigning the whole solution around batch windows.
The final layer is where streaming earns its keep: turning processed events into something people can see and trust. Dashboards, alerting and visualisation tools surface anomalies as they emerge, so operators can investigate a suspicious pattern before it spreads. Persisted tables hold the curated record for later analysis, while fresh aggregates feed live views. Combining a streaming backbone with warehousing, cloud storage and crawling or import steps lets teams blend real-time signals with longer historical context. Done well, this gives organisations a platform where data is not merely stored, but actively working in the moment it arrives.
Step 2: Ingest Streams of Data into a Data Store in Real-Time
Once fraud patterns are identified in the stream, results must be persisted for immediate querying and visualisation. MemSQL (now SingleStore) serves as the real-time operational data store. MemSQL Pipelines ingest data directly from Kafka topics with minimal latency, making the enriched fraud-detection output available for dashboards within sub-seconds of a transaction occurring.
Step 3: Visualise Your Data in Real-Time
Real-time visualisation closes the loop. Dashboards connected to MemSQL surface fraudulent transaction alerts as they happen, giving fraud analysts and decision makers an immediate view of threats. This rapid feedback enables intervention before financial losses accumulate — the essence of perishable insights.
A demonstration video walks through the full pipeline, from Kafka stream to live dashboard, showing how quickly the platform surfaces anomalies.