Streaming ingestion
Transactions score the moment they arrive, not on the next batch run.
All Projects
Backend · Real-time fraud detection
Getting a fraud alert to an analyst in under two seconds
A transaction-monitoring pipeline rebuilt around streaming ingestion, dropping alert latency from roughly an hour to under two seconds.
Start a Project- Client
- Vanta Line
- Industry
- Card & transaction fraud prevention
- Year
- 2024
- Duration
- 12 weeks
- Role
- Backend engineering + ML integration
- Team
- 4 engineers, 1 ML specialist
- Platform
- Backend
The challenge
Transactions landed in an hourly batch job, so an analyst reviewing a fraud alert could be looking at activity from before a compromised card had already been used again — sometimes several times. The detection model itself was accurate; it just never got a chance to act while the transaction was still reversible.
Approach & solution
Approach
We treated the batch window itself as the vulnerability, not the model's accuracy. The build was sequenced around collapsing that window first — streaming ingestion and scoring — with model tuning as a second pass once alerts could actually reach an analyst in time to matter.
Solution
A streaming ingestion layer that scores transactions as they arrive, an anomaly-detection service tuned against a year of confirmed fraud cases, and an analyst dashboard that shows current exposure instead of a stale batch pull.
What shipped
The capabilities behind the screens.
Streaming ingestion
Transactions score the moment they arrive, not on the next batch run.
Sub-two-second alerts
An analyst sees a flagged transaction while it's still reversible.
Tuned anomaly detection
Scored against a year of confirmed fraud cases, not a generic model.
Live exposure dashboard
Current risk, not a stale pull from the last batch window.
Prioritized alert queue
Highest-confidence flags reach an analyst first.
Case investigation view
An analyst traces a flag back through the account's recent activity.
Tech Stack & Architecture
Services, APIs, and business logic running the system end to end.
4 tools · 1 primary
Datastores and pipelines chosen for this engagement's consistency and throughput needs.
3 tools · 1 primary
Deployment, orchestration, and delivery tooling running the platform in production.
3 tools · 1 primary
System architecture
Technical challenges
Scoring at streaming volume without a false-positive flood
Moving from hourly batches to per-transaction scoring meant the model saw far noisier, less-aggregated input. Feature computation was reworked around rolling windows rather than daily aggregates, so noise didn't get mistaken for anomaly.
Holding a sub-two-second budget end to end
Ingestion, scoring and delivery all had to fit inside the window before a transaction could complete. Scoring was moved onto the ingestion path itself rather than a downstream consumer, cutting one network hop out of the budget.
Retraining the model without opening an alert gap
A model swap can't pause fraud detection while it deploys. Two scoring services run behind a shared queue, so a new model version warms up against live traffic before it takes over from the one it replaces.
How it was built
The stages this engagement ran
01
Brief & Strategy
1 week
Traced a sample of confirmed fraud cases through the batch pipeline to size the actual exposure window.
02
Strategy & Planning
1 week
Streaming architecture and the sub-two-second budget agreed before any model tuning began.
03
UX & UI Design
1 week
The analyst dashboard was wireframed against real flagged-transaction data, not mockups.
04
Development
5 weeks
Ingestion pipeline and scoring service built together, so the latency budget was tested from week one.
05
QA & Optimization
3 weeks
Load-tested against peak card-present volume, with the model swap-over rehearsed under live traffic.
06
Deployment & Support
1 week
Rolled out desk by desk, with the batch pipeline kept live as a fallback for the first week.
Results
What changed, in the numbers.
“An alert used to arrive after the damage was already done. Now the analyst sees it before the transaction clears.”
Grace Lindqvist · CTO · Vanta Line
Selected screens
Vanta Line, in the product
Related work
Works you may also like.
Technical briefingNot a sales desk