CLOUD + DATA
Cloud data systems.
Production data and distributed processing
See the system in motion.
Architecture walkthrough · 00:10
An abstracted event-to-warehouse path highlights queueing, failure handling and transformation responsibilities.
An illustrative composite of technologies from my client work; it does not represent a single deployment.
Open video ↗Application events and upstream datasets.
Background work fails quietly unless you design for it.
Production systems depend on background jobs and data pipelines that someone has to operate. When something fails, the failure has to be visible and recoverable.
Queues, workers and warehouses with explicit failure paths.
At Soluntech, I implemented SQS FIFO processing with dead-letter handling for a client platform with about 30,000 users, and built ingestion and transformation workflows with S3, Parquet, Glue, Redshift, dbt, BigQuery and Cloud Functions across client projects.
The unhappy path is part of the design.
Queue semantics, retries, failure capture and monitoring matter as much as the happy path. Client architecture stays private, so this case describes the patterns rather than a single deployment.
Apdex from 0.60 to 0.95, as a team.
After this work, that platform's Apdex rose from 0.60 to 0.95. It was a team result.
Professional work at Soluntech, with client details kept private. Code and internal links stay private. This page summarizes what those documents show.