Cloud cost optimisation is a continuous engineering discipline. Data engineering workloads — Spark jobs, warehouse queries, S3 storage, network egress — can generate surprisingly large bills if not designed with cost awareness from the start. The largest bills come from three sources: over-provisioned compute (clusters too large or running too long), unoptimised storage (data in the wrong class or never deleted), and unexpected data transfer charges (moving data between regions or cloud providers). Understanding the cost model before provisioning is the most effective control measure.
The AWS cost model follows a consistent pattern: compute is priced per instance-hour, storage per GB-month, and data transfer per GB. Spot Instances offer up to 90% discount over On-Demand by using spare capacity that AWS can reclaim with two minutes notice — appropriate for fault-tolerant batch jobs that can checkpoint and restart. Reserved Instances and Savings Plans offer 40–60% discount for a 1-year or 3-year commitment — appropriate for baseline compute that runs continuously. On-Demand is appropriate for unpredictable or short-lived workloads where commitment risk outweighs cost savings.