Cost Optimisation: Spot Instances and Model Compression

ML is expensive, training burns GPU-hours and serving runs hardware around the clock, and at scale these costs become a dominant operational concern that can make or break whether a model is viable in production. Cost optimisation is the discipline of delivering the same ML value for far less money, and it works on two fronts: cheaper infrastructure, above all spot instances that offer spare cloud capacity at steep discounts in exchange for the risk of interruption, and cheaper models, through compression techniques like quantisation, pruning, and distillation that shrink a model so it runs faster on smaller, cheaper hardware. The problem this solves is concrete: a serving fleet sized for peak on expensive on-demand GPUs, or a needlessly large model, can cost many times what an optimised setup does for identical results. Cost optimisation reclaims that waste. The central tension throughout is trade-offs, spot capacity is cheap but can vanish mid-job, and compression shrinks cost but can shave accuracy, so the skill is capturing the large savings while keeping the reliability and quality losses negligible, turning ML from a budget hazard into an efficient, sustainable operation.