Security: Model Theft and Adversarial Attacks

A deployed model is an asset and an attack surface: it is expensive intellectual property that competitors may try to steal, and a decision-maker that adversaries may try to fool, and unlike ordinary software whose vulnerabilities are bugs in code, ML systems have vulnerabilities in their learned behaviour. ML security is the discipline of defending models against these threats, and it centres on two broad classes: model theft, where an attacker reconstructs or extracts a model by querying it, and adversarial attacks, where carefully crafted inputs cause the model to make confident wrong predictions. The problem is distinctive because the threats exploit how models learn and generalise, not coding mistakes: a model can be perfectly implemented yet still be extractable through its API or foolable by inputs imperceptibly different from normal ones. The stakes are high, a stolen model leaks costly IP and the data patterns it encodes, and a fooled model can be made to misclassify a fraudulent transaction as legitimate or a stop sign as a speed limit. Understanding ML security matters because as models guard money, safety, and access, they become targets, and defences must address vulnerabilities that traditional application security was never designed to see.