Feature Engineering
Feature engineering is the process of using domain knowledge to select, transform, and create the input variables (features) that a machine learning model is trained on, in order to make patterns in the data easier for the model to learn.
Definition
Feature engineering is the process of using domain knowledge to select, transform, and create the input variables (features) that a machine learning model is trained on, in order to make patterns in the data easier for the model to learn.
Overview
Raw data — a database row, a log line, a sensor reading — is rarely in the ideal shape for a model to learn from directly. Feature engineering bridges that gap: it turns raw fields into informative inputs, such as deriving 'days since last purchase' from a timestamp, encoding a categorical variable like country as numeric indicators, or combining two columns into a ratio that captures a meaningful business signal a model would otherwise struggle to discover on its own. Classic feature engineering techniques include scaling numeric features to comparable ranges, encoding categorical variables (one-hot encoding, target encoding), extracting date and time components, creating interaction terms between features, and applying domain-specific transformations, like computing a rolling average for time-series data. Good feature engineering often has an outsized effect on model performance, particularly for classical machine learning algorithms like gradient-boosted trees, which don't automatically learn hierarchical feature representations the way deep learning models can. In deep learning, the need for manual feature engineering is reduced because neural networks learn their own internal feature representations directly from raw or lightly processed data — this is sometimes called 'representation learning,' and it's part of why techniques like embeddings have become central to modern NLP and recommendation systems. Still, thoughtful feature engineering remains valuable even for deep learning pipelines, especially for structured/tabular data. As teams scale, engineered features are frequently stored and served through a feature store so they can be reused consistently between training and production inference, avoiding training-serving skew.
Key Concepts
- Transforms raw data fields into informative inputs a model can learn from more easily
- Includes scaling, categorical encoding, date/time extraction, and interaction terms
- Has an outsized impact on performance for classical models like gradient-boosted trees
- Reduced in importance for deep learning, which can learn its own feature representations
- Closely tied to embeddings as a learned alternative to manual feature construction
- Requires domain knowledge to identify which derived signals matter for a given problem
- Engineered features are often centralized in a feature store for reuse across models