Feature Engineering
Feature engineering is the process of using domain knowledge to select, transform, and create the input variables (features) that a machine learning model is trained on, in order to make patterns in the data easier for the model to learn.
Definition
Feature engineering is the process of using domain knowledge to select, transform, and create the input variables (features) that a machine learning model is trained on, in order to make patterns in the data easier for the model to learn.
Overview
Raw data — a database row, a log line, a sensor reading — is rarely in the ideal shape for a model to learn from directly. Feature engineering bridges that gap: it turns raw fields into informative inputs, such as deriving 'days since last purchase' from a timestamp, encoding a categorical variable like country as numeric indicators, or combining two columns into a ratio that captures a meaningful business signal a model would otherwise struggle to discover on its own. Classic feature engineering techniques include scaling numeric features to comparable ranges, encoding categorical variables (one-hot encoding, target encoding), extracting date and time components, creating interaction terms between features, and applying domain-specific transformations, like computing a rolling average for time-series data. Good feature engineering often has an outsized effect on model performance, particularly for classical machine learning algorithms like gradient-boosted trees, which don't automatically learn hierarchical feature representations the way deep learning models can. In deep learning, the need for manual feature engineering is reduced because neural networks learn their own internal feature representations directly from raw or lightly processed data — this is sometimes called 'representation learning,' and it's part of why techniques like embeddings have become central to modern NLP and recommendation systems. Still, thoughtful feature engineering remains valuable even for deep learning pipelines, especially for structured/tabular data. As teams scale, engineered features are frequently stored and served through a feature store so they can be reused consistently between training and production inference, avoiding training-serving skew.
Key Concepts
- Transforms raw data fields into informative inputs a model can learn from more easily
- Includes scaling, categorical encoding, date/time extraction, and interaction terms
- Has an outsized impact on performance for classical models like gradient-boosted trees
- Reduced in importance for deep learning, which can learn its own feature representations
- Closely tied to embeddings as a learned alternative to manual feature construction
- Requires domain knowledge to identify which derived signals matter for a given problem
- Engineered features are often centralized in a feature store for reuse across models
Use Cases
Frequently Asked Questions
From the Blog
How to A/B Test an LLM Feature With Real Users
Judge model changes on behaviour, not on offline scores. Pick one primary behavioural metric, define guardrails that stop the experiment automatically, randomise at the unit users actually experience, and hold the test long enough for the slow signals — retention and follow-up rate — to arrive.
Read More AI & TechnologyBuilding a golden evaluation set for an LLM feature
Your golden set makes every later decision measurable. Learn to mine real inputs and past failures, size and stratify it, and keep it honest as the product changes.
Read More AI & TechnologyStaged rollout for an LLM feature: flags, cohorts and rollback
You cannot fully validate an LLM feature pre-launch, so the rollout is the test. Design flags, cohorts, guardrail metrics and rollback triggers before you ship.
Read More Data ScienceFeature Engineering: Turning Data Into Signal
Feature engineering turns raw columns into inputs a model can actually learn from. Learn the core techniques that often matter more than the algorithm itself.
Read More