Feature Engineering definition
Feature engineering is the process of turning raw data into input variables, called features, that help a machine learning model learn useful patterns. It includes cleaning values, combining fields, encoding categories, building time-based aggregates and removing misleading columns. Good features often improve model accuracy more than switching to a more complex algorithm.
Why feature engineering matters
A model only sees the columns you give it. Raw data is rarely shaped for learning: a transaction log holds timestamps and amounts, not "how often does this customer buy" or "is spending falling". Feature engineering builds those signals explicitly. On tabular business data, where most enterprise machine learning still lives, thoughtful features frequently matter more than the choice between two strong algorithms.
Worked example: a churn model fed raw login events learns little. Add features such as logins in the last 7 and 30 days, the ratio between the two, days since the last support ticket and the change in monthly spend versus the previous quarter. The model can now see a customer who is quietly drifting away, and the features also make the model easier to explain to the account team.
Common feature engineering techniques
- Encoding categories: one-hot, ordinal or target encoding for fields like city or plan type.
- Scaling: standardizing numeric ranges for distance-based and linear models.
- Aggregations: counts, sums and averages over windows such as 7, 30 or 90 days.
- Date and time features: day of week, holiday flags and time since an event.
- Ratios and interactions: debt to income, price per square foot.
- Text and image features: TF-IDF vectors or embeddings from pretrained models.
- Missing values: imputation plus a flag that records the value was missing.
- Binning: grouping continuous values such as age into bands when the relationship is stepwise.
Feature engineering vs feature selection
Feature engineering creates new inputs. Feature selection removes inputs that are irrelevant, redundant or risky. Selection methods include correlation checks, permutation importance, SHAP values and L1 regularization, which pushes the weights of weak features to zero. Fewer features usually mean faster training, lower risk of overfitting and models that are easier to explain to auditors and business users.
Selection should happen inside the cross-validation loop, not on the full dataset beforehand. Choosing features with knowledge of the test data leaks information and inflates scores, a mistake that is easy to make in notebooks and only shows up once the model meets live traffic.
Feature engineering in deep learning and production
Deep learning learns its own features from raw pixels, audio and text, which removes most manual work for unstructured data. Tabular problems still benefit from engineered features, and even deep models gain from good inputs such as clean timestamps and sensible units. Domain knowledge from finance, logistics or medicine is often the source of the most valuable features.
In production, features must be computed exactly the same way during training and serving. When they are not, the result is training-serving skew, a quiet cause of poor live accuracy. Feature stores such as Feast, Tecton and the Databricks Feature Store define each feature once and serve it to both paths. Nexzem defines features as versioned code for this reason, with tests that compare offline and online values.