Introduction to Feature Engineering
Chapter 1: The Role of Features in Machine Learning
Revisiting the Machine Learning Workflow
What Constitutes a Feature?
Impact of Feature Quality on Model Performance
Common Data Types and Their Challenges
Overview of Feature Engineering Tasks
Quiz for Chapter 1
Chapter 2: Handling Missing Data
Identifying Missing Values
Mechanisms of Missing Data (MCAR, MAR, MNAR)
Simple Imputation Strategies: Mean, Median, Mode
Creating Missing Value Indicators
Multivariate Imputation: KNN Imputer
Multivariate Imputation: Iterative Imputer
Comparing Imputation Methods
Hands-on Practical: Imputing Missing Data
Quiz for Chapter 2
Chapter 3: Encoding Categorical Features
Challenges with Categorical Data
Nominal vs. Ordinal Categories
One-Hot Encoding for Nominal Features
Ordinal Encoding for Ordered Features
Handling High Cardinality Features
Target Encoding (Mean Encoding)
Binary Encoding
Hashing Encoder
Comparing Encoding Methods
Hands-on Practical: Applying Encoding Techniques
Quiz for Chapter 3
Chapter 4: Feature Scaling and Transformation
The Need for Feature Scaling
Standardization (Z-score Scaling)
Normalization (Min-Max Scaling)
Scaling for Outliers
Log Transformation for Skewed Data
Box-Cox Transformation
Yeo-Johnson Transformation
Quantile Transformation
Choosing the Right Scaling/Transformation Method
Hands-on Practical: Scaling and Transforming Features
Quiz for Chapter 4
Chapter 5: Feature Creation
Motivation for Creating New Features
Interaction Features
Polynomial Features
Feature Creation from Date/Time Data
Binning Numerical Features
Domain-Specific Feature Engineering
Automated Feature Creation (Introduction)
Hands-on Practical: Engineering New Features
Quiz for Chapter 5
Chapter 6: Feature Selection
Importance of Feature Selection
Filter Methods Overview
Filter Methods: Variance Threshold
Filter Methods: Univariate Statistical Tests (ANOVA F-value, Chi-Squared)
Filter Methods: Correlation Analysis
Wrapper Methods Overview
Wrapper Methods: Recursive Feature Elimination (RFE)
Wrapper Methods: Sequential Feature Selection (SFS)
Embedded Methods Overview
Embedded Methods: Regularization (Lasso L1)
Embedded Methods: Tree-Based Feature Importance
Hands-on Practical: Selecting Features
Quiz for Chapter 6
Automated Feature Creation (Introduction)
Was this section helpful?
- Deep Feature Synthesis: Towards Automating Feature Engineering, Max Kanter, Kalyan Veeramachaneni, 2015 Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '15) (ACM) DOI: 10.1145/2783258.2783300 - Introduces Deep Feature Synthesis, a foundational algorithm for automated feature creation from relational data, and the basis for
Featuretools. - Featuretools Documentation, Alteryx, Inc., 2024 (Alteryx, Inc.) - Official documentation for the
FeaturetoolsPython library, providing practical guidance and examples for implementing Deep Feature Synthesis. - Genetic Programming for Feature Generation and Selection: A Survey, Sherif E. D. M. El-Din, E. H. H. Soliman, 2020 Applied Soft Computing, Vol. 92 (Elsevier) DOI: 10.1016/j.asoc.2020.106312 - A comprehensive survey on the application of Genetic Programming for automatically creating and selecting features, relevant to evolutionary approaches mentioned.
- Automated Machine Learning: Methods, Systems, Challenges, Frank Hutter, Lars Kotthoff, Joaquin Vanschoren, 2019 (Springer) DOI: 10.1007/978-3-030-05318-5 - This authoritative edited book provides a broad overview of Automated Machine Learning, including dedicated sections on automated feature engineering.