Introduction to Feature Engineering
Chapter 1: The Role of Features in Machine Learning
Revisiting the Machine Learning Workflow
What Constitutes a Feature?
Impact of Feature Quality on Model Performance
Common Data Types and Their Challenges
Overview of Feature Engineering Tasks
Quiz for Chapter 1
Chapter 2: Handling Missing Data
Identifying Missing Values
Mechanisms of Missing Data (MCAR, MAR, MNAR)
Simple Imputation Strategies: Mean, Median, Mode
Creating Missing Value Indicators
Multivariate Imputation: KNN Imputer
Multivariate Imputation: Iterative Imputer
Comparing Imputation Methods
Hands-on Practical: Imputing Missing Data
Quiz for Chapter 2
Chapter 3: Encoding Categorical Features
Challenges with Categorical Data
Nominal vs. Ordinal Categories
One-Hot Encoding for Nominal Features
Ordinal Encoding for Ordered Features
Handling High Cardinality Features
Target Encoding (Mean Encoding)
Binary Encoding
Hashing Encoder
Comparing Encoding Methods
Hands-on Practical: Applying Encoding Techniques
Quiz for Chapter 3
Chapter 4: Feature Scaling and Transformation
The Need for Feature Scaling
Standardization (Z-score Scaling)
Normalization (Min-Max Scaling)
Scaling for Outliers
Log Transformation for Skewed Data
Box-Cox Transformation
Yeo-Johnson Transformation
Quantile Transformation
Choosing the Right Scaling/Transformation Method
Hands-on Practical: Scaling and Transforming Features
Quiz for Chapter 4
Chapter 5: Feature Creation
Motivation for Creating New Features
Interaction Features
Polynomial Features
Feature Creation from Date/Time Data
Binning Numerical Features
Domain-Specific Feature Engineering
Automated Feature Creation (Introduction)
Hands-on Practical: Engineering New Features
Quiz for Chapter 5
Chapter 6: Feature Selection
Importance of Feature Selection
Filter Methods Overview
Filter Methods: Variance Threshold
Filter Methods: Univariate Statistical Tests (ANOVA F-value, Chi-Squared)
Filter Methods: Correlation Analysis
Wrapper Methods Overview
Wrapper Methods: Recursive Feature Elimination (RFE)
Wrapper Methods: Sequential Feature Selection (SFS)
Embedded Methods Overview
Embedded Methods: Regularization (Lasso L1)
Embedded Methods: Tree-Based Feature Importance
Hands-on Practical: Selecting Features
Quiz for Chapter 6
Impact of Feature Quality on Model Performance
Was this section helpful?
- Feature Engineering and Selection: A Practical Approach for Predictive Models, Max Kuhn, Kjell Johnson, 2019 (Chapman and Hall/CRC) DOI: 10.1201/9781315108230 - This comprehensive book systematically covers the creation and selection of features, directly addressing their impact on model accuracy, interpretability, and efficiency. (2nd edition)
- Machine Learning Yearning, Andrew Ng, 2017 (DeepLearning.AI) - This strategic guide emphasizes data quality and feature engineering for improving model performance and generalization in AI applications.
- Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, Aurélien Géron, 2022 (O'Reilly Media) - A widely-used practical textbook with chapters dedicated to data preprocessing and feature engineering, illustrating their fundamental role in achieving good model performance and generalization.
- A Survey on Feature Engineering Methods for Machine Learning, Navneet Singh, Jagdeep Singh Sohal Punia and Monika Partap Singh Bhatia, 2020 International Journal of Advanced Computer Science and Applications, Vol. 11 (Science and Information Organization) DOI: 10.14569/IJACSA.2020.110231 - This survey paper provides an overview of various feature engineering techniques, opening with a discussion on the importance of feature quality for model effectiveness.