不是模型不行，是你没做好特征工程（附完整步骤）

来源DeepHub IMBA 本文约1800字建议阅读5分钟本文介绍了特征工程全流程含数据处理、特征构造与选择。Feature engineering 是机器学习 pipeline 里最关键的一环。算法再好如果输入数据噪声大、不一致或者缺乏有意义的特征模型表现都不会很好。这篇文章用 Pandas 和 Scikit-learn把一条完整的 feature engineering pipeline 做个完整的介绍。什么是 Feature Engineering把原始数据转换成有意义的输入变量特征让机器学习模型表现更好——这就是 feature engineering。Step 1 — 探索性数据分析EDA动手做特征之前先把数据看明白。import pandas as pd import numpy as np np.random.seed(42) df pd.DataFrame({ Age: [25, 30, np.nan, 40, 35, 120, 28], Salary: [50000, 60000, 55000, 80000, np.nan, 1000000, 62000], Gender: [Male, Female, Female, np.nan, Male, Male, Female], City: [NY, LA, NY, SF, np.nan, LA, SF], Experience: [1, 3, 2, 10, 7, 25, 4], Date: pd.date_range(start2024-01-01, periods7), Target: [0, 1, 0, 1, 0, 1, 0] }) print(df) print(df.head()) print(df.info()) print(df.describe())检查缺失值print(df.isnull().sum())查看分布import matplotlib.pyplot as plt df.hist(figsize(12, 10)) plt.show()Step 2 — 缺失值填补缺失值会拉低模型准确率。均值 / 中位数填补from sklearn.impute import SimpleImputer imputer SimpleImputer(strategymedian) df[Age] imputer.fit_transform(df[[Age]])众数填补cat_imputer SimpleImputer(strategymost_frequent) df[City] cat_imputer.fit_transform(df[[City]])KNN 填补from sklearn.impute import KNNImputer knn_imputer KNNImputer(n_neighbors5) numeric_cols df.select_dtypes(include[int64, float64]) df[numeric_cols.columns] knn_imputer.fit_transform(numeric_cols)Step 3 — 类别编码模型只认数字不认文本。Label Encodingfrom sklearn.preprocessing import LabelEncoder encoder LabelEncoder() df[Gender] encoder.fit_transform(df[Gender])One-Hot Encodingdf pd.get_dummies(df, columns[City], drop_firstTrue)使用 Scikit-learn OneHotEncoderfrom sklearn.preprocessing import OneHotEncoder ohe OneHotEncoder(handle_unknownignore)Step 4 — 异常值检测与处理异常值会扭曲模型的学习过程。用 IQR 检测异常值Q1 df[Salary].quantile(0.25) Q3 df[Salary].quantile(0.75) IQR Q3 - Q1 lower Q1 - 1.5 * IQR upper Q3 1.5 * IQR outliers df[(df[Salary] lower) | (df[Salary] upper)] print(outliers)移除异常值df df[(df[Salary] lower) (df[Salary] upper)]Winsorization from scipy.stats.mstats import winsorize df[Salary] winsorize(df[Salary], limits[0.05, 0.05])Step 5 — 特征缩放与归一化不同量纲的特征会影响模型表现。StandardScalerfrom sklearn.preprocessing import StandardScaler scaler StandardScaler() df[[Age, Salary]] scaler.fit_transform(df[[Age, Salary]])MinMaxScalerfrom sklearn.preprocessing import MinMaxScaler minmax MinMaxScaler() df[[Age, Salary]] minmax.fit_transform(df[[Age, Salary]])RobustScaler数据里有异常值时使用。from sklearn.preprocessing import RobustScaler robust RobustScaler() df[[Age, Salary]] robust.fit_transform(df[[Age, Salary]])Step 6 — 特征构造与变换构造出有意义的特征往往是准确率拉升最明显的一步。日期特征抽取df[Date] pd.to_datetime(df[Date]) df[Year] df[Date].dt.year df[Month] df[Date].dt.month df[Day] df[Date].dt.day多项式特征from sklearn.preprocessing import PolynomialFeatures poly PolynomialFeatures(degree2) poly_features poly.fit_transform(df[[Age, Experience]])数值变量分箱df[Age_Group] pd.cut( df[Age], bins[0, 18, 35, 60, 100], labels[Teen, Young, Adult, Senior] )Step 7 — 特征选择挑出真正重要的特征可以减少过拟合也能让模型跑得更快。基于相关系数的选择import seaborn as sns corr df.corr(numeric_onlyTrue) sns.heatmap(corr, annotTrue)SelectKBestfrom sklearn.feature_selection import SelectKBest, f_classif X df.drop(Target, axis1) y df[Target] selector SelectKBest(score_funcf_classif, k5) X_new selector.fit_transform(X, y)递归特征消除RFEfrom sklearn.feature_selection import RFE from sklearn.linear_model import LogisticRegression model LogisticRegression() rfe RFE(model, n_features_to_select5) X_rfe rfe.fit_transform(X, y)Pipeline 把预处理自动化也能降低数据泄露的风险。完整 Pipeline 示例from sklearn.pipeline import Pipeline from sklearn.compose import ColumnTransformer from sklearn.preprocessing import StandardScaler, OneHotEncoder from sklearn.impute import SimpleImputer from sklearn.linear_model import LogisticRegression from sklearn.model_selection import train_test_split X df.drop(Target, axis1) y df[Target] numeric_features [Age, Salary, Experience] categorical_features [Gender, City] numeric_transformer Pipeline(steps[ (imputer, SimpleImputer(strategymedian)), (scaler, StandardScaler()) ]) categorical_transformer Pipeline(steps[ (imputer, SimpleImputer(strategymost_frequent)), (onehot, OneHotEncoder(handle_unknownignore)) ]) preprocessor ColumnTransformer( transformers[ (num, numeric_transformer, numeric_features), (cat, categorical_transformer, categorical_features) ] ) model_pipeline Pipeline(steps[ (preprocessor, preprocessor), (classifier, LogisticRegression(max_iter1000)) ]) X_train, X_test, y_train, y_test train_test_split( X, y, test_size0.2, random_state42 ) model_pipeline.fit(X_train, y_train) print(Model trained successfully)总结Feature engineering 是机器学习项目能否成立的基石。干净、变换过、有意义的特征往往胜过用劣质数据训练的复杂算法。上面把这些步骤都做扎实模型的准确率和稳健性都会上一个台阶。在真实的机器学习项目里feature engineering 往往比挑哪个模型更决定胜负。特征做得好预测自然好。by Dhivakar编辑于腾凯校对李嘉林关于我们数据派THU作为数据科学类公众号背靠清华大学大数据研究中心分享前沿数据科学与大数据技术创新研究动态、持续传播数据科学知识努力建设数据人才聚集平台、打造中国大数据最强集团军。新浪微博数据派THU微信视频号数据派THU今日头条数据派THU