scikit-learn 用户指南 - 监督学习 (User Guide: Supervised Learning)¶

来源: https://scikit-learn.org/stable/user_guide.html
翻译时间: 2026-06-01
状态: 关键章节翻译
原始作者: scikit-learn developers

目录 (Table of Contents)¶

监督学习概述
线性模型
支持向量机
决策树
集成方法

1. 监督学习概述 (Supervised Learning Overview)¶

英文原文:

Supervised learning consists of learning a function from labeled training data. The training data consist of pairs of input objects (typically a vector) and desired output values. The output of the function can be a continuous value (regression) or a class label (classification).

中文翻译:

监督学习包括从有标签的训练数据中学习一个函数。训练数据由输入对象（通常是向量）和期望的输出值对组成。函数的输出可以是连续值（回归）或类别标签（分类）。

关键术语: | 英文 | 中文 | 说明 | |------|------|------| | Supervised learning | 监督学习 | 从有标签数据学习 | | Labeled training data | 有标签训练数据 | 包含输入 - 输出对 | | Regression | 回归 | 预测连续值 | | Classification | 分类 | 预测离散类别 |

2. 线性模型 (Linear Models)¶

2.1 普通最小二乘法 (Ordinary Least Squares)¶

英文原文:

LinearRegression fits a linear model with coefficients w = (w1, ..., wp) to minimize the residual sum of squares between the observed targets in the dataset, and the targets predicted by the linear approximation.

中文翻译:

LinearRegression 拟合一个具有系数 w = (w1, ..., wp) 的线性模型，以最小化数据集中观测到的目标值与线性近似预测的目标值之间的残差平方和。

数学公式: $$ \min_{w} ||X w - y||_2^2 $$

代码示例:

from sklearn.linear_model import LinearRegression
import numpy as np

# 训练数据 (Training data)
X = np.array([[1, 1], [1, 2], [2, 2], [2, 3]])
y = np.dot(X, np.array([1, 2])) + 3

# 拟合模型 (Fit model)
reg = LinearRegression().fit(X, y)

# 预测 (Prediction)
print(f"Coefficients (系数): {reg.coef_}")  # [1. 2.]
print(f"Intercept (截距): {reg.intercept_}")  # 3.0
print(f"Prediction (预测): {reg.predict([[3, 5]])}")  # [16.]

2.2 岭回归 (Ridge Regression)¶

英文原文:

Ridge regression addresses some of the problems of Ordinary Least Squares by imposing a penalty on the size of the coefficients. The ridge coefficients minimize a penalized residual sum of squares.

中文翻译:

岭回归通过对系数大小施加惩罚来解决普通最小二乘法的一些问题。岭回归系数最小化带惩罚的残差平方和。

数学公式: $$ \min_{w} ||X w - y||_2^2 + \alpha ||w||_2^2 $$

其中 $\alpha$ 是正则化参数 (regularization parameter)

关键概念对比:

方法	优点	缺点	适用场景
Ordinary Least Squares	简单、快速	对多重共线性敏感	特征独立时
Ridge Regression	处理共线性、防止过拟合	不产生稀疏解	特征相关时

代码示例:

from sklearn.linear_model import Ridge
import numpy as np

# 数据 (Data)
X = np.random.rand(100, 10)
y = np.random.rand(100)

# 不同 alpha 值比较 (Compare different alpha values)
alphas = [0.01, 0.1, 1.0, 10.0]
for alpha in alphas:
    ridge = Ridge(alpha=alpha)
    ridge.fit(X, y)
    print(f"Alpha={alpha:.2f}, Coefficients norm={np.linalg.norm(ridge.coef_):.4f}")

2.3 Lasso 回归 (Lasso Regression)¶

英文原文:

The Lasso is a linear model that estimates sparse coefficients. It is useful in some contexts due to its tendency to prefer some solutions with fewer parameter values, effectively reducing the number of features upon which the given solution is dependent.

中文翻译:

Lasso 是一种估计稀疏系数的线性模型。在某些情况下它很有用，因为它倾向于选择参数值较少的解，有效地减少给定解所依赖的特征数量。

数学公式: $$ \min_{w} \frac{1}{2n} ||X w - y||_2^2 + \alpha ||w||_1 $$

L1 vs L2 正则化:

正则化	公式	效果
L1 (Lasso)	$\alpha \sum	w_i
L2 (Ridge)	$\alpha \sum w_i^2$	系数缩小，不稀疏

代码示例:

from sklearn.linear_model import Lasso
from sklearn.datasets import make_regression

# 生成数据 (Generate data)
X, y = make_regression(n_samples=100, n_features=20, 
                       n_informative=5, noise=0.1)

# Lasso 拟合 (Lasso fit)
lasso = Lasso(alpha=0.1)
lasso.fit(X, y)

# 查看稀疏性 (Check sparsity)
non_zero_coef = np.sum(lasso.coef_ != 0)
print(f"非零系数数量 (Non-zero coefficients): {non_zero_coef}/20")
# 输出：约 5-7 个（自动特征选择）

3. 支持向量机 (Support Vector Machines)¶

3.1 分类 (Classification)¶

英文原文:

Support Vector Machines (SVMs) are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis. Given a set of training examples, each marked as belonging to one or the other of two categories, an SVM training algorithm builds a model that assigns new examples to one category or the other.

中文翻译:

支持向量机 (SVM) 是带有相关学习算法的监督学习模型，用于分类和回归分析的数据分析。给定一组训练样本，每个样本被标记为属于两个类别之一，SVM 训练算法构建一个将新样本分配到一个类别或另一个类别的模型。

关键概念: - Support Vector (支持向量): 距离决策边界最近的样本点 - Margin (间隔): 支持向量到决策边界的距离 - Kernel (核函数): 将低维空间映射到高维空间

代码示例:

from sklearn import svm
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split

# 生成数据 (Generate data)
X, y = make_classification(n_samples=100, n_features=2, 
                           n_informative=2, n_redundant=0, random_state=42)

# 划分数据集 (Split dataset)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

# SVM 分类器 (SVM Classifier)
clf = svm.SVC(kernel='rbf', C=1.0, gamma='scale')
clf.fit(X_train, y_train)

# 评估 (Evaluate)
score = clf.score(X_test, y_test)
print(f"准确率 (Accuracy): {score:.2f}")

# 获取支持向量 (Get support vectors)
print(f"支持向量数量 (Number of support vectors): {len(clf.support_)}")

3.2 核函数 (Kernel Functions)¶

英文原文:

A kernel function takes the low-dimensional input space and transforms it into a higher dimensional space. In other words, a kernel function can be viewed as a similarity function between two data points.

中文翻译:

核函数将低维输入空间转换为高维空间。换句话说，核函数可以看作是两个数据点之间的相似度函数。

常用核函数:

核函数	公式	参数	适用场景
Linear (线性)	$K(x, x') = x^T x'$	无	线性可分数据
Polynomial (多项式)	$K(x, x') = (γ x^T x' + r)^d$	degree, gamma, coef0	图像处理
RBF (径向基)	$K(x, x') = \exp(-γ		x-x'
Sigmoid	$K(x, x') = \tanh(γ x^T x' + r)$	gamma, coef0	神经网络替代

代码示例 - 核函数对比:

from sklearn import svm
import numpy as np
import matplotlib.pyplot as plt

# 创建非线性可分数据 (Create non-linearly separable data)
np.random.seed(42)
n_samples = 100

# 圆形数据 (Circular data)
X = np.random.randn(n_samples, 2)
y = (X[:, 0]**2 + X[:, 1]**2 < 1).astype(int)

# 不同核函数比较 (Compare different kernels)
kernels = ['linear', 'poly', 'rbf', 'sigmoid']

fig, axes = plt.subplots(2, 2, figsize=(12, 10))
axes = axes.flatten()

for idx, kernel in enumerate(kernels):
    clf = svm.SVC(kernel=kernel, gamma=20)
    clf.fit(X, y)
    score = clf.score(X, y)

    axes[idx].scatter(X[:, 0], X[:, 1], c=y, cmap='coolwarm', alpha=0.6)
    axes[idx].set_title(f'{kernel} kernel\nAccuracy: {score:.2f}')
    axes[idx].axis('equal')

plt.tight_layout()
plt.show()

4. 决策树 (Decision Trees)¶

4.1 分类树 (Classification Tree)¶

英文原文:

Decision Trees (DTs) are a non-parametric supervised learning method used for classification and regression. The goal is to create a model that predicts the value of a target variable by learning simple decision rules inferred from the data features.

中文翻译:

决策树 (DT) 是一种用于分类和回归的非参数监督学习方法。目标是创建一个模型，通过学习从数据特征推断出的简单决策规则来预测目标变量的值。

关键概念: - Root Node (根节点): 树的起始点，包含所有样本 - Internal Node (内部节点): 测试某个特征的条件 - Leaf Node (叶节点): 最终的预测结果 - Splitting (分裂): 根据特征将数据分成子集

分裂标准: | 标准 | 公式 | 适用 | |------|------|------| | Gini Impurity | $1 - \sum p_i^2$ | 分类，默认 | | Entropy | $-\sum p_i \log_2 p_i$ | 分类，信息增益 | | Variance | $\frac{1}{N}\sum(y_i - \bar{y})^2$ | 回归 |

代码示例:

from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_iris
from sklearn.tree import plot_tree
import matplotlib.pyplot as plt

# 加载数据 (Load data)
iris = load_iris()
X, y = iris.data, iris.target

# 训练决策树 (Train decision tree)
clf = DecisionTreeClassifier(max_depth=3, random_state=42)
clf.fit(X, y)

# 可视化 (Visualize)
plt.figure(figsize=(12, 8))
plot_tree(clf, 
          feature_names=iris.feature_names,
          class_names=iris.target_names,
          filled=True)
plt.title('Decision Tree Visualization (决策树可视化)')
plt.show()

# 特征重要性 (Feature importance)
print("\n特征重要性 (Feature Importance):")
for name, importance in zip(iris.feature_names, clf.feature_importances_):
    print(f"  {name}: {importance:.4f}")

5. 集成方法 (Ensemble Methods)¶

5.1 随机森林 (Random Forest)¶

英文原文:

A random forest is a meta estimator that fits a number of decision tree classifiers on various sub-samples of the dataset and uses averaging to improve the predictive accuracy and control over-fitting.

中文翻译:

随机森林是一种元估计器，它在数据集的各种子样本上拟合多个决策树分类器，并使用平均来提高预测准确性和控制过拟合。

核心思想: - Bagging (Bootstrap Aggregating): 有放回抽样 - Feature Randomness: 每次分裂随机选择特征 - Majority Voting: 多数表决（分类）或平均（回归）

代码示例:

from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
from sklearn.model_selection import cross_val_score

# 生成数据 (Generate data)
X, y = make_classification(n_samples=1000, n_features=20,
                           n_informative=15, n_redundant=5, random_state=42)

# 随机森林 (Random Forest)
rf = RandomForestClassifier(n_estimators=100,  # 树的数量
                           max_depth=10,       # 最大深度
                           min_samples_split=5, # 最小分裂样本数
                           random_state=42)

# 交叉验证 (Cross-validation)
scores = cross_val_score(rf, X, y, cv=5)
print(f"交叉验证准确率 (CV Accuracy): {scores.mean():.3f} (+/- {scores.std():.3f})")

# 训练并查看特征重要性 (Train and check feature importance)
rf.fit(X, y)
importances = rf.feature_importances_

# 前 10 个重要特征 (Top 10 important features)
top_10_idx = importances.argsort()[-10:][::-1]
print("\n前 10 个重要特征 (Top 10 Features):")
for idx in top_10_idx:
    print(f"  Feature {idx}: {importances[idx]:.4f}")

5.2 梯度提升树 (Gradient Boosting Trees)¶

英文原文:

Gradient Boosting builds an additive model in a forward stage-wise fashion; it allows for the optimization of arbitrary differentiable loss functions. In each stage a regression tree is fit on the negative gradient of the given loss function.

中文翻译:

梯度提升以前向分步的方式构建加法模型；它允许优化任意可微的损失函数。在每个阶段，回归树被拟合到给定损失函数的负梯度上。

与随机森林对比:

特性	随机森林	梯度提升
树的构建	并行	串行
树的关系	独立	后树纠正前树错误
过拟合风险	较低	较高（需调参）
训练速度	快	慢
预测性能	好	通常更好

代码示例:

from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import GridSearchCV

# 梯度提升分类器 (Gradient Boosting Classifier)
gbt = GradientBoostingClassifier(n_estimators=100,
                                 learning_rate=0.1,
                                 max_depth=3,
                                 random_state=42)

# 网格搜索调参 (Grid Search for hyperparameter tuning)
param_grid = {
    'n_estimators': [50, 100, 200],
    'learning_rate': [0.01, 0.1, 0.2],
    'max_depth': [3, 5, 7]
}

grid_search = GridSearchCV(gbt, param_grid, cv=5, scoring='accuracy')
grid_search.fit(X, y)

print(f"最佳参数 (Best params): {grid_search.best_params_}")
print(f"最佳准确率 (Best accuracy): {grid_search.best_score_:.3f}")

📊 算法选择指南 (Algorithm Selection Guide)¶

英文原文:

Choosing the right estimator for your problem can be challenging. The following flowchart provides a high-level guidance.

中文翻译:

为你的问题选择合适的估计器可能具有挑战性。以下流程图提供高级指导。

graph TD
    A[数据量 > 50k 样本？] -->|是 | B[线性模型或线性核 SVM]
    A -->|否 | C{数据类型？}

    C -->|数值特征 | D{是否线性可分？}
    C -->|类别特征 | E[决策树/随机森林]

    D -->|是 | F[线性模型/Lasso]
    D -->|否 | G[RBF 核 SVM/梯度提升]

    B --> H[需要特征选择？]
    H -->|是 | I[Lasso/ElasticNet]
    H -->|否 | J[Ridge/线性回归]

    G --> K{需要概率输出？}
    K -->|是 | L[随机森林/梯度提升]
    K -->|否 | M[SVM]

    style A fill:#e3f2fd
    style C fill:#e3f2fd
    style D fill:#e3f2fd
    style H fill:#e3f2fd
    style K fill:#e3f2fd

🔑 关键要点总结 (Key Takeaways)¶

线性模型 (Linear Models)
简单快速，适合基线
Ridge 防止过拟合，Lasso 进行特征选择
支持向量机 (SVM)
高维空间有效
核函数处理非线性
参数调优关键
决策树 (Decision Trees)
易于解释
易过拟合，需剪枝
特征重要性直观
集成方法 (Ensemble Methods)
随机森林：稳定、快速
梯度提升：准确、需调参
实践中首选

翻译完成时间: 2026-06-01
翻译者: AI Assistant
审校状态: 待人工审校

**scikit-learn 用户指南 - 中英对照版** [返回顶部](#目录-table-of-contents)

核函数	公式	参数	适用场景
Linear (线性)	\(K(x, x') = x^T x'\)	无	线性可分数据
Polynomial (多项式)	\(K(x, x') = (γ x^T x' + r)^d\)	degree, gamma, coef0	图像处理
RBF (径向基)	$K(x, x') = \exp(-γ		x-x'
Sigmoid	\(K(x, x') = \tanh(γ x^T x' + r)\)	gamma, coef0	神经网络替代