卷积神经网络(LeNet)

1. LeNet 的历史地位

我们已经学完了神经网络中的基本组件。现在，我们可以尝试不再展平 Fashion-MNIST 数据集中的图像，而保留空间结构特征，进行模型训练和服装分类。

2. 网络架构设计

2.1 架构组成

2.2 PyTorch 实现

LeNet 的网络架构用 PyTorch 框架实现如下（去掉了最后一个全连接层的高斯激活函数）：

import torch
from torch import nn, Tensor

class LeNet(nn.Module):
    def __init__(self, num_classes=10):
        super(LeNet, self).__init__()
        
        # 特征提取器
        self.features = nn.Sequential(
            # 第一层：卷积 + Sigmoid + 池化
            nn.Conv2d(in_channels=1, out_channels=6, kernel_size=5, padding=2),
            nn.Sigmoid(),
            nn.AvgPool2d(kernel_size=2, stride=2),
            
            # 第二层：卷积 + Sigmoid + 池化
            nn.Conv2d(in_channels=6, out_channels=16, kernel_size=5),
            nn.Sigmoid(),
            nn.AvgPool2d(kernel_size=2, stride=2),
        )
        
        # 分类器
        self.classifier = nn.Sequential(
            nn.Flatten(),
            nn.Linear(in_features=16 * 5 * 5, out_features=120),
            nn.Sigmoid(),
            nn.Linear(in_features=120, out_features=84),
            nn.Sigmoid(),
            nn.Linear(in_features=84, out_features=num_classes)
        )

    def forward(self, x: Tensor) -> Tensor:
        x = self.features(x)
        x = self.classifier(x)
        return x

2.3 网络结构分析

为了方便理解各层的输出维度变化，可以用以下方式测试：

def test_lenet_architecture():
    """测试LeNet各层的输出维度"""
    model = LeNet(num_classes=10)
    test_input = torch.randn(1, 1, 28, 28)
    
    print(f"输入形状: {test_input.shape}")
    print("\n=== 特征提取部分 ===")
    
    x = test_input
    for i, layer in enumerate(model.features):
        x = layer(x)
        print(f"{type(layer).__name__} 输出形状: {tuple(x.shape)}")
    
    print("\n=== 分类器部分 ===")
    for i, layer in enumerate(model.classifier):
        x = layer(x)
        print(f"{type(layer).__name__} 输出形状: {tuple(x.shape)}")

# 运行测试
test_lenet_architecture()

输入形状: torch.Size([1, 1, 28, 28])

=== 特征提取部分 ===
Conv2d 输出形状: (1, 6, 28, 28)
Sigmoid 输出形状: (1, 6, 28, 28)
AvgPool2d 输出形状: (1, 6, 14, 14)
Conv2d 输出形状: (1, 16, 10, 10)
Sigmoid 输出形状: (1, 16, 10, 10)
AvgPool2d 输出形状: (1, 16, 5, 5)

=== 分类器部分 ===
Flatten 输出形状: (1, 400)
Linear 输出形状: (1, 120)
Sigmoid 输出形状: (1, 120)
Linear 输出形状: (1, 84)
Sigmoid 输出形状: (1, 84)
Linear 输出形状: (1, 10)

3. 模型训练与评估

3.1 训练环境配置

使用 TensorBoard 执行可视化，启用 TensorBoard 后端：

tensorboard --logdir ./runs

然后在浏览器中访问 http://localhost:6006/ 查看训练过程。

3.2 训练代码实现

继续使用 training_tools.py 中的工具来训练和评估LeNet模型：

import torch
from torch import nn, optim
from training_tools import fashionMNIST_loader, Trainer

def train_lenet():
    """训练LeNet模型"""
    # 超参数设置
    BATCH_SIZE = 256
    EPOCHS = 15  # 减少训练轮数用于演示
    LEARNING_RATE = 0.3
    
    # 创建模型和数据加载器
    model = LeNet(num_classes=10)
    train_loader, test_loader = fashionMNIST_loader(BATCH_SIZE)
    
    # 损失函数和优化器
    criterion = nn.CrossEntropyLoss()
    optimizer = optim.SGD(model.parameters(), lr=LEARNING_RATE)
    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    
    print(f"使用设备: {device}")
    print(f"模型参数数量: {sum(p.numel() for p in model.parameters()):,}")
    
    # 开始训练
    with Trainer(model, train_loader, test_loader, criterion, optimizer, device) as trainer:
        trainer.train(EPOCHS)

if __name__ == '__main__':
    train_lenet()

3.3 训练结果分析

使用设备: cuda
模型参数数量: 61,706

Epoch [  1/15] Train Loss: 2.3105, Train Acc: 10.09%, Test Loss: 2.3149, Test Acc: 10.00%
Epoch [  2/15] Train Loss: 2.2823, Train Acc: 13.79%, Test Loss: 2.1649, Test Acc: 24.98%
Epoch [  3/15] Train Loss: 1.5717, Train Acc: 42.73%, Test Loss: 1.2325, Test Acc: 50.35%
Epoch [  4/15] Train Loss: 1.1198, Train Acc: 56.12%, Test Loss: 1.0465, Test Acc: 58.59%
Epoch [  5/15] Train Loss: 0.9639, Train Acc: 62.85%, Test Loss: 0.9262, Test Acc: 66.43%
Epoch [  6/15] Train Loss: 0.8491, Train Acc: 68.36%, Test Loss: 0.8217, Test Acc: 70.62%
Epoch [  7/15] Train Loss: 0.7814, Train Acc: 70.95%, Test Loss: 0.8526, Test Acc: 67.93%
Epoch [  8/15] Train Loss: 0.7304, Train Acc: 72.75%, Test Loss: 0.7113, Test Acc: 73.10%
Epoch [  9/15] Train Loss: 0.6906, Train Acc: 73.90%, Test Loss: 0.7325, Test Acc: 72.51%
Epoch [ 10/15] Train Loss: 0.6545, Train Acc: 74.81%, Test Loss: 0.6730, Test Acc: 73.23%
...
Epoch [ 15/15] Train Loss: 0.6025, Train Acc: 76.48%, Test Loss: 0.6369, Test Acc: 76.09%

# 继续训练会看到最终收敛效果：
Epoch [100/200] Train Loss: 0.2545, Train Acc: 90.52%, Test Loss: 0.3077, Test Acc: 88.67%
Epoch [200/200] Train Loss: 0.1785, Train Acc: 93.33%, Test Loss: 0.2551, Test Acc: 91.06%

训练特点分析

4. 历史回顾与发展

即使是首个通过反向传播成功训练的 LeNet 提出，也并没有使卷积神经网络成为图像识别分类领域的主导。因为，LeNet 在更大、更真实的数据集上取得的突破并不比传统方法显著。

4.1 技术发展的两条路径

4.2 技术发展的瓶颈

4.3 ImageNet的推动作用