参考:http://caffe.berkeleyvision.org/gathered/examples/mnist.html

原笔记在印象笔记中,复制过来格式就乱了,原笔记地址:

http://app.yinxiang.com/l/AARGaZ2rlLdEn7ikhvPcGA7hd7_EnB2OlIo/

caffe安装目录:home/user/caffe-master

cd caffe-master
download MNIST dataset
注意:新版caffe都需要从根目录上执行,不然可能会遇到这个错误:
./create_mnist.sh: 16: ./create_mnist.sh: build/examples/mnist/convert_mnist_data.bin: not found
./data/mnist/get_mnist.sh

生成mnist-train-leveldb/ 和 mnist-test-leveldb/文件夹,这里包含了LDB格式的数据集
./examples/mnist/create_mnist.sh

LeNet:

caffe中用的模型结构是非常著名的手写体识别模型LeNet,唯一的区别是把其中的sigmoid激活函数换成了ReLU,整个结构中包含两个convolution layer、两个pooling layer和两个fully connected layer。结构定义在$caffe-master/examples/mnist/lenet_train_test.prototxt中。
定义MNIST Network:

该结构定义在lenet_train_test.prototxt中,需要对google protobuf有一定了解并且看过Caffe中protobuf的定义,其定义在$caffe-master/src/caffe/proto/caffe.proto。
protobuf简介:
protobuf是google公司的一个开源项目,主要功能是把某种数据结构的信息以某种格式保存及传递,类似微软的XML,但是效率较高。目前提供C++、java和python的API。
protobuf简介:http://blog.163.com/jiang_tao_2010/blog/static/12112689020114305013458/
使用实例 :http://www.ibm.com/developerworks/cn/linux/l-cn-gpb/
编写LeNet的protobuf:
1、首先给定一个名字
name: "LeNet"
2、数据层
从生成的lmdb文件中读取MNIST数据,定义数据层如下:
layers { name: "mnist" type: DATA data_param { source: "mnist_train_lmdb" backend: LMDB batch_size: 64 scale: 0.00390625 //归一化到[0,1) } top: "data" top: "label"}
3、卷积层

layers { name: "conv1" type: CONVOLUTION blobs_lr: 1. // weight learning rate, same as learning rate blobs_lr: 2. // bias learning rate, twice as learning rate convolution_param { num_output: 20 // 20 channels kernelsize: 5 stride: 1 weight_filler { type: "xavier" // xavier algorithm, automatically initialize weight filler } bias_filler { type: "constant" // initialize bias filler as constant, default value 0 } } bottom: "data" top: "conv1" }

4、pooling层
layers { name: "pool1" type: POOLING pooling_param { kernel_size: 2 stride: 2 pool: MAX } bottom: "conv1" top: "pool1"}
5、全连接层1
layers { name: "ip1" type: INNER_PRODUCT // caffe calls it an innerprodect layer blobs_lr: 1. blobs_lr: 2. inner_product_param { num_output: 500 // 500 outputs weight_filler { type: "xavier" } bias_filler { type: "constant" } } bottom: "pool2" top: "ip1"}
6、ReLU层
layers { name: "relu1" type: RELU bottom: "ip1" top: "ip1"}layers {7、全连接层2
layers { name: "ip2" type: INNER_PRODUCT blobs_lr: 1. blobs_lr: 2. inner_product_param { num_output: 10 // 10 outputs weight_filler { type: "xavier" } bias_filler { type: "constant" } } bottom: "ip1" top: "ip2"}
8、loss层(softmax_loss layer)
这一层实现了softmax和多项式回归误差计算。
layers { name: "loss" type: SOFTMAX_LOSS bottom: "ip2" bottom: "label"}
9、additional notes
还可以在网络定义中添加规则,如下:
layers { // ...layer definition... include: { phase: TRAIN }}
另外还有一个accuracy layer,用来在测试阶段每100次迭代报告模型准确率。
更多的配置和规则可以在$caffe-master/src/caffe/proto/caffe.proto中查看。
可以在$caffe-master/examples/mnist/lenet_solver.prototxt中查看查看protobuf的解释:
# The train/test net protocol buffer definitionnet: "examples/mnist/lenet_train_test.prototxt"# test_iter specifies how many forward passes the test should carry out.# In the case of MNIST, we have test batch size 100 and 100 test iterations,# covering the full 10,000 testing images.test_iter: 100# Carry out testing every 500 training iterations.test_interval: 500# The base learning rate, momentum and the weight decay of the network.base_lr: 0.01momentum: 0.9weight_decay: 0.0005# The learning rate policylr_policy: "inv"gamma: 0.0001power: 0.75# Display every 100 iterationsdisplay: 100# The maximum number of iterationsmax_iter: 10000# snapshot intermediate resultssnapshot: 5000snapshot_prefix: "examples/mnist/lenet"# solver mode: CPU or GPUsolver_mode: GPU
接下来就是训练和测试LeNet模型:
cd $caffe-master./examples/mnist/train_lenet.sh
出现这样的结果:
I1203 net.cpp:66] Creating Layer conv1I1203 net.cpp:76] conv1 <- dataI1203 net.cpp:101] conv1 -> conv1I1203 net.cpp:116] Top shape: 20 24 24I1203 net.cpp:127] conv1 needs backward computation. // 显示连接信息和输出信息
I1203 net.cpp:142] Network initialization done.I1203 solver.cpp:36] Solver scaffolding done.I1203 solver.cpp:44] Solving LeNet
I1203 solver.cpp:204] Iteration 100, lr = 0.00992565 // learning rate printed every 100 iterationsI1203 solver.cpp:66] Iteration 100, loss = 0.26044 // loss function ...I1203 solver.cpp:84] Testing netI1203 solver.cpp:111] Test score #0: 0.9785 // test accuracy every 1000 iterations I1203 solver.cpp:111] Test score #1: 0.0606671 // test loss function
I1203 solver.cpp:84] Testing netI1203 solver.cpp:111] Test score #0: 0.9897I1203 solver.cpp:111] Test score #1: 0.0324599I1203 solver.cpp:126] Snapshotting to lenet_iter_10000I1203 solver.cpp:133] Snapshotting solver state to lenet_iter_10000.solverstateI1203 solver.cpp:78] Optimization Done.
最终训练完的模型存储为一个二进制的protobuf文件:
lenet_iter_10000
Caffe默认的训练是在GPU上的,但是如果想改为CPU上,只需要在lenet_solver.prototxt中修改一行:
# solver mode: CPU or GPUsolver_mode: CPU