DeepLearnToolbox是一个简单理解CNN过程的工具箱，可以在github下载。为了理解卷积神经网络的过程，我特此对CNN部分源码进行了注释。公式的计算可以由上一篇blog推导得出。
注意：代码中没有的subsampling进行设置参数，将subsampling层的参数w就设置为了0.25,而偏置参数b设置为0。卷积层计算过程为上一层所有feature map的卷积的结果和，后再加一个偏置，再取sigmoid函数。而subsampling的计算过程为上一层对应的2*2的feature map的像素值求和再取平均，没有加上偏置和取sigmoid。最后一层隐藏层为4*4大小的12个feature map，因此最后能得到192维的特征，全连接就是192*10(分类数目)。。
此外net中一些参数进行说明：
net.fv： 最后一层隐藏层的特征矩阵，采用的是全连接方式
net.o: 最后输出的结果，每一列为一个样本结果
net.od: 最后一层输出层所对应的残差
net.fvd: 最后一层隐藏层所对应的误差(全连接的方式)
test_example_CNN.m
%function test_example_CNNaddpath D:\DeepLearning\DeepLearnToolbox-master\data\addpath D:\DeepLearning\DeepLearnToolbox-master\CNN\addpath D:\DeepLearning\DeepLearnToolbox-master\util\load mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255; % 训练集变成60000张28*28的图片大小 28*28*60000,像素点归一化到[0,1]test_x = double(reshape(test_x',28,28,10000))/255;% 测试集 28*28*10000train_y = double(train_y'); %10*6000 每列代表一个标签 softmax回归模型test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = {%%% 设置各层feature maps个数及卷积模板大小等属性struct('type', 'i') %input layerstruct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layerstruct('type', 's', 'scale', 2) %sub sampling layerstruct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layerstruct('type', 's', 'scale', 2) %subsampling layer};opts.alpha = 0.01; %迭代下降的速率opts.batchsize = 50; %每次选择50个样本进行更新 随机梯度下降，每次只选用50个样本进行更新opts.numepochs = 50; %迭代次数cnn = cnnsetup(cnn, train_x, train_y);%对各层参数进行初始化 包括权重和偏置cnn = cnntrain(cnn, train_x, train_y, opts); %训练的过程，包括bp算法及迭代过程[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);% assert(er<0.12, 'Too big error');
cnnsetup.m
function net = cnnsetup(net, x, y)%assert(~isOctave() || compare_versions(OCTAVE_VERSION, '3.8.0', '>='), ['Octave 3.8.0 or greater is required for CNNs as there is a bug in convolution in previous versions. See ?39314. Your version is ' myOctaveVersion]);inputmaps = 1; %输入图片数量 输入feature maps数量mapsize = size(squeeze(x(:, :, 1))); % 图片的大小 squeeze 要不要都行28 28for l = 1 : numel(net.layers) % layerif strcmp(net.layers{l}.type, 's')mapsize = mapsize / net.layers{l}.scale; %% sumsampling的featuremap长宽都是上一层卷积层featuremap的一半assert(all(floor(mapsize)==mapsize), ['Layer ' num2str(l) ' size must be integer. Actual: ' num2str(mapsize)]);for j = 1 : inputmapsnet.layers{l}.b{j} = 0; % 将偏置初始化0, 权重weight，，这段代码subsampling层将weight设为1/4 而偏置参数设为0，故subsampling阶段无需参数endendif strcmp(net.layers{l}.type, 'c')mapsize = mapsize – net.layers{l}.kernelsize + 1; % 得到当前层feature map的大小fan_out = net.layers{l}.outputmaps * net.layers{l}.kernelsize ^ 2; % fan_out与fan_in都是用来初始化kernel的,不知道whyfor j = 1 : net.layers{l}.outputmaps % output map 当前层feature maps的个数fan_in = inputmaps * net.layers{l}.kernelsize ^ 2;for i = 1 : inputmaps % input map共享权值，故kernel参数个数为inputmaps*outputmaps个数,每一个都是5*5net.layers{l}.k{i}{j} = (rand(net.layers{l}.kernelsize) – 0.5) * 2 * sqrt(6 / (fan_in + fan_out)); %% 初始化每个feature map对应的kernel参数 -0.5 再乘2归一化到[-1,1],最终归一化到[-sqrt(6 / (fan_in + fan_out)),+sqrt(6 / (fan_in + fan_out))] why??endnet.layers{l}.b{j} = 0; % 初始话feture map对应的偏置参数 初始化为0endinputmaps = net.layers{l}.outputmaps; % 修改输入feature maps的个数以便下次使用endend% 'onum' is the number of labels, that's why it is calculated using size(y, 1). If you have 20 labels so the output of the network will be 20 neurons.% 'fvnum' is the number of output neurons at the last layer, the layer just before the output layer.% 'ffb' is the biases of the output neurons.% 'ffW' is the weights between the last layer and the output neurons. Note that the last layer is fully connected to the output layer, that's why the size of the weights is (onum * fvnum)fvnum = prod(mapsize) * inputmaps;% S4最后结点个数即为特征的个数 用作全连接 12*4*4=192维特征onum = size(y, 1);%最终分类的个数 10类net.ffb = zeros(onum, 1);%softmat回归的偏置参数个数net.ffW = (rand(onum, fvnum) – 0.5) * 2 * sqrt(6 / (onum + fvnum)); %% softmaxt回归的权值参数 为10*192个 全连接end
cnntrain.m
打破原先的记录，生活没有预赛，要想登上它的领奖台，
