学习深度学习,不可避免的要选择一个适合于自己的框架,目前深度学习的主流框架有caffe,Theano,Torch7等,而由Yangqing Jia等人开发并维护的caffe框架,由于其代码简洁,可读性强,较高的运行效率以及CPU/GPU切换简单,并且拥有较大的user group,所以越来越受到学习者的重视.
caffe框架基于c++语言编写,并且具有licensed BSD,开放源码,具有matlab和python接口,关于caffe的详细介绍参考论文"Caffe:convolutional architecture for fast feature embedding",网址.本文先介绍caffe框架的网络定义.GitHub中有源码下载,源代码中的文件类型有.cpp,.prototxt,.sh,.m.,.py等,caffe框架的网络定义都放在.prototxt文件中.所以我们先分析此类文件.
1.Vision layers(头文件位置: ./include/caffe/vision_layers.hpp)
(i)卷积层
例子:
layers { name: "conv1" type: CONVOLUTION #层类型 bottom: "data" top: "conv1" blobs_lr: 1# learning rate multiplier for the filters blobs_lr: 2# learning rate multiplier for the biases weight_decay: 1# weight decay multiplier for the filters weight_decay: 0# weight decay multiplier for the biases convolution_param {num_output: 96# learn 96 filterskernel_size: 11 # each filter is 11x11stride: 4# step 4 pixels between each filter applicationweight_filler {type: "gaussian" # initialize the filters from a Gaussianstd: 0.01# distribution with stdev 0.01 (default mean: 0)}bias_filler {type: "constant" # initialize the biases to zero (0)value: 0} }}参数解释如下:
top和bottom:输出和输入
convolution_param:
必须:
num_output (c_o): 滤波器数目kernel_size (or kernel_h and kernel_w): 滤波器尺寸
建议:
weight_filler [default type: ‘constant’ value: 0]:滤波器权重
可选:
(ii)池化层
例子:
layers { name: "pool1" type: POOLING bottom: "conv1" top: "pool1" pooling_param {pool: MAXkernel_size: 3 # pool over a 3×3 regionstride: 2# step two pixels (in the bottom blob) between pooling regions }}参数解释如下
top和bottom:输出和输入
pooling_param:
必须:
kernel_size (or kernel_h and kernel_w): 指定滤波器尺寸
可选:
pool [default MAX]: 池化方法. 可选项为 MAX, AVE, or STOCHASTICpad (or pad_h and pad_w) [default 0]: 指定输入需要padding的尺寸stride (or stride_h and stride_w) [default 1]:指定滤波器滑动步长
(iii)局部响应正则化(Local Response Normalization)
参数解释如下
层类型:LRN
lrn_param:
可选:
(事实上,LRN操作是对图像区域执行了侧抑制,对每个输入值都乘以
).
2.Loss layers
神经网络能够执行运算的动力就是误差层,forward pass运算得到loss,backward pass运算利用loss计算gradient.
(i)softmax
层类型:SOFTMAX_LOSS
(ii)Sum-of-Squares / Euclidean
层类型: EUCLIDEAN_LOSS
(iii)Hinge / Margin
例子:
# L1 Normlayers { name: "loss" type: HINGE_LOSS #层类型 bottom: "pred" bottom: "label"}# L2 Normlayers { name: "loss" type: HINGE_LOSS bottom: "pred" bottom: "label" top: "loss" hinge_loss_param {norm: L2 }} 可选参数
norm [default L1]: 范数类型. 可选范数 L1, L2
(IV)Sigmoid Cross-Entropy
层类型:SIGMOID_CROSS_ENTROPY_LOSS
(V)Infogain
层类型:INFOGAIN_LOSS
(VI)Accuracy and Top-k
3.Activation / Neuron Layers
(i)ReLU / Rectified-Linear and Leaky-ReLU
例子:
layers { name: "relu1" type: RELU #层类型 bottom: "conv1" top: "conv1"}参数解释如下
relu_param:
可选
negative_slope [default 0]: 指定输入为负时的输出
(ReLU是常用的激活函数,因为收敛速度快,不易饱和.给定输入x,如果x>0,则输出x,否则输出negative_slope*x,如果未设定negative_slope的值,则等效于标准的relu操作,支持in-place运算,意味着bottom和top相同时可以避免内存的消耗.)
(ii)Sigmoid
例子
layers { name: "encode1neuron" bottom: "encode1" top: "encode1neuron" type: SIGMOID #层类型}(iii)TanH / Hyperbolic Tangent
例子
layers { name: "layer" bottom: "in" top: "out" type: TANH #层类型}(IV)Absolute Value
例子
layers { name: "layer" bottom: "in" top: "out" type: ABSVAL #层类型}(V)Power梦想不分高低贵贱，只要你心中有梦，乐观充实地过好每一天。
