Proto commits in OAID/Caffe-HRT

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

2017-08-26

add support acl batch normal,direct conv, local connect, concat layers

The documentation is generated from this commit.

2017-03-22

Clarify batch norm parameter documentation.

2017-02-16

Merge pull request #5210 from ftokarev/patches Obsolete reference to `bool solver` in caffe.proto

2017-01-21

Update a comment in caffe.proto

2017-01-06

Switched multi-GPU to NCCL

2016-11-27

fix many typos by using codespell

2016-11-17

sigmoid cross-entropy loss: normalize loss by different schemes sig-ce loss handles all the same normalizations as the softmax loss; refer to #3296 for more detail. this preserves the default normalization for sig-ce loss: batch size.

2016-06-28

add default value for rms_decay

2016-06-01

Add RecurrentLayer: an abstract superclass for other recurrent layer types

2016-05-05

add parameter layer for learning any bottom

2016-04-27

Fix an error in the example of ReshapeParameter. * this small mistake may confuse newer.

2016-04-05

Net: setting `propagate_down: true` forces backprop

2016-03-05

Extend Crop to N-D, changed CropParameter.

Crop: fixes, tests and negative axis indexing.

add CropLayer: crop blob to another blob's dimensions with offsets configure offset(s) through proto definition.

2016-02-25

deprecate input fields and upgrade automagically

add InputLayer for Net input Create an input layer to replace oddball Net `input` fields.

2016-02-20

tranpose parameter added to IP layer to support tied weights in an autoencoder. Arguments to matrix multiplication function are conditioned on this parameter, no actual transposing takes place. test ip gradient computation with transpose on

2016-01-23

Separation and generalization of ChannelwiseAffineLayer into BiasLayer and ScaleLayer. The behavior of ChannelwiseAffineLayer can be reproduced by a ScaleLayer with `scale_param { bias_term: true }`. BiasLayer and ScaleLayer each take 1 or 2 bottoms, with the output having the same shape as the first. The second input -- either another bottom or a learned parameter -- will have its axes (virtually) broadcast and tiled to have the same shape as the first, after which elementwise addition (Bias) or multiplication (Scale) is performed.

2016-01-22

Add ChannelwiseAffine for batch norm

Merge pull request #3388 from mohomran/exponential_linear_units Exponential Linear Units

2015-12-28

add short description of dilation to caffe.proto

add support for 2D dilated convolution

2015-12-04

ELU layer with basic tests

2015-11-22

Better normalization options for SoftmaxWithLoss layer.

2015-10-23

Merge pull request #3229 from cdoersch/batchnorm2 Yet another batch normalization PR

2015-10-22

Cleanup batch norm layer, include global stats computation

2015-10-17

Change solver type to string and provide solver registry

2015-10-16

Merge pull request #3160 from shelhamer/cudnnV3 Basic cuDNN v3 support

Initial cuDNN v3 support

2015-09-25

Add argmax_param axis

2015-09-19

Im2col and Convolution layers support N spatial axes

caffe.proto: generalize ConvolutionParameter to N spatial axes

2015-08-26

Add TileLayer

2015-08-25

Merge pull request #2032 from jeffdonahue/embed-layer Embed layer for lookup table of one hot encodings

2015-08-14

Adam solver This commit implements the Adam solver by Kingma et. al for CPU and GPU. All solver parameters are defined in the caffe.proto. This also adds an example for the MNIST dataset.

2015-08-13

Merge pull request #2903 from ronghanghu/multi_gpu Multi-GPU Data Parallelism

2015-08-12

Data Layers Parallel for Multi-GPU Allow data layers (and also PythonLayer when used as data layer) to be shared among worker solver's training net, and also test net for future-proof if one wants to do Multi-GPU testing. Data layers are locked during forward to ensure sequential forward.

2015-08-10

Implement AdaDelta; add test cases; add mnist examples

2015-08-09

Add DataReader for parallel training with one DB session - Make sure each solver accesses a different subset of the data - Sequential reading of DB for performance - Prefetch a configurable amount of data to host memory - Distribute data to solvers in round-robin way for determinism

Implement RMSProp Solver Implement RMSProp solver and cleaned up to adjust to new solver interface that uses accumulated gradients and refactored regularization.

2015-08-08

Add EmbedLayer for inner products with sparse input (one-hot vectors), with unit tests

temporarily switch the snapshot_format default back to BINARYPROTO out of anticipation for user issues due to issue #2885, which causes Caffe to crash when it attempts to snapshot nets with duplicate layer names

2015-08-07

Snapshot model weights/solver state to HDF5 files. Summary of changes: - HDF5 helper functions were moved into a separate file util/hdf5.cpp - hdf5_save_nd_dataset now saves n-d blobs, can save diffs instead of data - Minor fix for memory leak in HDF5 functions (delete instead of delete[]) - Extra methods have been added to both Net/Solver enabling snapshotting and restoring from HDF5 files - snapshot_format was added to SolverParameters, with possible values HDF5 or BINARYPROTO (default HDF5) - kMaxBlobAxes was reduced to 32 to match the limitations of HDF5

add double_data, double_diff to BlobProto for weights/snapshots saved when using Dtype == double

PythonLayer takes parameters by string

2015-08-04

ImageData layer default batch size of 1, and check for zero batch size

2015-06-03

Add ReductionLayer to reduce any number of "tail" axes to a scalar value Currently implements operations SUM, MEAN, ASUM (sum of absolute values), and SUMSQ (sum of squares)

FlattenLayer gets a FlattenParameter with an axis, end_axis

Add LogLayer

2015-05-30

Merge pull request #1977 from shelhamer/accum-grad Decouple the computational batch size and minibatch size by accumulating gradients

Merge pull request #2410 from sguada/datum_transform Datum transform

2015-05-27

zero-init param diffs and accumulate gradients (With layers whose backward accumulates gradients), this effectively decouples the computational batch from the SGD minibatch. Each iteration accumulates gradients over iter_size batches, then parameters are updated.

2015-05-26

Merge pull request #1946 from nickcarlevaris/msra_init Add MSRAFiller, an Xavier-like filler designed for use with ReLUs

Added MSRAFiller, an Xavier-like filler designed for use with ReLUs ...instead of tanh. Based on paper: He et al, "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification," 2015. - add VarianceNorm option to FillerParameters which allows one to normalize by fan_in, fan_out or their average. - update XavierFiller to use the VarianceNorm option (default behavior unchanged). - add tests for MSRAFiller and XavierFiller.

2015-05-18

clean up redundant message comments

2015-05-16

Merge pull request #2466 from ducha-aiki/mvn-less Remove unnecessary variance computation from backward in MVN layer

2015-05-15

Merge pull request #2095 from mtamburrano/skip_propagate_down_param Added param skip_propagate_down to LayerParameter

Remove unnecessary variance computation from backward in MVN layer

Added "propagate_down" param to LayerParameter

Add ReshapeParameter axis and num_axes to reshape only a particular span of the input shape

ReshapeLayer fixups for ND blobs

Added a Reshape layer for copying-free modification of blob dimensions.

Merge pull request #2177 from pgao/spp_layer Spatial Pyramid Pooling Layer

Spatial Pyramid Pooling Layer

2015-05-04

Added support for original implementation, using (margin - d^2), through the legacy_version parameter.

2015-04-09

Allow Transform of encoded datum. Allow initialize transformed_blob from datum or transform params. Allow force_color and force_gray as transform params.

2015-03-13

HDF5DataLayer shuffle: minor cleanup; clarification in HDF5DataParameter

shuffle data

2015-03-12

PReLU Layer and its tests described in Kaiming He et al, "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification", arxiv 2015. Belows are commit message histories that I had while developing. PReLULayer takes FillerParameter for init PReLU testing consistency with ReLU Fix : PReLU test concistency check PReLU tests in-place computation, and it failed in GPU Fix: PReLU in-place backward in GPU PReLULayer called an incorrect API for copying data (caffe_gpu_memcpy). First argment of `caffe_gpu_memcpy` should be size of memory region in byte. I modified to use `caffe_copy` function. Fix: style errors Fix: number of axes of input blob must be >= 2 Use 1D blob, zero-D blob. Rename: hw -> dim

2015-03-09

AccuracyLayer: add ignore_label param

Fixup AccuracyLayer like SoftmaxLossLayer in #1970 -- fixes #2063

2015-03-03

DummyDataLayer outputs blobs of arbitrary shape

ConcatLayer: generalized Blob axes

SliceLayer: generalized Blob axes

SoftmaxLayer: generalized Blob axes

Add BlobShape message; use for Net input shapes

InnerProductLayer can multiply along any axis

Blobs are ND arrays (for N not necessarily equals 4). vector<int> shape_ instead of (num, channels, height, width).

2015-02-20

Merge pull request #1910 from philkr/encoded add force_encoded_color flag to the data layer and warn about mixed encoding

added a force_encoded_color flag to the data layer. Printing a warning if images of different channel dimensions are encoded together

2015-02-17

give phase to Net and Layer Give the responsibility for phase to Net and Layer, making phase an immutable choice at instantiation and dropping it from the Caffe singleton.

[pycaffe] allow Layer to be extended from Python This is done by adding PythonLayer as a boost::python HeldType.

2015-02-14

Add gradient clipping -- limit L2 norm of parameter gradients

2015-02-06

fix for layer-type-str: loss_param and DECONVOLUTION type should have been included in V1LayerParameter, get upgraded

2015-02-05

get rid of NetParameterPrettyPrint as layer is now after inputs (whoohoo)

automagic upgrade for v1->v2

start layer parameter field IDs at 100 (always want them printed at the end, and want to allow more fields to be added in the future, so reserve fields 10-99 for that purpose)

'layers' -> 'layer'

restore upgrade_proto

add message ParamSpec to replace param name, blobs_lr, weight_decay, ...

Layer type is a string

2015-02-01

Merge pull request #1615 from longjon/deconv-layer Add deconvolution layer with refactoring of convolution layer to share code

2015-01-30

Merge pull request #1654 from longjon/softmax-missing-values Add missing value support to SoftmaxLossLayer

debug_info in NetParameter so it can be enabled outside training

2015-01-17

Make comments for sparse GaussianFiller match actual behavior (Fixes #1497 reported by @denizyuret)

2015-01-11

add DeconvolutionLayer, using BaseConvolutionLayer

2014-12-31

add spatial normalization option to SoftmaxLossLayer With missing values (and batches of varying spatial dimension), normalizing each batch across instances can inappropriately give different instances different weights, so we give the option of simply normalizing by the batch size instead.

add missing value support to SoftmaxLossLayer

2014-12-22

Added Multistep, Poly and Sigmoid learning rate decay policies Conflicts: include/caffe/solver.hpp src/caffe/proto/caffe.proto src/caffe/solver.cpp