Proto commits in tpys/face-recognition-caffe2

These 43 commits are when the Protocol Buffers files have changed:

2017-10-04

change device enums to be contiguous Summary: quick change Reviewed By: ajtulloch Differential Revision: D5976025 fbshipit-source-id: a5a1538a380edb7c3b0af76e74c2ccee09ecb928

The documentation is generated from this commit.

2017-10-02

add openglv2 to experimental Summary: only changes needing review are in proto_utils.cc and caffe2.proto Reviewed By: jerryzh168 Differential Revision: D5956743 fbshipit-source-id: e03fffaf5bc8413f2320c20a89a421f1a69b2870

2017-09-13

Add node_name to DeviceOption Summary: Allow for generalizing net transforms. Reviewed By: Yangqing Differential Revision: D5812140 fbshipit-source-id: e3f30acad362ae1f0614ee218d331b525710b88e

2017-08-23

Update proto definition Summary: Update Argument's definition to allow direct passing of NetDef Reviewed By: azzolini Differential Revision: D5681837 fbshipit-source-id: e6c618bff051f9bbc56075c796aeba0094fa97dd

2017-08-22

code cleanup: separate the several net implementations to separate files. Summary: TSIA. Reviewed By: harouwu Differential Revision: D5670906 fbshipit-source-id: 507e789978144341bf696fb20dc11f3c2d55493b

2017-08-18

Early design for a general Event abstraction cross-devices. Summary: There are ad-hoc efforts on avoiding excessive device synchronizations, such as async_dag, singlethread_async, etc. This diff aims to provide an early design for a general Event class, that can achieve the following: (1) It is device agnostic, essentially using a vtable to do cross device record, wait and synchronization. (2) Created new functions WaitEvent and Record in the Context class for interacting with Events. (3) Exposed the corresponding WaitEvent and Record functions in the OperatorBase class as well. An example use case is that, after potential future refactoring, one can achieve a real async execution per operator by running op.WaitEvent(previous_event); op.RunAsync(); op.RecordEvent(this_op_event); and the next op can do next_op.WaitEvent(this_op_event); Right now, I changed async_dag net implementation so that it uses the general event design. The old Event class is assimilated to the general Event class and the old Stream class is now essentially taken over by the Context class itself. Reviewed By: harouwu Differential Revision: D5648463 fbshipit-source-id: 58bd84d06e4a9977b0b835110ddb2f18be3b7cbc

2017-08-16

Support session in distributed realtime trainer Summary: Convert from PlanDef ProtoBuf into python Plan object by recursively creating Nets and ExecutionSteps. Also support running Plan object directly in Session. Reviewed By: azzolini Differential Revision: D5608393 fbshipit-source-id: c0ae3b6da743a759af6db3b614a5a3935fe0b34c

2017-08-02

Added functionality that allows users to store huge blobs Summary: Added functionality that allows users to store huge blobs of any type not only Tensors. Blob has to be divided into chunks in the same way as Tensor blob. Reviewed By: kennyhorror Differential Revision: D5432762 fbshipit-source-id: c171faacd99d209bfae6f9707ebde7c4e23ba3b9

2017-06-21

Allow tasks/execution_steps to be cloned at runtime Summary: Advantages of cloning the tasks/execution_steps at runtime: - Less complexity on the python side: no need to clone nets and add prefixes to blob names - Faster start-up: we had cases of complex plans that took up to 30min to be created. - Better isolation: each task cloned at runtime has its own child workspace, preventing false sharing of blobs. - Opens up possibility for dynamic scheduling: Number of threads per task can be increased on the fly, at runtime. Reviewed By: dzhulgakov Differential Revision: D5100730 fbshipit-source-id: 71b83193b135da4e6eaf2536d8fc266528e1fdcc

2017-06-19

Revert uuid change to OperatorDef protobuf Summary: a few issues: 1. Randomization hurts memoization 1. Even if we make it non random, then we can get key colisions when loading it back. 2. RNNs use prototxt for step net and apparently its not forward compatible like normal protobuf is I am thinking of a better less invasive solution now. Reviewed By: jamesr66a Differential Revision: D5272118 fbshipit-source-id: ab577fad04fbfc632e1fceffa923377a0d3da1be

2017-06-15

Fix a few typos and grammars in comment Summary: Fix a few typos and grammars in comment by using language-check, python library spell_checker source code is here : https://github.com/17-1-SKKU-OSS/011A/blob/master/spell_checker/spell_checker.py here is the text file which indicates what things should be fixed : https://github.com/17-1-SKKU-OSS/011A/tree/master/spell_checker/fix/caffe2 Closes https://github.com/caffe2/caffe2/pull/719 Differential Revision: D5165118 Pulled By: aaronmarkham fbshipit-source-id: 7fb8ef7a99d03cd5fd2f9ebdb01b9865e90fc37b

2017-06-14

Operator python traceback Summary: This is going to show a python Caffe2 user where a failed operator was created. Motivation for having this information not right in protobuf is to avoid having it too verboose and keep ability to read protobufs of a net after a simple print() call. Reviewed By: jamesr66a Differential Revision: D5226047 fbshipit-source-id: 7edfe850e05a2ec209577142aa3368664a57a108

2017-04-24

Expose operators execution statistics to python frontend. Summary: To expose operators execution statistics in python, profiling measurements collected in ProfDAGNet class is leveraged. In current implementation, a new operator is defined that outputs the statistic data in a protobuf message. In the frontend, OperatorStatsContainer works as a wrapper to print ProfDAGNet statistics. Differential Revision: D4923009 fbshipit-source-id: 18a6d76a405ef277a3fca7a312609051cf943207

2017-04-04

Constant string is generated from Protobuf instead of Thrift Summary: To make the predictor open souorce, move the constants that are generated from Thrift to Protobuf. Reviewed By: salexspb Differential Revision: D4656884 fbshipit-source-id: d4dbb3416e8396185e0981fcd9a090fbb054a18a

2017-03-31

Change the predictor to use Protobuf Reviewed By: salexspb Differential Revision: D4644798 fbshipit-source-id: 0cf96dfc9061f87978a57d2fedcfe4a0bb012405

2017-03-09

QTensor serialization/deserialization Summary: Added protobuf style serialization/deserialization w/o chunking for qtensors Reviewed By: salexspb Differential Revision: D4622677 fbshipit-source-id: 1f845ad773a61b7ae2c362ec31d8de04e4217f68

2017-03-03

Change the type of content in BlobProto from string to bytes Summary: We are converting MetaNetDef from thrift to protobuf. The protobuf is binary encoding. Since bytes is a superset of string. Change the field to bytes so that no warning is generated when compiling caffe2. Reviewed By: Yangqing Differential Revision: D4635581 fbshipit-source-id: 916b799e1fb9466658e1dd198bfb5c6928f22488

2017-03-01

add inference for gradient ops + a couple of missing shape inference functions + fix to scalars Summary: A bit too much stuff in one diff, so sorry: 1. Add inference for gradient types by using the fact that x_grad is gradient of x and must be of same shape. This is kind of awkward to use string matching, but in addition I rely on the operator being actually a gradient op. 2. dzhulgakov was write, scalar shape is () and not (1). Sorry, my claim easlier was #fakenews. 3. Added inference functions for MakeTwoClass, MomentumSGDUpdate and Cross entropy ops. Reviewed By: dzhulgakov Differential Revision: D4569758 fbshipit-source-id: 0db13f33819777fdddefe21d4b1ebf906fcaf98c

2017-02-22

Improve "reporter net" design Summary: Previously we had several limitations for a reporter net: - needed to be a net, not an execution step - only one allowed per execution step, with a single interval Now, "reporter nets" become repoter steps and multiple of them can be specified with different timeouts. Reviewed By: dzhulgakov Differential Revision: D4583686 fbshipit-source-id: ad7266e16f96e7829fd24dcc1f165f39e9db573d

2017-02-03

Shape and Type Inference Part1 Summary: This is a bit large diff, sorry about it. It includes basic shape and type inference functionality, based on YQ's Schema scaffolding. I added some helper functions to make it easier to write simple translations. Bigger refactoring was needed for ConvPoolBase so that we could use the shape inference already there in the schema. I annotated enough operators to be able to infer forward-pass of shapes for basic convnet, and added test for that. I intend to bootcamp some annotations and annotate enough to handle Resnets fully. Need to think about gradients, if they could be annotated in an easier way. Only shapes are now exposed to Python, types will follow later. Also the inference is not called yet anywhere but unit test. Also I am not sure if everything is in the best location in the code, but shouldn't be hard to move stuff around. Reviewed By: dzhulgakov Differential Revision: D4436818 fbshipit-source-id: eebee5937ccc9ac09c245465302388a1fae6933c

2016-11-29

Add the MKLDNN type to the tensor type strings and added proper docs. Summary: TSIA Reviewed By: dzhulgakov Differential Revision: D4217541 fbshipit-source-id: f68d1aba9c20af0fb0aed2cc1b2099961f6fa7a4

2016-11-18

fbsync at f5a877

2016-11-15

fbsync. TODO: check if build files need update.

2016-09-16

move third_party/google/protobuf to use git submodules.

2016-08-02

chunky sync again

2016-07-29

caffe translator with added back legacy pooling support

2016-07-28

sync

bring up caffe.proto to master

2016-07-21

add untracked files

chunky sync - build scripts to be written

2016-05-13

chunky sync

2016-03-18

minot changes

2016-01-14

misc updates

2015-12-06

deprecate legacy pooling

optionally use protobuf lite

2015-11-28

various updates

2015-10-22

A few changes: (1) cudnn for conv (2) cublas: after going through the work I feel it's beter to use HOST pointer mode, so changed it. (3) storage order: despite that googlenet and multibox uses NHWC, it seems better to be still using NCHW as default to be consistent with caffe and cudnn; moved to NCHW as default.

2015-10-12

A consolidation of a couple random weekend work. (1) various bugfixes. (2) Tensor is now a class independent from its data type. This allows us to write easier type-independent operators. (3) code convention changes a bit: dtype -> T, Tensor<*Context> -> Tensor* alias. (4) ParallelNet -> DAGNet to be more consistent with what it does. (5) Caffe's own flags library instead of gflags. (6) Caffe's own logging library instead of glog, but glog can be chosen with compile-time definition -DCAFFE2_USE_GOOGLE_GLOG. As a result, glog macros like CHECK, DCHECK now have prefix CAFFE_, and LOG(*) now becomes CAFFE_LOG_*. (7) an optional protobuf inclusion, which can be chosen with USE_SYSTEM_PROTOBUF in build_env.py.

2015-07-18

(1) minidb bugfix (2) blob serialization comments (3) cudnn: putting it under a separate device name so we can explicitly choose cudnn instead of having CUDA device prioritizing it. (4) note that mint is not available with ipython due to zeromq conflict (5) db_throughput utility (6) added gprofiler

2015-07-02

[style] Finishing name changes for the rest of the fields in the protobuf.

2015-07-01

[style] Massive name change from plural to singular. This one changes operator’s “inputs” and “outputs” to “input” and “output”, and "args" to "arg".

GoogleNet adaption - added yet another legacy padding support.

2015-06-25

A clean init for Caffe2, removing my earlier hacky commits.