Proto commits in petewarden/tensorflow_ios

These 52 commits are when the Protocol Buffers files have changed:

2016-03-09

Prototype of an in-process gRPC server for TensorFlow/Python. Adds support for binding a TensorFlow server to any port, to support single-process testing. This interface is a work in progress. In particular, it supports launching a server, but the support for clean shutdown is incomplete. Change: 116593644

The documentation is generated from this commit.

2016-03-04

Moved config.proto from core/framework to core/protobuf. Change: 116396958

2016-03-03

Added timeout support for Session APIs. To use in Python Session API: sess = tf.Session( config=tf.ConfigProto(session_timeout_in_ms=your_timeout_value)) Change: 116288637

Adds an in-process gRPC server with support for clean shutdown. This CL refactors the `grpc_server_lib` library such that it exports a `ServerInterface` class with `Start()`, `Stop()`, and `Join()` methods; and a factory method that takes a new `ServerDef` proto for configuring the server. Also fixes a bug in "master.cc" whereby a `~Master()` would hang if GC was disabled. TODO(mrry): Add a SWIG wrapper so that this can be instantiated in Python. Change: 116194792

2016-03-02

Use java_generate_equals_and_hash=true for Features and Examples. Without it, generic reflective implementations of equals() and hashCode() are used, which are much slower. Change: 116150839

Add optional comprehensive logging of memory allocation/deallocation events. When enabled, the following events are recorded: The start of a step, with the numerical step_id and a textual handle describing the step. A Tensor allocation, including the step_id, the name of the OpKernel, the data type, shape, allocation size, allocation_id, data pointer location, and allocator used (the allocation_id is local to an allocator). A Tensor deallocation, including the allocation_id and allocator used. A raw memory allocation, including the step_id, the name of the component (e.g. Eigen), the number of bytes, data pointer location, allocation_id and allocator used. A raw memory deallocation, including the step_id, the name of the component (e.g. Eigen), allocation_id and allocator used. For now many Tensor allocations show 'unknown' for the kernel and step_id. These mostly come from Tensors allocated by the system from protocol buffers, and Tensors allocated by Ops using the Tensor constructor directly instead of calling OpKernelContext::allocate_temp. The latter can in principle be cleaned up one by one as necessary. The former would require some plumbing to associate an allocation with the appropriate step_id. With this CL memory logging is enabled by raising the VLOG level to 1. Once there is an ability to set process-wide options programmatically it would make sense to update the machinery to do that. Currently recorded events are logged as INFO, and they can all be retrieved by filtering the log for lines including __LOG_MEMORY__. Some example lines are as follows: I0301 13:38:55.797563 81179 log_memory.cc:18] __LOG_MEMORY__ MemoryLogTensorAllocation { step_id: -6 kernel_name: "Unknown (from Proto)" tensor { dtype: DT_FLOAT shape { } allocation_description { requested_bytes: 4 allocated_bytes: 4 allocator_name: "cuda_host" allocation_id: 2 has_single_reference: true ptr: 8717861408 } } } I0301 13:38:55.802245 81179 log_memory.cc:18] __LOG_MEMORY__ MemoryLogTensorAllocation { step_id: -6 kernel_name: "Unknown" tensor { dtype: DT_FLOAT shape { } allocation_description { requested_bytes: 4 allocated_bytes: 256 allocator_name: "gpu_bfc" allocation_id: 1 has_single_reference: true ptr: 47378989056 } } } I0301 13:38:55.802347 81179 log_memory.cc:18] __LOG_MEMORY__ MemoryLogTensorDeallocation { allocation_id: 2 allocator_name: "cuda_host" } [...] I0301 13:38:55.806454 81179 log_memory.cc:18] __LOG_MEMORY__ MemoryLogStep { step_id: 1 handle: "->/init;0" } I0301 13:38:55.806659 81220 log_memory.cc:18] __LOG_MEMORY__ MemoryLogTensorOutput { step_id: 1 kernel_name: "random_normal/shape" tensor { dtype: DT_INT32 shape { dim { size: 4 } } allocation_description { requested_bytes: 16 allocated_bytes: 16 allocator_name: "cuda_host" allocation_id: 1 ptr: 8717860896 } } } [...] I0301 13:38:56.362898 81218 log_memory.cc:18] __LOG_MEMORY__ MemoryLogTensorAllocation { step_id: 1 kernel_name: "conv1/truncated_normal" tensor { dtype: DT_FLOAT shape { dim { size: 11 } dim { size: 11 } dim { size: 3 } dim { size: 96 } } allocation_description { requested_bytes: 139392 allocated_bytes: 139520 allocator_name: "gpu_bfc" allocation_id: 36 has_single_reference: true ptr: 47379030016 } } } I0301 13:38:56.362894 81217 log_memory.cc:18] __LOG_MEMORY__ MemoryLogTensorDeallocation { allocation_id: 24 allocator_name: "gpu_bfc" } I0301 13:38:56.362903 81213 log_memory.cc:18] __LOG_MEMORY__ MemoryLogTensorOutput { step_id: 1 kernel_name: "conv5/truncated_normal/mul" tensor { dtype: DT_FLOAT shape { dim { size: 3 } dim { size: 3 } dim { size: 1024 } dim { size: 1024 } } allocation_description { requested_bytes: 37748736 allocated_bytes: 37748736 allocator_name: "gpu_bfc" allocation_id: 34 ptr: 48512711168 } } } [...] I0229 16:39:57.482980 76558 log_memory.cc:18] __LOG_MEMORY__ MemoryLogRawAllocation { step_id: 13 operation: "xentropy/EigenAllocator" num_bytes: 64 ptr: 47386857472 allocation_id: 625 allocator_name: "gpu_bfc" } I0229 16:39:57.483147 76558 log_memory.cc:18] __LOG_MEMORY__ MemoryLogRawDeallocation { step_id: 13 operation: "xentropy/EigenAllocator" allocation_id: 625 allocator_name: "gpu_bfc" deferred: true } I0229 16:39:57.483197 76558 log_memory.cc:18] __LOG_MEMORY__ MemoryLogRawDeallocation { step_id: 13 operation: "xentropy/EigenAllocator" allocation_id: 625 allocator_name: "gpu_bfc" } Change: 116065112

2016-02-26

Adds RunWithOpts() that takes a per-step RunOptions and RunOutputs to C++ Session API. Use it for optionally turning tracing on for a step and returning profiling info collected via StepStats. For DirectSession only. Example usage: RunOptions run_options; run_options.set_trace_level(RunOptions::FULL_TRACE); RunOutputs run_outputs; ASSERT_TRUE(!run_outputs.has_step_stats()); Status s = session->RunWithOpts(run_options, inputs, output_names, target_nodes, &outputs, &run_outputs); ASSERT_TRUE(run_outputs.has_step_stats()); Change: 115693287

Final fix to TestReporter (hopefully). Change: 115675044

Initial version of the open-source distributed TensorFlow runtime. This includes a gRPC server (grpc_tensorflow_server) that can serve as both the master of a distributed TensorFlow computation, and an individual worker in the computation. The GrpcSession class is included to allow client programs (including Python clients) to interact with a server. See tensorflow/core/distributed_runtime/README.md for usage instructions. This change partially addresses issue #23. Change: 115634191

2016-02-25

Rollback of "TestReporter is back in. Maybe also fixed the Android build." Test fails. Change: 115602477

TestReporter is back in. Maybe also fixed the Android build. Change: 115589642

TensorFlow: fix bug in StringPiece::contains which made it always return true. Add a unittest to catch this type of regression in the future. Change: 115573280

Added TestReporter and test / benchmark reporting tools. These tools are meant to allow recording of benchmark & unit test structured output to pbtxt files in a directory only when the environment variable TEST_REPORT_FILE_PREFIX is set. For now, only saving of C++ microbenchmark output is supported. Change: 115518303

2016-02-18

Added documentation for import_meta_graph and export_meta_graph. Change: 115005379

2016-02-11

Add versions to checkpoints Checkpoints now have a version scheme analogous to that for GraphDefs. We have no plans to ever deprecate a checkpoint version, but it's good to have the scheme in place in case we need to. Change: 114364388

2016-02-10

Enables java_multiple_files for all tf protos, and sets the outer class name consistently to <FileName>Protos. Also specifies the java namespace as org.tensorflow.*. This enables compiling tf protos with nano proto for Android (which currently does not work because of file/message name clashes) and follows the same convention for proto3 as used by the API platform. Change: 114279703

Added the use of google.protobuf.Any in meta_graph.proto to allow packing any any user protos. Change: 114256308

2016-02-09

Add the Ops for the producer to MetaGraphDef. Change: 114175259

2016-02-07

Added export_meta_graph() and import_meta_graph() for serializing/de-serializing the graph and other Python objects necessary restarting training, running eval, or running inference into a MetaGraphDef protocol buffer. MetaGraphDef contains the following: - MetaInfoDef: For storing version and other meta data associated with the meta graph. - GraphDef: The Graph. - SaverDef: The Saver. - CollectionDef * Int64List * FloatList * BytesList * NodeList * AnyList These are evolving APIs and subject to change. Change: 114026857

2016-02-03

Refactor the logic to apply optimization into a common module. Change: 113692577

Add assert_same_float_dtype. Change: 113646664

2016-02-01

Get rid of mysterious comment previously needed for syncing with the Google-internal version. Change: 113552310

2016-01-27

Add a new 'log message' event type. This will be used in a subsequent change to allow Python code to log to the events file so that it can be viewed in TensorBoard and so logging events can be associated with models more easily. Change: 113194631

- Added optimizer_options field to GraphOptions, moved graph optmization options there. - Deprecated the existing skip_common_subexpression_elimination field. Change: 113194182

2016-01-26

Change GraphDef versions to use version, min_consumer, min_producer Since all GraphDef versions to date are forwards compatible, we make no attempt to merge old-style and new-style GraphDef version information. The old int32 version field is ignored (and deprecated) and the new VersionDef versions field defaults to producer 0, min_consumer 0 if left out. As a benefit, once this CL is in we can immediately bump the scalar strictness GraphDef version, since consumers will either ignore the new versions (before this CL) or be forward compatible (after this CL). Later, I'll use the same mechanism and protobuf to add versioning to checkpoints. Change: 113091281

Delete obsolete comment. Change: 113024053

2016-01-25

Added constant folding optimization pass. - Graph* -> Graph* pass - Creates a local executor and executes a copy of the constant "slice" of the original graph, and replaces nodes in original graph with constant nodes. Change: 112971745

Adds UINT16 type to TensorFlow. Change: 112970147

Running our linter on a lot of files. Change: 112920860

2016-01-18

Fix typos in core/framework.

Merge commit for internal changes

Adds enough auditing to make it possible to track tensor buffers throughout an execution, and build a cost model of memory usage. There are two main components: 1) GPU allocators now assign to each allocated tensor buffer a unique ID so its use can be tracked within and across steps. 2) The checkin cleans up the tracking of usage of Tensor buffers, and makes it work for both sync and async kernels (async kernels did not previously track gpu memory correctly). Each use is now tracked by the OpKernelContext (for allocators that need this support) in a single uniquified set of TensorReferences. When the kernel finishes, the executor retrieves the list of references, logs it if needed in the nodeexecstats, then passes it to the device, which may add an additional reference to keep the memory from being reused until the execution completes. When the tensor is logged in the nodeexecstats a flag is set if there is a single remaining reference to the buffer, which means that the memory will be freed once the Op completes. Change: 112375683

2016-01-12

Merge commit for internal changes

Add GraphDef version 6 implementing scalar strictness, but don't use it The kAllowLegacyScalars flag is now gone. Instead, scalar strictness now depends on the GraphDef version: we are lenient below 6 and strict with 6 and above. The current GraphDef version is still 5; new graphs will not yet use the new version by default. Outside of PLATFORM_GOOGLE, this change has no effect since the code was already scalar strict. The TensorShapeUtils versions of IsLegacyScalar and IsLegacyVector are also gone; users should now get them from OpKernel. The GraphDef version history has been moved to core/public/version.h to minimize the number of files that must be changed in future CLs. Change: 111947842

2016-01-11

Merge commit for internal changes

Add note regarding when we can make parsing of AttrValue's holding an empty list more strict. We had some convenient GraphDef version bumps at the relevant time. Change: 111863523

Fix broken reference to TensorBase::Serialize

2016-01-08

TensorFlow: Add more graphdef validation, this time against OpDefs. The Graph construction code assumes that the GraphDef's NodeDefs match the OpDef specifications. When creating NodeDefs using NodeDef builder, we call ValidateNodeDef() function in node_def_util, but when loading an arbitrary protobuf, we do not. This changes the code to do some important sanity checking during Session::Create/Extend. Bumps the version of graph_def to only do this for new graphs. Also fixes placeholder's shape attr to really be optional, which it wasn't. Fix one test that was using an invalid graph. Change: 111657920

2016-01-07

Add PartialTensorShape C++ class. Wraps the TensorShapeProto, but supports a dimension value of -1 as meaning an "unknown" value. Contains compatibility and merging capabilities similar to the Python version of TensorShape. Future changes will add: * a bool flag to the TensorShape proto that signifies an unknown rank. * serialization/deserialization of Python TensorShape None values to -1 in the Proto. * A new Op attr type "partial_shape" that deserializes to PartialTensorShape. Change: 111622281

2016-01-06

Change: 111522571

New GraphDef version 4: TensorFlow is now scalar strict The kAllowLegacyScalars flag is now gone. Instead, scalar strictness now depends on the GraphDef version: we are lenient below 4 and strict with 4 and above. All new graphs should use version >= 4. The internal Google version of tensorflow is now as scalar strict as the open source release. Note that outside of PLATFORM_GOOGLE, this change has no effect, since the code was already scalar strict. Change: 111467133

2016-01-05

Added 'logging' import to control_flow_ops which is used in the file but not imported. Change: 110842260

2015-12-21

Change: 110592065

2015-12-12

TensorFlow: merge changes from internal Change 110055925 Clean up interface for adjust_contrast and adjust_brightness. - Simplify kernel for adjust_contrast and remove all min/max and casts. - Change semantics of delta arg to adjust_brightness (always in [0,1)), and adjust users. - Add saturate_cast for casting images without over/underflow problems. - Add new numbers for adjust_contrast benchmark. This CL makes two changes to the public API: - It changes the semantics of the delta parameter of adjust_brightness, which was in the same range as the input image before, and now is always in [0,1). - It changes the semantics of adjust_contrast (the cc op), which wasn't hidden, but was shadowed by the python wrapper in image_ops. It's a little questionable whether this function was part of the public API. It definitely shouldn't have been. It is now hidden, although now it could be part of the public API, albeit with a different name. Change 110054427 update ci_build * add PYTHON_BIN_PATH and always run ./configure in ci_build * rename ci_build cache directory to bazel-ci_build-cache * sync ci_build/Dockerfile.cpu with docker/Dockerfile.devel * use "FROM nvidia/cuda:..." for gpu container * therefore no need of the tensorflow_extra_deps directory anymore * share install code between containers using ./install/*.sh scripts * do not inherit (and override FROM clausule in dockerfiles anymore) * print bazel test errors to stderr Change 110047126 Update ops.pbtxt. Change 110046428 Simplify the example for the Fill op. Base CL: 110056265

2015-12-08

TensorFlow: upstream changes to git. Change 109695551 Update FAQ Change 109694725 Add a gradient for resize_bilinear op. Change 109694505 Don't mention variables module in docs variables.Variable should be tf.Variable. Change 109658848 Adding an option to create a new thread-pool for each session. Change 109640570 Take the snapshot of stream-executor. + Expose an interface for scratch space allocation in the interface. Change 109638559 Let image_summary accept uint8 input This allows users to do their own normalization / scaling if the default (very weird) behavior of image_summary is undesired. This required a slight tweak to fake_input.cc to make polymorphically typed fake inputs infer if their type attr is not set but has a default. Unfortunately, adding a second valid type to image_summary *disables* automatic implicit conversion from np.float64 to tf.float32, so this change is slightly backwards incompatible. Change 109636969 Add serialization operations for SparseTensor. Change 109636644 Update generated Op docs. Change 109634899 TensorFlow: add a markdown file for producing release notes for our releases. Seed with 0.5.0 with a boring but accurate description. Change 109634502 Let histogram_summary take any realnumbertype It used to take only floats, not it understands ints. Change 109634434 TensorFlow: update locations where we mention python 3 support, update them to current truth. Change 109632108 Move HSV <> RGB conversions, grayscale conversions, and adjust_* ops back to tensorflow - make GPU-capable version of RGBToHSV and HSVToRGB, allows only float input/output - change docs to reflect new size constraints - change HSV format to be [0,1] for all components - add automatic dtype conversion for all adjust_* and grayscale conversion ops - fix up docs Change 109631077 Improve optimizer exceptions 1. grads_and_vars is now a tuple, so must be wrapped when passed to format. 2. Use '%r' instead of '%s' for dtype formatting Base CL: 109697989

2015-12-04

TensorFlow: upstream changes to git Change 109366961 TensorFlow BUILD: now that we have an ops library, set linkstatic to 1. This fixes a breakage in the would-be opensource build, and it *might* mean we can get rid of all of the RequireDefaultOps() calls in our code. The ops library is much smaller than the kernels library that was previously linked together. We set linkstatic=0 presumably since we didn't want to package a static copy of the kernels (very large) everywhere. But the op definitions are small, so this seems like a safe change to make. Time to build the various tests was not any longer after this change, and inspecting the example_trainer binary showed no large increase. Change 109363613 TensorFlow: new graph_def_builder_test needs to RequireDefaultOps. Change 109362569 Split ":ops" out of ":kernels" target in tensorflow/core. Change 109360666 Catch dtype and some shape errors sooner in `QueueBase`. Some avoidable errors were not being caught (e.g. the dtypes of the enqueue components were not checked against the queue's dtypes in Python), leading to cryptic messages at runtime. After this CL, they will be caught earlier. Change 109359569 TensorFlow: Expect g_ != nullptr in test Change 109350735 Add a version number to GraphDef We would like to be able to deprecate behavior in newly generated graphs without invalidating tensorflow's ability to read and evaluate old graphs. For this purpose, GraphDef now has a version field which can be checked inside op kernels to determine how backwards compatible to be. version.h defines TF_GRAPHDEF_VERSION_MIN and TF_GRAPHDEF_VERSION_MAX specifying the range of supported GraphDef versions in the current version of tensorflow. Also expose tf.__version__ and tf.__graph_def_version{,_min,_max}__ for Python interrogation purposes. Whenever we want to deprecate or change some GraphDef semantics, we will proceed as follows: 1. Bump TF_GRAPHDEF_VERSION_MAX, leaving TF_GRAPHDEF_VERSION_MIN unchanged. Describe the change in graph.proto, include the date introduced. 2. In each relevant kernel, implement the new behavior if the GraphDef version is new, but preserve the old behavior for previous GraphDef versions. 3. Wait six months or so (we need to formalize this somewhere). 4. Bump TF_GRAPHDEF_VERSION_MIN and remove the backwards compatibility. The GraphDef version is distinct from the open source version, but at least (4) and possibly (1) correspond to major version number bumps. The first GraphDef version bump is the upcoming scalar strictness change, which affects Google users only since open source is already scalar strict. This commit does not yet plumb the version number into OpKernelConstruction so that ops can access it. That will follow. Change 109350260 Made TensorShapeProto implicitly convertible to TensorShape. Base CL: 109366982

2015-12-02

TensorFlow: Upstream changes to git. Change 109240606 Fix typo Change 109240358 Fix bug in Concat's shape inference due to legacy scalar handling. The shape function was inadvertently converting outputs of unknown shape (rank=None) to vectors of unknown length (rank=1), due to inability to distinguish between legacy scalars and vectors, because `max(1, None)` is 1. Change 109237152 Remove numarray requirement in python_config. Change 109234003 Fix typo in elu documentation. Change 109232946 Python must now be configured via ./configure script Change 109232134 Backported fixes to the tensor comparison operators from the public Eigen repository Change 109231761 Test invalid inputs to softmax_cross_entropy_with_logits. Change 109230218 Backported fixes to the tensor comparison operators from the public Eigen repository Change 109229915 Correct comments in seq2seq to show the right input types for embedding models. (Thanks to hugman@github for bringing this up.) Change 109229118 Fix resize_images example in documentation and allow resize_images to run on a single image with partially-known shape. Change 109228940 Fix demo and node add/remove button spacing Change 109227909 Include Elu in the NN docs. Change 109227059 Adds variable_op_scope and makes variable_scope always add a name_scope. This creates an op scope for variables that makes it easy to create independent operations with a default name by making that name unique for the current scope and it allows explicit names that are not made unique. Change 109224492 Streamline yuv -> rgb conversion to be done in one pass in native code. The entire process now takes ~2ms (including the ByteBuffer.get() calls), down from 10+ ms when the arrays were being interleaved in Java prior to conversion. Also abstracting common yuv->rgb color conversion into helper method. Change 109224389 Add ability to move nodes in and out of auxiliary nodes in graph. Change 109217177 Update generated Op docs. Change 109215030 Implementation of the ELU activation function: http://arxiv.org/abs/1511.07289 Change 109209848 When GPUBFCAllocator runs out of memory, also log a summary of chunks in use by size. Change 109206569 Switched to the public version of the Eigen::sign method since it supports complex numbers. Change 109199813 Modify tensorflow.SequenceExample to support multiple-length sequences. Base CL: 109241553

2015-11-18

TensorFlow: more features, performance improvements, and doc fixes. Changes: - Add Split/Concat() methods to TensorUtil (meant for convenience, not speed) by Chris. - Changes to linear algebra ops interface by Rasmus - Tests for tensorboard by Daniel - Fix bug in histogram calculation by Cassandra - Added tool for backwards compatibility of OpDefs. Tool Checks in history of opdefs and their changes, checks for backwards-incompatible changes. All done by @josh11b - Fix some protobuf example proto docs by Oliver - Add derivative of MatrixDeterminant by @yaroslavvb - Add a priority queue queue by @ebrevdo - Doc and typo fixes by Aurelien and @dave-andersen - Speed improvements to ConvBackwardFilter by @andydavis - Improve speed of Alexnet on TitanX by @zheng-xq - Add some host memory annotations to some GPU kernels by Yuan. - Add support for doubles in histogram summary by @jmchen-g Base CL: 108158338

2015-11-16

TensorFlow: Doc and linter fixes, some additional tests and error handling, updates to website. Changes: - Removes redundant reshape from image models by @mrry - Default TensorBoard to localhost by @danmane - Reformatting of tensorflow/core by @josh11b - Make tutorials backwards compatible to 0.5.0 by @girving - Improve print documentation (md files not updated). - Add proper scrolling to sitemap by @martinwicke Base CL: 107956254

2015-11-12

TensorFlow: upstream changes from the afternoon. Changes: - futurize --stage2 changes for Python 3 compatibility by @girving. - Small updates to documentation by @vrv, schuster and others - Account for failure of std::thread::hardware_concurrency by @ebrevdo. - More changes for backwards-compatibility tests by Josh - Updates to python op doc generation by Josh - Added support for using the best-fit allocator via ConfigProto by @vrv. - Rename LocalSession to DirectSession, since local was a bad name for it. - Enable tf.nn.moments() to work with tensors of unknown shape by @mrry. GITHUB_ISSUE: 139 - Changes for Android build by Andrew. Base CL: 107645181

2015-11-07

TensorFlow: Upstream latest changes to Git. Changes: - Updates to installation instructions. - Updates to documentation. - Minor modifications and tests for word2vec. Base CL: 107284192

TensorFlow: Initial commit of TensorFlow library. TensorFlow is an open source software library for numerical computation using data flow graphs. Base CL: 107276108