Proto commits in uwescience/myria

These 45 commits are when the Protocol Buffers files have changed:

2017-07-21

Revert "Merge pull request #870 from uwescience/enable_worker_sysgc" This reverts commit 770e652b6b53e0b0e44f9d5b2069547292fbf72c, reversing changes made to 412855281b99b6f9235cab09550481fb78b314f4.

The documentation is generated from this commit.

2017-07-20

merge from master

2017-07-12

Include REEF container exceptions in Myria query response (#899) * serialize REEF exceptions and rethrow * address review feedback

The documentation is generated from this commit.

2017-05-12

make server trigger sysgcs after query completion

2017-01-30

addressessing code review comments addressing code review comments, part 2 format and remove comments

2017-01-04

rename bytes/bytebuffer to blob; make batch size member of tb,tbb and mutable tb

2016-12-15

adding blob data type and tests

2016-05-17

adding new blob data type

2016-02-05

Hijack CSV export to push protobufs

2016-01-15

move all failure handling to driver

2015-12-04

Added Myria Matrix type to protobuf/column.proto and recompiled

2015-11-08

move all failure handling to driver

2015-05-22

Added MYRIAMATRIX_TYPE and MyriaMatrix class. Added associated columns for them. Not tested.

2015-01-30

Merging Victor's scale out code with master so I can use postgres on EC2

2014-12-11

Add a ResourceStats protobuf message.

2014-07-25

add ResourceStats to protobuf

2014-05-21

protobuf: add subquery ID to control messages There will be a subsequent commit to make the server and worker control structures handle this well.

2013-08-22

Add ADD_WORKER_ACK message into protobuf. This message is sent from a worker to the master when the worker received a ADD_WORKER message from master.

Add REMOVE_WORKER_ACK message into protobuf. This message is sent from a worker to the master when the worker received a REMOVE_WORKER message from master.

Add QUERY_RECOVER message into protobuf. This message is sent from master to all the other workers when a recovery worker is launched and ready to execute the query. When other workers get this message, they will start recovery tasks to send buffered data to the new worker.

2013-08-14

protobuf: update package name, recompile, add @SuppressWarnings("unused") - fix package name and recompile - Also, there's apparently a new version of protobuf so bump the library while we're here. - after, suppress warnings in the auto-generated code. Signed-off-by: Daniel Halperin <dhalperi@cs.washington.edu> Signed-off-by: Daniel Halperin <dhalperi@cs.washington.edu>

2013-07-19

Add datetime java/proto types

2013-07-16

Add 2 protobuf messages: REMOVE_WORKER and ADD_WORKER. When a worker is treated as dead by the server, server needs to tell other workers to remove the connections to the dead worker. Also, when a new worker is started and sends the first heartbeat, the server needs to tell other workers to add this new one.

change WORKER_ALIVE to WORKER_HEARTBEAT.

2013-07-09

Remove IPC meta messages from Protobuf These messages will be processed by the IPC layer itself.

2013-05-19

refactor StringColumn the old implementation has too much overheads in getString (basically a copy of the substring from StringBuilder)

2013-04-21

change the elapse to a statistics class

2013-04-20

refactor protobuf. Move query related messages to query proto. Add query report to support report back of query executions from workers to the master.

2013-03-31

move query related control protobuf definition together and rename START_QUERY to QUERY_START so that all query related protobuf messages all start with prefix QUERY.

add query control protobuf messages

2013-03-18

add some comment

2013-03-15

add a BOS message, which means the beginning of a data stream. And remove the redudent numTuple field in each column.

2013-01-24

merge from master

2013-01-22

myriad: add query id to queries and messages Also fix a few enums. Signed-off-by: Daniel Halperin <dhalperi@cs.washington.edu>

myriad: add query completed message and use it to fix SplitDataTest Now there's a Server.queryCompleted(queryId) function that tells whether all workers have returned a QUERY_COMPLETED message. Signed-off-by: Daniel Halperin <dhalperi@cs.washington.edu>

myriad: make Server actually wait until all workers are alive Get rid of races in SystemTests. Signed-off-by: Daniel Halperin <dhalperi@cs.washington.edu>

2013-01-21

add EOI message to protobuf

2013-01-02

add disconnect message to protobuf

2012-10-22

Sanitized pure python protobuf. Fixed namespace issue, such that all internal modules import protobuf, not google.protobuf. The latter conflicts with Google App Engine.

2012-10-21

Merge branch 'dhalperi-add-datalog-support' of https://github.com/uwescience/myriad into dhalperi-add-datalog-support Conflicts: protobuf/cpp/logical_ra.pb.cc protobuf/cpp/logical_ra.pb.h protobuf/java/edu/washington/escience/myriad/proto/LogicalRaProto.java protobuf/logical_ra.proto protobuf/python/logical_ra_pb2.py

Added select operator

2012-10-16

protobuf: pluralize repeated variable names, join->equijoin, add cross It makes more sense, I think, to name the repeated fields (which inherently represent sets/bags of values) to plural, e.g., column to columns in equijoin and project. I also renamed join to equijoin to be clear, and added cross. Do we need a specific theta-join operator too? Signed-off-by: Daniel Halperin <dhalperi@cs.washington.edu>

2012-10-10

datalog-support: add proto files for "Logical RA" operation See protobuf/python/logical_ra_example.py to test. Signed-off-by: Daniel Halperin <dhalperi@cs.washington.edu>

2012-10-03

First full version of protobuf

2012-08-29

myriad: add protobuf and serialize columns into it Includes the proto object, the probuf-java jar. The Column interface now mandates that implementations be able to serialize themselves as a ColumnMessage object. Signed-off-by: Daniel Halperin <dhalperi@cs.washington.edu>