Proto commits in UWSysLab/tapir

These 23 commits are when the Protocol Buffers files have changed:

2018-06-07

fixing protobuf bug and adding hoard

2018-05-07

updating to protobuf 3

2018-03-26

Recovery & view change implementation + bug fixes. Overview ======== This PR implements the view change and recovery portion of IR. It also fixes a couple of important bugs and introduces a new mechanism with with to unit test IR. What's New? =========== This PR includes the following, in roughly descending order of importance: - The main contribution of this PR is the implementation of view changes and recovery. The implementation required some of the following things: - I introduce DoViewChangeMessage and StartViewMessage protobufs as well as protobufs for serializing records. - IR replicas have logic to periodically initiate a view change or initiate a view change upon recovery. They also have logic to handle DoViewChangeMessage and StartViewMessage messages. - IR replicas persist their view information, which is needed for recovery, using PeristentRegisters which persist data on disk. - When a client issues a PropseInconsistentMessage or ProposeConsistentMessage, the reply indicating whether the request has already been finalized. If it has, the clients respond appropriately. - For conensus requests, clients make sure that the majority of replies and majority of confirms come from the same view. - I fixed a slow path/fast path bug in IR clients. Previously, after a timeout, clients would transition from the fast path into the slow path. But, they would not wait for a majority of responses from replicas. Instead, they would call decide on whatever responses they had at the moment. - Previously, decide functions took in a set of replies, but they needed to take in a multiset of replies so that the decide function could do things like count the frequency of each reply. - I introduced a new simulated transport ReplTransport. It's pretty neat. It launches a command line REPL that you can use to manually control the execution of the system. You decide what timers to trigger and what messages to deliver. Then, you can save your execution as a unit test to run for later. I used the ReplTransport a lot to debug the view change and recovery algorithms since a lot of the corner cases are hard to trigger with the existing transport layers. I added some ReplTransport unit tests for the lock server. - I implemented Sync and Merge for the lock server. The lock server is now fully functional. - I generalized timeout callbacks to error callbacks which are invoked whenever an error (not just a timeout) is encountered. This was necessary to nicely handle some of the failure scenarios introduced by view changes (e.g. a client gets majority replies and majority confirms in different views for a consensus request). - I performed some miscellaneous cleanup here and there, fixing whitespace, changing raw pointers to smart pointers, stuff like that. What's Left? ============ There are still some things left that this PR doesn't implement: - Currently during a view change, replicas send their entire records to one another. I'm guessing there are more efficient ways to transfer records between replicas, similar to how VR has some optimization tricks for log shipping. - I have not implemented Sync or Merge for TAPIR, only for the lock server. - Clients do not notify replicas of view changes when they receive replies from older views. Similarly, a replica never detects that its in a stale view or requests a master record from replica in a higher view. It just eventually does a view change to stay up-to-date. - None of the code has been profiled or optimized or anything like that. This PR focuses only on correctness, not performance. - There could be more unit tests.

The documentation is generated from this commit.

2017-02-08

fixing bugs in tests and removing Naveen's test client

2017-01-19

making proto files v3 compatible

2017-01-17

minimal code for threading error

2016-08-10

porting Jialin's changes and handling multiple requests

2016-01-08

fixing a bug in versioning reads in tapirstore

piping proposed transaction timestimp through to server and back for TAPIR

2016-01-05

checking in a big code clean up and reorg

2015-12-28

Porting ORStore to TAPIR

2015-12-27

Moving common store files out of txnstore and fixing Rules.mk

2015-10-02

adding a lockserver on top of IR

2015-08-28

adding basic IR code

2015-08-21

adding files to implement the IR record

2015-06-26

cleaning up transactional store code for gitlab

2015-06-25

fixing some bugs and checking in new test cases for backend stores

updating copyright notice on files

updating make files for store common files

2015-06-24

reorganizing some storage files and renaming

moving and adding some more storage level files

2015-06-23

reorganizing and updating some VR code

2015-06-20

Adding transport library