Proto commits in basho/machi

These 52 commits are when the Protocol Buffers files have changed:

2016-01-05

WIP

2015-12-31

Add namespace info to wedge_status API call; add namespace enforcement @ machi_flu1_net_server

The documentation is generated from this commit.

2015-12-29

Use 'bool' type in PB spec where feasible

Clean up internal protocol<->tuple mappings for correct epoch checking

Remove straggler CoC items in code

read_chunk API refactoring; all tests pass; todo tasks remain

trim_chunk API refactoring; all tests pass; todo tasks remain

write_chunk API refactoring; all tests pass; todo tasks remain

append_chunk API refactoring; all tests pass; todo tasks remain

2015-12-18

Merge slf/flu-config-rcd-style

2015-12-07

WIP: machi_chain_manager1:set_chain_members() API change, all tests pass, yay

2015-12-02

All eunit tests now passing again

2015-10-29

Change trigger_gc default -> 0

2015-10-28

Trim command and GC prototype implementation * maybe_gc/2 is triggered at machi_file_proxy, when chunk is deleted and the file is larger than `max_file_size` * A file is deleted if all chunks except 1024 bytes header are trimmed * If a file is going to be deleted, file_proxy notifies metadata_mgr to remember the filename persistently, whose filename is `known_files_<FluName>` * Such trimmed filenames are stored in a machi_plist file per flu * machi_file_proxy could not be started if the filename is in the manager's list. Consequently, any write, read and trim operations cannot happen against deleted file. * After the file was trimmed, any read request to the file returns `{error, trimmed}` * Disclaimer: no tests written yet and machi_plist does not support any recovery from partial writes. * Add some thoughts as comments for repairing trims. * State diagram of every byte is as follows: ``` state\action| write/append | read_chunk | trim_chunk ------------+----------------+------------------+--------------- unwritten | -> written | fail (+repair) | -> trimmed written | noop or repair | return content | -> trimmed trimmed | fail | fail | noop ```

2015-10-22

Update read_chunk() PB protocol to return trimmed chunks

2015-10-19

Allow read_chunk() to return partial chunks This is simply a change of read_chunk() protocol, where a response of read_chunk() becomes list of written bytes along with checksum. All related code including repair is changed as such. This is to pass all tests and not actually supporting partial chunks.

2015-10-14

Fix typo in comment

2015-10-13

Add surface of trim to scrub

2015-10-08

Change flag_checksum -> flag_no_checksum for consistency

2015-09-25

WIP: broken

2015-09-24

Implement disable read repair

2015-08-05

Oops, fix PB stuff to add witnesses

2015-07-02

Add kick_projection_reaction, implement yo:tell_author_yo()

2015-07-01

Ha, oops! Add bad_epoch code, derp 1

2015-06-30

WIP: refactoring & edoc'ing

2015-06-29

WIP: giant hairball 12

WIP: giant hairball 01

2015-06-26

WIP: hairball, but: Failed: 6. Skipped: 0. Passed: 13

WIP: hairball, but flu_smoke_test() works!

WIP: giant hairball 5

WIP: giant hairball 3

WIP: giant hairball, bleh, low-level checksum_list() barely working

2015-06-25

machi.proto definition for low-level protocol ops

timing_pb_encoding_test_... speed factor=35.95 [2.730 s] ok So, the PB style encoding of the Mpb_LL_WriteProjectionReq message is about 35-36 times slower than using Erlang's term_to_binary() and binary_to_term(). {sigh}

All projection store protocol operations are now using Protocol Buffers! So, there's some cheating going on, because some of the parts of the #projection_v1{} and #p_srvr{} records aren't fully specified. Those parts are being specified as "opaque" in the field names, e.g. optional bytes opaque_flap = 10; optional bytes opaque_inner = 11; required bytes opaque_dbg = 12; required bytes opaque_dbg2 = 13; The serialization that's being used is erlang term sexprs. That isn't portable. So if/when we really need to deal with a non-Erlang language, we'll have to straighten this out further.

WIP: bugfixes, add {error, written}

WIP: bugfixes, machi_flu1_test still broken

2015-06-24

WIP: dinnertime, machi_flu1_test still broken

WIP: 1/2 of low-level projection proto finished, machi_flu1_test fails

Complete PB round-trip for #projection_v1{}, bleh

WIP: encoding #p_srvr and #projection_v1, just starting. Damn tedious.

2015-06-23

WIP: list_files end-to-end!

WIP: checksum_list incomplete implementation....

WIP: read_chunk #1

WIP: write_chunk #1

WIP: append_chunk #1

2015-06-19

{sigh} add filter-dialyzer-dep-warnings

Add test/machi_pb_test.erl, finish PB refactoring

Overhaul the 0.1 PB definition. Again. Many thanks to @seancribbs for a suggestion to avoid the PB design mistake/feature of the original Riak KV PB API.

2015-06-18

WIP: PB wrestling

WIP: PB wrestling

2015-06-17

First attempt at Protocol Buffers .proto + infrastructure