Proto commits in harvard-acc/smaug

These 22 commits are when the Protocol Buffers files have changed:

2021-10-22

core: Remove the AllCache memory policy. (#105) AllCache as a memory policy doesn't make sense. A MemoryPolicy specifies *how* the data is moved, not where it gets stored after it is moved. The latter is specified in the Aladdin config files (scratchpads or caches). This option, when used, causes simulations to fail because the logic to handle this case was not added for HybridDatapath::handleCacheMemoryOp; however, in practice it is almost the same as using ACP, which *does* handle it. So, we just drop this policy as an option altogether. Fixes issue #102.

The documentation is generated from this commit.

2021-07-17

Add a padding operator. (#95) The implemented operator supports padding in all dimensions and asymmetric padding. And it is implemented in software only, no accelerated version is available. The code is tested in both CPP and python API. In the CPP test, I test it with different input sizes (2D, 4D) and different padding patterns (asymmetric, symmetric). The python API testing is mainly focused on it can work in a graph. Example Python code. ```python # consider input_tensor is a 4D tensor, we pad the last 2 dimension with 1 out = array_ops.padding(input_tensor, [0, 0, 0, 0, 1, 1, 1, 1], "padding") # consider input_tensor is a 4D tensor, we pad the 2nd dimension with symmetric # padding with size 1 and the 3rd dimension with asymmetric padiing of size 1 and 2. out = array_ops.padding(input_tensor, [0, 0, 0, 0, 1, 1, 1, 2], "padding") ``` Fixes issue https://github.com/harvard-acc/smaug/issues/94.

2020-07-29

core: Get rid of the children field of NodeProto. The children field is extraneous because we can infer the parent/child relationship using the parents field already. This field also creates extra maintainance burden especially for graph post-processings. TESTED=all tests are still passing. Also updated the model files in the experiments submodule.

2020-06-06

operators: Add a merge operator. The added merge operator forwards the value of an available tensor from inputs to output. All input tensors should be dead except one. This operator will be used as the merge point after divergent control flow paths. Using the merge operator and the previously added switch operator, a conditional operator will be introduced in the next commit. TESTED=unit Change-Id: I8187e3e84488420e8735c62a4482e51a0dd78310

operators: Add a switch operator. This switch operator is used to forward input tensor to the output tensor determined by a given pred tensor. For example: pred = Tensor(data_layout=N, tensor_data= np.array([True], dtype=bool)) res_f, res_t = switch(input, pred) If the value of pred is true, input will be forwarded to res_t. Otherwise, res_f will be the output. This commit also introduces the concept of a dead tensor, which is a tensor on the untaken branch of a control flow. Like in the above example, since pred is true, res_f will become a dead tensor. When an operator is scheduled, if any of its inputs is dead, the run() method won't be invoked and instead, its outputs will be marked as dead tensors. TESTED=unit Change-Id: I143bc1251f388d11b6cc0a3e6d4401e2f1fc173f

2020-05-29

operators: Add less, less_equal, greater and greater_equal. This adds these four arithmetic operators for the Reference and SMV backends. These will be useful in implementing control flow operators. TESTED=unit Change-Id: I1db26d70a482e8b517c543fb96bb5e10a20856dc

core: Add a Bool type. This adds a Bool type as some of the operators now require a bool data type. Change-Id: Ia04d76d7879ede79423d8382aa14c16aef1e5d11

2020-05-15

core: Add support for serializing Tensor into TensorProto. This adds an asTensorProto() method to the Tensor class, which returns a TensorProto by serializing the Tensor. This feature is used to put the output tensor of the network into a proto buffer file. Ultimately, we can use this to communicate any intermediate results between Python and C++ runtime. Change-Id: I4a8e1cab472497a77372ac61121f454b07d04515

2020-05-14

common: Make an universal import/include convention. Now we use absolute import/include as much as possible. This makes the build folder looks like the below: build/ - bin/ - smaug # The built binary. - smaug/ # The source folder is copied to here. - operators/ - core/ - utility/ - ... - gem5 # The gem5 source files. - tracer # The tracer header file. The proto generated C++ files will also be copied to the build folder, while the generated Python files are in the original source folder. All tests still pass. Change-Id: Ie151c3f28108753f578b601b3970e96f70c01c8c

common: Reorganize the directory structure. This changes the directory structure into: smaug - build - docker - experiments - smaug # This was named nnet_lib. - arch - core - operators - python - utility - smaug.cpp - make # This was previously named common under nnet_lib. - third_party # This was in nnet_lib/src. The motivation is to expose SMAUG as a nice package to the outside world, so we can import SMAUG in Python files like the below: import smaug as sg from smaug.core import types_pb2 from smaug.python.graph import Graph from smaug.python.tensor import Tensor All the tests still pass. Change-Id: I938a4cf849f39abe57bb9a3cefe1adbd13074b05

2020-04-30

operators: Add a repeat operator. This operator constructs a tensor by repeating each dimension of the given tensor. TESTED=unit Change-Id: I0eee893ef65b3800bd619f6493e4a89d6c7eeca3

operators: Add a reshape operator. This changes the shape of a tensor. The tensor data remains intact. TESTED=unit Change-Id: If8a8f1602b6516016009c8b1830823aa5a4152e7

core: Add NCT and NTC layouts. The T dimension represents the time dimension that is required by RNN models. The inputs to an RNN model is ususally in [batch, depth, time] or [batch, time, depth] layouts, thus these two layouts are added. This also adds reordering support for NCT->NTC and NTC->NCT. Change-Id: Ia214b5cb1699957676a02c572c1110241a735666

2020-04-29

python,core: Fix proto when a node has multiple output tensors. The Node proto has parents/children field to indicate the connections between nodes. To restore the dataflow graph on the C++ side, the network_builder sets input tensors of a node to the corresponding output tensors of its parent nodes. It has the assumption that every node only has one output tensor, which isn't true anymore since we have operators like split. To fix this, we added a src_tensors_indices field in the Node proto, thus the network_builder can use it to correctly set the input tensors. TESTED=unit Change-Id: I721e310ad0eda6149b759847ffe8f295db29a6c7

2020-04-24

operators: Add a split operator. This adds a split operator. This operator splits an input tensor into sub tensors, which does the opposite of the previously added concat operator. LSTM also requires this operator for splitting the inputs into timesteps. TESTED=unit Change-Id: Icdf930a28b9abc0276c5d3d945eaeb6937f4b4b8

operators: Add a concatenate operator. This adds an operator for concatenation of tensors. The implementation supports arbitrary number of input tensors, which are concatenated on the specified axis. This is needed in LSTM. TESTED=unit Change-Id: Id060334fd45acc246b44769878018f13126013dc

operators: Add an elementwise multiplication operator. This operator is very common in the NLP models. This commit adds the kernel implementations for the reference and SMV backends. TESTED=unit Change-Id: Iac774571862f0e4c9a769d75a841c926488345a4

2019-09-25

python: Add support for specifying the host memory access policy in Python. This commit adds a HotMemoryAccessPolicy proto, which currently has 4 policies (i.e., AllDma, AllAcp, AllCache and AllAcpWithDmaForWeights). The user can specify the policy along with the model. Change-Id: I83a8c3d3e0c1726592e48822f27cf34e8f4c8f20

2019-06-05

python: Separated parameters from the network topology. This adds a proto (TensorDataArray) to separately store the parameters of the network. This separates the parameters from the network topology, so that we can dump them into two different files. The topology file will be a much smaller txt file, which can be easily reviewed. Change-Id: I7691da4b1dd648a8ead519b18b98118d29c8e7e4

2019-05-23

python: Add support for activation function fusion. This first adds the missing activation functions in the Python client, and then adds an activation option to the three operators that may need activation function fusion: convolution, inner product, and batch norm. The user is also allowed to change the parameters used by the fused activation functions via the activation parameters option, otherwise default values will be used. TESTED=unit Change-Id: I370518f8f918aab5d22d0febd4ddc91c284d239e

2019-05-08

core: Use reorder operator for weights transposition. This makes inner product weights transposition done ahead of the operator creation. Specifically, it does the following things: 1) Add a new DataLayout called CN. 2) Move the 2D tensor transposition into the reorder operator. For a reorder operator, if the source layout = NC (or CN) and the target layout = CN (or NC), the 2D tensor will be transposed. 3) At the Python side, we were already automatically inserting reorder ops for layout transformation. This commit augments this logic for weights transposition. Basically, if the current weight layout is different from the exepected one, a reorder op will be added. 4) Remove the weight transpositions in the SMV inner product tests. This also helps the next commit that adds the reference transposed inner product implementation. Testing is done together with that commit. Change-Id: I482704e8e7a3db932cf6944597734354c06a17db

2019-03-08

python-client (1/5): Add protos and change the Makefile. This adds the proto files and updates the build flow in Makefile. The generated C++ code is put into the build directory, like other C/C++ source. The generated Python code is in the src/python directory, from there we will add other Python client code in the subsequent patches. Also, rename graph.h to typedefs.h since we have a graph.proto which will generate a graph.pb.h. Change-Id: I01a3baefaf2e1f47bca3048e3439428e132cd652