A protobuf (https://developers.google.com/protocol-buffers/) compiler plugin designed to simplify the API calls needed to perform simple object persistence tasks. This is currently accomplished by creating a secondary .pb.gorm.go file which contains sister objects to those generated in the standard .pb.go file satisfying these criteria:
The protobuf compiler (protoc) is required.
Get the golang protobuf code generator:
Before Go 1.17
go get -u github.com/golang/protobuf/protoc-gen-go
Starting from Go 1.17
go install github.com/golang/protobuf/protoc-gen-go@latest
Retrieve and install the vendored dependencies for this project with go mod:
go mod tidy
To use this tool, install it from code with make install, go install github.com/infobloxopen/protoc-gen-gorm@latest directly, or go get github.com/infobloxopen/protoc-gen-gorm.
Once installed, the --gorm_out=. or --gorm_out=${GOPATH}src option can be specified in a protoc command to generate the .pb.gorm.go files.
Any message types with the option (gorm.opts).ormable = true will have the following autogenerated:
[(gorm.field).tag = {..., tag: value, ...}].option (gorm.opts) = {include: []}, either of a built-in type e.g. {type: "int32", name: "secret_key"}, or an imported type, e.g. {type: "StringArray", name: "array", package:"github.com/lib/pq"}.Any services with the option (gorm.server).autogen = true will have basic grpc server generated:
Create|Read|Update|Delete generated implementation will call basic CRUD handlers.return &MethodResponse{}, nil stub is generated.For CRUD methods to be generated correctly you need to follow specific conventions:
payload, for Read and Delete methods an id field is required. Nothing is required in the List request.result and for List a repeated Ormable Type named results.(gorm.method).object_type option to indicate which Ormable Type it should delete, and has no response type requirements.To customize the generated server, embed it into a new type and override any desired functions.
If conventions are not met stubs are generated for CRUD methods. As seen in the feature_demo/demo_service example.
To leverage DB specific features, specify the DB engine during generation using the --gorm_out="engine={postgres,...}:{path}". Currently only Postgres has special type support, any other choice will behave as default.
The generated code can also integrate with the grpc server gorm transaction middleware provided in the atlas-app-toolkit using the service level option option (gorm.server).txn_middleware = true.
Example .proto files and generated .pb.gorm.go files are included in the 'example' directory. The user file contains model examples from GORM documentation, the feature_demo/demo_types demonstrates the type handling and multi_account functions, and the feature_demo/demo_service shows the service autogeneration.
Running make example will recompile all these test proto files, if you want to test the effects of changing the options and fields.
Within the proto files, the following types are supported:
uint32, uint64, int32, int64, float, double, bool, string map to the same type at ORM levelgoogle.protobuf.StringValue, .BoolValue, .UInt32Value, .FloatValue, etc. map to pointers of the internal type at the ORM level, e.g. *string, *bool, *uint32, *floatgoogle.protobuf.Timestamp maps to time.Time type at the ORM levelgorm.types.UUID and gorm.types.UUIDValue, which wrap strings and convert to a uuid.UUID and *uuid.UUID at the ORM level, from https://github.com/satori/go.uuid. A null or missing gorm.types.UUID will become a ZeroUUID (00000000-0000-0000-0000-000000000000) at the ORM level.gorm.types.JSONValue, which wraps a string in protobuf containing arbitrary JSON and converts to custom types.Jsonb type if Postgres is the selected DB engine, otherwise it is currently dropped.gorm.types.InetValue, which wraps a string and will convert to the types.Inet type at ORM level, which uses the golang net.IPNet type to hold an ip address and mask, IPv4 and IPv6 compatible, with the scan and value functions necessary to write to DBs. Like JSONValue, currently dropped if DB engine is not PostgresThe plugin supports the following GORM associations:
Note: polymorphic associations are currently not supported.
Association is defined by adding a field of some ormable message type(either single or repeated).
message Contact {
option (gorm.opts).ormable = true;
uint64 id = 1;
string name = 2;
repeated Email emails = 3;
Address home_address = 4;
}
Has-One is a default association for a single message type.
message Contact {
option (gorm.opts).ormable = true;
uint64 id = 1;
string first_name = 2;
string middle_name = 3;
string last_name = 4;
Address address = 5;
}
message Address {
option (gorm.opts).ormable = true;
string address = 1;
string city = 2;
string state = 3;
string zip = 4;
string country = 5;
}
Set (gorm.field).belongs_to option on the field in order to define Belongs-To.
message Contact {
option (gorm.opts).ormable = true;
uint64 id = 1;
string first_name = 2;
string middle_name = 3;
string last_name = 4;
Profile profile = 5 [(gorm.field).belongs_to = {}];
}
message Profile {
option (gorm.opts).ormable = true;
uint64 id = 1;
string name = 2;
string notes = 3;
}
Has-Many is a default association for a repeated message type.
message Contact {
option (gorm.opts).ormable = true;
uint64 id = 1;
string first_name = 2;
string middle_name = 3;
string last_name = 4;
repeated Email emails = 5;
}
message Email {
option (gorm.opts).ormable = true;
string address = 1;
bool is_primary = 2;
}
Set (gorm.field).many_to_many option on the field in order to define Many-To-Many.
message Contact {
option (gorm.opts).ormable = true;
uint64 id = 1;
string first_name = 2;
string middle_name = 3;
string last_name = 4;
repeated Group groups = 5 [(gorm.field).many_to_many = {}];
}
message Group {
option (gorm.opts).ormable = true;
uint64 id = 1;
string name = 2;
string notes = 3;
}
For each association type, except Many-To-Many, foreign keys pointing to primary keys(association keys) are automatically created if they don't exist in proto messages, their names correspond to GORM default foreign key names. GORM association tags are also automatically inserted.
foreignKey and references options.disable_association_autoupdate, disable_association_autocreate options, which effectively produces statements to prevent Association auto behavior (much like in tags were used in gormV1). Check out official association docs for more information.preload option, which generates required code (with the same meaning gormV1 preload tag was working). Check out GORM docs.append ([GORM]https://gorm.io/docs/associations.html#Append-Associations), clear ([GORM]https://gorm.io/docs/associations.html#Clear-Associations) and replace ([GORM]https://gorm.io/docs/associations.html#Replace-Associations).position_field so additional field is created if it doesn't exist in proto message to maintain association ordering. Corresponding CRUDL handlers do all the necessary work to maintain the ordering.foreignkey_tag and position_field_tag options.jointable, joinForeignKey and joinReferences options.Check out user to see a real example of associations usage.
Currently only proto3 is supported.
This project is currently in development, and is expected to undergo "breaking" (and fixing) changes
Pull requests are welcome!
Any new feature should include tests for that feature.
Before pushing changes, run the tests with:
make gentool-test
This will run the tests in a docker container with specific known versions of dependencies.
Before the tests run, they generate code. Commit any new and modified generated code as part of your pull request.