Proto commits in firecracker-microvm/firecracker-containerd

These 36 commits are when the Protocol Buffers files have changed:

2024-05-02

Upgrade containerd to v1.7 Signed-off-by: Henry Wang <henwang@amazon.com>

The documentation is generated from this commit.

2022-05-02

Allow configuration of block_device cache strategy in firecracker (#615) Signed-off-by: Swagat Bora <sbora@amazon.com> Co-authored-by: Swagat Bora <sbora@amazon.com>

2022-03-17

Add VSockPath to GetVmInfoResponse Firecracker's virtio-vsock implementation bypasses the the host kernel by mediating between AF_UNIX on the host and AF_VSOCK in the guest. Firecracker-containerd places the host AF_UNIX socket path in the internal shim storage directory when launching a VM. Vsock is currently only used for communication between the shim and the in-VM agent. For some use-cases (e.g. remote snapshotters) it would be useful to have additional services listening on vsock inside the VM with corresponding host services to communicate with them. This change adds VSockPath to GetVmInfoResponse to allow a host service to get the host AF_UNIX socket path to open additional connections to vsock services in the guest. Signed-off-by: Kern Walster <walster@amazon.com>

2021-05-24

Memory ballooning support on firecracker-containerd Implement the memory ballooning APIs on firecracker-containerd Signed-off-by: Royce Zhao <qiqinzha@amazon.com>

2021-03-05

Re-establish a connection when vsock breaks IOProxy struct connects the container's stdio inside a micro VM to the FIFO files on its host. containerd clients consume the FIFO files to access the container's stdio. However, IOProxy was stopping itself when there was an error from the streams, such as EPIPE. In that case, reading from the FIFO files blocks indefinitely since there were no writers. This change adds ioProxyHandler and uses the handler on State() to make sure IOProxy is running. If not, ioProxyHandler#Attach() creates a new IOProxy instance. Fixes #482. Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>

2021-02-19

update proto and add pause, resume Signed-off-by: Plamen Petrov <plamb0brt@gmail.com>

2020-10-14

Make LogFifoPath and MetricsFifoPath configurable Before this change, firecracker-containerd was returning the paths of the files as a part of CreateVM's response. However the files are also useful to investage when CreateVM doesn't finish successfully. Fixes #449 Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>

2020-08-24

Make runc_jailer's behavior configurable regarding drive files runc_jailer implicitly copies all drive files to a jail directory before calling runc. While copying provides a stronger guarantee of isolation, some clients would want have more control on the drive files (e.g. use shred(1) for removing files). DriveExposePolicy allows client to use bind-mount instead of copying. Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>

2020-05-20

Make CreateVM's timeout configurable As like StopVM, CreateVM's timeout should be configurable to accomodate other root fs images and slower environments. Fixes #423. Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>

2020-05-12

Adding CgrouPath to CreateVMRequest This commit allows for configuration of the cgroup path. If no value is provided the default value of /firecracker-containerd will be used making the full cgroup path of /sys/fs/cgroup/cpu/firecracker-containerd/<vmID> Signed-off-by: xibz <impactbchang@gmail.com>

2019-12-13

Add support for UID and GID when jailing This adds UID and GID to the jailer config used in the CreateVMRequest object. This has the caveat that UID and GID cannot be zero and if they are an error will be returned Signed-off-by: xibz <impactbchang@gmail.com>

2019-12-04

Added Release() method to the to unmount and return stub to freeStubs list Signed-off-by: bpopovschi <zyqsempai@mail.ru>

2019-12-03

Add numa cpuset support This change adds support for specifying a set of cpu and memory nodes during the CreateVM call. This will run the jailed process on those set of cpus and restrict access to the specified nodes. This change also adds the cpuset.Builder which allows for easy creation of cpus and mems strings Signed-off-by: xibz <impactbchang@gmail.com>

2019-11-19

Support rate-limiters and read-only for drive mounts. Signed-off-by: Erik Sipsma <sipsma@amazon.com>

2019-11-18

Adds cgroup path to CreateVMResponse This commit adds a response to CreateVM. The CreateVMResponse contains the CgroupPath field that will be returned when jailing is enabled. Signed-off-by: xibz <impactbchang@gmail.com>

Add support for netns This change allows for users to pass in a custom network namespace in the create vm request. This will then pass that netns to the SDK which will run the VM in that netns. Signed-off-by: xibz <impactbchang@gmail.com>

2019-10-31

Add {Get,Update}VMMetadata calls to the fc-control api. This enables callers of the fc-control api to patch metadata instead of requiring a user to submit all the metadata everytime. Signed-off-by: Cody Roseborough <cdr@amazon.com>

2019-10-25

Support DriveMount API in CreateVM. This implements the proposal found in docs/drive-mounts-proposal.md, with the exception for supporting RateLimiters and IsReadOnly in the DriveMount objects, due to the need for further refactoring of the internal stub drive code (will be followed up in #296). Signed-off-by: Erik Sipsma <sipsma@amazon.com>

2019-10-24

Add jailer config to proto generation This commit adds the jailer config to the proto defintion as well as regenerating of the models. Signed-off-by: xibz <impactbchang@gmail.com>

2019-10-15

IsReadOnly -> IsWritable In proto3, all omitted boolean values are considered as false! This change allows makes the default mode of a drive read-only, even if the flag is omitted. Signed-off-by: Kazuyoshi Kato <katokazu@amazon.com>

2019-10-09

Added timeout for VMshutdown Signed-off-by: bpopovschi <zyqsempai@mail.ru>

2019-09-24

Add support for CNI-configured VM network interfaces. Signed-off-by: Erik Sipsma <sipsma@amazon.com>

2019-06-27

Exit shim upon Task delete, not exit. Issue #87 was previously addressed by having our shim process exit when all tasks it was managing exited. However, it turns out that is not the expected behavior for a containerd shim. The expected behavior is that a shim continue running until all tasks have been *deleted* by the client (via the containerd Delete API call). When any task has been deleted (or create of a task fails) containerd will send the shim a Shutdown API call, at which time the shim is expected to exit if it *all* tasks it was managing have been deleted. This commit fixes brings our shim in line with that expected behavior. It, however, also retains the existing FCControl parameter "ExitAfterAllTasksGone" (though now renamed to "ExitAfterAllTasksDeleted") to give users the ability to specify whether the default containerd shim behavior should be ignored and instead allow the Shim+VM to keep running even after all Tasks have been deleted. The majority of the changes here actually end up just being a refactor of the TaskManager interface to safely handle Create/Delete task APIs alongside checking whether the shim has any tasks left being managed (which is uses to decide if it should shutdown). The refactorization also ensures that all IO is flushed before a Delete call returns (which is a better solution to handling I/O races than the temporary fix applied in 1e36219). Signed-off-by: Erik Sipsma <sipsma@amazon.com>

2019-06-25

Adding drive ID to extra data message Adds drive ID to protobuf definition along with regeneration of the proto definitions. Signed-off-by: xibz <impactbchang@gmail.com>

2019-05-15

Replace FirecrackerConfig with CreateVM. Clients now just get the default VM options if they create a task without pre-creating a VM. For custom VM options, they use the CreateVM call. Signed-off-by: Erik Sipsma <sipsma@amazon.com>

Integrate fccontrol plugin with shim. This is a fairly significant reorganization that combines the work done on the fccontrol plugin with the shim. Significant aspects include: 1. The shim implements the fccontrol interface (over ttrpc) and handles the actual VM operations. The plugin is now only responsible for starting shims and forwarding fccontrol requests from clients to the correct shim (based on VMID). 2. The fccontrol plugin is now completely stateless (as a result of the above change). 3. A shim and its VM are now started together, so the shim start routine no longer directly starts a shim process; it just gets a VMID and issues a CreateVM call to the plugin. This lets the plugin take care of looking to see if the shim for the incoming VM already exists (and starting it if not). This change was biased towards keeping the behavior and interface unchanged, but there are a few significant updates from the previous state of things: 1. The shim no longer by default shuts down when all tasks have exited; instead it continues to run until either StopVM is called or Shutdown is called when no tasks are left. Its lifecycle and the VM lifecycle are now the same essentially. * This results in less surprising behavior; a shim is created via a CreateVM call and can exist without any tasks, so it makes sense that it should be able to continue to exist after each task has exited to. * Otherwise, it's easy for users to, for example, create a VM and issue several CreateTask calls for that VM, but the first Task is created and exits quickly before the other tasks were setup, which results in the shim exiting and the rest of the tasks failing to be created. * For situations where the previous behavior is more desirable, a new CreateVM parameter, ExitAfterAllTasksGone, can be set to true. This is used by the shim start routine when the client did not specify any VMID (and thus don't care about VMs, just their task). 2. You can no longer pass Firecracker machine options as part of the CreateTask extra data field. If you create a task with a VMID that doesn't exist yet or no VMID at all, the VM will be created using the default options as configured in the runtime's config file. * This greatly simplifies the amount of branches our code has to handle and also greatly simplifies the interface we are giving to users without reducing the functionality. If users want custom VM options, they just need to use the CreateVM call. If they don't care about custom VM options, they can continue to just issue a CreateTask request and get the defaults specified in our runtime config file. Signed-off-by: Erik Sipsma <sipsma@amazon.com>

2019-05-07

Re-use proto messages for both ttrpc and grpc. Before this change, each protobuf message type was generated for both grpc and ttrpc. However, if both definitions get loaded into the same process (i.e. a process that's translating between the two such as the fccontrol plugin), the protobuf names will conflict with each other. This change results in protobuf message types being generated only once. Now only the service definitions are generated separately for both grpc and ttrpc. Signed-off-by: Erik Sipsma <sipsma@amazon.com>

2019-04-26

Support VM Agent managing multiple containers. Signed-off-by: Erik Sipsma <sipsma@amazon.com>

2019-04-23

Implement get VM info endpoint Signed-off-by: Maksym Pavlenko <makpav@amazon.com>

2019-04-15

Publish containerd events Signed-off-by: Maksym Pavlenko <makpav@amazon.com>

Expect VM ID from client instead of generating Signed-off-by: Maksym Pavlenko <makpav@amazon.com>

2019-04-08

Update protobuf definitions Add GetFifoPath and SetVMMetadata Signed-off-by: Maksym Pavlenko <makpav@amazon.com>

2019-04-02

Add protobuf definitions for GRPC plugin Signed-off-by: Maksym Pavlenko <makpav@amazon.com>

2019-02-21

add firecracker VM options as per #76 This commit extends the FirecrackerConfig protobuf message to add the following fields: * CPU count * CPU template * Memory * Additional storage devices (with rate limiters) * Network devices (with rate limiters) * Kernel Signed-off-by: Anirudh Aithal <aithal@amazon.com>

2019-02-15

use CreateTaskRequest's Options field to pass firecracker config args A new proto message called FirecrackerConfig has been added to let clients pass in various firecracker VM config options to firecracker-containerd. Currently it consists of: 1. runc options 2. firecracker network config This can be extended in the future to pass in additional arguments. Signed-off-by: Anirudh Aithal <aithal@amazon.com>

2018-11-21

Write config.json into the guest Write config.json into the guest by passing it via existing Options field inside CreateTaskRequest. Signed-off-by: Cody Roseborough <crrosebo@amazon.com>