Proto commits in colinmarc/magic-mirror

These 34 commits are when the Protocol Buffers files have changed:

2025-02-12

docs(protocol): fix some spelling errors

The documentation is generated from this commit.

2025-02-10

chore(mm-protocol): bump version

feat!: send hierarchical_layer as video frame metadata The motivation for this (breaking) change is that the `frame_optional` field doesn't provide enough information to the decoder on all platforms. In particular, VideoToolbox on Mac, when confronted with a dropped frame, breaks if you send it any frames depending on it until it's recovered. If there are more than two layers, the middle layer isn't exactly optional, but it's only necessary to exclude higher layers until we recover.

2025-01-24

feat!: implement forward error correction Forward error correction, or FEC, allows us to spend bandwidth and a little bit of CPU and latency to increase the robustness of the video stream in the face of lost packets. In this first simple implementation, we use an inexpensive scheme called RaptorQ, set at a constant level of overhead for each video layer in the server config.

feat!: implement forward error correction Forward error correction, or FEC, allows us to spend bandwidth and a little bit of CPU to increase the robustness of the video stream in the face of lost packets. In this first simple implementation, we use an inexpensive scheme called RaptorQ to protect the base layer of a video stream (assuming hierarchical coding has created multiple layers). This causes us to send 15% more data for those frames, with the advantage that the frames are recoverable with up to 15% packets missing.

feat: explicit video refresh Instead of regularly sending headers and keyframes every few seconds, the idea here is to send an endless stream of P frames, each referencing the previous one. This saves some bandwidth, but means that a client which "falls off" the chain has no way of (automatically) recovering. To fix that, the client needs an explicit way to request a keyframe/header packet, which is added here as an attachment message. This is much more robust for the client than just waiting for a refresh to come along, and results in shorter pauses when an important frame is dropped. Because we use hierarchical coding (for h265), we take care to indicate if a frame is actually important, so that clients can avoid requesting refreshes when the stream is already recoverable.

2024-12-12

docs: wrap line

docs: fix markdown link

docs: generate internal links for protocol reference

2024-12-05

feat: add header images to the application list

2024-11-30

feat: add header images to the application list

2024-11-28

refactor: rename application 'name' to 'id' This should not break existing configs, and is wire compatible.

feat: add 'app_path' for organizing apps

2024-10-24

fix(protocol): use the attachment coordinate space This is cleaner from a protocol design perspective, and currently it makes no difference, since we only support attaching at the session's native resolution.

2024-10-21

feat: refactor out most of mmclient into a UniFFI rust lib The new lib, called "mm-client-common", is intended to be used in more clients.

chore(protocol): use a custom timestamp struct instead of google's This is wire-compatible with the old version, but doesn't require the dependency. We want to export protocol structs as UniFFI records, which is difficult with imported types.

fix(client): correctly invert joystick direction gilrs (the rust input library we're using) inverts up and down. This is also underspecified at the protocol level. Finally, we were also sending the right trigger incorrectly.

2024-10-15

feat: refactor out most of mmclient into a UniFFI rust lib The new lib, called "mm-client-common", is intended to be used in more clients.

chore(protocol): use a custom timestamp struct instead of google's This is wire-compatible with the old version, but doesn't require the dependency. We want to export protocol structs as UniFFI records, which is difficult with imported types.

2024-10-14

fix(client): correctly invert joystick direction gilrs (the rust input library we're using) inverts up and down. This is also underspecified at the protocol level. Finally, we were also sending the right trigger incorrectly.

feat: allow specifying 'permanent' gamepads for a session

2024-09-13

docs(protocol): be picky about indentatation

2024-09-12

fix(protocol): rename Gamepad* enums to reduce the possibility of collision

feat: plumb controller input through to the server This doesn't actually do anything with it yet, though.

2024-09-08

docs(protocol): be consistent around events sent on the att stream

2024-08-01

feat(protocol): allow attachments to be configured for HDR10 output

2024-05-07

feat: add --preset, for setting quality/bandwidth usage dynamically In particular, this makes it a first-class concept, and allows clients to set the quality level.

feat: cursor locking and relative motion First-person games generally work by locking the cursor to the center of the screen and using relative motion to look around. This requires some additions to the protocol, as well as two wayland protocols.

2024-04-30

feat(protocol): allow attachments to be configured for HDR10 output

2024-04-24

docs(protocol): add a note about protocol semver

2024-04-19

feat: cursor push This allows the server to push the app's custom cursor all the way to the client. It works for both "named" cursors and cursor images (like custom cursors in games) but only via the shm wayland interface at the moment.

2024-04-18

feat: support scroll wheel and touchpad input

2024-04-14

fix(protocol): remove server_side_cursor_enabled It's not checked anywhere.

2024-03-08

Initial commit