Proto commits in livekit/rust-sdks

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

Commit:bddbe06
Author:Ryan Gaus
Committer:Ryan Gaus

feat: allow data stream compress option to be toggled via livekit-ffi

Commit:c92a56f
Author:Ryan Gaus

feat: allow data stream compress option to be toggled via livekit-ffi

Commit:51a3ada
Author:Jacob Gelman

Merge remote-tracking branch 'origin/main' into ladvoc/schema-metadata

Commit:18033b9
Author:shijing xian

fix: correct DegradationPreference docstrings to match WebRTC semantics The docstrings for MaintainFramerate and MaintainResolution were swapped. Per the W3C WebRTC spec: - MaintainFramerate: Degrade resolution to maintain framerate (prioritize smooth motion) - MaintainResolution: Degrade framerate to maintain resolution (prioritize image clarity) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Commit:2e83ff6
Author:xianshijing-lk
Committer:GitHub

chore: remove deprecated DEGRADATION_PREFERENCE_DISABLED from proto (#1201) ## Summary - Remove the deprecated `DEGRADATION_PREFERENCE_DISABLED` (value 3) from the `DegradationPreference` enum in `room.proto` - Update the conversion code in `room.rs` to remove the `Disabled` match arm ## Rationale The `Disabled` value was deprecated in favor of `MAINTAIN_FRAMERATE_AND_RESOLUTION` which provides the same behavior. Since `DegradationPreference` is a new feature being added to the FFI protocol, there's no backwards compatibility concern - no existing clients are using the `Disabled` value. This aligns the proto enum with the Rust `libwebrtc::rtp_parameters::DegradationPreference` enum which only has: - `Balanced` - `MaintainFramerate` - `MaintainResolution` - `MaintainFramerateAndResolution` ## Test plan - [x] `cargo check -p livekit-ffi` passes πŸ€– Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

The documentation is generated from this commit.

Commit:dd7e1d3
Author:shijing xian

chore: remove deprecated DEGRADATION_PREFERENCE_DISABLED from proto Remove the deprecated DEGRADATION_PREFERENCE_DISABLED (value 3) from the DegradationPreference enum in room.proto. This value was deprecated in favor of MAINTAIN_FRAMERATE_AND_RESOLUTION which provides the same behavior. Since DegradationPreference is a new feature being added to the FFI protocol, there's no backwards compatibility concern - no existing clients are using the Disabled value. This aligns the proto enum with the Rust libwebrtc::rtp_parameters::DegradationPreference enum which only has: Balanced, MaintainFramerate, MaintainResolution, and MaintainFramerateAndResolution. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The documentation is generated from this commit.

Commit:db712fa
Author:xianshijing-lk
Committer:GitHub

feat: add MaintainFramerateAndResolution to DegradationPreference enum (#1200) ## Summary Aligns the DegradationPreference enum with WebRTC M144 changes: - Add `MAINTAIN_FRAMERATE_AND_RESOLUTION` as the new recommended value - Deprecate `DISABLED` (maps to `MAINTAIN_FRAMERATE_AND_RESOLUTION` for backwards compatibility) Reference: https://github.com/webrtc-sdk/webrtc/blob/m144_release/api/rtp_parameters.h#L125 ## Changes - **room.proto**: Add `DEGRADATION_PREFERENCE_MAINTAIN_FRAMERATE_AND_RESOLUTION` enum value - **webrtc-sys**: Rename `Disabled` to `MaintainFramerateAndResolution` in the repr(i32) enum - **libwebrtc**: Add `MaintainFramerateAndResolution` variant, deprecate `Disabled` as a const alias - **livekit-ffi**: Update conversion to handle both old `DISABLED` and new `MAINTAIN_FRAMERATE_AND_RESOLUTION` proto values ## Backwards Compatibility - Existing code using `DISABLED` will continue to work (mapped to `MaintainFramerateAndResolution`) - FFI clients can use either value, both map to the same internal behavior ## Test plan - [x] `cargo build -p livekit-ffi` passes - [x] `cargo build -p livekit` passes - [x] `cargo test -p livekit --lib -- degradation` passes πŸ€– Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:d697b47
Author:xianshijing-lk
Committer:GitHub

Fix initial video quality blurriness for all codecs (#1197) Summary Fixes initial video quality blurriness by: 1. Applying x-google-start-bitrate SDP hint for all video codecs (VP8, VP9, AV1, H264, H265) 2. Defaulting DegradationPreference to MaintainResolution for all video tracks Problem Users report video starting blurry for 5-15 seconds before reaching full quality (https://www.loom.com/share/9a34ae565f6949b891a155a436b3d573). This happens because: 1. Slow bandwidth ramp-up: x-google-start-bitrate was only set for VP9/AV1, so VP8/H264 streams start at WebRTC's conservative default (~300 kbps) and ramp up slowly. 2. Resolution degradation: Default DegradationPreference was Balanced, causing the encoder to reduce resolution during the ramp-up period. Changes livekit/src/rtc_engine/peer_transport.rs - Extended x-google-start-bitrate SDP munging to all video codecs (VP8, VP9, AV1, H264, H265) - Updated tests libwebrtc/src/rtp_parameters.rs - Added DegradationPreference enum - Added set_degradation_preference() and degradation_preference() methods to RtpParameters livekit/src/room/options.rs - Added degradation_preference field to TrackPublishOptions - Added get_default_degradation_preference() that defaults to MaintainResolution - Re-exported DegradationPreference for users livekit/src/room/participant/local_participant.rs - Apply degradation preference to video tracks after creating the transceiver Behavior β”‚ Setting β”‚ Before β”‚ After β”‚ β”‚ x-google-start-bitrate β”‚ VP9/AV1 only β”‚ All video codecs β”‚ β”‚ DegradationPreference β”‚ Not set (WebRTC default) β”‚ MaintainResolution β”‚ Users can override degradation_preference in TrackPublishOptions if they prefer Balanced or MaintainFramerate. Testing - Added unit tests for DegradationPreference defaults - Updated x-google-start-bitrate tests for all codecs - Manual testing with local_video example --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:903dc56
Author:shijing xian

added ffi changes

Commit:d8c646b
Author:David Chen
Committer:GitHub

add user_data trailer type (#1180)

Commit:f87ac95
Author:Jacob Gelman

Expose custom encoding identifiers over FFI

Commit:6f108fd
Author:lukasIO
Committer:GitHub

harden reconnect behaviour (#1148) ### Before you submit your PR Make sure the following is true before submitting your PR: - [ ] I have read the [contributing guidelines](https://github.com/livekit/rust-sdks/blob/main/CONTRIBUTING.md) and validated that this PR will be accepted. - [ ] I have read and followed the principles regarding breaking changes, testing, and code quality. ### PR description Describe the changes in this PR. Explain what the PR is meant to solve and how to reproduce the issue in the first place. ### Breaking changes If this PR introduces breaking changes, list them here and document the rationale for introducing such a change. ### MSRV If the PR modifies the crate's MSRV (Minimum Supported Rust Version), document it here. ### Testing Ideally, unit test the code you add, but ensure you're not repeating existing test cases. Use as many already written scaffolding, utilities as possible; write your own, when needed. If external services, APIs, tokens are required (e.g., running an LK server instance), provide the necessary information. Make sure your tests perform useful, context-aware assertions and do not simply emulate "happy paths". ### Async We want the project to be runtime-agnostic, so please reuse what's already in [livekit-runtime](https://github.com/livekit/rust-sdks/blob/main/livekit-runtime/) and feel free to add anything missing. It's ok to use Tokio directly, when writing unit tests, if necessary. When testing, do not use artificial delays for the state to "catch up"; instead, respect the event flow and subscribe properly using channels or other mechanisms. --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:14af23c
Author:Jacob Gelman
Committer:Jacob Gelman

Merge remote-tracking branch 'origin/main' into ladvoc/schema-metadata

Commit:3a37ba6
Author:David Chen
Committer:David Chen

add user_data trailer type

Commit:fb65760
Author:David Chen
Committer:GitHub

rename Packet Trailer to Frame Metadata (#1179) - rename the PacketTrailerFeatures enum to FrameMetadataFeatures for clarity

Commit:0579e4f
Author:David Chen

add user_data trailer type

Commit:e63fb1e
Author:David Chen

rename all Packet Trailer to Frame Metadata

Commit:049a4ed
Author:lukasIO

Add DisconnectSignalOnResume scenario to FFI variants

Commit:a271c9d
Author:Jacob Gelman
Committer:Jacob Gelman

Expose data track fields over FFI

Commit:d76e0c5
Author:David Chen

reliable data track experiment

Commit:127288a
Author:Stephen DeRosa
Committer:Stephen DeRosa

webrtc-sys: cache and auto-prepend H.264/H.265 parameter sets Encoded track source now scans incoming frames for SPS/PPS (H.264) or VPS/SPS/PPS (H.265), caches the latest seen set, and prepends them to any keyframe that arrives without inline params. This makes hardware encoders and camera feeds that only emit parameter sets on stream start usable as-is, without requiring producers to replicate them on every IDR. Producers still get a clear warning if the very first keyframe has no parameter sets and the cache is empty. The caller-supplied has_sps_pps flag becomes a hint only; the scanner is the source of truth so double-prepending is impossible. Also fix a stale `src->get()` reference left over from the SetRates refactor in PassthroughVideoEncoder::Encode. examples: H.264, H.265, VP8, AV1. VP9 not supported yet

Commit:da3bba9
Author:Stephen DeRosa
Committer:Stephen DeRosa

livekit: publish encoded video tracks Dispatch RtcVideoSource::Encoded through the new PCF path in LocalVideoTrack, and normalize TrackPublishOptions for encoded sources in LocalParticipant::publish_track β€” simulcast is forced off and the codec is pinned to the source's codec, with warnings on override.

Commit:48cd8a1
Author:Jacob Gelman

Expose over FFI

Commit:b1cc204
Author:Max Heimbrock

platform_audio: add device-selection capability + explicit mobile no-op Device selection is only meaningful on desktop; on iOS/Android the OS owns audio routing (AVAudioSession / AudioManager). Previously the "no-op on mobile" was accidental β€” a fallback in the native layer selected index 0 whenever a GUID didn't match β€” which also silently swallowed a real desktop error (a saved device that no longer exists). - proto: add PlatformAudioInfo.device_selection_supported. - PlatformAudio::supports_device_selection() β€” false on iOS/Android, true on desktop. Lets clients hide device-picker UI where selection has no effect. - set_/switch_ recording/playout device short-circuit to Ok(()) on mobile before touching native, so the no-op is explicit rather than dependent on native device-reporting quirks (iOS RecordingDeviceName -1; Android empty GUID). - ffi server populates the new field from the accessor. - webrtc-sys: drop the index-0 fallback in set_*_device_by_guid so an unknown GUID returns false -> DeviceNotFound on desktop; mobile never reaches it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Commit:a5252b4
Author:David Chen
Committer:GitHub

adding support to setting preferred encoder on video track publication (#1126) - Add field to `TrackPublishOptions` that allows the user to set a preferred video encoder when there are multiple options. - Options can be `auto`, `software`, `hardware`, `nvenc`, `vaapi`, `videotoolbox`. --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:2434b45
Author:Ryan Gaus
Committer:GitHub

Add max round trip latency (#1127) Adds a new `max_round_trip_latency` field to `PerformRpcData`, adding similar behavior as was added to the corresponding [swift](https://github.com/livekit/client-sdk-swift/pull/1023) and [android](https://github.com/livekit/client-sdk-android/pull/953) pull requests. Note that this is a **breaking api change**. Per @ladvoc's guidance, this was deemed to be ok because there isn't really a great way to add this new field to the `PerformRpcData` struct in a backwards compatible way. Given this is a breaking change, I opted to break it in a way where if we have to add _more_ fields in here in the future, it could be done in a non breaking way by making `PerformRpcData` use a builder pattern. That results in something that looks like: ```rust let output = caller_room.local_participant().perform_rpc( PerformRpcData::new("identity", "method") // <-- New builder .with_payload("payload") .with_response_timeout(Duration::from_secs(5)) ).await?; ``` --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: lukasIO <mail@lukasseiler.de>

Commit:940e9a8
Author:David Chen

Merge branch 'main' into dc/feature/track_encoder_option

Commit:8c12be6
Author:lukasIO
Committer:GitHub

bump protocol to v1.46.4 and expose client_protocol via FFI (#1121) ### PR description - bumps the protocol to v1.46.4 and adds the `livekit_analytics.proto` messages to the generated stubs. - exposes participant `client_protocol` via FFI - Adds a `remove_participant_with_options` method that allows to pass new remove options to the API without a previous `remove_participant` call breaking. ### Breaking changes for the `livekit-api` crate I've added a `derive(default)` and an additional (option) field to the EgressListOptions struct. I don't see a good way around this though. Open for suggestions. --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:3bfb373
Author:David Chen

adding support to setting preferred encoder on video track publication

Commit:76d6008
Author:David Chen

rename to video_encoder

Commit:f3ec2f7
Author:David Chen
Committer:David Chen

add API to select video encoder

Commit:dbc50db
Author:xianshijing-lk
Committer:GitHub

fix Android PlatformAudio (#1104) ### Before you submit your PR Make sure the following is true before submitting your PR: - [x] I have read the [contributing guidelines](https://github.com/livekit/rust-sdks/blob/main/CONTRIBUTING.md) and validated that this PR will be accepted. - [x] I have read and followed the principles regarding breaking changes, testing, and code quality. ### PR description ⏺ Android PlatformAudio: Problems & Solutions Problem 1: ADM Creation Failed Symptom: CreateAudioDeviceModule returned nullptr Failed to initialize PlatformAudio Root Cause: WebRTC's CreateAndroidAudioDeviceModule internally calls GetAppContext() to get the Android application context. But GetAppContext() returns null unless ContextUtils.initialize(context) was called first. CreateAndroidAudioDeviceModule() └── GetAppContext() └── Returns null ❌ (ContextUtils not initialized) Solution: Pass the Android Context from Unity to Rust, then call ContextUtils.initialize(context) before any WebRTC audio initialization. Unity Rust/C++ ───── ──────── GetApplicationContext() ──────► livekit_ffi_initialize_android_context() β”‚ β”œβ”€β”€ do_android_init(vm) // JVM init └── ContextUtils.initialize(context) // Now GetAppContext() works! Files changed: - livekit-ffi/src/cabi.rs - Added livekit_ffi_initialize_android_context() - webrtc-sys/src/android.cpp - Added init_android_context() - client-sdk-unity/.../FFIClient.cs - Calls the init with context - client-sdk-unity/.../NativeMethods.cs - P/Invoke declaration --- Problem 2: Wrong ADM Factory Function Symptom: AdmProxy: All Android audio layers failed! Root Cause: The code was using the generic CreateAudioDeviceModule() which doesn't work properly on Android. Android requires CreateAndroidAudioDeviceModule() from the Android-specific header. Solution: Changed adm_proxy.cpp to use the correct function: // Before (wrong): #include "modules/audio_device/include/audio_device.h" platform_adm_ = webrtc::AudioDeviceModule::Create(...); // After (correct): #include "sdk/android/native_api/audio_device_module/audio_device_android.h" platform_adm_ = webrtc::CreateAndroidAudioDeviceModule(...); File changed: - webrtc-sys/src/adm_proxy.cpp --- Problem 3: Empty GUID Validation Failed Symptom: Recording device at index 0 has no GUID Root Cause: Unity SDK threw an exception when device GUID was empty. But on Android, devices don't have GUIDs - WebRTC returns empty strings. Solution: Allow empty GUIDs and pass them to native code, which falls back to index 0: // Before: if (string.IsNullOrEmpty(deviceId)) throw new InvalidOperationException("has no GUID"); // After: SetRecordingDevice(deviceId ?? ""); // Empty string triggers fallback File changed: - client-sdk-unity/.../PlatformAudio.cs --- Problem 4: Device Not Found with Empty GUID Symptom: Failed to set recording device: Device not found Root Cause: Native code searched for a device matching the empty GUID string, found nothing. Solution: Added fallback in C++ - if GUID is empty and devices exist, use index 0: bool AudioDeviceController::set_recording_device_by_guid(rust::String guid) const { int16_t count = adm_proxy_->RecordingDevices(); // Android fallback: empty GUID β†’ use index 0 if (guid.empty() && count > 0) { return adm_proxy_->SetRecordingDevice(0) == 0; } // ... normal GUID lookup for desktop } File changed: - webrtc-sys/src/audio_device_controller.cpp ### Breaking changes N/A ### MSRV N/A ### Testing Tested with Unity ### Async N/A --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:1972db7
Author:Jacob Gelman
Committer:GitHub

Improved data tracks depacketizer to support mutliple in flight packets (#1101) Equivalent to the implementation in: https://github.com/livekit/client-sdk-js/pull/1923 Closes BOT-342 --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:2dee8d3
Author:Alan George

WIP wiring up for flushing Rust logs

Commit:d74892d
Author:Jacob Gelman
Committer:GitHub

FFI room events: only sent after receiving ReadyForRoomEventRequest after initial connection (#1068) Resolves BOT-344 --------- Co-authored-by: Stephen DeRosa <stephen.derosa@livekit.io> Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:a548c0e
Author:Jacob Gelman
Committer:GitHub

Return EOS event from data track stream read request (#1093) Resolves BOT-347 --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:03ea826
Author:xianshijing-lk
Committer:GitHub

change the sample rate and num of channel optional in the proto (#1097) ### Before you submit your PR Make sure the following is true before submitting your PR: - [x] I have read the [contributing guidelines](https://github.com/livekit/rust-sdks/blob/main/CONTRIBUTING.md) and validated that this PR will be accepted. - [x] I have read and followed the principles regarding breaking changes, testing, and code quality. ### PR description change the sample rate and num of channel optional in the proto Those fileds are not needed by the PlatformAudio source ### Breaking changes No ### MSRV N/A ### Testing CI ### Async N/A --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:038c7b9
Author:xianshijing-lk
Committer:GitHub

get webrtc adm into rust (#1037) Summary This PR implements Platform Audio support for the LiveKit Rust SDK, enabling WebRTC's built-in audio device handling with microphone capture and speaker playout. The implementation introduces a handle-based PlatformAudio API that coexists with the existing NativeAudioSource for manual audio pushing. Key Features - Two Audio Source Types: - RtcAudioSource::Native (default): Manual audio push via NativeAudioSource for TTS, file streaming, agents - RtcAudioSource::Device: WebRTC handles mic capture & speaker playout with echo cancellation (AEC) - Handle-based API: Create PlatformAudio instances that enable ADM recording; drop to release - Reference Counting: Multiple PlatformAudio instances share the same underlying ADM - Device Enumeration & Selection: List and select recording/playout devices - Hot-swap Device Switching: switch_recording_device() / switch_playout_device() for changing devices during active sessions - Audio Processing Configuration: AEC, AGC, NS with hardware/software preference - WebRTC Patching: external_audio_source.patch prevents audio mixing conflicts between device and manual sources Design Document See docs/ADM_PROXY_DESIGN.md for full architecture details including: - Recording gate pattern - WebRTC patching explanation - FFI API documentation API Overview use livekit::prelude::*; // Create PlatformAudio instance (enables ADM recording) let audio = PlatformAudio::new()?; // Enumerate and select devices for i in 0..audio.recording_devices() as u16 { println!("Mic [{}]: {}", i, audio.recording_device_name(i)); } audio.set_recording_device(0)?; // Connect and publish let (room, _) = Room::connect(&url, &token, RoomOptions::default()).await?; let track = LocalAudioTrack::create_audio_track("mic", audio.rtc_source()); room.local_participant().publish_track(LocalTrack::Audio(track), opts).await?; // Cleanup - just drop the handle room.close().await?; drop(audio); // ADM recording disabled when all handles released Testing Run Standalone Tests (no LiveKit server required) # Set custom WebRTC build path export LK_CUSTOM_WEBRTC="/path/to/webrtc-sys/libwebrtc/mac-arm64-debug" # Run standalone PlatformAudio tests cargo test -p livekit --test platform_audio_test test_platform_audio_standalone -- --nocapture # Run FFI request handler tests cargo test -p livekit-ffi requests::tests -- --nocapture Run E2E Integration Tests (requires LiveKit server) # Start a local LiveKit server first, then: LIVEKIT_URL=ws://localhost:7880 \ LIVEKIT_API_KEY=devkey \ LIVEKIT_API_SECRET=secret \ cargo test -p livekit --test platform_audio_test --features __lk-e2e-test -- --nocapture Test Coverage Category │ Tests │ Description │ Standalone - Creation │ 1 │ PlatformAudio creation, device enumeration Standalone - Ref Counting │ 1 │ Clone, sharing, drop behavior Standalone - Device Selection │ 1 │ Set devices, invalid index handling Standalone - Processing │ 1 │ AEC/AGC/NS configuration, hardware availability Standalone - Reset │ 1 │ reset_platform_audio() function Standalone - Lifecycle │ 1 │ Full create→configure→use→release cycle FFI - Handlers │ 6 │ NewPlatformAudio, GetDevices, SetDevice, handle lifecycle E2E - Room Connection │ 4+ │ Platform audio with room, two participants, device switching All tests handle missing audio devices gracefully (CI-friendly). Run the Example List Audio Devices cargo run -p basic_room -- --list-devices Connect with Platform Audio (microphone capture) LIVEKIT_URL=wss://your-server.livekit.cloud \ LIVEKIT_API_KEY=your-key \ LIVEKIT_API_SECRET=your-secret \ cargo run -p basic_room -- --platform-audio Connect with File Audio cargo run -p basic_room -- --file path/to/audio.raw Connect with Both Platform Audio and File cargo run -p basic_room -- --platform-audio-and-file path/to/audio.raw WebRTC Build Requirements The external_audio_source.patch must be applied to WebRTC. The patch is automatically applied by all platform build scripts: - build_macos.sh - build_ios.sh - build_linux.sh - build_android.sh - build_windows.cmd For local development, set LK_CUSTOM_WEBRTC to point to your patched WebRTC build. Known Limitations Limitation │ Description Process-global │ Audio configuration affects all rooms in the process Device indices │ May change on hot-plug; match by name for persistence Single device track │ One device audio track per ADM (use NativeAudioSource for additional streams) --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:ca618eb
Author:CloudWebRTC
Committer:GitHub

feat: add scalability mode for AV1/VP9. (#1076) Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:8bd506a
Author:David Zhao
Committer:GitHub

chore: add LocalTrackRepublished event for FFI clients (#1072) during a reconnect, all published tracks will be republished and receive new TrackID. there is currently no way for FFI to learn about the new TrackID. --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:f317f51
Author:David Zhao
Committer:GitHub

support SimulateScenario through FFI to improve testing (#1069) also fixed a few edge cases with connection resumption. added tests --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:d165044
Author:David Chen

Merge remote-tracking branch 'origin/main' into dc/local_video_metal

Commit:8d7d24c
Author:Jacob Gelman
Committer:GitHub

Expose data track subscribe error message for FFI (#1057) Exposes the previously unused `SubscribeDataTrackError` message added in #981 and clarifies behavior for FFI implementors. Resolves BOT-341 --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:d62cbbf
Author:David Chen

add metal zero copy rendering pipeline & test pattern generator

Commit:43b8a5c
Author:David Chen
Committer:GitHub

Add support for frame level user timestamp (#890) - Add support to attach/parse frame level timestamps to VideoTracks as a custom payload trailer. --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: stavied <40528896+stephen-derosa@users.noreply.github.com>

Commit:f6f9918
Author:David Chen

wire up rtp timestamp

Commit:fed3c65
Author:David Chen

wip

Commit:19b9e1b
Author:David Chen

rename user_timestamp to user_timestamp_us for consistency w/ data tracks

Commit:c1261a9
Author:David Chen

Merge branch 'main' into dc/feature/user_timestamp

Commit:95346d0
Author:Jacob Gelman
Committer:GitHub

Data tracks FFI interface (#905) Initial support for data tracks over FFI.

Commit:690f0d5
Author:David Chen

Merge branch 'main' into dc/feature/user_timestamp

Commit:b0c8c3b
Author:xianshijing-lk
Committer:GitHub

use the bounded buffer for video stream (#956) Before this PR, it uses an unbounded buffer for video stream, that will cause multiple problems: 1, video will be lagged behind if rendering is slow or just wake up from background 2, it will be out of sync with audio

Commit:0f593b1
Author:David Chen

Merge branch 'main' into dc/feature/user_timestamp

Commit:5a15baa
Author:stavied
Committer:GitHub

user_timestamp FFI (#980) Expose user_timestamp for video frames to the FFI

Commit:1e4ac2b
Author:David Zhao
Committer:GitHub

feat: expose participant active event, state, and joined at (#978) Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Jacob Gelman <3182119+ladvoc@users.noreply.github.com>

Commit:c3f44e8
Author:David Chen

add frame ID to ffi

Commit:b9cc8f1
Author:David Chen

Merge branch 'main' into dc/feature/user_timestamp

Commit:c863be7
Author:Onestacked
Committer:GitHub

E2EE: allow setting key_ring_size and key_derivation_algorithm (#921) * Allow setting keyring-size from Rust * Implement using HKDF as the KDF. * Reexport KeyDerivationAlgorithm, use const for default KeyDerivationAlgorithm. * Update webrtc * Add changeset * Don't wrongly to m144 change, update webrtc to webrtc-24f6822-2

Commit:c3ec0ea
Author:ThΓ©o Monnom
Committer:GitHub

add disconnect reason to Room::close (#925)

Commit:03875c8
Author:xianshijing-lk
Committer:GitHub

add bounded buffer to audio_stream, and use 10 frames as the default (#945) * add bounded buffer to audio_stream, and use 10 frames as the default * run cargo fmt * switched over to rtrb and added some tests * Create add_bounded_buffer_to_audio_stream_and_use_10_frames_as_the_default.md added change msg

Commit:de9cd2b
Author:David Chen

adding the PayloadTrailerFeatures to the ffi protos

Commit:9a928b8
Author:xianshijing-lk
Committer:GitHub

turn single peerconnection off by default (#897)

Commit:9f15dcd
Author:ThΓ©o Monnom
Committer:GitHub

add is_screencast to VideoSource (#896)

Commit:9196330
Author:David Zhao
Committer:GitHub

ensure signal connections times out properly and retries (#895) * ensure signal connections times out properly and retries * generated protobuf --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:8b3f35a
Author:xianshijing-lk
Committer:GitHub

added Single Peer Connection support to Rust (#888) * added Single Peer Connection support to Rust * fix the code to make it work for both v0 and v1 (single peerconnection) signaling, add tests * fix the README and make the tests run with localhost, as it will fallback to V0 * cargo fmt * addressed the comments * some improvements on corner cases and added more tests * fix the tests on CIs, as the CIs are running local server, which does not support v1 by default * fix the tests * generated protobuf --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:17f914a
Author:David Chen

add simulcast support

Commit:1778ec8
Author:ThΓ©o Monnom
Committer:GitHub

add can_manage_agent_session permission (#870)

Commit:0a707a2
Author:David Chen

h264 tcp ingest

Commit:a98566d
Author:Jacob Gelman

Format

Commit:fdbca12
Author:Jacob Gelman
Committer:GitHub

Upgrade protocol to v1.44.0 (#857)

Commit:58f342c
Author:Jacob Gelman

Initial prototype

Commit:cb8f8a8
Author:Jacob Gelman

Integrate

Commit:51d1468
Author:GuuJiang
Committer:GitHub

Expose participant's permission to ffi layer (#824) * Expose participant's permission to ffi layer * Init participant's permission

Commit:ff6041e
Author:xianshijing-lk
Committer:GitHub

Add a request_async_id to the async requests (#842) * Add a request_async_id to the async requests * remove the unnedded clone() * revert the livekit-protocol/protocol changes * revert livekit-protocol/protocol * a try to fix the node generate protobuf job * generated protobuf --------- Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

Commit:ba8a988
Author:Anunay Maheshwari
Committer:GitHub

feat(ParticipantInfo): export kind details (#813)

Commit:47eec80
Author:Anunay Maheshwari
Committer:GitHub

feat(connector): initial service impl (#790) * feat(connector): initial service impl * build fixes from proto upgrade * fmt * bump protocol * pr feedback

Commit:6689a3c
Author:lukasIO
Committer:GitHub

Add RoomEvent::TokenRefreshed (#803)

Commit:971c72b
Author:Jacob Gelman
Committer:Jacob Gelman

Fix protocol integration

Commit:6464e2e
Author:Jacob Gelman
Committer:GitHub

Expose set video quality (#759) * Expose set video quality * Update wgpu_room example * Expose over FFI

Commit:2c7d5c4
Author:Hamdan
Committer:GitHub

add h265 codec support (#762)

Commit:d707572
Author:Jacob Gelman
Committer:Jacob Gelman

Expose set video quality over FFI

Commit:50e5938
Author:Jacob Gelman
Committer:GitHub

chore: License check (#746)

Commit:f60e539
Author:Ryan Gaus

fix: add PARTICIPANT_KIND_CONNECTOR to livekit-ffi proto definition

Commit:dda0d57
Author:Jacob Gelman

License format

Commit:785a7bb
Author:lukasIO
Committer:GitHub

feature(e2ee): add data channel encryption (#708) * chore: Upgrade libwebrtc to m137. * fix compile issue for linux. * fix. * Add g++ for aarch64 to Ubuntu dependencies * fix build for linux arm64. * Remove goma usage and switch to autoninja * fix android build. * fix build on linux arm64 for gcc. * Update Windows OS version in workflow configuration * Add GCC 14 installation steps for Ubuntu * Update webrtc-builds.yml * Change Windows OS to latest and add SDK installation * Update webrtc-builds.yml * Update webrtc-builds.yml * Update webrtc-builds.yml * Update webrtc-builds.yml * Update build_linux.sh * Update webrtc-builds.yml * Update webrtc-builds.yml * Update build_windows.cmd * cargo fmt. * Update build_windows.cmd * Add new encryption room options field and forward to e2ee_manager * libwebrtc ffi * fix ffi * add data packet cryptor to webrtc crate * add data frame cryptor handling on rtc session * data packet type encryption handling * cleanup * ffi encryption option * add way to retrieve latest key index * fix key provider last index * Allow passing test room options * add dc e2ee test * use shared key * Implement encryption test receiver side * rename test * Add encrypted text stream example * revert merge artifacts * whitespace * whitespace * format * cleanup * address comments * simplify conversion and recursive data handling * cleanup * more explicit conversion * Update test case * wip expose encryption_type * track participant's encryption status based on publications and incoming data messages * use is_encrypted getter on participants * update deprecation notice --------- Co-authored-by: cloudwebrtc <duanweiwei1982@gmail.com> Co-authored-by: Jacob Gelman <3182119+ladvoc@users.noreply.github.com>

Commit:451eca9
Author:lukasIO

use is_encrypted getter on participants

Commit:3b382bf
Author:lukasIO

wip expose encryption_type

Commit:ad2f7af
Author:lukasIO

Merge branch 'main' into lukas/dc-e2ee

Commit:56178e3
Author:Ben Cherry
Committer:GitHub

Add send_bytes method (#691) * Add send_bytes * Document integration testing * Add data stream tests * Reorganize E2E tests * Define FFI messages for send bytes * Implement FFI bindings * Create example for send bytes * Warn if total_length option specified --------- Co-authored-by: Jacob Gelman <3182119+ladvoc@users.noreply.github.com>

Commit:9407cb6
Author:lukasIO

ffi encryption option

Commit:d621a50
Author:Jacob Gelman
Committer:Jacob Gelman

Define FFI messages for send bytes

Commit:289b036
Author:David Chen
Committer:GitHub

Add option to enable audio preconnect buffer in SDK & FFI (#700) * add preconnect buffer option to ffi room proto * set AudioTrackFeatures on AddTrackRequest * add preconect to conversion from proto to TrackPublishOptions

Commit:67f00fe
Author:David Chen

add preconnect buffer option to ffi room proto

Commit:bdc0ffb
Author:David Zhao
Committer:GitHub

Expose room updates, support MoveParticipant (protocol 15) (#662) * Expose room updates, support MoveParticipant (protocol 15) giving clients other information about the room so they could receive * also fix sid updates * remove dead var

Commit:dd4c3ff
Author:David Zhao

also fix sid updates

Commit:4554454
Author:David Zhao

Expose room updates, support MoveParticipant (protocol 15) giving clients other information about the room so they could receive

Commit:81a3151
Author:ThΓ©o Monnom
Committer:GitHub

add frame-size-ms to AudioStream (#658)

Commit:fd33fe8
Author:Anunay Maheshwari
Committer:Anunay Maheshwari

pr comments

Commit:de3a3e1
Author:lukasIO
Committer:GitHub

Bump protocol to v1.37.1 and add audio_features handling (#634)

Commit:743b9d2
Author:lukasIO

Add ffi support for audio track features