Proto commits in slint-ui/slint

These 54 commits are when the Protocol Buffers files have changed:

Commit:823fa0a
Author:Robin Cramer
Committer:Robin Cramer

testing: Answer element tree requests over system testing The system-testing agent now answers `RequestGetElementTree`, which used to be MCP-only, so the debugger can show the element tree of a preview. `GetElementTreeResponse` lists the elements in pre-order with their depth, handle, and type names and ids. Without an element handle, the request covers every window that still shows a component. `debug_info_available` is false when the application was built without debug info, since the tree is empty then. == Implementation Notes == The tree comes from a new `ElementHandle::visit_tree`, since `visit_descendants` reports no depth. It includes clipped-away elements, so the tree doesn't change when the user scrolls, and it visits a popup after the element that shows it. MCP's `get_element_tree` keeps its flat list. One request without a root handle avoids chaining window list, window properties and tree requests. The live preview shows each recompiled instance in the window of the previous one. So showing a tracked window again now updates its root instead of tracking it twice, and keeps its handle. Showing a window also forgets the windows that were dropped. Eviction spares the window roots and keeps a fixed number of other handles. A response holds at most that many, so minting late handles can't evict early ones. `max_elements: 0` requests that maximum.

Commit:a30079d
Author:Robin Cramer
Committer:Robin Cramer

testing: Answer inspectable property requests over system testing The system-testing agent now answers `RequestElementInspectableProperties` with every property of the element that exists at runtime and its current value, so the debugger can show them for a preview. `available` is false when the application doesn't support inspection, which is the case for generated Rust and C++ code. == Implementation Notes == It's a request of its own rather than part of `ElementPropertiesResponse`, which MCP embeds in many results and would grow by every property. MCP doesn't expose it. The response reuses `TestableProperty`, since an inspectable property has the same name, type and debug-encoded value. The previews answer it without further changes: the interpreter always compiles with debug info.

Commit:c138812
Author:Robin Cramer
Committer:Robin Cramer

testing: Push element tree and property changes over system testing A debugger can now subscribe to the inspectable properties of an element, or to an element tree, and receive what changed until it unsubscribes, instead of polling. `RequestSubscribeElementProperties` and `RequestSubscribeElementTree` return the initial state and a subscription ID, and `RequestUnsubscribe` ends the subscription. Unsubscribing succeeds for a subscription that already ended. Each change is a diff against the previous one, pushed at most once per event-loop pass through the sink that `handle_encoded_request` now takes. A subscription ends when the element it observes is destroyed. A tree subscription also ends when the tree grows larger than one response can hold. In both cases, the sink receives `Ended`. Like `GetElementTreeResponse`, the response to a tree subscription reports whether the application has debug info. Only the debugger transport has a sink, so TCP system testing rejects the requests, and MCP doesn't expose them. Generated Rust and C++ code doesn't support inspection, so a property subscription there reports `available: false`. == Implementation Notes == Each subscription is a `ChangeTracker` whose evaluation takes a property snapshot or walks the tree, so it depends on everything it reads. The tree walk runs the instantiation pass itself, without running change handlers, so it depends on the repeater and conditional models. Opening and closing popups and adding windows bump generation properties that the walk reads. The response to a subscribe request comes from the tracker's first evaluation, so subscribing evaluates once. Dropping a property doesn't mark its dependents dirty, so a property snapshot or a subtree walk alone doesn't notice its element being destroyed. The evaluation also runs the instantiation pass on the item trees that hold the element, and reads the popups generation. It then re-evaluates when a repeater, conditional or popup destroys the element. The tree diff keys elements by their handle. Repeaters and conditionals only insert and remove whole instances, so one pass over both lists finds the removed and inserted runs; it falls back to replacing the whole tree if the survivors reorder. A tree subscription pins the handles of its elements, so eviction keeps them, and they don't count toward the handle cap. Subscribing to a tree fails when it has more elements than one response can hold, and the subscription ends once the tree outgrows that. This bounds the handles each subscription pins, but not their total across subscriptions. `live-preview` passes a sink that drops every notification for now.

Commit:340d3e1
Author:Robin Cramer
Committer:Robin Cramer

testing: Answer element tree requests over system testing The system-testing agent now answers `RequestGetElementTree`, which used to be MCP-only, so the debugger can show the element tree of a preview. `GetElementTreeResponse` lists the elements in pre-order with their depth, handle, and type names and ids. Without an element handle, the request covers every window that still shows a component. `debug_info_available` is false when the application was built without debug info, since the tree is empty then. == Implementation Notes == The tree comes from a new `ElementHandle::visit_tree`, since `visit_descendants` reports no depth. It includes clipped-away elements, so the tree doesn't change when the user scrolls, and it visits a popup after the element that shows it. MCP's `get_element_tree` keeps its flat list. One request without a root handle avoids chaining window list, window properties and tree requests. The live preview shows each recompiled instance in the window of the previous one. So showing a tracked window again now updates its root instead of tracking it twice, and keeps its handle. Showing a window also forgets the windows that were dropped. Eviction spares the window roots and keeps a fixed number of other handles. A response holds at most that many, so minting late handles can't evict early ones. `max_elements: 0` requests that maximum.

Commit:c79faa9
Author:Robin Cramer
Committer:Robin Cramer

testing: Answer inspectable property requests over system testing The system-testing agent now answers `RequestElementInspectableProperties` with every property of the element that exists at runtime and its current value, so the debugger can show them for a preview. `available` is false when the application doesn't support inspection, which is the case for generated Rust and C++ code. == Implementation Notes == It's a request of its own rather than part of `ElementPropertiesResponse`, which MCP embeds in many results and would grow by every property. MCP doesn't expose it. The response reuses `TestableProperty`, since an inspectable property has the same name, type and debug-encoded value. The previews answer it without further changes: the interpreter always compiles with debug info.

Commit:3d0f673
Author:Robin Cramer
Committer:Robin Cramer

testing: Push element tree and property changes over system testing A debugger can now subscribe to the inspectable properties of an element, or to an element tree, and receive what changed until it unsubscribes, instead of polling. `RequestSubscribeElementProperties` and `RequestSubscribeElementTree` return the initial state and a subscription ID, and `RequestUnsubscribe` ends the subscription. Unsubscribing succeeds for a subscription that already ended. Each change is a diff against the previous one, pushed at most once per event-loop pass through the sink that `handle_encoded_request` now takes. A subscription ends when the element it observes is destroyed. A tree subscription also ends when the tree grows larger than one response can hold. In both cases, the sink receives `Ended`. Like `GetElementTreeResponse`, the response to a tree subscription reports whether the application has debug info. Only the debugger transport has a sink, so TCP system testing rejects the requests, and MCP doesn't expose them. Generated Rust and C++ code doesn't support inspection, so a property subscription there reports `available: false`. == Implementation Notes == Each subscription is a `ChangeTracker` whose evaluation takes a property snapshot or walks the tree, so it depends on everything it reads. The tree walk runs the instantiation pass itself, without running change handlers, so it depends on the repeater and conditional models. Opening and closing popups and adding windows bump generation properties that the walk reads. The response to a subscribe request comes from the tracker's first evaluation, so subscribing evaluates once. Dropping a property doesn't mark its dependents dirty, so a property snapshot or a subtree walk alone doesn't notice its element being destroyed. The evaluation also runs the instantiation pass on the item trees that hold the element, and reads the popups generation. It then re-evaluates when a repeater, conditional or popup destroys the element. The tree diff keys elements by their handle. Repeaters and conditionals only insert and remove whole instances, so one pass over both lists finds the removed and inserted runs; it falls back to replacing the whole tree if the survivors reorder. A tree subscription pins the handles of its elements, so eviction keeps them, and they don't count toward the handle cap. Subscribing to a tree fails when it has more elements than one response can hold, and the subscription ends once the tree outgrows that. This bounds the handles each subscription pins, but not their total across subscriptions. `live-preview` passes a sink that drops every notification for now.

Commit:644cd77
Author:Till Adam
Committer:Simon Hausmann

testing/mcp: read `@testable` properties from ElementHandle and the MCP server Verify UI state from a test or an MCP client by reading an element's `@testable` properties back, instead of diffing screenshots. `ElementHandle` gains `testable_properties()` and `testable_property_value()` in the Rust and C++ testing APIs. `get_element_properties` and `get_element_tree` report `testableProperties` with typed JSON values: booleans and numbers as JSON values, colors as `#rrggbbaa`, enums as their `.slint` spelling. When debug info is missing, `get_element_properties` says so in a note instead of returning an empty list. The same source and expected strings run on generated Rust, generated C++ and the interpreter. Closes #13251

Commit:06f0451
Author:Simon Hausmann
Committer:Simon Hausmann

testing: match elements by accessible label in queries Add ElementQuery::match_accessible_label() and a matching match_element_accessible_label query instruction to the system testing protocol, and list it in the MCP tool descriptions. slint-testing 0.4 matches labels on the Python side: it fetches every candidate's properties with a separate request. Evaluating the match in the application makes a label query a single request. The Visual Editor UI tests look up most elements by label. ElementHandle::find_by_accessible_label() now uses the new match instead of its own predicate.

Commit:fcbbb9f
Author:Nigel Breslaw

Add native click targeting and automatic scrolling to Python tests Negotiate a read-only native left-click target contract and reveal clipped controls through nested Flickables. Check again after hover callbacks before pressing, preserve explicit unsupported policies, and keep older binaries on labeled basic readiness. Record target checks and scrolling in Studio actions. Validate against a standalone native fixture, the legacy Python runtime, and the editor. Keep the protobuf descriptor checked against the Rust schema and include it in Studio's preflight cache identity. Validation: 655 editor UI tests, 213 editor Rust tests, 77 backend tests, 59 generic Python tests, and 68 Studio tests passed. Native Studio reran four inspector cases and displayed their captures. Ruff, ty, repository autofixes, and strict backend/root-workspace Clippy passed. Full repository Clippy is blocked by the examples workspace's FFmpeg/header incompatibility. ChangeLog: Improve Python UI test clicks with native target checks and automatic scrolling.

Commit:d00053d
Author:Till Adam
Committer:Simon Hausmann

mcp: hold modifiers during move_pointer too hover_element and move_pointer both dispatch a PointerMoveEvent and differ only in target (element handle vs. raw position), same as the scroll_element/ dispatch_pointer_scroll pair, which already takes modifiers on both variants. move_pointer was the odd one out.

Commit:e029af3
Author:Till Adam
Committer:Simon Hausmann

mcp: hold keyboard modifiers during a pointer gesture A shift-click was three calls: a key press of '\u{0010}', the click, then the release. Every embedder had to rediscover that Shift is that code, and a modifier-sensitive gesture couldn't be expressed as one call. The pointer tools now take a `modifiers` object. Slint reads the modifiers of a pointer event from the window's key state rather than from the event, so the keys are pressed before the gesture and released after it. A guard does the releasing on drop, so a gesture that fails in between doesn't leave the window with a key stuck down. `click_element` and `drag_element` carry the field into the systest transport as well, since both transports share the dispatch functions. ChangeLog: MCP pointer tools accept a `modifiers` object, so a shift-click or a ctrl+wheel zoom is one call.

Commit:4e64a0b
Author:Till Adam
Committer:Simon Hausmann

mcp: add wheel events to the tool surface A client could move, click and drag the pointer but never scroll it, so a Flickable or ScrollView couldn't be scrolled and a wheel-driven canvas zoom wasn't reachable at all. Working around it meant finding the application's own scroll or zoom controls in the element tree and clicking them repeatedly. `ElementHandle::scroll` already dispatches `PointerScrolled` at an element's center, so the two tools are thin: `scroll_element` takes a handle and `dispatch_pointer_scroll` an arbitrary point, mirroring the existing `hover_element` and `move_pointer` pair. ChangeLog: The MCP server has new `scroll_element` and `dispatch_pointer_scroll` tools, to send mouse wheel events.

Commit:220ae7c
Author:Till Adam
Committer:Simon Hausmann

testing: Add hover_element and move_pointer tools to the MCP server `click_element` and `drag_element` both press a button, so there was no way to produce a plain hover: any attempt to move the pointer over an element also delivered a press to it. Hover states, tooltips and menu highlighting were therefore out of reach over MCP. `hover_element` moves the pointer to an element's center, reusing the same `absolute_center` that `click_element` targets so a hover lands where a click would. `move_pointer` moves it to a position in the window, which is what clearing a hover needs. Neither presses a button. They are two tools rather than one with alternative targeting fields so that each `inputSchema` states exactly which arguments are required, as every other tool with arguments already does. Both boil down to dispatching a PointerMoved window event, which the systest wire protocol can already do via RequestDispatchWindowEvent. So RequestMovePointer/RequestHoverElement are kept only as free-standing message shapes for the MCP tools' input schemas, not added to the RequestToAUT/AUTResponse oneofs, and systest.rs no longer special-cases them. ChangeLog: The MCP server has new `hover_element` and `move_pointer` tools, to hover an element or a position without pressing a button.

Commit:3d75487
Author:Olivier Goffart
Committer:Olivier Goffart

Add a title-bar accessible role Maps to AccessKit's TitleBar and Qt's TitleBar role, so a custom title bar in a frameless window can expose the right semantics.

Commit:43615eb
Author:Olivier Goffart
Committer:Olivier Goffart

Add a title-bar accessible role Maps to AccessKit's TitleBar and Qt's TitleBar role, so a custom title bar in a frameless window can expose the right semantics.

Commit:4fe001f
Author:Simon Hausmann
Committer:Simon Hausmann

Move WindowEventDispatchResult to slint::platform The enum now sits next to WindowEvent, where backends look for it. Drop the Ignored variant. To a backend, an event that nothing handled and one that a handler refused mean the same thing, so Rejected covers both. The Node.js port follows: the enum moves into the platform namespace, and the docs sidebar now lists that namespace's enums. RecordedEventResult in the system testing protocol loses its Ignored value too, since nothing can produce it any more. cc #13066

Commit:bf9f065
Author:Olivier Goffart

Add a title-bar accessible role Maps to AccessKit's TitleBar and Qt's TitleBar role, so a custom title bar in a frameless window can expose the right semantics.

Commit:9c1ff7f
Author:Martin Marmsoler
Committer:Martin Marmsoler

add missing accessibility properties ( item_count, item_index, ...) to the testing crate. (#12892) (cherry picked from commit 3f269063d5f53e3774301fe204716293cbf7e2b6)

Commit:3f26906
Author:Martin Marmsoler
Committer:GitHub

add missing accessibility properties ( item_count, item_index, ...) to the testing crate. (#12892)

Commit:b3279cd
Author:Simon Hausmann
Committer:Olivier Goffart

system testing / MCP: Expose the window scale factor in the window properties (cherry picked from commit a7f9413f5a084dcd32d088910de516d7464eba75)

Commit:a7f9413
Author:Simon Hausmann
Committer:Simon Hausmann

system testing / MCP: Expose the window scale factor in the window properties

Commit:ee963ac
Author:Simon Hausmann

system testing / MCP: Expose the window scale factor in the window properties

Commit:4bc189b
Author:Olivier Goffart
Committer:GitHub

accessibility: Add landmark accessible roles (#11831) Add banner, complementary, content-info, form, main, navigation, region, and search roles to AccessibleRole with mappings for the AccessKit, Qt, and testing backends. Use navigation and main landmarks in the gallery demo.

Commit:d37ca00
Author:Till Adam
Committer:GitHub

Add MCP event log for processed window events (#11665) - add an internal window-event dispatch hook and shared testing-backend event ring buffer - expose get_event_log and clear_event_log MCP tools, plus systest responses for the same log - record pointer/key/window events with sequence cursors, dropped counts, and key processing results - preserve extended pointer buttons and keep event-log pagination/clear cursors monotonic

Commit:1ddcc3b
Author:Ioan Michnea
Committer:GitHub

widgets: Add RadioGroup (#11141) * feat: add RadioGroup and docs/example updates

Commit:bb9d6aa
Author:Till Adam
Committer:Simon Hausmann

Drive MCP tool definitions and dispatch from proto messages Replace hand-written JSON schemas and manual string parsing with proto-driven generation: - Add RequestGetElementTree, RequestDispatchKeyEvent, and KeyEventType to slint_systest.proto so all 12 MCP tools have proto counterparts - Generate JSON schemas from proto descriptors at build time in build.rs - Replace 160 lines of hand-written JSON with a declarative TOOLS table mapping tool names to proto request types - Unify all tool handlers to use deserialize_params with proto types, eliminating manual string matching for enums and field extraction

Commit:a0415da
Author:Till Adam
Committer:Simon Hausmann

Add drag gesture simulation to testing backend and MCP server Add drag() and mock_drag() methods to ElementHandle that simulate a pointer drag from the element's center to a target position. The pointer is pressed at center, moved in interpolated ~5px steps (below the 8px DISTANCE_THRESHOLD), then released at the target. Zero-distance drags short-circuit to a press-release without spurious move events. Expose the new capability through both transports: - systest: RequestElementDrag / ElementDragResponse proto messages - MCP: drag_element tool with targetX, targetY, optional button Also adds MCP server documentation to the testing backend README and two unit tests covering normal drags and zero-distance edge case.

Commit:1b79aed
Author:David Faure
Committer:David Faure

Rename FlexBoxLayout to FlexboxLayout and LayoutKind::FlexBox -> LayoutKind::FlexboxLayout for consistency with the other enum values in LayoutKind In internal/backends/testing/slint_systest.proto, move the comment about Unknown to the AccessibleRole where it initially belonged (commit 2cad49eb9b32c inserted enum LayoutKind between the comment and the AccessibleRole enum) Fixes #11152

Commit:a337105
Author:Till Adam
Committer:Simon Hausmann

Address review: use LayoutKind enum, rename proto variants, add tests - Replace SharedString return with a proper LayoutKind enum in ElementHandle::layout_kind() - Rename proto enum variants to match element names (HorizontalLayout, VerticalLayout, GridLayout) - Add test covering all layout kinds and non-layout elements

Commit:2cad49e
Author:Till Adam
Committer:Simon Hausmann

Expose layout kind through the test protocol Encode the layout kind (h-box, v-box, grid, flex-box) as a third field in the debug info string emitted by the compiler, and surface it through ItemRc, ElementHandle, and the systest protobuf protocol. This allows test tooling to distinguish layout elements from plain rectangles.

Commit:2949410
Author:Arnold Loubriat
Committer:Simon Hausmann

Add `AccessibleRole.radio-button`

Commit:f0c3df8
Author:Simon Hausmann
Committer:Simon Hausmann

system-testing: Make the image format for snapshots configurable Let the image crate pick an encoder by mime-type, otherwise fall back to png. Renaming the field in the response is compatible, as only the tag numbers are encoded. Adding a field to the request is also compatible as every field is optional.

The documentation is generated from this commit.

Commit:ccf651c
Author:Simon Hausmann

system testing: Expose accessible-enabled and accessible-read-only

Commit:864fbf0
Author:Simon Hausmann

system testing: Expose accessible-placeholder-text in the element properties

Commit:90c337f
Author:Arnold Loubriat
Committer:GitHub

Add `AccessibleRole::Image` and use it in the `AboutSlint` widget (#7593) * Add `AccessibleRole::Image` * Improve accessibility of the `AboutSlint` widget * Filter out some images from the accessibility tree

Commit:358e42a
Author:Arnold Loubriat
Committer:Simon Hausmann

Add the `accessible-action-expand` action

Commit:aa33220
Author:Simon Hausmann
Committer:Simon Hausmann

Rename AccessibleRole.GroupBox to AccessibleRole.Groupbox While unfortunate in casing, this is consistent with Checkbox, Combobox, and Spinbox.

Commit:3e94bd2
Author:Olivier Goffart
Committer:Olivier Goffart

Janitor: Remove trailing whitespaces from all files `git grep -I -l -O'sed -i "s/[[:space:]]*$//"' -e ''`

Commit:4919672
Author:Arnold Loubriat
Committer:Simon Hausmann

Add `AccessibleRole::GroupBox`

Commit:3304df0
Author:Arnold Loubriat
Committer:Simon Hausmann

Add `AccessibleRole::TabPanel`

Commit:b2873a3
Author:Simon Hausmann
Committer:Simon Hausmann

ui-testing: Add support for reading the computed opacity of elements This is useful when the UI shows for example labels that appear and the screenshot for referenceing is meant to be taken when the opacity animation is complete and the value is (near) 1.0.

Commit:f014202
Author:Arnold Loubriat
Committer:Simon Hausmann

Changelog: Introduce `AccessibleRole::ListItem`

Commit:12f5343
Author:Simon Hausmann
Committer:Simon Hausmann

Add the window's root element handle to the window properties

Commit:69e5a8a
Author:Simon Hausmann
Committer:Simon Hausmann

System Testing: Add descendent and element id/type/role matching messages

Commit:9e8fccb
Author:Simon Hausmann
Committer:Simon Hausmann

system testing: Add support for dispatching pointer and key window events

Commit:e35d05f
Author:Simon Hausmann
Committer:Simon Hausmann

API Review: Remove SharedImageBuffer from public API and rename Window::grab_window() to take_snapshot() Use SharedPixelBuffer as return value for take_snapshot() and provide counter-parts to from_rgb* in Image as to_rgb*

Commit:7f05b8a
Author:Simon Hausmann
Committer:Simon Hausmann

Changed ElementHandle click API to a single_click and double_click function taking the button That way one can also simulate double right click, for example. This is a little less convenient, more generic, but a reasonable compromise (perhaps :-)

Commit:e78e68a
Author:Simon Hausmann
Committer:Simon Hausmann

System Testing: Add support for clicking on elements

Commit:155156a
Author:Simon Hausmann
Committer:Simon Hausmann

System testing: Add support for grabbing window contents as PNG encoded image

Commit:142b1eb
Author:Simon Hausmann
Committer:Simon Hausmann

System Testing: Expose the accessibility role of elements

Commit:1c6aade
Author:Simon Hausmann
Committer:Simon Hausmann

Add support for setting the accessible value

Commit:e071ba9
Author:Simon Hausmann
Committer:Simon Hausmann

Add a way to invoke accessibility actions

Commit:97343b2
Author:Simon Hausmann
Committer:Simon Hausmann

Testing: Add the ability to locate elements by id and expose various properties

Commit:4591ad8
Author:Simon Hausmann
Committer:Simon Hausmann

Add plumbing for system testing