Proto commits in sozu-proxy/sozu

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

2026-06-04

feat(udp): add first-class UDP load balancing (#1273) Add a `udp` listener type alongside tcp/http/https so Sōzu can load-balance datagram services (DNS, syslog, NTP, generic UDP) without a second load balancer in front of it, reusing the control plane, metrics/logging and e2e harness. Architecture: a two-level sans-io core (lib/src/protocol/udp/: UdpManager admission/flow-table/generation-token timer-wheel over per-flow UdpFlow, pure and property-tested) driven by an I/O shell (lib/src/udp.rs) on the existing single-threaded mio loop. Static per-listener worker ownership; scale across cores by running multiple listeners. Flows reset on hot-upgrade (listener fd handed off via SCM, flow state not migrated). - Control plane: UdpListenerConfig + RequestUdpFrontend proto, TOML config parsing, ConfigState with diff/generate_requests/activate replay symmetry, SCM listener-fd hand-off, ctl CLI + master-process routing, and sozu top/listener-list/frontend-list/cluster display parity. - Datapath: virtual 4-tuple flows, three-knob teardown (responses / idle timeout / requests), symmetric NAT return via a per-flow connected upstream socket, bounded per-flow/per-listener write-queue egress backpressure with writable re-arm, max_flows fd-pressure shedding. No panic on network input (EMFILE/ENFILE/oversize/invalid -> drop + metric). - Load balancing: round-robin + HRW/rendezvous (default) + Maglev (opt-in), selectable per cluster via a key: Option<u64> threaded through the LB trait; the Maglev table is rebuilt only on backend-set change, never per datagram. - PROXY protocol v2 to the backend (first-datagram default, per-cluster every-datagram opt-in; DGRAM transport byte 0x12/0x22). - Active health checks bound to the endpoint: companion TCP probe (primary) + app-level UDP probe (secondary), rise/fall hysteresis, fail-open. - Metrics (udp.*) with no gauge underflow on any teardown path, and per-flow access logs under the UDP / UDP-FLOW log tag. - Tests: sans-io quickcheck/state-machine core tests, 14 e2e scenarios (RR/HRW/Maglev affinity, NAT return, teardown, truncation, PPv2, health/fail-open, hot reconfig, upgrade fd hand-off) with a mock UDP backend/client, and a fuzz_udp_flow cargo-fuzz target. - Docs: doc/configure.md UDP listener/cluster/health section + metrics, CHANGELOG, README, architecture. Plaintext only. Single-listener multi-core / eBPF SK_REUSEPORT / userland dispatcher / DSR / io_uring / DTLS / QUIC / UDP-over-HTTP / flow-state migration across upgrade are explicit non-goals. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

fix(mux): mitigate the HTTP/2 bomb (HPACK header-field cap + window-stall reaping) Closes both halves of the HTTP/2 memory-amplification DoS class (calif.io disclosure; same family as Apache CVE-2026-49975). HPACK header bomb ----------------- Thousands of 1-byte HPACK indexed references — or a single `cookie` header split into many crumbs (RFC 9113 §8.2.3) — each materialize a `Pair` of per-entry bookkeeping, amplifying wire bytes into allocation. `SETTINGS_MAX_HEADER_LIST_SIZE` was accounted as name+value bytes only, omitting the RFC 9113 §6.5.2 mandated 32-octet per-field overhead, and nothing capped the field count. - `pkawa.rs`: account `+32` octets/field per §6.5.2 and enforce a per-block materialized-field-count cap in `decode_headers_with_budget` and `handle_trailer`; cookie crumbs count individually. Over-limit blocks are rejected with `RST_STREAM` / `GOAWAY(ENHANCE_YOUR_CALM)`. - New per-listener `h2_max_header_fields` knob (default 128), plumbed through `H2FloodConfig`, `command.proto`, `config.rs`, the http/https flood-config getters, live-update patch-apply + state replay + validation, the sozuctl CLI, and listener `Display`. Window-stall ------------ A peer that holds its receive window shut while a response is buffered can pin the stream and its `MAX_CONCURRENT_STREAMS` slot. The bidirectional liveness timer (`stream_last_activity_at`) is refreshed by any inbound frame, so a 1-byte inbound DATA drip kept a stalled stream warm indefinitely. - New per-stream flow-control-stall deadline (`stream_fc_stalled_since`): armed only when a stream holds buffered response data it cannot send because its effective send window `min(stream.window, connection.window)` is exhausted, cleared on real outbound progress (`consumed > 0`), and never refreshed by inbound DATA/HEADERS — so the inbound-drip vector cannot keep it warm. Reaped by `cancel_timed_out_streams` (governed by `h2_stream_idle_timeout_seconds`) via a deduped union with the idle guard (`collect_timed_out_streams`). - The reaper runs from both `readable()` and `MuxState::timeout`, so even a fully-silent peer's stalled stream is reaped (slot freed + queued `RST_STREAM(CANCEL)` flushed). Direction-correct: a slow upload has no buffered response, a socket-bound slow link keeps a positive window, and a legitimate slow download clears the deadline on every window grant — none arm it. No new config knob (reuses `h2_stream_idle_timeout_seconds`). Tests: `pkawa` unit tests (`+32` accounting, field cap, cookie-crumb cap, legit request) and `collect_timed_out_streams` unit tests; e2e `e2e_h2_parser_reject_header_field_bomb`, `test_h2_window_stall_response_reaped`, `test_h2_window_stall_inbound_drip_reaped`, `test_h2_window_stall_silent_reaped`, with `test_h2_active_upload_survives_idle_timeout` as a false-positive guard. Validated: `cargo build --all-features`, `clippy -D warnings`, nightly fmt, the full h2 e2e suite (222), lib + command unit tests, and the cargo-fuzz hpack / frame targets. Docs: `doc/configure.md`, `CHANGELOG.md`, mux `LIFECYCLE.md`. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

2026-06-03

feat(udp): add first-class UDP load balancing (#1273) Add a `udp` listener type alongside tcp/http/https so Sōzu can load-balance datagram services (DNS, syslog, NTP, generic UDP) without a second load balancer in front of it, reusing the control plane, metrics/logging and e2e harness. Architecture: a two-level sans-io core (lib/src/protocol/udp/: UdpManager admission/flow-table/generation-token timer-wheel over per-flow UdpFlow, pure and property-tested) driven by an I/O shell (lib/src/udp.rs) on the existing single-threaded mio loop. Static per-listener worker ownership; scale across cores by running multiple listeners. Flows reset on hot-upgrade (listener fd handed off via SCM, flow state not migrated). - Control plane: UdpListenerConfig + RequestUdpFrontend proto, TOML config parsing, ConfigState with diff/generate_requests/activate replay symmetry, SCM listener-fd hand-off, ctl CLI + master-process routing, and sozu top/listener-list/frontend-list/cluster display parity. - Datapath: virtual 4-tuple flows, three-knob teardown (responses / idle timeout / requests), symmetric NAT return via a per-flow connected upstream socket, bounded per-flow/per-listener write-queue egress backpressure with writable re-arm, max_flows fd-pressure shedding. No panic on network input (EMFILE/ENFILE/oversize/invalid -> drop + metric). - Load balancing: round-robin + HRW/rendezvous (default) + Maglev (opt-in), selectable per cluster via a key: Option<u64> threaded through the LB trait; the Maglev table is rebuilt only on backend-set change, never per datagram. - PROXY protocol v2 to the backend (first-datagram default, per-cluster every-datagram opt-in; DGRAM transport byte 0x12/0x22). - Active health checks bound to the endpoint: companion TCP probe (primary) + app-level UDP probe (secondary), rise/fall hysteresis, fail-open. - Metrics (udp.*) with no gauge underflow on any teardown path, and per-flow access logs under the UDP / UDP-FLOW log tag. - Tests: sans-io quickcheck/state-machine core tests, 14 e2e scenarios (RR/HRW/Maglev affinity, NAT return, teardown, truncation, PPv2, health/fail-open, hot reconfig, upgrade fd hand-off) with a mock UDP backend/client, and a fuzz_udp_flow cargo-fuzz target. - Docs: doc/configure.md UDP listener/cluster/health section + metrics, CHANGELOG, README, architecture. Plaintext only. Single-listener multi-core / eBPF SK_REUSEPORT / userland dispatcher / DSR / io_uring / DTLS / QUIC / UDP-over-HTTP / flow-state migration across upgrade are explicit non-goals. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

fix(mux): mitigate the HTTP/2 bomb (HPACK header-field cap + window-stall reaping) Closes both halves of the HTTP/2 memory-amplification DoS class (calif.io disclosure; same family as Apache CVE-2026-49975). HPACK header bomb ----------------- Thousands of 1-byte HPACK indexed references — or a single `cookie` header split into many crumbs (RFC 9113 §8.2.3) — each materialize a `Pair` of per-entry bookkeeping, amplifying wire bytes into allocation. `SETTINGS_MAX_HEADER_LIST_SIZE` was accounted as name+value bytes only, omitting the RFC 9113 §6.5.2 mandated 32-octet per-field overhead, and nothing capped the field count. - `pkawa.rs`: account `+32` octets/field per §6.5.2 and enforce a per-block materialized-field-count cap in `decode_headers_with_budget` and `handle_trailer`; cookie crumbs count individually. Over-limit blocks are rejected with `RST_STREAM` / `GOAWAY(ENHANCE_YOUR_CALM)`. - New per-listener `h2_max_header_fields` knob (default 128), plumbed through `H2FloodConfig`, `command.proto`, `config.rs`, the http/https flood-config getters, live-update patch-apply + state replay + validation, the sozuctl CLI, and listener `Display`. Window-stall ------------ A peer that holds its receive window shut while a response is buffered can pin the stream and its `MAX_CONCURRENT_STREAMS` slot. The bidirectional liveness timer (`stream_last_activity_at`) is refreshed by any inbound frame, so a 1-byte inbound DATA drip kept a stalled stream warm indefinitely. - New per-stream flow-control-stall deadline (`stream_fc_stalled_since`): armed only when a stream holds buffered response data it cannot send because its effective send window `min(stream.window, connection.window)` is exhausted, cleared on real outbound progress (`consumed > 0`), and never refreshed by inbound DATA/HEADERS — so the inbound-drip vector cannot keep it warm. Reaped by `cancel_timed_out_streams` (governed by `h2_stream_idle_timeout_seconds`) via a deduped union with the idle guard (`collect_timed_out_streams`). - The reaper runs from both `readable()` and `MuxState::timeout`, so even a fully-silent peer's stalled stream is reaped (slot freed + queued `RST_STREAM(CANCEL)` flushed). Direction-correct: a slow upload has no buffered response, a socket-bound slow link keeps a positive window, and a legitimate slow download clears the deadline on every window grant — none arm it. No new config knob (reuses `h2_stream_idle_timeout_seconds`). Tests: `pkawa` unit tests (`+32` accounting, field cap, cookie-crumb cap, legit request) and `collect_timed_out_streams` unit tests; e2e `e2e_h2_parser_reject_header_field_bomb`, `test_h2_window_stall_response_reaped`, `test_h2_window_stall_inbound_drip_reaped`, `test_h2_window_stall_silent_reaped`, with `test_h2_active_upload_survives_idle_timeout` as a false-positive guard. Validated: `cargo build --all-features`, `clippy -D warnings`, nightly fmt, the full h2 e2e suite (222), lib + command unit tests, and the cargo-fuzz hpack / frame targets. Docs: `doc/configure.md`, `CHANGELOG.md`, mux `LIFECYCLE.md`. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

2026-06-01

fix(otel): capture wall-clock start time for accurate span reconstruction Access-log consumers reconstructing OTel spans computed the request start timestamp as `precise_time - request_time`. This subtraction mixes two unsynchronised clock sources: - `precise_time` is a wall-clock reading (`CLOCK_REALTIME` via `OffsetDateTime::now_utc()`), captured at log-emission time. - `request_time` is a monotonic duration (`CLOCK_MONOTONIC` via `Instant::now() - start`), captured in `SessionMetrics` *before* the wall-clock reading. The two clocks drift independently (NTP adjustments affect CLOCK_REALTIME but not CLOCK_MONOTONIC), and they are sampled at different instants within the log-emission path. On short-lived requests the resulting error can exceed the request duration itself, producing spans whose computed start time falls after a subsequent request's start — visibly broken traces. This commit adds a `start_wall: Option<SystemTime>` field to `SessionMetrics`, captured at the same point as the existing monotonic `start: Option<Instant>`. A new `start_wall_ns()` accessor converts it to nanoseconds-since-epoch for the access log. Wire changes: - `ProtobufAccessLog.start_time` (field 30, optional `Uint128`) in `command.proto` — backwards-compatible: old consumers ignore it, new consumers with old producers see `None` and can fall back to the subtraction. - `RequestRecord.start_time_ns: Option<i128>` plumbed through all four access-log emission sites (kawa_h1, mux/stream, pipe, tcp). Consumers should now prefer `start_time` over `time - request_time` whenever the field is present. Signed-off-by: miton18 <remi@collignon-ducret.fr>

2026-05-21

refactor(command,server): remove the proto_version capability handshake Drop the proto_version field and the SetMetricDetail capability partition that gated the dispatch on worker proto version. Production deployments keep master + workers in sync via the existing UpgradeMain hot-upgrade flow, so the mixed-version-fleet state the field was designed for does not occur. The implementation was structurally broken anyway: the master stamped every WorkerInfo.proto_version from its own SOZU_PROTO_VERSION constant at fork time, so the field always read as the master's version and MIN_PROTO_VERSION_FOR_SET_METRIC_DETAIL was effectively unconditional. The proto contract is additive-only; a worker that does not recognise tag 55 (SetMetricDetail) returns WorkerResponse::error("unknown request type") which already surfaces in the standard fan-out error tally (extras.fanout.workers_err counter). MetricDetailStatus.unsupported_workers becomes redundant and is also removed. Deletions: - WorkerInfo.proto_version (tag 4) and MetricDetailStatus.unsupported_workers (tag 5); both replaced by `reserved` markers per project convention. - sozu_command_lib::SOZU_PROTO_VERSION constant. - WorkerSession.proto_version field. - MIN_PROTO_VERSION_FOR_SET_METRIC_DETAIL constant and the capability partition in set_metric_detail_request; SetMetricDetail now fans out unconditionally via the standard scatter path. - Capability-handshake paragraphs in CHANGELOG.md and doc/sozu-top.md. - print_metric_detail_status block that rendered the unsupported_workers table row. Old workers without tag 55 surface as 'succeeded with errors' (the normal fan-out failure shape) rather than as a dedicated capability-skip list. Operator visibility is preserved. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(command,lib): per-worker WorkerMetricDetailStatus payload Final piece of the PR #1256 follow-through. Previously the capability-aware dispatcher in `bin/src/command/requests.rs` (`SetMetricDetailTask::on_finish`) synthesised `MetricDetailStatus.workers[<worker_id>]` using the master's aggregator view as a stand-in for each worker because workers replied with `WorkerResponse::ok(message.id)` carrying no payload. Each worker holds an independent `Aggregator` with its own lease table, so that stand-in obscured real per-worker drift (different configured floors, different active lease counts after a partial fan-out). Wire it properly end-to-end: - New `ResponseContent::WorkerMetricDetailStatus` oneof variant (tag 17 — proto additive). Carries the worker's own `(configured, effective, previous_effective, active_lease_count)` quartet, semantically distinct from the aggregated `MetricDetailStatus` at tag 16. - New `lib/src/server.rs::worker_metric_detail_status_content` helper that builds the response payload from a `(configured, effective, previous_effective, lease_count)` snapshot captured BEFORE the `METRICS.borrow_mut` scope ends (so the per- request snapshot is consistent with the transition that just happened). - The three ok-paths in the worker's SetMetricDetail arm (clear-Cleared, clear-NotFound, apply-Applied) now reply via `WorkerResponse:: ok_with_content` with the freshly-built payload instead of the payload-less `ok`. The `clear-NotFound` path reports `previous_effective == effective` (no transition). - Master-side `SetMetricDetailTask::on_finish` collects the per-worker payload from `response.content` and only falls back to skipping the worker entry when the response has no payload (e.g. an older worker that never went through `ok_with_content`). Removes the master-view stand-in noted as a follow-up in commit `70cd24af` (`set_metric_detail_request`). - `command/src/proto/display.rs` adds a silent OK match arm for the new variant — the per-worker payload flows master-side and is never printed directly on the operator's terminal. Build/clippy clean; 1075/1075 workspace tests pass. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(server,command): wire worker→master audit IPC for METRIC_DETAIL_CHANGED Previously, worker-local cardinality-lease transitions (TTL janitor expiry, worker-arm apply/clear) left no audit trail because the worker had no IPC path to the master's audit pipeline. Only operator-initiated transitions audited from `bin/src/command/requests.rs::worker_request` produced an audit row. A SOC analyst correlating "who elevated metrics cardinality" could see the apply but not the implicit clear. Close the gap by reusing the existing worker→master `Event` channel: - New proto `MetricDetailTransition` carrying `previous_effective`, `effective`, `transition_kind`, and an optional `client_id` for explicit apply/clear. Folded into `Event.metric_detail` (tag 5) so `EventKind::METRIC_DETAIL_CHANGED` events now carry their full payload through the existing fan-out plumbing. - Worker emits the event from three sites in `lib/src/server.rs::notify`: the polled `lease_tick` janitor (transition_kind = "lease_tick_expired" and `client_id = None` because the janitor may retire multiple leases at once), the SetMetricDetail worker arm on apply ("lease_apply"), and on clear ("lease_clear"). All three callers gate on `previous != effective` so the helper itself is a defence-in-depth no-op when nothing actually changed. - Master's `handle_worker_response` recognises METRIC_DETAIL_CHANGED events and routes them through a dedicated `audit_worker_metric_detail_transition` helper that writes to both audit sinks (text + JSON). The worker is its own actor — `worker_id` takes the `client_id` slot in the envelope; `actor_role=worker`, `actor_comm=sozu-worker`. Subscriber fan-out is unchanged. - The 11 existing `push_event(Event { … })` constructors get `metric_detail: None` to stay compatible with the new field; only the worker's lease-transition emitter populates it. Build/clippy clean; 53/53 TUI unit tests pass. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(command): worker proto version handshake, scaffolding for unsupported_workers Wire the proto-capability layer that the `MetricDetailStatus.unsupported_workers` field needs to be usefully populated. Previously the field was declared but always empty because the master had no way to know which workers could decode the new SetMetricDetail (tag 55) verb. This commit lays the rails; the dispatch-time gating itself remains a follow-up because it requires a dedicated `WorkerTask` impl for SetMetricDetail that synthesises a MetricDetailStatus reply rather than the current generic worker_request flow. The scaffolding shipped here: - New `sozu_command_lib::SOZU_PROTO_VERSION = 1` constant, baked into every Sōzu binary at compile time. Bumped any time a new wire- affecting `RequestType` or proto field needs capability gating. Version 1 covers `SetMetricDetail`, `METRIC_DETAIL_CHANGED`, the worker→master audit IPC, and per-lease peer-credential binding. - New `WorkerInfo.proto_version` proto field (tag 4, optional uint32) so TUI / status consumers can observe each worker's version directly. - New `WorkerSession.proto_version` master-side field, snapshotted at fork time from the binary's `SOZU_PROTO_VERSION` constant. The `to_info()` / `list_workers` paths populate the new proto field. - Proto comment on `MetricDetailStatus.unsupported_workers` updated to describe the capability-gate model AND the inherited-after-UpgradeMain caveat: a re-exec master does NOT yet re-query inherited workers' versions, so they retain the new master's compile-time value until the planned per-worker capability handshake on Status reply lands. Build/clippy clean; 53/53 TUI unit tests pass. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(metrics,command): bind lease ownership to peer credentials Replace the unauthenticated `lease_clear(client_id)` API with a binding- aware variant so one same-UID operator cannot clear another operator's lease by guessing the `client_id` format. The binding pairs the master- side `actor_pid` (captured via `SO_PEERCRED` at command-socket accept) with the per-connection session ULID; both halves must match the apply- time binding for the worker to authorise the clear. Implementation: - New `PeerBinding { pid, session_ulid }` + `LeaseEntry` + `LeaseClearOutcome` in `lib/src/metrics/mod.rs`. `lease_apply` records the binding alongside `(level, expires_at)`; `lease_clear` returns `Cleared`/`NotFound`/ `Unauthorized` depending on the apply-time binding vs the presented one. - A "binding unknown" apply (pre-binding caller or platform without `SO_PEERCRED`) preserves backward compat: the worker accepts any clear. A fully-known apply rejects every clear whose presented binding does not match, including the default ("unknown") clear. - New proto fields `SetMetricDetail.peer_pid` (tag 6) and `SetMetricDetail.peer_session_ulid` (tag 7), additive. Clients leave them empty; the master populates them in `bin/src/command/requests.rs::worker_request` from the connecting `ClientSession` before fan-out. - Worker parses the presented ULID via `rusty_ulid::Ulid::from_str` with a `0x…` hex fallback, then routes the `LeaseClearOutcome::Unauthorized` outcome to `WorkerResponse::error` so the operator gets a loud failure rather than a silent no-op. - Six new unit tests cover authorised clear, unauthorised mismatch, unknown-apply accepts-any, known-apply rejects default clear, the existing apply/clear/tick paths through the new signature. Build/clippy clean; 26/26 metric tests pass. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

style(top,proto): silence the last clippy and rustdoc warnings Clears the 14 warnings remaining after the simplify pass: - Five dead-code items: drop `render_placeholder` (all panes ship, no placeholder needed); drop `ClusterRow.errors_5xx_total` (the renderer reads `error_rate_pct`, never the raw count); drop `ThresholdTable.conn_warn_pct` (no consumer); drop `BackendRow.bytes_in` / `bytes_out` (the BACKENDS pane reads `back_bytes_in` / `back_bytes_out`). `Skin.categorical` gets a short doc + `#[allow(dead_code)]` because the field is read by tests and is the TOML surface for operator skins — the production consumer (cluster-row categorical tinting) lands in a follow-up. - Nine rustdoc warnings in the prost-generated `command.rs`: my earlier H2 commit's numbered-list comment in the `SetMetricDetail` preamble used 4-space indentation on the continuation lines, which prost-build expanded to overindented in the rendered docstring. Trim to 3 spaces so the rendered comment lands on the rustdoc list-item-alignment rule (3 spaces aligns with the text after `1. ` markers). - One rustdoc warning in `bin/src/ctl/top/mod.rs::run_top`: the doc comment had a `+` at line-start that markdown parsed as a list bullet, then complained the next two lines were unindented. Rephrase to join with "and" so the `+` no longer leads a line. Build is now warning-free under `cargo clippy --all-features --all-targets --locked`. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

docs(proto,changelog): scope METRIC_DETAIL_CHANGED to operator-initiated transitions The proto documented "every effective-level transition emits an `EventKind::METRIC_DETAIL_CHANGED` event", but only operator-initiated transitions emit today: the master-side audit-log wiring covers the SetMetricDetail fan-out path, but worker-local transitions (lease expiry on the polled janitor, post-fan-out apply/clear in the worker arm) leave a silent gap. Replicating the audit emission inside the worker `notify` arm needs a new IPC back to the master and is deferred to a follow-up. Update the `SetMetricDetail` doc comment and the `EventKind::METRIC_DETAIL_CHANGED` declaration to scope the contract to operator-initiated transitions, and add a CHANGELOG caveat so audit-log consumers know which transitions are not yet surfaced. The three previous `// TODO(sozu-top week 2):` markers in the worker arm now point to the proto doc as the source of truth for the scope. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(proto,command,server): SetMetricDetail TTL lease verb + plumbing Adds a runtime cardinality lease verb so `sozu top` can elevate the metrics drain to `MetricDetailLevel::Backend` for the duration of an interactive session. The lease design (TTL-bounded, `client_id`-keyed, self-expiring) is crash-safe and composes with multiple concurrent clients — see `Aggregator` lease bookkeeping in the previous commit. Proto (additive, backwards-compat): - `Request.request_type::SetMetricDetail = 55` carries `{ client_id, detail?, ttl_seconds?, clear?, reason? }`. - `ResponseContent::metric_detail_status = 16` returns `MetricDetailStatus { configured, effective, previous_effective, workers: map<id, WorkerMetricDetailStatus>, unsupported_workers[] }` for mixed-version-fleet safety. - `EventKind::METRIC_DETAIL_CHANGED = 30` on the `SubscribeEvents` audit stream; distinct from `METRICS_CONFIGURED` (Enabled/Disabled /Clear) since the cause is different. - `command/build.rs` re-attaches `Hash, Eq` for `MetricDetailStatus` (the embedded `map<string, WorkerMetricDetailStatus>` strips the prost auto-derive, which propagates to `ResponseContent.content_type` and `Request.request_type`). - `command/src/proto/display.rs` adds arms for `RequestType:: SetMetricDetail`, `ContentType::MetricDetailStatus` (with a prettytable renderer that lists per-worker configured/effective /previous_effective + unsupported workers), and `EventKind:: MetricDetailChanged`. - `command/src/request.rs` routes `SetMetricDetail` through the worker-level dispatch group (mirrors `ConfigureMetrics`). Master + worker plumbing: - `bin/src/command/requests.rs::is_mutating_verb` learns the new verb so the master brackets it with `RELOADING=1`/`READY=1` systemd hints. - The dispatch match routes through the existing `worker_request` fan-out path (same shape as `ConfigureMetrics`); per-worker `MetricDetailStatus` aggregation lands in week 2 when the TUI starts consuming it. - `lib/src/server.rs::notify` adds two hooks: a polled lease-expiry janitor at the top of every dispatch (gated by `lease_tick_due` so it only walks the lease table every 5 s), and the `SetMetricDetail` arm itself. The arm clamps the TTL via the `Aggregator` setter, decodes the `MetricDetail` enum defensively, and acks with `WorkerResponse:: ok()` for now (week 2 will return the per-worker `WorkerMetricDetailStatus` payload). `MetricDetailChanged` audit emission is left as a `TODO(sozu-top week 2)` at both `lease_apply`/`lease_clear` call sites and the janitor — plumbed through `bin/src/command/requests.rs::audit_emit_inline` once the master collects per-worker effective levels back. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

fix(command): default missing repeated/map fields when loading older state files `bin/src/command/requests.rs::load_state` reads each `\n\0`-separated JSON record via `command::parser::parse_several_requests::<WorkerRequest>`, which calls `serde_json::from_slice` per record. The prost-build config in `command/build.rs` attaches `Serialize`/`Deserialize` derives to every generated message but did not attach `#[serde(default)]` anywhere, so missing `repeated`/`map` fields rejected the record (`Vec<T>` and `BTreeMap<K, V>` are required-by-serde without an explicit default). Post-1.1.1 schema additions (`Cluster.answers`, `Cluster.authorized_hashes`, `RequestHttpFrontend.headers`, plus the listener-level `answers` / `alpn_protocols` fields) therefore broke `LoadState` for any older client (e.g. proxy-manager pinned to `sozu-command-lib = "1.1.1"`): the first `AddCluster` or `AddHttpFrontend` failed to deserialize, `parse_several_requests`'s `many0(complete(...))` left the unparsed bytes as the remainder, and the read loop reported `"Error consuming load state message"` to the client at EOF. Each post-1.1.1 `repeated`/`map` field on a state-file-emittable message now carries `#[serde(default)]` via a `field_attribute(... )` line in `command/build.rs`, so missing fields default to empty (mirroring the protobuf wire-format default). Required scalars stay strict on purpose: `ConfigState::add_cluster` keys the cluster map by `cluster_id` without a non-empty check, so a struct-level blanket would silently insert a bogus `""`-keyed entry. The field-level annotation preserves that defense-in-depth. The new contract is documented at the top of `command/src/command.proto` and inline in `command/build.rs`: any new `repeated` or `map` field on a message reachable from a `SaveState`/`LoadState` JSON file (anything emitted by `ConfigState::generate_requests`, or carried in a `RequestType` an external client may build and feed through `LoadState`) must add the matching `field_attribute(...)` line. Regression tests in `command/tests/state_compat_v1_1_1.rs` pin both the 1.1.1-shaped fixture round-trip and the missing-required-scalar rejection contract. Signed-off-by: Florentin Dubois <florentin.dubois@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(hsts): force-replace mode + Set replaces Delete Adds an operator opt-in to override backend-supplied `Strict-Transport-Security` instead of preserving it. Also folds in the broader cleanup that drops `HeaderEditMode::Delete` (zero producers in the diff) in favour of the new `Set` mode that subsumes its semantics via delete-then-insert. - New `HstsConfig.force_replace_backend` proto field (tag 5). Default RFC 6797 §6.1 backend-wins behaviour is unchanged (`HeaderEditMode::SetIfAbsent`); operators flip the field to `true` when a stale or weak upstream HSTS policy needs hardening at the proxy edge (`HeaderEditMode::Set`). - New `FileHstsConfig.force_replace_backend` field, plumbed through `to_proto` to the proto value; documented in `bin/config.toml` and `doc/configure.md`. - New `--hsts-force-replace-backend` CLI flag on `sozu frontend https/http add`. Treated as an enabling flag — `--hsts-force-replace-backend` alone enables HSTS with the canonical default `max-age = DEFAULT_HSTS_MAX_AGE`. Mutually exclusive with `--hsts-disabled`. Two new unit tests cover the force-replace cells. - New `HeaderEditMode::Set` (drop `Delete`). `Set` is delete-then-insert in one entry: the retain pass drops every header with the matching name, then the insert pass appends the new value unconditionally. The legacy empty-`val` Append delete encoding is preserved verbatim for backwards compatibility. - `Frontend::new` chooses `Set` vs `SetIfAbsent` based on `cfg.force_replace_backend`. The single materialiser site is the only consumer; the helper handles both modes uniformly. - Two new `apply_response_header_edits` unit tests cover the `Set` path (replaces existing header / inserts when absent). Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(command,lib): add HstsConfig proto + HeaderEditMode for SetIfAbsent Introduce the HSTS (RFC 6797) typed-config foundation: - new HstsConfig proto message with enabled/max_age/include_subdomains/ preload, attached as optional field on HttpsListenerConfig (tag 46), UpdateHttpsListenerConfig (tag 41), and RequestHttpFrontend (tag 16); inline rustdoc cites RFC 6797 §6.1, §7.2, §8.1, §11.4, §14.2 - new HeaderEditMode { Append, Delete, SetIfAbsent } and a `mode` field on HeaderEditSnapshot; apply_response_header_edits learns SetIfAbsent so upstream-supplied Strict-Transport-Security passes through unchanged (RFC 6797 §6.1 single-header requirement) - legacy empty-val Delete encoding preserved via Append+empty fallback so no existing call site needs updating in this commit - unit tests cover SetIfAbsent skip-when-present and insert-when-absent No behaviour change on the wire yet — the materialisation path (router → headers_response) lands in a follow-up commit. Existing struct literals updated only to add 'hsts: None' / 'mode: HeaderEditMode::Append'. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(events): add ClusterRecovered event (proto tag 29) Pairs with the existing NoAvailableBackends event (tag 2) so dashboards can plot per-cluster recovery as well as the all-down transition. Highest existing tag was 28 (HealthCheckUnhealthy); 29 was free. Refs #892 Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(redirect): add 302 (Found) and 308 (Permanent Redirect) policies Closes #1009. Sōzu's `RedirectPolicy` previously only supported `Permanent` (301) and `Unauthorized` (401) at the frontend. Two new variants: - `RedirectPolicy::Found` → 302 Found (RFC 9110 §15.4.3) — a temporary redirect; user agents MAY rewrite POST → GET on follow. - `RedirectPolicy::PermanentRedirect` → 308 Permanent Redirect (RFC 9110 §15.4.9) — like 301 but the HTTP method MUST be preserved on follow (no GET-rewrite on POST). Wire-level changes: - `command/src/command.proto::RedirectPolicy` gains `FOUND = 3` and `PERMANENT_REDIRECT = 4`. Tags 0..2 unchanged so v1.x clients still decode `FORWARD` / `PERMANENT` / `UNAUTHORIZED` correctly. - `HttpContext` gains `redirect_status: Option<u16>` stashed by `Router::route_from_request` per resolved policy. The answer engine reads it in `mux/mod.rs` to pick the matching `http.{301,302,308}.redirection` template; the legacy `cluster.https_redirect = true` path keeps its 301 default (the field is `None`). - `DefaultAnswer` gains `Answer302` and `Answer308` variants alongside `Answer301`; `set_default_answer_with_retry_after` dispatches all three through one redirect-shaped path. - Default templates `default_302` and `default_308` ship in-tree — `Connection: close`, `Sozu-Id: %REQUEST_ID`, single `Location: %REDIRECT_LOCATION`. Operator-supplied templates flow through the renamed `HttpAnswers::render_inline_redirect(code, …)`; `render_inline_301` is preserved as a thin wrapper for binary / source compatibility. - Per-status counters `http.302.redirection` and `http.308.redirection` fire alongside the existing `http.301.redirection`, both labelled by cluster + backend, mirroring the 301 emission path. Two e2e regression tests in `redirect_rewrite_auth_tests.rs`: - `try_redirect_found_h1_emits_302`: H1 frontend with `RedirectPolicy::Found` must emit 302 + HTTPS Location, no backend contact. - `try_redirect_permanent_redirect_h1_emits_308`: same but for 308. H2 frontend cells, plus the inline-template H2 path, remain to be backfilled in a follow-up; the H1 path covers the routing decision + answer-engine plumbing end-to-end. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

fix(router,mux,answers): close PR #1162 follow-up gaps on main Four gaps identified after the redirect/rewrite/answer-template/auth stack landed on `main`. PR #1162 ("Redirection and URL rewrite", @Wonshtrum, base `main`) is structurally superseded by the unified mux implementation but its last commit `448ece33 Rewriting fixes` carried two genuine correctness fixes that did not make it to `main`. The clusterless-redirect ordering bug and the doc/code drift on `%STATUS_CODE` are independent gaps surfaced by the same review pass. * Clusterless `RedirectPolicy::Permanent` now reachable. In `lib/src/protocol/mux/router.rs::route_from_request` the `RedirectPolicy::Permanent` branch is moved ahead of the `Unauthorized || cluster_id.is_none()` deny, so a frontend declared with `redirect = permanent` and no backing cluster (the canonical "this hostname has moved, no service remains" shape from #1161) emits 301 instead of 401. The `Permanent` block does not read `cluster_id`; the cluster-derived knobs already default to safe sentinels at the cluster lookup when `cluster_id` is `None`. The `let Some(cluster_id) = cluster_id else { unreachable!() }` binding moves below the deny block; its invariant still holds. * Non-trie host regex anchored at both ends. In `lib/src/router/mod.rs::convert_regex_domain_rule` the compiled regex now opens with `\A` and closes with `\z`. Without anchors, `Regex::is_match` is unanchored, so an operator's `/example\.com/` matched any hostname containing `example.com` as a substring, including `attacker.example.com.evil.org` — letting an attacker-controlled domain reach a frontend that should only serve `example.com` (CWE-1023, routing bypass). The trie path (`lib/src/router/pattern_trie.rs`) was already anchored. * Inner `/`-finding loop terminates on first match. Same function; the inner loop now `break`s after `found = true`, so a multi-segment regex hostname like `/seg1/.foo./seg2/.com` no longer overwrites `index` on every later `/` and the literal `.` separators between regex segments are kept in their correct position (`\Aseg1\.foo\.seg2\.com\z`). * `%STATUS_CODE` doc/code drift removed. The placeholder was advertised in `doc/configure.md`, the `redirect-template` CLI help in `bin/src/cli.rs`, and the `redirect_template` doc comments in `command/src/command.proto` and `command/src/config.rs`, but `lib/src/protocol/kawa_h1/answers.rs::HttpAnswers::template` never defined a `STATUS_CODE` variable. References are stripped so operators no longer see a promised variable that silently no-ops. Implementation deferred to the same change that adds 302/308 for #1009. Tests: `convert_regex` (updated for the anchored shape) plus three new unit / e2e regressions — `regex_domain_rule_rejects_suffix_and_prefix`, `regex_domain_rule_multi_segment_segments_are_isolated`, and `try_clusterless_permanent_redirect_emits_301` (asserts 301 + Location on the rewritten host). Closes parts of #1161 (proposal — clusterless permanent redirect was the last reachable gap) and #1154 (modify Host header — already addressed by `rewrite_host`, now closeable). Leaves #1009 partially open: 302 (`Temporary`) and 308 are still missing on `main`. Validation: * cargo build --all-features --locked * cargo clippy --all-targets --locked * cargo +nightly fmt --all -- --check * cargo test --workspace --locked (955 passed, 7 ignored) * cargo test -p sozu-e2e -- redirect (19 passed) Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Démolis <43861898+Wonshtrum@users.noreply.github.com> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

chore(comments,docs): drop review-process leakage; align stale h2c doc comments Inline comments and the proto file referenced PR numbers and 'Codex finding' from the cross-model review pipeline. Replace each with durable technical rationale so the committed text stands on its own merit and ages with the code rather than the review log. Also realign the doc comments on FileClusterConfig.health_check and HttpClusterConfig.health_check that still claimed HTTP/1.1-only probes — this PR adds h2c support, the proto already documents 'probe wire follows cluster.http2', the Rust-side config doc comments should match. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

refactor(health-check): derive h2c probe from cluster.http2; drop is_h2c The probe wire format and the data-plane backend connection both need to choose between HTTP/1.1 and HTTP/2 (h2c) for the same backends. Carrying that decision twice — once on `Cluster.http2` (read by the mux router at `protocol/mux/router.rs::Router::connect`) and once on `HealthCheckConfig.is_h2c` — invites the two flags to drift, so an h2c-only backend gets probed with HTTP/1.1 (or vice versa) the moment an operator updates one without the other. Collapse to a single source of truth: derive the probe wire from `cluster.http2` directly. The probe and the data-plane backend connection now share one switch and cannot diverge. Wire / API surface - `HealthCheckConfig.is_h2c` (proto field 7) is removed. Field 7 remains reserved-by-omission; new fields will start at 8. - `FileHealthCheckConfig.is_h2c` removed; `to_proto` no longer carries the field. - `--h2c` CLI flag on `sozu cluster health-check set` removed. - `BackendMap` gains a `cluster_http2: HashMap<ClusterId, bool>` populated from `Cluster.http2` on every `AddCluster`. The health-check probe reads the entry at probe-creation time and records it on the `InFlightCheck` (`h2c: bool`) so the response parser stays consistent even if the operator flips the flag mid-probe. Documentation - `doc/health_checks.md` rewrites the wire-format paragraph: probe follows `cluster.http2`; no `is_h2c` knob. - The configuration-parameters table drops the `is_h2c` row and gains a paragraph explaining the lockstep with `cluster.http2`. - `CHANGELOG.md` Added bullet rephrased: "cluster-derived h2c probes" instead of "opt-in h2c probes". Validation - cargo build --all-features --locked: pass. - cargo +nightly fmt --all -- --check: pass. - cargo clippy --all-targets --all-features --locked: 0 errors, 0 warnings. - cargo test -p sozu-lib health_check::: 12 passed (existing h2c unit tests still cover the parser; they construct configs without the dropped field). Signed-off-by: Florentin Dubois <florentin.dubois@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(health-check): h2c prior-knowledge probe support Add an opt-in HTTP/2 prior-knowledge (cleartext, h2c) probe path so operators can health-check backends configured with `cluster.http2 = true` without co-locating an HTTP/1.1 endpoint. Wire / API surface: - `HealthCheckConfig.is_h2c` (proto field 7, optional bool, default false). The TOML key on `[clusters.<id>.health_check]` and the `FileHealthCheckConfig` struct gain a matching field. - `sozu cluster health-check set --h2c` flag forwards the bool through the request-builder validator. Probe wire: - 24-byte connection preface `PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n`. - Empty client SETTINGS frame (0-byte payload, type 0x04, flags 0, stream 0). - HEADERS frame on stream 1 with END_STREAM | END_HEADERS (flags 0x05) carrying a hand-rolled HPACK header block: * `:method GET` indexed (static idx 2 → 0x82). * `:scheme http` indexed (static idx 6 → 0x86). * `:path <uri>` literal w/o indexing, name idx 4 (0x04). * `:authority <host:port>` literal w/o indexing, name idx 1 (0x01). Length octets use the HPACK 7-bit-prefix integer form, including the multi-byte continuation chain for values > 127 bytes. Response parser (`try_parse_h2c_status`): - Walks frames in the buffered response, ignoring SETTINGS, SETTINGS ACK, DATA, etc. until it finds a HEADERS frame on stream 1. - Strips PADDED and PRIORITY prefixes correctly. - Decodes `:status` from either: * Static-table indexed forms 0x88..0x8E (200, 204, 206, 304, 400, 404, 500), or * Literal-with-indexed-name forms 0x08 / 0x18 (literal w/o indexing or never-indexed for name index 8) followed by length + 3-byte ASCII status code. - A GOAWAY frame is treated as a probe failure. - Returns `None` while the buffer is truncated mid-frame so the caller keeps reading. Dispatch in `progress_checks`: a single `parse_probe_response` helper chooses HTTP/1.1 status-line parsing or h2c frame walking based on `config.is_h2c`. The HTTP/1.1 path is unchanged. Tests (in `lib/src/health_check.rs::tests`): - `build_h2c_probe_starts_with_preface_and_settings` - `h2c_indexed_status_200_is_healthy_for_any_2xx` - `h2c_indexed_status_500_fails_default_2xx_check` - `h2c_literal_status_503_matches_expected_503` - `h2c_goaway_marks_unhealthy` - `h2c_truncated_buffer_returns_none` - `h2c_padded_headers_strips_pad_length_octet` Documentation: `doc/health_checks.md` updates the "HTTP/1.1 only" note to describe the new `is_h2c` opt-in, the wire shape, and the parser coverage. HTTPS (h2 over TLS) probes remain a follow-up. Validation: - cargo build --no-default-features --features crypto-ring: pass. - cargo +nightly fmt --all -- --check: pass. - cargo clippy --all-targets --no-default-features --features crypto-ring: 0 errors, 1 cosmetic warning unrelated. - cargo test -p sozu-lib --no-default-features --features crypto-ring health_check::: 11 passed (5 prior + 6 new h2c). Signed-off-by: Florentin Dubois <florentin.dubois@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(health-check): non-blocking HTTP health checks with fail-open and CLI Add HTTP/1.1 backend health-check probing that runs inside the existing single-threaded mio event loop. No additional threads, no async runtime. Key changes: - New lib/src/health_check.rs (~480 LOC) with HealthChecker, threshold- based state machine, jittered intervals, response size cap (4 KB), CRLF-sanitised URIs, per-backend in-flight tracking. Sockets are registered with mio in a dedicated bounded token namespace [HEALTH_CHECK_TOKEN_BASE, HEALTH_CHECK_TOKEN_BASE + HEALTH_CHECK_TOKEN_CAPACITY) = [1<<24, 1<<24 + 1<<16) so the upper bound never falsely claims the mux GOAWAY sentinel Token(usize::MAX). The allocator picks slot offsets modulo the capacity and skips offsets matching in-flight checks; if the table is full it logs an error and returns None rather than silently colliding. - Fail-open routing in lib/src/load_balancing.rs: when ALL backends for a cluster are unhealthy, route to the Normal backends rather than returning 503 (Amazon health-check paper recommendation). - Backend.health::HealthState machine in lib/src/backends.rs with consecutive success/failure counters and Up/Down event emission. - Server event loop integration in lib/src/server.rs: poll the health checker each iteration; dispatch ready tokens; pass Registry for register/deregister. - Server-side validation (command/src/request.rs) rejects zero interval/timeout/thresholds and bad URIs. - Proto: HealthCheckConfig (fields 1-6), SetHealthCheck (cluster_id, config), QueryHealthChecks (optional cluster_id), HealthChecksList (map<cluster, config>) at command/src/command.proto. - CLI: cluster health-check {set,remove,list} subcommands in bin/src/cli.rs with --uri/--interval/--timeout/--healthy-threshold /--unhealthy-threshold/--expected-status flags. CLI-side validation rejects URIs missing a leading '/' and rejects \r, \n, NUL, and any C0 control byte (RFC 9110 §5.1) — not just CR/LF. - Master dispatch in bin/src/command/requests.rs broadcasts state-mutating health-check requests and aggregates worker responses on query. - ConfigState dispatch + persistence in command/src/state.rs. - Tabular display in command/src/proto/display.rs. - Metrics health_check.{success,failure,up,down,healthy_backends}. - Log envelope: every emit in health_check.rs prefixes its format string with "{}, log_context!()" so the lib/tests/log_layout.rs regression guard recognises the canonical HEALTH-CHECK tag (mirrors the hyphenated MUX-H2 / PROXY-RELAY / TLS-RESOLVER convention). Adaptations vs the original PR #1191 commit (rebase onto post-1209 main): - Proto field renumbering. Post-1209 main occupies several carriers PR #1191 originally used; the rebased commit retargets: * Request.set_health_check 47 -> 52 * Request.remove_health_check 48 -> 53 * Request.query_health_checks 49 -> 54 * Cluster.health_check 8 -> 15 * ResponseContent.health_checks_list 14 -> 15 * EventKind.HEALTH_CHECK_HEALTHY 5 -> 27 * EventKind.HEALTH_CHECK_UNHEALTHY 4 -> 28 - Token namespace tightened (bounded modulo allocator + bounded owns_token range) to avoid claiming the mux GOAWAY sentinel. - URI sanitisation hardened to reject every C0 control byte. - ClusterCmd::HealthCheck variant placed alongside ClusterCmd::H2 (PR #1191's structure), not nested under ClusterH2Cmd. Validation: - cargo build --no-default-features --features crypto-ring: pass - cargo +nightly fmt --all -- --check: pass - cargo clippy --all-targets --no-default-features --features crypto-ring: 0 errors, 1 cosmetic warning (redundant_closure on Criterion bench setup, unrelated to this commit). - cargo build --tests --no-default-features --features crypto-ring: pass. Signed-off-by: Florentin Dubois <florentin.dubois@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(splice): operator-tunable kernel-pipe capacity Make the splice(2) kernel-pipe capacity per direction configurable via a new `splice_pipe_capacity_bytes` field on ServerConfig, instead of the hard-coded 64 KiB constant. The override flows through the same path as the existing `basic_auth_max_credential_bytes` knob: declared in `command/src/command.proto` (ServerConfig field 22), wired through `FileConfig`, the `ConfigBuilder` mapper, the runtime `Config`, and the proto round-trip via `From<&Config> for ServerConfig`. Linux-only storage at the lib layer: a `OnceLock<usize>` in `lib/src/splice.rs` populated once per worker boot from `Server::try_new_from_config` (cfg-gated), with a setter that no-ops on `0` so an explicit zero does not collapse the pipe to PAGE_SIZE. `SplicePipe::new` now applies the configured capacity via `fcntl(F_SETPIPE_SZ)` on each pipe and reads the realised value back with `fcntl(F_GETPIPE_SZ)`, storing the smaller of the two sizes in a new `capacity` field. This handles the kernel's behaviour: it rounds up to PAGE_SIZE and clamps at `/proc/sys/fs/pipe-max-size` (default 1 MiB unprivileged; CAP_SYS_RESOURCE goes higher). On `F_SETPIPE_SZ` failure the kernel keeps the previous capacity (typically 64 KiB), the failure is logged at `warn!`, and SplicePipe continues with that realised value — splice still works, just at the kernel default. `splice_in` gains a `len: usize` parameter so callers thread the realised capacity through; pipe.rs reads `splice_pipe.capacity` for both the per-call `len` and the "pipe is full" backpressure check (replaces the deleted `SPLICE_PIPE_CAPACITY` const). CHANGELOG, doc/getting_started.md (feature-flag table), and doc/configure.md (global parameters table) updated to document the new knob and its kernel-side limits. Signed-off-by: Florentin Dubois <florentin.dubois@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(http,https,mux): add X-Real-IP injection and anti-spoof elision (H1+H2) Two listener-scoped opt-in bool flags, both default false and independently combinable. Configurable on `HttpListenerConfig` / `HttpsListenerConfig` and runtime-patchable through the new `Update*ListenerConfig` partial-update verbs. - elide_x_real_ip: when true, any client-supplied `X-Real-IP` header is stripped from the request before forwarding (anti-spoofing). - send_x_real_ip: when true, a proxy-generated `X-Real-IP` header carrying the connection peer IP is appended. The IP is read from the post-PROXY-v2 `session_address`, so deployments terminating PROXY v2 surface the original client IP, not the upstream proxy's. Both H1 and H2 are covered by a single elision branch in `HttpContext::on_request_headers`, dispatched from `pkawa::handle_header` for H2 initial HEADERS frames. H2 trailer HEADERS frames take a separate path through `pkawa::handle_trailer` and would otherwise bypass the elision; `handle_trailer` now drops `x-real-ip` trailer pairs when the listener flag is set, closing that gap. Five e2e tests cover the four-flag matrix on H1 (including PROXY-v2 unwrap with original client IP) plus an H2 trailer regression placeholder. Mirrors the `strict_sni_binding` precedent for trait accessor / mux Context propagation / Update*ListenerConfig apply path. Closes #1113 Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Démolis <43861898+Wonshtrum@users.noreply.github.com> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

feat(server,mux,tcp,cli): per-cluster per-IP connection limit with HTTP 429 + graceful TCP close Caps the number of simultaneous frontend connections one source IP may hold against a given cluster. Replaces the simpler per-source-IP variant the previous PR #1193 commits proposed (those did not survive the H2 mux unification). Closes #890, #1057. Wire shape (proto): - ServerConfig.max_connections_per_ip = 22 [optional, default 0] - ServerConfig.retry_after = 23 [optional, default 60] - Cluster.max_connections_per_ip = 13 [optional, override] - Cluster.retry_after = 14 [optional, override] - CustomHttpAnswers.answer_429 = 12 [optional, custom 429 template] - Request.set_max_connections_per_ip = 50 (uint64) - Request.query_max_connections_per_ip = 51 (QueryMaxConnectionsPerIp {}) - ResponseContent.max_connections_per_ip_limit = 14 (MaxConnectionsPerIpLimit { limit: u64 }) Override semantics: cluster `None` inherits the global default, `Some(0)` is explicit "unlimited for this cluster", `Some(n > 0)` overrides. Source IP is the parsed PROXY-protocol source when present, else `peer_addr`. Enforcement: - HTTP/HTTPS via the unified mux at protocol/mux/router.rs::connect: after cluster resolution, before backend selection. The check fires AFTER auth/redirect/SNI decisions so a 401/421/redirect frontend never trips the limit. On hit: stash the resolved Retry-After on the stream context and return BackendConnectionError::TooManyConnectionsPerIp, which the mux converts into a 429 default answer through the new set_default_answer_with_retry_after path. Covers H1 and H2 — the unified mux serves both protocols. - H2 multiplex semantics: SessionManager keeps a per-token HashSet<(cluster, ip)>. Multiple streams to the same (cluster, ip) from the same H2 connection share one slot — the limit governs distinct frontend connections, not requests. Decrement is wholesale on session close (untrack_all_cluster_ip). - TCP at tcp.rs::connect_to_backend: rejection produces a graceful FIN via SessionResult::Close (no SO_LINGER trick). Answer engine (lib/src/protocol/kawa_h1/answers.rs + lib/src/protocol/mux/answers.rs): - Answer429 variant on DefaultAnswer with retry_after: Option<u32>. - 429 template registered alongside 421/503/etc. - New %RETRY_AFTER variable with `or_elide_header = true`: when the resolved value is 0/None, the engine drops the entire `Retry-After:` line. `Retry-After: 0` would invite an immediate retry that defeats the limit, so we omit instead of rendering literal 0. - legacy_to_map flattens CustomHttpAnswers.answer_429 into the per-listener answers map. - default_answer_for_code(429, ...) builds the variant. - set_default_answer_with_retry_after threads the resolved retry value through to the rendered answer. Proxy-protocol fixes (lib/src/protocol/proxy_protocol/{expect,relay}.rs): - ExpectProxyProtocol::into_pipe and RelayProxyProtocol::into_pipe now use ProxyAddr::source() from the parsed v2 header instead of the raw TCP peer_addr. Without this fix the pipe phase records the upstream PROXY-emitter (an LB / edge proxy / health-check probe), not the originating client — which means the per-(cluster, source-IP) limit would have keyed on the LB's IP. Relay mode previously discarded the parsed addresses entirely; we now stash them on the struct. SessionManager (lib/src/server.rs): - max_connections_per_ip: u64 + retry_after: u32 fields. - connections_per_cluster_ip: HashMap<(String, IpAddr), usize> + cluster_ip_tracks: HashMap<Token, HashSet<(String, IpAddr)>>. - cluster_ip_at_limit (token-aware: a token already holding a slot is never at the limit), track_cluster_ip (idempotent within a token), untrack_all_cluster_ip (drains on session close), effective_max_connections_per_ip / effective_retry_after (override resolution), clear_cluster_ip_tracking (runtime disable). - HttpProxy / HttpsProxy / TcpSession close paths drain via untrack_all_cluster_ip on the frontend token before the slab slot is reused. ProxySession trait (lib/src/lib.rs): - New cluster_id() and session_address() default-implemented as None. - New L7Proxy::sessions() so the mux router can reach the SessionManager from a Rc<RefCell<dyn L7Proxy>>. - New BackendConnectionError::TooManyConnectionsPerIp variant with cluster_id payload. CLI (bin/src/{cli,ctl/mod,ctl/request_builder}.rs): - `sozu connection-limit set <N>` / `remove` / `show` patches the global limit at runtime. The setter is non-sticky (workers reset to the TOML-configured value on restart); operators must mirror the change in the config to make it durable. - `--answer-429 <path>` flag added to `sozu listener {http,https} update`. - New build_http_answers helper threads answer_429 through the existing CustomHttpAnswers builder. Config (command/src/config.rs): - DEFAULT_MAX_CONNECTIONS_PER_IP = 0 (disabled), DEFAULT_RETRY_AFTER = 60. - FileConfig + Config + ServerConfig From conversion threaded. - FileClusterConfig + HttpClusterConfig + TcpClusterConfig + ListenerBuilder + CustomHttpAnswers gain the new fields. Metric: connections.rejected_per_cluster_ip (counter, labelled by cluster_id, backend_id always empty — rejection happens before backend selection). Replaces the simpler `connections.rejected_per_ip` from the older PR #1193 commits — single name covers H1, H2, TCP. Tests: - e2e/src/tests/cluster_ip_limit_tests.rs covers H1 global limit (429 + Retry-After), H1 Retry-After: 0 elides the header, H1 per-cluster override (unlimited cluster coexists with capped), TCP graceful close on limit. All four pass `cargo test -p sozu-e2e cluster_ip_limit`. Validation: - cargo build --all-features --locked - cargo +nightly fmt --all -- --check - cargo clippy --all-features --all-targets --locked -- -D warnings - cargo test --workspace --locked Docs: bin/config.toml, doc/configure.md (parameters + metric), CHANGELOG.md. Co-authored implementation guidance from Codex on the H2 multiplex semantics, Retry-After: 0 elision, PROXY-protocol Relay-mode address stashing, and rolling-upgrade safety (optional proto fields, not required). Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Démolis <43861898+Wonshtrum@users.noreply.github.com> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

feat(server): evict least-active sessions when accept queue is full Add an opt-in `evict_on_queue_full` knob (proto `ServerConfig` field 22, default `false`) that, when set, evicts the oldest 1% of non-listener sessions to make room for queued sockets when `SessionManager::check_limits` refuses a new accept. Selection uses `select_nth_unstable_by_key` (introselect, O(n) average) over `Session::last_event()` to partition the candidate slab in place, avoiding an O(n log n) sort. The cap loop in `Server::create_sessions` keeps the existing `incr!("listener.connection_capped")` counter (so dashboards stay meaningful regardless of eviction outcome), runs the eviction batch, and re-checks limits before continuing. A new `sessions.evicted` counter is emitted only when the mitigation fires. Eviction is deliberately skipped during graceful `shutting_down`: forcing sessions closed there defeats the shutdown semantics and is wasted work since the worker is winding down anyway. Default `false` because during a DDoS the active sessions are more likely to be legitimate clients than the queue contents — evicting them would serve attackers. Operators dominated by normal traffic spikes can flip it on. Closes #644, closes #916. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

chore(proto): drop preemptive `reserved` blocks on Cluster and RequestHttpFrontend The `reserved 13 to 19` (Cluster) and `reserved 16 to 19` (RequestHttpFrontend) blocks were defensive "reserve room for future fields" — not idiomatic protobuf. The `reserved` keyword is meant to guard against tag REUSE after a field is deleted, so an old serialised message doesn't alias into a new field with an incompatible type. We didn't delete anything; we only added new fields. The next person adding a field would just pick the next free tag (13 / 16) anyway, and removing the `reserved` line would be a single-line edit if those numbers were genuinely needed. Drops the blocks; the regenerated `command.rs` shrinks accordingly. No wire-format change, no downstream impact. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(config,auth): expose basic_auth_max_credential_bytes + warn at 33% buffer The maximum length of a base64-decoded `Authorization: Basic` payload the worker accepts was a hard-coded 4 KiB. Exposing it in the main TOML config lets operators on hardened tenants lower it (256 / 512 typical) to bound the per-failed-auth allocation tighter against hostile peers sending large tokens. Surfaces: - New `ServerConfig.basic_auth_max_credential_bytes` proto field (tag 21, optional uint64), threaded from `FileConfig.basic_auth_max_credential_bytes` through `Config` so the value rides the existing config-load and state-restore plumbing. - `lib::protocol::mux::auth` replaces the const cap with a `OnceLock<usize>` overlay over a built-in default (4096). The override is committed once on each worker at boot via `lib::server::Server::try_new_from_config` calling `set_max_decoded_credential_bytes` — a single set-once handoff with no per-request atomic. Subsequent attempts to mutate are no-ops (`OnceLock::set` rejects), and an explicit `0` is treated as "use default" so a typo cannot disable the cap by accident. New unit test pins the zero-is-noop semantics. - Config validator emits a `warn!` at boot when the operator-set cap is `>= buffer_size / 3`. At that point a single failed-auth attempt can pin ~33% of the per-frontend buffer's worth of bytes; combined with in-flight request/response framing the buffer trends toward back-pressure under load. Informational only — operators with a deliberate threat model can keep the value, but the surprise stays visible in the boot log. - `doc/configure.md` "HTTP Basic authentication" section gains a "Tuning the credential decode cap" subsection covering both the knob and the 33% warning. `CHANGELOG.md` mentions both. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

fix(cli,router,auth): clear /review's deferred Medium/Low/Nit backlog Closes the items the prior `fix(answers,auth,router)` pass deferred: * M2 — `sozu frontend {http,https} add` gains the per-frontend policy flags (`--redirect`, `--redirect-scheme`, `--redirect-template`, `--rewrite-host`, `--rewrite-path`, `--rewrite-port`, `--required-auth`, `--header <position>=<name>=<value>`). The two `frontend …` siblings now share a single `build_http_frontend_add` helper so they cannot drift; each policy field is validated up-front (range-checked, scheme/policy parsed against the proto enum) so a malformed input surfaces as a typed `CtlError::ArgsNeeded` instead of reaching the worker. * M3 — `Route::Frontend(Rc<Frontend>)` is reachable. `HttpFrontend` carries the new policy fields all the way through `RequestHttpFrontend::to_frontend`, and `Router::add_http_front` flips onto the rich path whenever any policy field is non-default (otherwise still produces the legacy `Route::ClusterId` / `Route::Deny` shapes). The dead-arm warning is no longer hypothetical — the variant is exercised end-to-end on the live routing path. * L4 — `mux::auth::canonicalize_basic_credentials` tightens the whitespace grammar to `1*SP` per RFC 7235 §2.1 / RFC 9110 §11.4. SP before and between scheme and token68 is tolerated; HTAB and zero-spaces are rejected. * L5 — `ConfigError::InvalidHeaderPosition` becomes `{ index, position }` so a multi-entry config pinpoints the bad row. `parse_header_edit` takes the array index from `entries.iter().enumerate()`. * L6 — new `template_fill_adjacent_body_variables` regression test pins the `body_size` accounting against an `[%ROUTE%ROUTE]` body so a future tweak to the inter-chunk arithmetic cannot silently break back-to-back placeholders. * N2 — `Cluster` reserves field numbers 13-19 and `RequestHttpFrontend` reserves 16-19. A future migration that accidentally aliases an old field number now fails at proto compile time. Knock-on: every test fixture and example builder constructing `HttpFrontend { … }` directly inherits the eight new `Option`/`Vec` fields; existing in-tree call sites are populated with `None` / `Vec::new()` (see `lib/src/http.rs::frontend_from_request_test`). The 4 cluster-add CLI tests added in c1e0b1a6 plus the new `template_fill_adjacent_body_variables` bring sozu-lib to 436 passing tests; sozu-command-lib stays at 66/5 ignored. Build, clippy `-D warnings`, fmt --check, log-layout regression all clean. Out of scope for this pass * The header-injection runtime path on the mux side still needs to consume `Frontend.headers_request` / `headers_response` at `mux/h1.rs::writable` and `mux/h2.rs::write_streams`. The plumbing is in place (the routing layer builds and stashes the slices) — the actual emission is the next round of work. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

fix(answers,auth,router): act on /review findings Apply the High and tractable Medium/Low/Nit items the read-only review surfaced. Build, clippy --all-targets -D warnings, fmt --check, sozu-lib (435 pass) and sozu-command-lib (66 pass) all green. High * answers.rs: stop unconditionally splicing a synthetic `Content-Length` header into rendered responses. The engine now detects an operator-supplied `Content-Length:` line during the parse pass and routes its value through the existing `ContentLength` placeholder, so the rendered response carries exactly one header with the auto-computed body size. Previously a template with a literal `Content-Length:` line ended up with two headers — RFC 9110 §8.6 / RFC 7230 §3.3.2 request-smuggling vector. * mux/auth.rs: pad both candidate and stored hash into a fixed `[u8; AUTH_COMPARE_PAD_LEN + 8]` envelope (256-byte body + 8-byte little-endian length) before `subtle::ConstantTimeEq::ct_eq`, so the per-entry compare loop iterates the full padded length even when lengths differ. `subtle`'s slice `ct_eq` short-circuits on length mismatch — the padding here defeats that leak. Closes the realm-size and matching-username-length channels (CWE-208 family). * kawa_h1/editor.rs: `HttpContext::reset()` now clears `redirect_location` and `www_authenticate` between pipelined H1 requests so a future 301/401 default-answer path that bypasses routing cannot inherit a stale Location / realm from a prior request. Backed by the existing `test_reset_clears_request_response_state` regression, extended with the two new field assertions. Medium * command.proto: `HeaderPosition` gains an explicit `HEADER_POSITION_UNSPECIFIED = 0` variant; the proto-default-encoded shape now deserialises into a typed "unset" instead of failing `HeaderPosition::try_from(0)`. The runtime drops `Header { position: Unspecified, … }` entries with a `warn!` rather than guessing a position. * mux/auth.rs: replaces the hand-rolled `to_hex` (which carried two `expect("hi/lo nibble")` panics on an auth path) with `hex::encode` from the existing `hex` workspace dep. Drops the corresponding unit test and the `_METHOD_USED` workaround that kept the otherwise- unused `Method` import alive. * kawa_h1/answers.rs: `HttpAnswers::new` propagates `TemplateError` from the bundled-default-template parse path via `?` instead of `expect(...)`. The new `default_templates_all_parse` regression test guards the invariant. * router/pattern_trie.rs: new `segment_regex_rejects_partial_matches` test pins the `\A...\z` anchoring contract — `cdn[0-9]+` matches `cdn1` and `cdn123` but rejects `cdn1xxx`, `xxxcdn1`, and `cdnabc`. Without the test the next clippy or refactor pass could silently revert anchoring. Low / Nit * request_builder.rs: validate `--https-redirect-port` against `1..=65535` before sending so a typo doesn't render `Location: https://host:70000/...` on the wire. Lifts `looks_like_authorized_hash` to module scope and adds 6 unit tests covering canonical form, missing colon, short hex, uppercase, empty username, non-alnum username. * router/mod.rs: drop the dead-defensive `from_utf8(b).unwrap_or_default()` in `RewritePart`; `pattern` is `template.as_bytes()` and the split is on the ASCII byte `$`, so `template[start..i]` lies on char boundaries and indexes safely. * mux/router.rs: gate the legacy `cluster.https_redirect` `redirect_location` stash on `proxy.kind() == ListenerType::Http` so an HTTPS listener never carries a stale URL into a downstream default-answer path. Replaces `cluster_id.expect(...)` with a `let-else { unreachable!() }` form per project style. * router/mod.rs: tighten the `log_module_context!` doc-comment to reflect the single-call-site reality (`Frontend::new`'s warn). Out of scope (deferred to follow-up) * /review M2: per-frontend `--redirect`, `--rewrite-*`, `--header` CLI flags (cluster-level flags shipped in c1e0b1a6). * /review M3: making the `Route::Frontend(Rc<Frontend>)` variant reachable from `add_http_front` — needs the Wave 1c bridge that converts `RequestHttpFrontend` policy fields into a `Frontend`. * /review L4: `Authorization` whitespace tolerance (RFC 7235 §2.1 permits SP only; current accepts SP and HT). * /review L5: `parse_header_edit` failure does not include array index — minor diagnostic polish. * /review L6: `body_size -= variable.name.len() + 1` accounting fuzz test for adjacent `%VAR%VAR` patterns. * /review N2: `reserved` declarations in `Cluster` / `RequestHttpFrontend` / listener configs. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(proto): add answer templates, redirect, rewrite, headers, auth schema Lay the wire schema for the bundled feature work that brings PR #1206..#1210 onto current main. Wave 1 of a multi-wave implementation; subsequent waves land the runtime engine, mux integration, auth helper, e2e tests, and docs. Cluster gains four fields: per-cluster `answers` (status code → template body) at field 9, `https_redirect_port` at 10, `authorized_hashes` at 11, and `www_authenticate` realm at 12. RequestHttpFrontend gains eight: `redirect` policy at 8, `required_auth` at 9, `redirect_scheme` at 10, `redirect_template` at 11, `rewrite_host`/`rewrite_path`/`rewrite_port` at 12-14, and a repeated `headers` list at 15. Listener configs gain a parallel `answers` map at 31 (HttpListenerConfig) and 43 (HttpsListenerConfig); the legacy `CustomHttpAnswers http_answers` field is preserved on the wire so existing state files round-trip — the runtime will read both for one minor. New top-level enums RedirectPolicy / RedirectScheme / HeaderPosition and a Header message land alongside. An empty `Header.val` deletes the named header (HAProxy `del-header` parity). prost stops auto-deriving Hash/Eq once a message holds a map field, so the build script gains explicit `#[derive(Hash, Eq)]` for Cluster, HttpListenerConfig, HttpsListenerConfig, UpdateHttpListenerConfig, and UpdateHttpsListenerConfig. Existing initializers grow a `..Default::default()` fallthrough so this commit is a pure additive scaffolding step — no runtime behaviour changes yet. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(config): expose slab_entries_per_connection knob (default 4, [2,32]) The slab capacity multiplier was a private constant SLAB_ENTRIES_PER_CONNECTION = 4 introduced by the H2 mux work to accommodate stream multiplexing (1 frontend + up to 3 backend connections per session). Operators with topologies that fan out across more than 4 backends per session had no recourse short of a recompile, and slab exhaustion presents as "accept refused" with no telemetry pointing at the slab. Adds an optional uint64 slab_entries_per_connection field to ServerConfig (proto tag 20) and a matching Config field. Effective value flows through ServerConfig::effective_slab_entries_per_connection which clamps to [MIN_SLAB_ENTRIES_PER_CONNECTION = 2, MAX_SLAB_ENTRIES_PER_CONNECTION = 32]; absent or 0 falls back to DEFAULT_SLAB_ENTRIES_PER_CONNECTION = 4 so existing deployments keep their current capacity. Documented in doc/configure.md alongside max_buffers / buffer_size. A `worker.slab.utilization_pct` operator-facing gauge is left as a follow-up (gauge wiring lives in the worker hot path; this commit is config plumbing only). PR #1209 consolidated review (MED-6). Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

refactor(command): finish audit log enrichment — ts, actor_role, connect_ts, boot_generation, build_git_sha, JSON sink Closes the deferred items from 6b21c9c4. After this commit, the only audit-log gaps left are architectural decisions that need their own designs: OTel trace_id/span_id (proto bump on Request to carry tracing context end-to-end) and Linux auditd integration (new kernel-side sink, large scope). `source_ip=` stays permanently N/A while the command channel is unix-only. New mandatory fields on every `Command(...)` line ------------------------------------------------- * `ts=<RFC3339 UTC microseconds>` — in-body timestamp. Lets operators extract a single audit line and still know when it fired without cross-referencing the outer logger prefix. Implemented with a std-only Hinnant `civil_from_days` converter — no `time` or `chrono` dep added. * `actor_role=root|system|user|unknown` — buckets the actor uid so SOC dashboards can route on it without parsing UID values per host (uid=0 → root, 1..1000 → system, 1000+ → user, missing → unknown). * `connect_ts=<RFC3339 UTC>` — wall-clock `accept(2)` time of the client connection, stamped on `ClientSession.connect_ts` via `SystemTime::now()`. Forensic windowing — "all verbs from connections that opened in the 30s before the incident". * `boot_generation=<u32>` — counter incremented at every `MAIN_UPGRADED` re-exec, persisted across the re-exec via `UpgradeData.boot_generation`. Disambiguates post-upgrade sessions from pre-upgrade ones — PIDs reset, but `(boot_generation, session_ulid)` is a durable correlation pair. * `build_git_sha=<12hex>` — short git SHA embedded by `bin/build.rs` via a `cargo:rustc-env=SOZU_BUILD_GIT_SHA=…` directive. Falls back to `unknown` outside a git tree (vendored tarballs, sysroots). `bin/build.rs` re-runs only when `.git/HEAD` or `.git/refs` move so cached cargo builds stay fast. JSON sink — `audit_logs_json_target` ------------------------------------ Mirrors every audit line as a single-line JSON object to a dedicated file. Same `O_APPEND | O_CREAT | 0o640` lifecycle as the human sink. Schema is stable; missing values are JSON `null` so SIEM pipelines can flatten without conditional fields. The `audit` block groups actor identity (uid/gid/pid/user/comm/role) and the `extras` block groups completion-time fields (elapsed_ms, fanout, error_code, reason, request_sha256). Both sinks (text + JSON) are independent — operators can set both for tail-friendly + machine-parseable. Plumbed through `FileConfig` → `Config` → `ServerConfig` (proto tag 19) so workers see the field even though only the main process writes to it. Server gains `audit_log_json_writer: Option<RefCell<File>>` opened at boot via the same `open_audit_log_file` helper as the text sink. `Server.boot_generation` + `UpgradeData.boot_generation` ------------------------------------------------------- * New `pub boot_generation: u32` on `Server`, `0` on first boot, `saturating_add(1)` at every `upgrade_main` invocation BEFORE the re-exec serialises `UpgradeData` so the new main starts at the bumped value. * `UpgradeData.boot_generation` carries it across the re-exec boundary (`#[serde(default)]` so old upgrade payloads decode as `0`). * The `MainUpgraded` audit line records the new generation in its `target=` field for the audit-trail itself. Documentation + retention policy (LISA-013) ------------------------------------------ * `doc/observability.md` documents the JSON schema with a worked example. * New retention-policy section: PCI-DSS 10.7 calls for ≥ 1 year of audit retention with the most recent 3 months immediately available. Recommended shape: `logrotate` daily compress, off-host archive after 90 days, 400-day cold retention. Sōzu does NOT rotate audit files itself — delegated to the OS-level rotator with `copytruncate` since sōzu keeps the file handle open. (`SIGHUP` re-open is a TODO.) * Sample configs (`bin/config.toml`, `os-build/config.toml`) document `audit_logs_json_target` next to `audit_logs_target`. Drift-guard ----------- * `audit_format_tests` extended with three new tests: `actor_role_buckets`, `rfc3339_utc_round_numbers` (epoch + Y2K+1day with microsecond fraction), and `build_git_sha_format` (either 12 hex chars or `unknown`). * Regex updated to anchor on the new mandatory fields (`ts`, `actor_role`, `connect_ts`, `build_git_sha`, `boot_generation`) and accept the optional extras in any order between `result=…` and `sozu_version=…`. * `TestServer` stub mirrors `Server.boot_generation` so the macro expansion compiles in tests without the full Poll / listener ceremony. Macro signature change ---------------------- `audit_log_context!` now takes `($server, $client, $request_id, $entry, $result)` instead of `($client, $request_id, $entry, $result)`. The only in-tree caller is `audit_emit` which already has `server: &mut Server` in scope; updated trivially. Test stubs got a matching `TestServer { boot_generation }` minimal struct. Followups (truly architectural — not in this commit) ---------------------------------------------------- * OTel `trace_id` / `span_id` propagation needs a `Request` proto bump to carry tracing context end-to-end from sozuctl through the command channel. Worth its own design. * Linux auditd subsystem integration would be a third sink with kernel-side persistence — new dep + integration design. * `source_ip=` stays permanently N/A while the command channel is unix-only. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

refactor(command): audit log P2+P3 — ucred/user/socket, dedicated sink, diff before→after, request sha256, cert-replace new fingerprint Follow-up to 53b51e03. Takes the audit log from "MUX-layout line with the core five fields" to an operationally-complete control-plane trail that satisfies PCI-DSS 10.5 routing and records everything a SOC analyst needs to reconstruct a control-plane incident without external context. New fields on `Command(...)` ---------------------------- * `actor_user=<username>` — resolved NSS account name (`getpwuid_r(uid)` at accept time via nix `user` feature). Distinct from `actor_uid` which stays the primary attribution key. `unknown` when lookup fails. * `socket=<path>` — command-socket path the client connected through. Propagated via `Arc<str>` on `CommandHub` and cloned onto each accepted `ClientSession`. Disambiguates multi-instance deployments that share a SIEM sink. * `request_sha256=<16hex>` — truncated (64-bit, first 16 hex chars) SHA-256 of the proto `Request` wire-encoding. Set on verbs that flow through `worker_request`; useful for replay / dedupe detection. Uses the workspace's existing `sha2` dep (same crate already hashes certificates in `command/src/certificate.rs`). Dedicated sink (`audit_logs_target`) ------------------------------------ New optional config field on `FileConfig` / `Config` / `ServerConfig` (+ proto tag 18). When set to a filesystem path (e.g. `/var/log/sozu/audit.log`), every audit line is *also* appended to that file opened `O_APPEND | O_CREAT` with mode `0o640` so granting an `audit` group tail-only access via filesystem ACL is one step away. ANSI escape sequences are stripped before writing to the dedicated sink via a single-pass `strip_ansi` helper so the file stays SIEM-parseable regardless of `log_colored`. Write failures log a warning and never block the mutation. `None` (default) keeps audit lines routed only through `log_target`. PCI-DSS 10.5 ("protect audit trails") can now be met with a simple filesystem ACL and logrotate config. Sample configs (`bin/config.toml`, `os-build/config.toml`) document the option under `[General]` next to `access_logs_target`. Smarter `target=` contents -------------------------- * **`UpdateHttp/Https/TcpListener`** — `format_patch_diff_*` now takes an optional snapshot of the pre-patch listener state and emits each patched field as `field=old→new` instead of just `field=new`. Falls back to `field=?→new` when no current listener is known (e.g. the patch arrived before the listener was registered), so the audit line never swallows a change. * **`ReplaceCertificate`** — the new cert's fingerprint is computed at audit time via `sozu_command_lib::certificate::calculate_fingerprint` and included alongside the old one: `target=certificate:<addr>:old=<fp>:new=<fp>`. Forensic win: cert rotation patterns + substituted-cert detection. Dep --- * `nix` gains the `user` feature (for `User::from_uid`). * `sha2` added as an explicit `bin/` dep (was already a workspace dep used by `command/src/certificate.rs`). Drift-guard ----------- Regex extended to cover the four new mandatory fields (`actor_user`, `socket`) and the new optional `request_sha256` extras slot. All four existing tests still green. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

refactor(command): enrich audit log — MUX layout, full ucred, fanout, injection defence Follow-up to 0c2dc869 — takes the audit log from a proof-of-concept into a compliance-grade control-plane log. Driven by parallel codex / lisa analyses (`tasks/audit-log-enhancements/*.md`). Layout ------ * Drop the trailing `\t >>>` continuation marker — the line is self-contained, nothing follows the closing paren. * Rename the keyword from `Session(...)` to `Command(...)` — the payload describes a control-plane command, not a proxy session. Retains the MUX-family bracket + tab + uppercase `AUDIT` tag layout. Taxonomy (proto EventKind additions — 10 new variants) ------------------------------------------------------ * `STATE_LOADED`, `STATE_SAVED` — `LoadState`/`SaveState` emit at task completion with `ok:<n> errors:<n>` in `target=`. * `LISTENER_ADDED`, `LISTENER_REMOVED` — `AddHttp/Https/TcpListener` and `RemoveListener`. * `SOZU_STOP_REQUESTED` — SoftStop / HardStop; `target=stop:soft|hard`. * `MAIN_UPGRADED`, `WORKER_UPGRADED` — hot-upgrade entry points. * `EVENTS_SUBSCRIBED` — SubscribeEvents subscribers. Actor identification -------------------- * `peer_cred_from_stream` captures the full SO_PEERCRED triple (uid, gid, pid), not just uid. * `peer_comm(pid)` reads `/proc/<pid>/comm` at accept time so operators can tell `sozuctl` apart from ad-hoc shells sharing a UID. * `ClientSession` gains `actor_gid`, `actor_pid`, `actor_comm`. Completion-time audit + timing ------------------------------ Worker-fanning verbs emit two audit lines — attempt-time (accepted by main state) and completion-time (applied across workers). The completion line carries `fanout=ok|partial|timeout|local_only`, `workers=<ok>/<err>/<expected>`, `elapsed_ms`, and on err paths `error_code` + `reason`. Structured error taxonomy ------------------------- New `AuditErrorCode` enum (dispatch_error, worker_failure, worker_timeout, peer_cred_unavailable, invalid_input, io_error, other). Wired at dispatch rejection, worker fan-out failures/timeouts, logging filter parse errors, and state save/load I/O errors. Log-injection defence --------------------- New `sanitize_for_audit` helper replaces ASCII control chars with `?`. Applied at render time to every attacker-influenced field (target, actor_comm, reason) so embedded tabs/newlines/ANSI escapes cannot forge additional audit lines. Drift-guard test covers the injection scenario. Dependency ---------- `rustls-webpki` 0.103.12 → 0.103.13 (RUSTSEC-2026-0104 reachable panic in CRL parsing). Out of audit codepath, flagged by CVE sweep. Followups (not in this commit) ------------------------------ * Dedicated tamper-resistant audit sink (`audit_logs_target` config + O_APPEND file) — PCI-DSS 10.5. * `socket_path=` field for multi-instance SIEM disambiguation. * `actor=<username>` via getpwuid_r cached at accept. * `request_sha256=` for dedupe / tamper-detection hints. * `ReplaceCertificate` target currently carries old fingerprint only — add new fingerprint too. * Patch-diff formatters log new values only — include old values. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(command): sozu listener {http,https,tcp} update runtime-patch verb Introduces an in-place update verb for non-bind-only listener settings, so operators can tune CVE-related H2 flood thresholds, SNI binding, disable_http11, ALPN, graceful-shutdown deadline, stream-0 WINDOW_UPDATE cap, sozu_id_header, custom HTTP answers, and timeouts under attack without cycling the listening socket. Wire protocol: - UpdateHttpListenerConfig, UpdateHttpsListenerConfig, UpdateTcpListenerConfig (+ AlpnProtocols wrapper so absent/empty is unambiguous) in command.proto, RequestType tags 47/48/49, EventKind::LISTENER_UPDATED = 18. Control plane (command/src/state.rs): - ConfigState::update_{http,https,tcp}_listener with field-mask merge. - merge_custom_http_answers preserves per-field http_answers so a patch that sets only answer_503 does NOT wipe answer_401/404/etc (regression of the earlier HttpAnswers blocker). - Server-side validation (validate_h2_flood_knobs_*, validate_alpn_*, validate_sozu_id_header) enforces H2 flood knobs >= 1, h2_stream_shrink_ratio >= 2, lifetime/header caps >= 1, ALPN values in {h2, http/1.1}, and RFC 9110-approximating token grammar on sozu_id_header. Raw protobuf clients cannot bypass via LoadState. Worker plane (lib/src/http.rs, https.rs, tcp.rs, server.rs): - *Listener::update_config + *Proxy::update_listener + Server-level notify_update_* routing. - HttpsListener rebuilds rustls ServerConfig on ALPN patch via the existing create_rustls_context (pure over (&config, resolver)) so the MioTcpListener/token/resolver are preserved. - HttpAnswers::replace_defaults rewrites listener-default templates per-field and leaves cluster_custom_answers untouched, fixing the silent-data-loss blocker that naive HttpAnswers::new swap would cause. - Defense-in-depth: worker-side update_config also runs the pub validators so a raw protobuf client or state replay cannot bypass. - ListenerError::InvalidValue maps StateError::InvalidValue through to the WorkerResponse failure path. CLI + audit (bin/src/cli.rs, ctl/request_builder.rs, command/requests.rs): - Per-protocol Update variants mirroring add/remove/activate/deactivate. - Paired --foo / --no-foo boolean flags via ArgAction::SetTrue + overrides_with; --alpn-protocols / --reset-alpn pair; answer-file paths loaded client-side. - worker_request dispatch + audit_entry_for emit EventKind::LISTENER_ UPDATED with the field-diff string on the audit log only (Event has no free-form field). Tests (e2e/src/tests/listener_update_tests.rs): - 14 e2e tests (5 passing, 9 currently #[ignore] with TODO notes) plus 26 new state.rs unit tests + replace_defaults_preserves_* in answers.rs. - Passing: test_flood_knob_validation, test_http_answers_replace_ preserves_cluster_overrides (the codex HIGH must-pass), test_not_ found, test_disable_http11_toggle, test_disable_http11_inflight_ keepalive. Ignored tests document follow-up needs on the e2e command-channel query path and timing harness. Docs: - doc/configure.md: new "Runtime patch" section with mutability-class table, CVE references, and worked examples. - CHANGELOG.md: Added + Changed entries. Known follow-ups (review notes in the commit, not blockers): - ALPN rebuild master/worker divergence rollback (M-1 codex review). - Full RFC 9110 token tokenizer for sozu_id_header. - Un-ignoring the 9 e2e tests once command-channel query + timing harness land. Verification: - cargo build --locked --all-features: clean - cargo +nightly fmt --all: clean - cargo clippy -p sozu-command-lib -p sozu-lib -p sozu --all-targets --locked: clean - cargo test -p sozu-command-lib --lib: 63/63 - cargo test -p sozu-lib --lib --test-threads=1: 382/382 - cargo test -p sozu --bin sozu: 2/2 - cargo test -p sozu-e2e listener_update --test-threads=1: 5/5 passed, 9 ignored with TODO rationale. Plan: /home/florentin/.claude/plans/i-want-you-to-precious-sun.md Cross-reviewed by codex + /review + /guidelines + /simplify. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(listener): configurable Sozu-Id correlation header name Adds a new listener knob `sozu_id_header` that lets operators rename the per-request correlation header Sozu injects into every request and response. Default stays `"Sozu-Id"`; common rebrands include `"X-Request-Trace"` or `"X-Edge-Id"`. The value is plumbed end-to-end: * `command/src/command.proto` — new field on both `HttpListenerConfig` (= 30) and `HttpsListenerConfig` (= 42), `optional string` so the default is only applied when absent. * `command/src/config.rs` — matching `Option<String>` on the builder with plumbing through `to_http` / `to_tls`. * `command/src/proto/display.rs` — `sozuctl`-style output shows the knob when set. * `lib/src/lib.rs` — `L7ListenerHandler` trait gains `fn get_sozu_id_header(&self) -> &str` with default `"Sozu-Id"` so call sites that haven't been updated keep the legacy value. * `lib/src/http.rs` + `lib/src/https.rs` — concrete listener implementations honour the config, falling back to the literal `"Sozu-Id"` when the field is `None` or empty. * `lib/src/protocol/kawa_h1/editor.rs` — `HttpContext` gains `sozu_id_header: String`, initialised from the listener at stream creation. Both the request-side and response-side writers use it instead of a hard-coded `kawa::Store::Static(b"Sozu-Id")`. * `lib/src/protocol/mux/mod.rs` — H2 stream creation reads the name from the listener and passes it to `HttpContext::new`. * `lib/src/protocol/kawa_h1/mod.rs` — H1 `Http::new` reads from the listener and passes it to `HttpContext::new` (the listener handle is already in scope, no upstream call-site change). * `doc/configure.md` — documents the knob. Tests: * `test_sozu_id_header_default_name_stored_on_context` — default path. * `test_sozu_id_header_custom_name_stored_on_context` — operator override stored verbatim. Hot-reload semantics: the value is cached on `HttpContext` per connection, so a config change takes effect on NEW connections only. Existing keep-alive connections continue to emit the old header name until they close — consistent with how sozu handles other listener-level config (connect_timeout, sticky_name, etc.) and with typical HTTP-proxy expectations. Addresses issue #1145 (§B3-t in the 2026-04-21 H2 triage plan). Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(h2): add flood counter for connection-level WINDOW_UPDATE (stream 0) Adds a new per-sliding-window counter in `H2FloodDetector` that tracks non-zero stream-0 WINDOW_UPDATE frames and triggers GOAWAY(ENHANCE_YOUR_CALM) when the configurable threshold is exceeded. Context: the pre-existing detector tracked RST_STREAM, PING, SETTINGS, empty DATA, and CONTINUATION floods plus a generic glitch counter, but non-zero stream-0 WINDOW_UPDATE frames were uncounted. Zero-increment stream-0 WINDOW_UPDATEs already short-circuit into GOAWAY(PROTOCOL_ERROR) per RFC 9113 §6.9, but legal non-zero increments have no per-frame cost limit and a peer could burn proxy CPU by sending millions of them. Changes: * `command/src/command.proto` — new optional field `h2_max_window_update_stream0_per_window` on both `HttpListenerConfig` (= 29) and `HttpsListenerConfig` (= 41). * `command/src/config.rs` — matching `Option<u32>` on `ListenerBuilder` with plumbing through `to_http` / `to_tls`. * `command/src/proto/display.rs` — extends `add_h2_flood_rows` with the new field so `sozuctl`-style output shows it when set. * `lib/src/protocol/mux/h2.rs`: - new constant `DEFAULT_MAX_WINDOW_UPDATE_STREAM0_PER_WINDOW = 100` (mirrors the other per-window defaults). - `H2FloodConfig` gains `max_window_update_stream0_per_window: u32` with matching default, `new()` arg, and `.max(1)` clamp. - `H2FloodDetector` gains `window_update_stream0_count: u32`, initialised to 0, halved in `maybe_reset_window`, and checked in `check_flood` with metric key `h2.flood.violation.window_update_stream0_window`. - `handle_window_update_frame` increments the counter (saturating) on every non-zero stream-0 WINDOW_UPDATE before the arithmetic, and calls `check_flood` so a burst is stopped before we pay the cost. * `lib/src/{http,https}.rs` — wire the listener-config field into `get_h2_flood_config` with defaults fallback. * `doc/configure.md` — document the knob in the flood-thresholds table and the TOML example. Tests: * `test_flood_detector_window_update_stream0_trips_at_threshold` asserts strict greater-than semantics + correct violation metadata. * `test_flood_detector_window_update_stream0_honours_default` asserts the default counter config value matches the documented constant. * `test_flood_detector_half_decay_on_window_expiry` extended to exercise the new counter alongside the existing four. Addresses Codex finding G2 from the 2026-04-21 H2 triage (~/.claude/plans/ask-h2-issues-triage-plan.md §Un-ticketed Gaps Matrix and §Implementation Outcome follow-up list). Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Démolis <43861898+Wonshtrum@users.noreply.github.com> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

feat(h2): make soft_stop graceful-shutdown deadline configurable Introduces per-listener knob `h2_graceful_shutdown_deadline_seconds` (proto field on `HttpListenerConfig` and `HttpsListenerConfig`, TOML key on `[[listeners]]`). When a worker receives `soft_stop`, the existing `Mux::shutting_down()` now arms a forced-close deadline at the moment `graceful_goaway()` first transitions the connection into draining; once the budget elapses the session is torn down even with Linked/Unlinked streams still in flight. Default: 5 seconds (preserves historic intent). `0` maps to `None`, which disables the forced-close branch entirely and reverts to the old behavior of waiting indefinitely for streams to drain. Wiring: - `command.proto`: `h2_graceful_shutdown_deadline_seconds = 28` on `HttpListenerConfig`, `= 40` on `HttpsListenerConfig`. - `command/src/config.rs`, `proto/display.rs`: ListenerBuilder field + `to_http`/`to_tls` propagation + display row. Mirrors the sibling H2 knob wiring (`h2_max_rst_stream_per_window` template). - `lib/src/lib.rs`: new trait method `L7ListenerHandler::get_h2_graceful_shutdown_deadline` with a `Some(Duration::from_secs(5))` default. - `lib/src/http.rs`, `lib/src/https.rs`: HttpListener / HttpsListener implementations. Value `0` maps to `None`. - `lib/src/protocol/mux/h2.rs`: `H2DrainState` gains `started_at` (armed once by `graceful_goaway`) and `graceful_shutdown_deadline`. Peer-initiated GOAWAYs (via `handle_goaway_frame`) deliberately do NOT arm the timer — the budget applies only to the proxy's own soft-stop. Adds `ConnectionH2::graceful_shutdown_deadline_elapsed`. - `lib/src/protocol/mux/connection.rs`: `new_h2_server` / `new_h2_client` plumb the deadline; enum-level `Connection::graceful_shutdown_deadline_elapsed` (H1 returns `false` — no multiplex to drain). - `lib/src/protocol/mux/mod.rs::Mux::shutting_down`: checks `graceful_shutdown_deadline_elapsed` right after `drive_frontend_shutdown_io` and returns `true` on expiry so the server loop closes the session. - `lib/src/protocol/mux/router.rs`, `lib/src/https.rs`: plumb the deadline through `new_h2_client` / `new_h2_server` call sites. - `lib/src/protocol/mux/LIFECYCLE.md`: documents the armed timer and forced-close branch in §8.3 Session drain. - `doc/configure.md`: adds the knob to the H2 connection tuning table and TOML example. Tests (e2e/src/tests/h2_tests.rs): - `test_h2_graceful_shutdown_timeout_forces_close` — default 5 s budget fires between 3 s and 15 s after soft_stop with a held request. - `test_h2_graceful_shutdown_deadline_configurable_short` (deadline=1 s) — worker stops in under 4 s. - `test_h2_graceful_shutdown_deadline_configurable_long` (deadline=60 s) — no premature close within 10 s, then completes on release. Uses `resolve_request_timeout(60 s)` so hyper does not abort the in-flight stream inside the assertion window. Validation: `cargo build --all-features --locked`, `cargo clippy --all-targets --locked`, `cargo +nightly fmt --all -- --check`, `cargo test --workspace --locked --lib`, focused `cargo test -p sozu-e2e --locked -- --test-threads=1 test_h2_graceful` — all pass. Resolves feat/h2-mux audit Q1 (2026-04-21): the soft_stop deadline is intentional, must remain configurable, default preserved. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Démolis <43861898+Wonshtrum@users.noreply.github.com> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

docs(cluster): clarify http2 flag is a backend-capability hint cluster.http2 = true signals that the BACKEND speaks HTTP/2 (h2c or h2+TLS). It does NOT gate H2 acceptance at the frontend — frontend H2 is negotiated via TLS ALPN on the listener (alpn_protocols) and is fully independent of per-cluster configuration. Per user decision: feat/h2-mux / PR #1209 planning, 2026-04-21 (audit Q8 — clarify http2 field semantics). Files updated: - doc/configure.md: added callout box under "HTTP/2 backend connections" - command/src/command.proto: expanded Cluster.http2 doc comment - command/src/config.rs: expanded ClusterConfig.http2 doc comment - CLAUDE.md: tightened H2 config knobs bullet Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(logging): surface TLS metadata and XFF chain on access logs Extend the `ProtobufAccessLog` wire schema with five additional optional fields (tags 25–29) capturing the per-connection TLS handshake metadata and the upstream-attested forwarded chain. Follows the same pattern as `x_request_id` (c9ec90cf): plumb through `mux::Context` → `HttpContext` → `RequestRecord` → proto, populated at every access-log emit site. Fields: - `tls_version` (`&'static str`): short label from `rustls_version_label` (e.g. `TLSv1.3`). New helper alongside the existing metric-prefixed `rustls_version_str` so the log records `TLSv1.3` rather than the dotted metric key. - `tls_cipher` (`&'static str`): short label from `rustls_ciphersuite_label` (e.g. `TLS_AES_128_GCM_SHA256`). Paired with `tls_version` in `lib/src/https.rs::upgrade_handshake`. - `tls_sni` (`&str`): borrowed from `HttpContext.tls_server_name`, same pre-lowercased value the routing layer uses to enforce the SNI ↔ `:authority` binding (CWE-346 / CWE-444). - `tls_alpn` (`&'static str`): on-the-wire ALPN label (`h2`, `http/1.1`) captured alongside the existing `AlpnProtocol` match. - `xff_chain` (`&str`): verbatim `X-Forwarded-For` value snapshotted in `editor.rs::on_request_headers` *before* Sōzu appends its own peer hop — the log records the upstream-attested chain, not the rewritten header Sōzu forwards. TLS fields are connection-scoped (stamped once on `mux::Context` at handshake completion, propagated to every per-stream `HttpContext` via `Context::create_stream`); `HttpContext::reset()` intentionally preserves them across H1 keep-alive so request N+1 still carries the handshake metadata. `xff_chain` is per-request and resets. Coverage across session types: - H1 + H2 mux: populated from the shared `HttpContext`. - WSS post-upgrade pipe: `Pipe` grows `set_tls_metadata`, called from `https.rs::upgrade_mux` so the WebSocket access log inherits the handshake metadata. - Plain TCP / WS / TCP+proxy-protocol: always `None` (no TLS termination on those paths). Also: - `HttpContext` gains corresponding fields + reset-preservation test. - `doc/configure.md` documents the five new access-log fields with their wire tags and source of truth. Signed-off-by: Florentin Dubois <florentin.dubois@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(metrics): metrics.detail cardinality knob (foundation) Foundation for per-listener / per-cluster / per-backend metric label opt-in, mirroring HAProxy's `process|frontend|backend|server` extra-counters knob. Adding labels to the StatsD keyspace under load (e.g. per-listener bytes_in on a host with many listeners) can blow up the keyspace of any statsd aggregator; HAProxy solved this by making the detail level an explicit operator choice. This commit lands the same control surface on Sōzu. Added: - `MetricDetail` proto enum in `command/src/command.proto` (`DETAIL_PROCESS=0 | FRONTEND=1 | CLUSTER=2 | BACKEND=3`, each a superset of the previous). `ServerMetricsConfig` gains an optional `detail` field (tag 4) so workers built before this lands default to DETAIL_CLUSTER on the lib side and preserve historical behaviour. - `MetricDetailLevel` Rust enum in `command/src/config.rs` with `serde(rename_all = "lowercase")` so operators write `detail = "frontend"` in the TOML config. Doc-comment calls out the superset relationship and the HAProxy analogue. Deferred (intentional — scope hard stop per the brief): - Wiring the new macros / labels into the existing `incr!`/`gauge!` call sites. That's >50 call sites and risks either a breaking API change to the macro or a parallel `incr_listener!` family — either shape wants a dedicated follow-up MR so the macro surface stays reviewable. - Touching `lib/src/metrics/network_drain.rs` to actually honour the detail level on the wire. Depends on the macro decision. - The accept-path telemetry (commit 4378101c) already labels by listener address regardless of this knob; once the wiring lands, that path becomes opt-in via DETAIL_FRONTEND and collapses to DETAIL_PROCESS otherwise. Shipping the proto + config enum first means the follow-up MR is a pure labelling change against a stable config shape — operators can already set the knob in their TOML files today; the value is simply ignored by the current drain until wiring lands. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(command): extend EventKind with control-plane mutation variants Foundation for a control-plane audit trail. Today the SubscribeEvents bus carries only four backend-health events (BACKEND_DOWN, BACKEND_UP, NO_AVAILABLE_BACKENDS, REMOVED_BACKEND_HAS_NO_CONNECTIONS). Add 14 new EventKind variants for the mutation surface so subscribers (audit shims, SIEMs, compliance ingestion) can react to cluster / frontend / certificate / listener / configuration / worker / logging changes without polling state diffs. New variants (numeric tags 4..17, existing tags unchanged for wire compatibility): - CLUSTER_ADDED, CLUSTER_REMOVED - FRONTEND_ADDED, FRONTEND_REMOVED - CERTIFICATE_ADDED, CERTIFICATE_REMOVED, CERTIFICATE_REPLACED - LISTENER_ACTIVATED, LISTENER_DEACTIVATED - CONFIGURATION_RELOADED - WORKER_KILLED, WORKER_RELAUNCHED - LOGGING_LEVEL_CHANGED, METRICS_CONFIGURED `Display for Event` (`command/src/proto/display.rs`) gains the matching human-readable strings. `bin/Cargo.toml` enables the `nix` `socket` feature so the follow-up that wires emit sites can use `nix::sys::socket::sockopt::PeerCredentials` to capture the SO_PEERCRED actor UID off the unix socket. The feature flag has no runtime cost when unused. Deferred to follow-up (intentionally not in this commit): - Emit sites in `bin/src/command/{requests,server}.rs`. Each mutating request handler needs to push an `Event` with the matching kind and log a structured audit line. Touches ~10 handlers. - SO_PEERCRED capture at the unix-socket accept site, plumbed through `ClientSession` into the audit log line. - Per-verb `incr!("config.<verb>", ...)` counters (depend on a cardinality decision: per-actor labels need scoping rules first). - doc/configure.md update describing the new event taxonomy. Compliance regimes (PCI-DSS 10.2, ISO 27001 A.8.15, SOC 2) require an immutable audit trail of privileged mutations; this commit is the wire- format prerequisite for that work to land without breaking subscribers. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(http): propagate x-request-id and log it on access log Add end-to-end `x-request-id` handling. The header is the de-facto correlation key used by every modern LB (Envoy, HAProxy, most cloud load-balancers) — before this change Sōzu dropped incoming values and never synthesised one, so request flows couldn't be correlated across the proxy. Behaviour: - Incoming request with `x-request-id`: value preserved verbatim in `HttpContext.x_request_id`, forwarded unchanged to the backend, `incr!("http.x_request_id.propagated")`. - Incoming request without `x-request-id`: generate a header value from the request ULID (`self.id`), inject it into the block list, store the value in `HttpContext.x_request_id`, `incr!("http.x_request_id.generated")`. Works for both H1 and H2 because `pkawa.rs` decodes HPACK into a kawa-H1 representation and dispatches into the shared `on_headers` callback in `editor.rs`. Also surfaced on access logs: - `x_request_id: Option<&str>` added to `RequestRecord`. - `optional string x_request_id = 24;` added to `ProtobufAccessLog` (wire-compatible append). - Populated from `HttpContext.x_request_id` on H1 and H2 mux paths; always None on pure-TCP / WebSocket paths. - Reset test updated. doc/configure.md gains a `Request-ID propagation` subsection. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(h2): expose h2_max_rst_stream_emitted_lifetime as a listener config knob The MadeYouReset (CVE-2025-8671) mitigation added in 7953696d hard-coded the 500 ceiling as a compile-time `DEFAULT_MAX_RST_STREAM_EMITTED_LIFETIME`. Operators on very busy or very quiet frontends may want to tune it — mirror the plumbing used by the existing `h2_max_rst_stream_lifetime` and `h2_max_rst_stream_abusive_lifetime` knobs: - `command/src/command.proto`: new optional `h2_max_rst_stream_emitted_lifetime` on `HttpListenerConfig` (#27) and `HttpsListenerConfig` (#39); prost-build regenerates the proto module at build time (gitignored). - `command/src/proto/display.rs`: new argument on `add_h2_flood_rows` + the two listener-Display callers, with a dedicated row label. - `command/src/config.rs::ListenerBuilder`: new `Option<u64>` field wired through the `None` default constructor and both `to_http` / `to_https` forwarders. - `lib/src/http.rs` + `lib/src/https.rs` `get_h2_flood_config`: read the listener value, fall back to `defaults.max_rst_stream_emitted_lifetime`. - `doc/configure.md`: renamed the "RST_STREAM lifetime caps" section to cover both directions, added the new row + a MadeYouReset note + updated the TOML example. - `bin/config.toml`: commented block showing all three lifetime caps. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Démolis <43861898+Wonshtrum@users.noreply.github.com> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

feat(mux): propagate per-session ULID through the protocol stack + log-context schema Introduces a stable per-connection identity that survives protocol upgrades (ExpectProxy → TLS handshake → H1/H2) and rewrites the access log-context block so operators can grep a whole TCP/TLS session (`session_id`) independently of a single HTTP exchange (`request_id`). **Log-context schema** (`command/src/logging/access_logs.rs`, `display.rs`): - `LogContext { session_id: Ulid, request_id: Option<Ulid>, cluster_id, backend_id }`. Rendered as `[<session_id> <request_id_or_-> <cluster_id_or_-> <backend_id_or_->]`. - `session_id` is minted once per accepted socket and copied across every protocol state change. - `request_id` becomes optional: present for H1 keep-alive exchanges and per H2 stream, absent for pre-upgrade events that belong to the session but not to any single request. - `RequestRecord` serialisation now emits both ids so downstream access-log consumers can correlate either axis. **`SessionTcpStream` wrapper** (`lib/src/socket.rs`, +287 lines): New `SocketHandler::session_ulid() -> Option<Ulid>` method with a default `None` for raw `mio::TcpStream`. `SessionTcpStream` is a thin `mio::TcpStream` wrapper that carries the owning `Ulid` and returns it from `session_ulid()` — used by every frontend socket path so error logs inside `SocketHandler` implementations can stamp the session prefix without threading it through every call site. **Propagation**: the session ULID flows from the outer `HttpSession` / `HttpsSession` constructors (`lib/src/{http,https}.rs`) through: - `Connection::new_h1_server(session_ulid, socket, …)`, `new_h2_{server,client}` - `ConnectionH1 { …, session_ulid: Ulid }` and the per-stream generation of `request_id` - `Pipe { …, session_id, request_id }` (pipe protocol now tracks both) - `Context::new(session_ulid, pool, listener, …)` on the mux side - ProxyProtocol `expect`/`relay`/`send` handlers pick up the ULID when the upgrade transfers the socket forward. **`log_context!` macros** (`mux/{h1,h2,mod,router}.rs`, `kawa_h1/{mod,editor}.rs`, `pipe.rs`, `rustls.rs`, `tcp.rs`): read the new `LogContext.session_id`/`request_id.unwrap_or(session_id)` fields when formatting the bracketed prefix. No behaviour change for logs that were already session-scoped; adds the second ULID slot everywhere else. **proto** (`command/src/command.proto`): access-log wire format gains a `session_id` field alongside the existing `request_id`. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Démolis <43861898+Wonshtrum@users.noreply.github.com> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

feat(config): add h2_max_header_table_size listener knob Make the SETTINGS_HEADER_TABLE_SIZE cap configurable per-listener. Wire through config.rs, command.proto, display.rs, http.rs, https.rs, H2FloodConfig, and doc/configure.md. Default: 65536 (64 KB). Also documents previously missing H2 knobs: h2_max_rst_stream_lifetime, h2_max_rst_stream_abusive_lifetime, h2_stream_idle_timeout_seconds, strict_sni_binding, disable_http11. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

fix(h2): convert per-stream idle timeout from lifetime cap to true idle timeout The h2_stream_idle_timeout_seconds guard (introduced in 77260267) measured time since stream creation, not time since last activity. This caused active uploads exceeding 30s to be RST_STREAM(CANCEL)'d even with data actively flowing — matching a customer report of PHP uploads interrupted at ~30s. Rename stream_opened_at to stream_last_activity_at and refresh the timestamp on each non-empty inbound DATA frame and on HEADERS for existing streams (response headers, trailers). Empty DATA frames (CVE-2019-9518 vector) do NOT reset the timer, preserving the slow-multiplex Slowloris defense. Add two e2e tests: - active upload trickles DATA past timeout → stream survives (200 response) - idle stream with only PINGs → RST_STREAM(CANCEL) after timeout Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Démolis <43861898+Wonshtrum@users.noreply.github.com> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

fix(h2): add per-stream idle timeout to prevent slow-multiplex Slowloris The connection-level TimeoutContainer resets on every HTTP/2 frame, so a peer that sends periodic DATA/HEADERS across many streams can keep the connection alive indefinitely while pinning up to the listener's configured `h2_max_concurrent_streams` slots — a slow-multiplex Slowloris variant (audit Pass 4 Medium #3 / Pass 3 Low #6). Introduce a per-stream deadline: - Timestamp each stream on open (server `create_stream`, client `start_stream`) via `ConnectionH2::stream_opened_at`. - Clear the entry everywhere the stream leaves the map (`remove_dead_stream`, RST_STREAM handler, GOAWAY retry path, discard on oversized data). - `cancel_timed_out_streams()` scans the open streams on every `readable()` call; any stream older than `stream_idle_timeout` is queued as RST_STREAM(CANCEL) through the existing `pending_rst_streams` path and marked in `rst_sent` to prevent duplicate frames. Expose the knob as `h2_stream_idle_timeout_seconds` on both the HTTP and HTTPS listener configs (proto fields 25 / 37), plumb it through `ListenerBuilder`, `get_h2_stream_idle_timeout()` (default 30 seconds), and `Connection::new_h2_server/client`. Unset behavior matches the previous default-30s ceiling, so existing deployments stay within a sensible safety envelope without any config change. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Démolis <43861898+Wonshtrum@users.noreply.github.com> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

feat(https): reject ALPN None on H2-only listener Pass 5 Medium #4 of the security audit: when a listener is configured as H2-only, a client that fails to negotiate `h2` via TLS ALPN (including one that omits ALPN entirely) was silently downgraded to HTTP/1.1, bypassing protections reserved for H2 traffic on that listener. - Add HttpsListenerConfig::disable_http11 (proto field 36, default false). When true, the listener only accepts `h2`; clients that negotiate `http/1.1` or send no ALPN are dropped at upgrade_handshake instead of being handed to the H1 state machine. - Mirror the field on ListenerBuilder, the to_tls() builder, and the Display row for operator visibility. - Expose HttpsListener::is_http11_disabled() and consult it in upgrade_handshake; rejected connections bump a dedicated `https.alpn.rejected.http11_disabled` counter and warn with the negotiated ALPN (Some / None) for forensics. Returning `None` closes the session through the existing FailedUpgrade path. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(config): expose H2 flood thresholds as per-listener config knobs Extend HttpListenerConfig and HttpsListenerConfig with three additional H2 flood-detection knobs so operators can tune Rapid Reset and HPACK header-size protections per listener without a rebuild: - h2_max_rst_stream_lifetime (u64, default 10_000) — absolute ceiling on RST_STREAM frames received over a single connection (CVE-2023-44487). - h2_max_rst_stream_abusive_lifetime (u64, default 50) — pre-response-start Rapid Reset signature ceiling, trips well before the generic lifetime cap. - h2_max_header_list_size (u32, default 65536) — accumulated HPACK-decoded header list size per request (RFC 9113 §6.5.2 SETTINGS_MAX_HEADER_LIST_SIZE). Wire the fields through protobuf (HttpListenerConfig fields 22-24, HttpsListenerConfig fields 32-34), sozu-command-lib ListenerBuilder, the http.rs / https.rs get_h2_flood_config() helpers, H2FloodConfig, and pkawa::handle_header. H2FloodDetector now reads all thresholds from its configured H2FloodConfig instead of the module-level compile-time constants, so unset knobs fall back to the previous defaults and behavior is unchanged when the config is omitted. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(config): add strict_sni_binding listener knob for SNI-authority enforcement Phase 1D hard-coded the TLS SNI ↔ HTTP `:authority` binding (CWE-346 / CWE-444) on every HTTPS listener. This exposes the behavior as an opt-out knob so operators with legitimate cross-SNI routing needs can relax the check per listener while the default remains the safe "enforce" setting. - Add HttpsListenerConfig::strict_sni_binding (proto field 35, default true) plus a matching ListenerBuilder field and Display row. - Add L7ListenerHandler::get_strict_sni_binding() with a true default; override in https.rs to read the listener config (HTTP has no SNI to check, so it uses the default). - Capture the flag once on mux::Context at handshake time and mirror it onto each HttpContext in Context::create_stream, avoiding a per-stream listener borrow. - `route_from_request` now gates the existing exact-match SNI check on HttpContext::strict_sni_binding; plaintext listeners still short-circuit on tls_server_name: None regardless of the flag. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(mux): introduce 421 Misdirected Request template for SNI-authority mismatch Phase 1D mapped the TLS SNI ↔ `:authority` mismatch to a 401 Unauthorized response because Sōzu did not ship a 421 template. RFC 9110 §15.5.20 defines 421 Misdirected Request as the semantically correct status (the target authority does not belong to this TLS connection) and lets the client retry on a fresh TLS connection with a matching SNI. - Add `Answer421 {}` variant + `set_answer` metric + `u16` mapping to kawa_h1::DefaultAnswer. - Ship `default_421()` body, the `answer_421` template entry in `HttpAnswers::template()` / `ListenerAnswers`, and the `get()` arm that binds `route`/`request_id` variables — styled to match the existing 400/401/404 pages. - Expose `CustomHttpAnswers::answer_421` (proto field 11), the matching ListenerBuilder field, `get_http_answers()` loader, and the Display row for operator overrides. - Register 421 in the mux default-answer helper (`default_answer_for_code` + `http.421.errors` metric key) and rewire the `RetrieveClusterError::SniAuthorityMismatch` branch in `mux/mod.rs` to call `set_default_answer(..., 421, ...)`. This covers both H1 and H2 frontends going through mux; the `http.sni_authority_mismatch` metric continues to be the durable operator signal. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Démolis <43861898+Wonshtrum@users.noreply.github.com> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

feat(h2): add per-listener connection_window, max_concurrent_streams, stream_shrink_ratio Thread three new configurable H2 parameters through the 5-layer config stack (proto → config → listener → trait → runtime): - h2_initial_connection_window: connection-level receive window size (default 1MB, replaces hardcoded ENLARGED_CONNECTION_WINDOW) - h2_max_concurrent_streams: SETTINGS_MAX_CONCURRENT_STREAMS (default 100, was hardcoded) - h2_stream_shrink_ratio: threshold ratio for recycled stream Vec shrinking (default 2, was hardcoded) All values are optional per-listener with safe compile-time defaults. Validation clamps to minimums (window >= 65535, streams >= 1, ratio >= 1). Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Démolis <43861898+Wonshtrum@users.noreply.github.com> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

feat(h2): make flood detection thresholds configurable via listener config Add 6 optional fields to HttpListenerConfig and HttpsListenerConfig proto messages for tuning H2 flood detection thresholds at runtime: - h2_max_rst_stream_per_window (default: 100) - h2_max_ping_per_window (default: 100) - h2_max_settings_per_window (default: 50) - h2_max_empty_data_per_window (default: 100) - h2_max_continuation_frames (default: 20) - h2_max_glitch_count (default: 100) Implementation: - H2FloodConfig struct with Default matching original constants - L7ListenerHandler::get_h2_flood_config() trait method with default impl - HttpListener and HttpsListener extract optional values from proto config - H2FloodDetector uses config values instead of compile-time constants - Config wired through Connection::new_h2_server and new_h2_client All fields are optional with safe defaults — no breaking change to existing configurations. Closes #1211 Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Démolis <43861898+Wonshtrum@users.noreply.github.com> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

feat(config): add per-listener ALPN protocol configuration Add `alpn_protocols` field to HTTPS listeners allowing operators to control which protocols are advertised during TLS handshake. Defaults to ["h2", "http/1.1"] (prefer HTTP/2). Supports strict validation at config load time, HashSet-based deduplication, and warns when http/1.1 is excluded. Changes: - Proto: add `repeated string alpn_protocols = 22` to HttpsListenerConfig - Config: DEFAULT_ALPN_PROTOCOLS constant, ConfigError::InvalidAlpnProtocol, ListenerBuilder field + validation in to_tls() - TLS: replace hardcoded SERVER_PROTOS with config-driven ALPN in create_rustls_context() - Display: show alpn_protocols in listener table output - Tests: 6 unit tests (default, custom, invalid, empty, dedup, order) + 2 e2e tests (H1-only listener, reversed preference order) - Docs: configure.md ALPN reference, getting_started.md H2 section, config.toml commented examples Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(h2): extract mux module from mux_v1 branch Extract the HTTP/2 multiplexing module from the mux_v1 branch and adapt it to compile against the current main API surface. Changes: - Extract 7 mux module files into lib/src/protocol/mux/ - Adapt generic P: L7Proxy to dyn L7Proxy (dynamic dispatch) - Update Route::Cluster -> Route::ClusterId - Fix HttpContext::new parameter order for main's API - Add protocol() and public_address() to ListenerHandler trait - Add MaxBuffers and HttpsRedirect error variants - Add http2 field to Cluster protobuf message - Add backend_address, extract_route, get_route, websocket_context to HttpContext - Remove old lib/src/protocol/h2/ directory - Replace H2 ALPN path with TODO for task #5 (mux integration) Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Demolis <eloi.demolis@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

fix(h2): address review findings from PR #1209 - Use standard HTTP reason phrases in default answers (e.g., "Not Found" for 404, "Bad Gateway" for 502) instead of generic "Sozu Default Answer" - Use HTTP/1.1 version in default answers instead of H2 - Remove extra Content-Length: 0 from error responses to match old behavior - Fix cookie header matching in HPACK decoder: compare against "cookie" instead of debug artifact "cookie---", forward cookies as regular headers - Add #[repr(u32)] with explicit RFC 7540 error codes to H2Error enum - Remove stale commented-out nom macro parser code - Fix typo "trat" -> "treat" in parser comment - Document the http2 proto field on Cluster message Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: Eloi Demolis <eloi.demolis@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud> Co-authored-by: hcaumeil <78665596+hcaumeil@users.noreply.github.com>

2026-05-20

docs(proto,changelog): scope METRIC_DETAIL_CHANGED to operator-initiated transitions The proto documented "every effective-level transition emits an `EventKind::METRIC_DETAIL_CHANGED` event", but only operator-initiated transitions emit today: the master-side audit-log wiring covers the SetMetricDetail fan-out path, but worker-local transitions (lease expiry on the polled janitor, post-fan-out apply/clear in the worker arm) leave a silent gap. Replicating the audit emission inside the worker `notify` arm needs a new IPC back to the master and is deferred to a follow-up. Update the `SetMetricDetail` doc comment and the `EventKind::METRIC_DETAIL_CHANGED` declaration to scope the contract to operator-initiated transitions, and add a CHANGELOG caveat so audit-log consumers know which transitions are not yet surfaced. The three previous `// TODO(sozu-top week 2):` markers in the worker arm now point to the proto doc as the source of truth for the scope. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(server,command): wire worker→master audit IPC for METRIC_DETAIL_CHANGED Previously, worker-local cardinality-lease transitions (TTL janitor expiry, worker-arm apply/clear) left no audit trail because the worker had no IPC path to the master's audit pipeline. Only operator-initiated transitions audited from `bin/src/command/requests.rs::worker_request` produced an audit row. A SOC analyst correlating "who elevated metrics cardinality" could see the apply but not the implicit clear. Close the gap by reusing the existing worker→master `Event` channel: - New proto `MetricDetailTransition` carrying `previous_effective`, `effective`, `transition_kind`, and an optional `client_id` for explicit apply/clear. Folded into `Event.metric_detail` (tag 5) so `EventKind::METRIC_DETAIL_CHANGED` events now carry their full payload through the existing fan-out plumbing. - Worker emits the event from three sites in `lib/src/server.rs::notify`: the polled `lease_tick` janitor (transition_kind = "lease_tick_expired" and `client_id = None` because the janitor may retire multiple leases at once), the SetMetricDetail worker arm on apply ("lease_apply"), and on clear ("lease_clear"). All three callers gate on `previous != effective` so the helper itself is a defence-in-depth no-op when nothing actually changed. - Master's `handle_worker_response` recognises METRIC_DETAIL_CHANGED events and routes them through a dedicated `audit_worker_metric_detail_transition` helper that writes to both audit sinks (text + JSON). The worker is its own actor — `worker_id` takes the `client_id` slot in the envelope; `actor_role=worker`, `actor_comm=sozu-worker`. Subscriber fan-out is unchanged. - The 11 existing `push_event(Event { … })` constructors get `metric_detail: None` to stay compatible with the new field; only the worker's lease-transition emitter populates it. Build/clippy clean; 53/53 TUI unit tests pass. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(command,lib): per-worker WorkerMetricDetailStatus payload Final piece of the PR #1256 follow-through. Previously the capability-aware dispatcher in `bin/src/command/requests.rs` (`SetMetricDetailTask::on_finish`) synthesised `MetricDetailStatus.workers[<worker_id>]` using the master's aggregator view as a stand-in for each worker because workers replied with `WorkerResponse::ok(message.id)` carrying no payload. Each worker holds an independent `Aggregator` with its own lease table, so that stand-in obscured real per-worker drift (different configured floors, different active lease counts after a partial fan-out). Wire it properly end-to-end: - New `ResponseContent::WorkerMetricDetailStatus` oneof variant (tag 17 — proto additive). Carries the worker's own `(configured, effective, previous_effective, active_lease_count)` quartet, semantically distinct from the aggregated `MetricDetailStatus` at tag 16. - New `lib/src/server.rs::worker_metric_detail_status_content` helper that builds the response payload from a `(configured, effective, previous_effective, lease_count)` snapshot captured BEFORE the `METRICS.borrow_mut` scope ends (so the per- request snapshot is consistent with the transition that just happened). - The three ok-paths in the worker's SetMetricDetail arm (clear-Cleared, clear-NotFound, apply-Applied) now reply via `WorkerResponse:: ok_with_content` with the freshly-built payload instead of the payload-less `ok`. The `clear-NotFound` path reports `previous_effective == effective` (no transition). - Master-side `SetMetricDetailTask::on_finish` collects the per-worker payload from `response.content` and only falls back to skipping the worker entry when the response has no payload (e.g. an older worker that never went through `ok_with_content`). Removes the master-view stand-in noted as a follow-up in commit `70cd24af` (`set_metric_detail_request`). - `command/src/proto/display.rs` adds a silent OK match arm for the new variant — the per-worker payload flows master-side and is never printed directly on the operator's terminal. Build/clippy clean; 1075/1075 workspace tests pass. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

refactor(command,server): remove the proto_version capability handshake Drop the proto_version field and the SetMetricDetail capability partition that gated the dispatch on worker proto version. Production deployments keep master + workers in sync via the existing UpgradeMain hot-upgrade flow, so the mixed-version-fleet state the field was designed for does not occur. The implementation was structurally broken anyway: the master stamped every WorkerInfo.proto_version from its own SOZU_PROTO_VERSION constant at fork time, so the field always read as the master's version and MIN_PROTO_VERSION_FOR_SET_METRIC_DETAIL was effectively unconditional. The proto contract is additive-only; a worker that does not recognise tag 55 (SetMetricDetail) returns WorkerResponse::error("unknown request type") which already surfaces in the standard fan-out error tally (extras.fanout.workers_err counter). MetricDetailStatus.unsupported_workers becomes redundant and is also removed. Deletions: - WorkerInfo.proto_version (tag 4) and MetricDetailStatus.unsupported_workers (tag 5); both replaced by `reserved` markers per project convention. - sozu_command_lib::SOZU_PROTO_VERSION constant. - WorkerSession.proto_version field. - MIN_PROTO_VERSION_FOR_SET_METRIC_DETAIL constant and the capability partition in set_metric_detail_request; SetMetricDetail now fans out unconditionally via the standard scatter path. - Capability-handshake paragraphs in CHANGELOG.md and doc/sozu-top.md. - print_metric_detail_status block that rendered the unsupported_workers table row. Old workers without tag 55 surface as 'succeeded with errors' (the normal fan-out failure shape) rather than as a dedicated capability-skip list. Operator visibility is preserved. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(proto,command,server): SetMetricDetail TTL lease verb + plumbing Adds a runtime cardinality lease verb so `sozu top` can elevate the metrics drain to `MetricDetailLevel::Backend` for the duration of an interactive session. The lease design (TTL-bounded, `client_id`-keyed, self-expiring) is crash-safe and composes with multiple concurrent clients — see `Aggregator` lease bookkeeping in the previous commit. Proto (additive, backwards-compat): - `Request.request_type::SetMetricDetail = 55` carries `{ client_id, detail?, ttl_seconds?, clear?, reason? }`. - `ResponseContent::metric_detail_status = 16` returns `MetricDetailStatus { configured, effective, previous_effective, workers: map<id, WorkerMetricDetailStatus>, unsupported_workers[] }` for mixed-version-fleet safety. - `EventKind::METRIC_DETAIL_CHANGED = 30` on the `SubscribeEvents` audit stream; distinct from `METRICS_CONFIGURED` (Enabled/Disabled /Clear) since the cause is different. - `command/build.rs` re-attaches `Hash, Eq` for `MetricDetailStatus` (the embedded `map<string, WorkerMetricDetailStatus>` strips the prost auto-derive, which propagates to `ResponseContent.content_type` and `Request.request_type`). - `command/src/proto/display.rs` adds arms for `RequestType:: SetMetricDetail`, `ContentType::MetricDetailStatus` (with a prettytable renderer that lists per-worker configured/effective /previous_effective + unsupported workers), and `EventKind:: MetricDetailChanged`. - `command/src/request.rs` routes `SetMetricDetail` through the worker-level dispatch group (mirrors `ConfigureMetrics`). Master + worker plumbing: - `bin/src/command/requests.rs::is_mutating_verb` learns the new verb so the master brackets it with `RELOADING=1`/`READY=1` systemd hints. - The dispatch match routes through the existing `worker_request` fan-out path (same shape as `ConfigureMetrics`); per-worker `MetricDetailStatus` aggregation lands in week 2 when the TUI starts consuming it. - `lib/src/server.rs::notify` adds two hooks: a polled lease-expiry janitor at the top of every dispatch (gated by `lease_tick_due` so it only walks the lease table every 5 s), and the `SetMetricDetail` arm itself. The arm clamps the TTL via the `Aggregator` setter, decodes the `MetricDetail` enum defensively, and acks with `WorkerResponse:: ok()` for now (week 2 will return the per-worker `WorkerMetricDetailStatus` payload). `MetricDetailChanged` audit emission is left as a `TODO(sozu-top week 2)` at both `lease_apply`/`lease_clear` call sites and the janitor — plumbed through `bin/src/command/requests.rs::audit_emit_inline` once the master collects per-worker effective levels back. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(command): worker proto version handshake, scaffolding for unsupported_workers Wire the proto-capability layer that the `MetricDetailStatus.unsupported_workers` field needs to be usefully populated. Previously the field was declared but always empty because the master had no way to know which workers could decode the new SetMetricDetail (tag 55) verb. This commit lays the rails; the dispatch-time gating itself remains a follow-up because it requires a dedicated `WorkerTask` impl for SetMetricDetail that synthesises a MetricDetailStatus reply rather than the current generic worker_request flow. The scaffolding shipped here: - New `sozu_command_lib::SOZU_PROTO_VERSION = 1` constant, baked into every Sōzu binary at compile time. Bumped any time a new wire- affecting `RequestType` or proto field needs capability gating. Version 1 covers `SetMetricDetail`, `METRIC_DETAIL_CHANGED`, the worker→master audit IPC, and per-lease peer-credential binding. - New `WorkerInfo.proto_version` proto field (tag 4, optional uint32) so TUI / status consumers can observe each worker's version directly. - New `WorkerSession.proto_version` master-side field, snapshotted at fork time from the binary's `SOZU_PROTO_VERSION` constant. The `to_info()` / `list_workers` paths populate the new proto field. - Proto comment on `MetricDetailStatus.unsupported_workers` updated to describe the capability-gate model AND the inherited-after-UpgradeMain caveat: a re-exec master does NOT yet re-query inherited workers' versions, so they retain the new master's compile-time value until the planned per-worker capability handshake on Status reply lands. Build/clippy clean; 53/53 TUI unit tests pass. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(metrics,command): bind lease ownership to peer credentials Replace the unauthenticated `lease_clear(client_id)` API with a binding- aware variant so one same-UID operator cannot clear another operator's lease by guessing the `client_id` format. The binding pairs the master- side `actor_pid` (captured via `SO_PEERCRED` at command-socket accept) with the per-connection session ULID; both halves must match the apply- time binding for the worker to authorise the clear. Implementation: - New `PeerBinding { pid, session_ulid }` + `LeaseEntry` + `LeaseClearOutcome` in `lib/src/metrics/mod.rs`. `lease_apply` records the binding alongside `(level, expires_at)`; `lease_clear` returns `Cleared`/`NotFound`/ `Unauthorized` depending on the apply-time binding vs the presented one. - A "binding unknown" apply (pre-binding caller or platform without `SO_PEERCRED`) preserves backward compat: the worker accepts any clear. A fully-known apply rejects every clear whose presented binding does not match, including the default ("unknown") clear. - New proto fields `SetMetricDetail.peer_pid` (tag 6) and `SetMetricDetail.peer_session_ulid` (tag 7), additive. Clients leave them empty; the master populates them in `bin/src/command/requests.rs::worker_request` from the connecting `ClientSession` before fan-out. - Worker parses the presented ULID via `rusty_ulid::Ulid::from_str` with a `0x…` hex fallback, then routes the `LeaseClearOutcome::Unauthorized` outcome to `WorkerResponse::error` so the operator gets a loud failure rather than a silent no-op. - Six new unit tests cover authorised clear, unauthorised mismatch, unknown-apply accepts-any, known-apply rejects default clear, the existing apply/clear/tick paths through the new signature. Build/clippy clean; 26/26 metric tests pass. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

style(top,proto): silence the last clippy and rustdoc warnings Clears the 14 warnings remaining after the simplify pass: - Five dead-code items: drop `render_placeholder` (all panes ship, no placeholder needed); drop `ClusterRow.errors_5xx_total` (the renderer reads `error_rate_pct`, never the raw count); drop `ThresholdTable.conn_warn_pct` (no consumer); drop `BackendRow.bytes_in` / `bytes_out` (the BACKENDS pane reads `back_bytes_in` / `back_bytes_out`). `Skin.categorical` gets a short doc + `#[allow(dead_code)]` because the field is read by tests and is the TOML surface for operator skins — the production consumer (cluster-row categorical tinting) lands in a follow-up. - Nine rustdoc warnings in the prost-generated `command.rs`: my earlier H2 commit's numbered-list comment in the `SetMetricDetail` preamble used 4-space indentation on the continuation lines, which prost-build expanded to overindented in the rendered docstring. Trim to 3 spaces so the rendered comment lands on the rustdoc list-item-alignment rule (3 spaces aligns with the text after `1. ` markers). - One rustdoc warning in `bin/src/ctl/top/mod.rs::run_top`: the doc comment had a `+` at line-start that markdown parsed as a list bullet, then complained the next two lines were unindented. Rephrase to join with "and" so the `+` no longer leads a line. Build is now warning-free under `cargo clippy --all-features --all-targets --locked`. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

2026-05-04

fix(command): default missing repeated/map fields when loading older state files `bin/src/command/requests.rs::load_state` reads each `\n\0`-separated JSON record via `command::parser::parse_several_requests::<WorkerRequest>`, which calls `serde_json::from_slice` per record. The prost-build config in `command/build.rs` attaches `Serialize`/`Deserialize` derives to every generated message but did not attach `#[serde(default)]` anywhere, so missing `repeated`/`map` fields rejected the record (`Vec<T>` and `BTreeMap<K, V>` are required-by-serde without an explicit default). Post-1.1.1 schema additions (`Cluster.answers`, `Cluster.authorized_hashes`, `RequestHttpFrontend.headers`, plus the listener-level `answers` / `alpn_protocols` fields) therefore broke `LoadState` for any older client (e.g. proxy-manager pinned to `sozu-command-lib = "1.1.1"`): the first `AddCluster` or `AddHttpFrontend` failed to deserialize, `parse_several_requests`'s `many0(complete(...))` left the unparsed bytes as the remainder, and the read loop reported `"Error consuming load state message"` to the client at EOF. Each post-1.1.1 `repeated`/`map` field on a state-file-emittable message now carries `#[serde(default)]` via a `field_attribute(... )` line in `command/build.rs`, so missing fields default to empty (mirroring the protobuf wire-format default). Required scalars stay strict on purpose: `ConfigState::add_cluster` keys the cluster map by `cluster_id` without a non-empty check, so a struct-level blanket would silently insert a bogus `""`-keyed entry. The field-level annotation preserves that defense-in-depth. The new contract is documented at the top of `command/src/command.proto` and inline in `command/build.rs`: any new `repeated` or `map` field on a message reachable from a `SaveState`/`LoadState` JSON file (anything emitted by `ConfigState::generate_requests`, or carried in a `RequestType` an external client may build and feed through `LoadState`) must add the matching `field_attribute(...)` line. Regression tests in `command/tests/state_compat_v1_1_1.rs` pin both the 1.1.1-shaped fixture round-trip and the missing-required-scalar rejection contract. Signed-off-by: Florentin Dubois <florentin.dubois@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(hsts): force-replace mode + Set replaces Delete Adds an operator opt-in to override backend-supplied `Strict-Transport-Security` instead of preserving it. Also folds in the broader cleanup that drops `HeaderEditMode::Delete` (zero producers in the diff) in favour of the new `Set` mode that subsumes its semantics via delete-then-insert. - New `HstsConfig.force_replace_backend` proto field (tag 5). Default RFC 6797 §6.1 backend-wins behaviour is unchanged (`HeaderEditMode::SetIfAbsent`); operators flip the field to `true` when a stale or weak upstream HSTS policy needs hardening at the proxy edge (`HeaderEditMode::Set`). - New `FileHstsConfig.force_replace_backend` field, plumbed through `to_proto` to the proto value; documented in `bin/config.toml` and `doc/configure.md`. - New `--hsts-force-replace-backend` CLI flag on `sozu frontend https/http add`. Treated as an enabling flag — `--hsts-force-replace-backend` alone enables HSTS with the canonical default `max-age = DEFAULT_HSTS_MAX_AGE`. Mutually exclusive with `--hsts-disabled`. Two new unit tests cover the force-replace cells. - New `HeaderEditMode::Set` (drop `Delete`). `Set` is delete-then-insert in one entry: the retain pass drops every header with the matching name, then the insert pass appends the new value unconditionally. The legacy empty-`val` Append delete encoding is preserved verbatim for backwards compatibility. - `Frontend::new` chooses `Set` vs `SetIfAbsent` based on `cfg.force_replace_backend`. The single materialiser site is the only consumer; the helper handles both modes uniformly. - Two new `apply_response_header_edits` unit tests cover the `Set` path (replaces existing header / inserts when absent). Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(command,lib): add HstsConfig proto + HeaderEditMode for SetIfAbsent Introduce the HSTS (RFC 6797) typed-config foundation: - new HstsConfig proto message with enabled/max_age/include_subdomains/ preload, attached as optional field on HttpsListenerConfig (tag 46), UpdateHttpsListenerConfig (tag 41), and RequestHttpFrontend (tag 16); inline rustdoc cites RFC 6797 §6.1, §7.2, §8.1, §11.4, §14.2 - new HeaderEditMode { Append, Delete, SetIfAbsent } and a `mode` field on HeaderEditSnapshot; apply_response_header_edits learns SetIfAbsent so upstream-supplied Strict-Transport-Security passes through unchanged (RFC 6797 §6.1 single-header requirement) - legacy empty-val Delete encoding preserved via Append+empty fallback so no existing call site needs updating in this commit - unit tests cover SetIfAbsent skip-when-present and insert-when-absent No behaviour change on the wire yet — the materialisation path (router → headers_response) lands in a follow-up commit. Existing struct literals updated only to add 'hsts: None' / 'mode: HeaderEditMode::Append'. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(events): add ClusterRecovered event (proto tag 29) Pairs with the existing NoAvailableBackends event (tag 2) so dashboards can plot per-cluster recovery as well as the all-down transition. Highest existing tag was 28 (HealthCheckUnhealthy); 29 was free. Refs #892 Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

2026-05-02

feat(redirect): add 302 (Found) and 308 (Permanent Redirect) policies Closes #1009. Sōzu's `RedirectPolicy` previously only supported `Permanent` (301) and `Unauthorized` (401) at the frontend. Two new variants: - `RedirectPolicy::Found` → 302 Found (RFC 9110 §15.4.3) — a temporary redirect; user agents MAY rewrite POST → GET on follow. - `RedirectPolicy::PermanentRedirect` → 308 Permanent Redirect (RFC 9110 §15.4.9) — like 301 but the HTTP method MUST be preserved on follow (no GET-rewrite on POST). Wire-level changes: - `command/src/command.proto::RedirectPolicy` gains `FOUND = 3` and `PERMANENT_REDIRECT = 4`. Tags 0..2 unchanged so v1.x clients still decode `FORWARD` / `PERMANENT` / `UNAUTHORIZED` correctly. - `HttpContext` gains `redirect_status: Option<u16>` stashed by `Router::route_from_request` per resolved policy. The answer engine reads it in `mux/mod.rs` to pick the matching `http.{301,302,308}.redirection` template; the legacy `cluster.https_redirect = true` path keeps its 301 default (the field is `None`). - `DefaultAnswer` gains `Answer302` and `Answer308` variants alongside `Answer301`; `set_default_answer_with_retry_after` dispatches all three through one redirect-shaped path. - Default templates `default_302` and `default_308` ship in-tree — `Connection: close`, `Sozu-Id: %REQUEST_ID`, single `Location: %REDIRECT_LOCATION`. Operator-supplied templates flow through the renamed `HttpAnswers::render_inline_redirect(code, …)`; `render_inline_301` is preserved as a thin wrapper for binary / source compatibility. - Per-status counters `http.302.redirection` and `http.308.redirection` fire alongside the existing `http.301.redirection`, both labelled by cluster + backend, mirroring the 301 emission path. Two e2e regression tests in `redirect_rewrite_auth_tests.rs`: - `try_redirect_found_h1_emits_302`: H1 frontend with `RedirectPolicy::Found` must emit 302 + HTTPS Location, no backend contact. - `try_redirect_permanent_redirect_h1_emits_308`: same but for 308. H2 frontend cells, plus the inline-template H2 path, remain to be backfilled in a follow-up; the H1 path covers the routing decision + answer-engine plumbing end-to-end. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

2026-04-30

fix(router,mux,answers): close PR #1162 follow-up gaps on main Four gaps identified after the redirect/rewrite/answer-template/auth stack landed on `main`. PR #1162 ("Redirection and URL rewrite", @Wonshtrum, base `main`) is structurally superseded by the unified mux implementation but its last commit `448ece33 Rewriting fixes` carried two genuine correctness fixes that did not make it to `main`. The clusterless-redirect ordering bug and the doc/code drift on `%STATUS_CODE` are independent gaps surfaced by the same review pass. * Clusterless `RedirectPolicy::Permanent` now reachable. In `lib/src/protocol/mux/router.rs::route_from_request` the `RedirectPolicy::Permanent` branch is moved ahead of the `Unauthorized || cluster_id.is_none()` deny, so a frontend declared with `redirect = permanent` and no backing cluster (the canonical "this hostname has moved, no service remains" shape from #1161) emits 301 instead of 401. The `Permanent` block does not read `cluster_id`; the cluster-derived knobs already default to safe sentinels at the cluster lookup when `cluster_id` is `None`. The `let Some(cluster_id) = cluster_id else { unreachable!() }` binding moves below the deny block; its invariant still holds. * Non-trie host regex anchored at both ends. In `lib/src/router/mod.rs::convert_regex_domain_rule` the compiled regex now opens with `\A` and closes with `\z`. Without anchors, `Regex::is_match` is unanchored, so an operator's `/example\.com/` matched any hostname containing `example.com` as a substring, including `attacker.example.com.evil.org` — letting an attacker-controlled domain reach a frontend that should only serve `example.com` (CWE-1023, routing bypass). The trie path (`lib/src/router/pattern_trie.rs`) was already anchored. * Inner `/`-finding loop terminates on first match. Same function; the inner loop now `break`s after `found = true`, so a multi-segment regex hostname like `/seg1/.foo./seg2/.com` no longer overwrites `index` on every later `/` and the literal `.` separators between regex segments are kept in their correct position (`\Aseg1\.foo\.seg2\.com\z`). * `%STATUS_CODE` doc/code drift removed. The placeholder was advertised in `doc/configure.md`, the `redirect-template` CLI help in `bin/src/cli.rs`, and the `redirect_template` doc comments in `command/src/command.proto` and `command/src/config.rs`, but `lib/src/protocol/kawa_h1/answers.rs::HttpAnswers::template` never defined a `STATUS_CODE` variable. References are stripped so operators no longer see a promised variable that silently no-ops. Implementation deferred to the same change that adds 302/308 for #1009. Tests: `convert_regex` (updated for the anchored shape) plus three new unit / e2e regressions — `regex_domain_rule_rejects_suffix_and_prefix`, `regex_domain_rule_multi_segment_segments_are_isolated`, and `try_clusterless_permanent_redirect_emits_301` (asserts 301 + Location on the rewritten host). Closes parts of #1161 (proposal — clusterless permanent redirect was the last reachable gap) and #1154 (modify Host header — already addressed by `rewrite_host`, now closeable). Leaves #1009 partially open: 302 (`Temporary`) and 308 are still missing on `main`. Validation: * cargo build --all-features --locked * cargo clippy --all-targets --locked * cargo +nightly fmt --all -- --check * cargo test --workspace --locked (955 passed, 7 ignored) * cargo test -p sozu-e2e -- redirect (19 passed) Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

chore(comments,docs): drop review-process leakage; align stale h2c doc comments Inline comments and the proto file referenced PR numbers and 'Codex finding' from the cross-model review pipeline. Replace each with durable technical rationale so the committed text stands on its own merit and ages with the code rather than the review log. Also realign the doc comments on FileClusterConfig.health_check and HttpClusterConfig.health_check that still claimed HTTP/1.1-only probes — this PR adds h2c support, the proto already documents 'probe wire follows cluster.http2', the Rust-side config doc comments should match. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

refactor(health-check): derive h2c probe from cluster.http2; drop is_h2c The probe wire format and the data-plane backend connection both need to choose between HTTP/1.1 and HTTP/2 (h2c) for the same backends. Carrying that decision twice — once on `Cluster.http2` (read by the mux router at `protocol/mux/router.rs::Router::connect`) and once on `HealthCheckConfig.is_h2c` — invites the two flags to drift, so an h2c-only backend gets probed with HTTP/1.1 (or vice versa) the moment an operator updates one without the other. Collapse to a single source of truth: derive the probe wire from `cluster.http2` directly. The probe and the data-plane backend connection now share one switch and cannot diverge. Wire / API surface - `HealthCheckConfig.is_h2c` (proto field 7) is removed. Field 7 remains reserved-by-omission; new fields will start at 8. - `FileHealthCheckConfig.is_h2c` removed; `to_proto` no longer carries the field. - `--h2c` CLI flag on `sozu cluster health-check set` removed. - `BackendMap` gains a `cluster_http2: HashMap<ClusterId, bool>` populated from `Cluster.http2` on every `AddCluster`. The health-check probe reads the entry at probe-creation time and records it on the `InFlightCheck` (`h2c: bool`) so the response parser stays consistent even if the operator flips the flag mid-probe. Documentation - `doc/health_checks.md` rewrites the wire-format paragraph: probe follows `cluster.http2`; no `is_h2c` knob. - The configuration-parameters table drops the `is_h2c` row and gains a paragraph explaining the lockstep with `cluster.http2`. - `CHANGELOG.md` Added bullet rephrased: "cluster-derived h2c probes" instead of "opt-in h2c probes". Validation - cargo build --all-features --locked: pass. - cargo +nightly fmt --all -- --check: pass. - cargo clippy --all-targets --all-features --locked: 0 errors, 0 warnings. - cargo test -p sozu-lib health_check::: 12 passed (existing h2c unit tests still cover the parser; they construct configs without the dropped field). Signed-off-by: Florentin Dubois <florentin.dubois@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(health-check): h2c prior-knowledge probe support Add an opt-in HTTP/2 prior-knowledge (cleartext, h2c) probe path so operators can health-check backends configured with `cluster.http2 = true` without co-locating an HTTP/1.1 endpoint. Wire / API surface: - `HealthCheckConfig.is_h2c` (proto field 7, optional bool, default false). The TOML key on `[clusters.<id>.health_check]` and the `FileHealthCheckConfig` struct gain a matching field. - `sozu cluster health-check set --h2c` flag forwards the bool through the request-builder validator. Probe wire: - 24-byte connection preface `PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n`. - Empty client SETTINGS frame (0-byte payload, type 0x04, flags 0, stream 0). - HEADERS frame on stream 1 with END_STREAM | END_HEADERS (flags 0x05) carrying a hand-rolled HPACK header block: * `:method GET` indexed (static idx 2 → 0x82). * `:scheme http` indexed (static idx 6 → 0x86). * `:path <uri>` literal w/o indexing, name idx 4 (0x04). * `:authority <host:port>` literal w/o indexing, name idx 1 (0x01). Length octets use the HPACK 7-bit-prefix integer form, including the multi-byte continuation chain for values > 127 bytes. Response parser (`try_parse_h2c_status`): - Walks frames in the buffered response, ignoring SETTINGS, SETTINGS ACK, DATA, etc. until it finds a HEADERS frame on stream 1. - Strips PADDED and PRIORITY prefixes correctly. - Decodes `:status` from either: * Static-table indexed forms 0x88..0x8E (200, 204, 206, 304, 400, 404, 500), or * Literal-with-indexed-name forms 0x08 / 0x18 (literal w/o indexing or never-indexed for name index 8) followed by length + 3-byte ASCII status code. - A GOAWAY frame is treated as a probe failure. - Returns `None` while the buffer is truncated mid-frame so the caller keeps reading. Dispatch in `progress_checks`: a single `parse_probe_response` helper chooses HTTP/1.1 status-line parsing or h2c frame walking based on `config.is_h2c`. The HTTP/1.1 path is unchanged. Tests (in `lib/src/health_check.rs::tests`): - `build_h2c_probe_starts_with_preface_and_settings` - `h2c_indexed_status_200_is_healthy_for_any_2xx` - `h2c_indexed_status_500_fails_default_2xx_check` - `h2c_literal_status_503_matches_expected_503` - `h2c_goaway_marks_unhealthy` - `h2c_truncated_buffer_returns_none` - `h2c_padded_headers_strips_pad_length_octet` Documentation: `doc/health_checks.md` updates the "HTTP/1.1 only" note to describe the new `is_h2c` opt-in, the wire shape, and the parser coverage. HTTPS (h2 over TLS) probes remain a follow-up. Validation: - cargo build --no-default-features --features crypto-ring: pass. - cargo +nightly fmt --all -- --check: pass. - cargo clippy --all-targets --no-default-features --features crypto-ring: 0 errors, 1 cosmetic warning unrelated. - cargo test -p sozu-lib --no-default-features --features crypto-ring health_check::: 11 passed (5 prior + 6 new h2c). Signed-off-by: Florentin Dubois <florentin.dubois@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(health-check): non-blocking HTTP health checks with fail-open and CLI Add HTTP/1.1 backend health-check probing that runs inside the existing single-threaded mio event loop. No additional threads, no async runtime. Key changes: - New lib/src/health_check.rs (~480 LOC) with HealthChecker, threshold- based state machine, jittered intervals, response size cap (4 KB), CRLF-sanitised URIs, per-backend in-flight tracking. Sockets are registered with mio in a dedicated bounded token namespace [HEALTH_CHECK_TOKEN_BASE, HEALTH_CHECK_TOKEN_BASE + HEALTH_CHECK_TOKEN_CAPACITY) = [1<<24, 1<<24 + 1<<16) so the upper bound never falsely claims the mux GOAWAY sentinel Token(usize::MAX). The allocator picks slot offsets modulo the capacity and skips offsets matching in-flight checks; if the table is full it logs an error and returns None rather than silently colliding. - Fail-open routing in lib/src/load_balancing.rs: when ALL backends for a cluster are unhealthy, route to the Normal backends rather than returning 503 (Amazon health-check paper recommendation). - Backend.health::HealthState machine in lib/src/backends.rs with consecutive success/failure counters and Up/Down event emission. - Server event loop integration in lib/src/server.rs: poll the health checker each iteration; dispatch ready tokens; pass Registry for register/deregister. - Server-side validation (command/src/request.rs) rejects zero interval/timeout/thresholds and bad URIs. - Proto: HealthCheckConfig (fields 1-6), SetHealthCheck (cluster_id, config), QueryHealthChecks (optional cluster_id), HealthChecksList (map<cluster, config>) at command/src/command.proto. - CLI: cluster health-check {set,remove,list} subcommands in bin/src/cli.rs with --uri/--interval/--timeout/--healthy-threshold /--unhealthy-threshold/--expected-status flags. CLI-side validation rejects URIs missing a leading '/' and rejects \r, \n, NUL, and any C0 control byte (RFC 9110 §5.1) — not just CR/LF. - Master dispatch in bin/src/command/requests.rs broadcasts state-mutating health-check requests and aggregates worker responses on query. - ConfigState dispatch + persistence in command/src/state.rs. - Tabular display in command/src/proto/display.rs. - Metrics health_check.{success,failure,up,down,healthy_backends}. - Log envelope: every emit in health_check.rs prefixes its format string with "{}, log_context!()" so the lib/tests/log_layout.rs regression guard recognises the canonical HEALTH-CHECK tag (mirrors the hyphenated MUX-H2 / PROXY-RELAY / TLS-RESOLVER convention). Adaptations vs the original PR #1191 commit (rebase onto post-1209 main): - Proto field renumbering. Post-1209 main occupies several carriers PR #1191 originally used; the rebased commit retargets: * Request.set_health_check 47 -> 52 * Request.remove_health_check 48 -> 53 * Request.query_health_checks 49 -> 54 * Cluster.health_check 8 -> 15 * ResponseContent.health_checks_list 14 -> 15 * EventKind.HEALTH_CHECK_HEALTHY 5 -> 27 * EventKind.HEALTH_CHECK_UNHEALTHY 4 -> 28 - Token namespace tightened (bounded modulo allocator + bounded owns_token range) to avoid claiming the mux GOAWAY sentinel. - URI sanitisation hardened to reject every C0 control byte. - ClusterCmd::HealthCheck variant placed alongside ClusterCmd::H2 (PR #1191's structure), not nested under ClusterH2Cmd. Validation: - cargo build --no-default-features --features crypto-ring: pass - cargo +nightly fmt --all -- --check: pass - cargo clippy --all-targets --no-default-features --features crypto-ring: 0 errors, 1 cosmetic warning (redundant_closure on Criterion bench setup, unrelated to this commit). - cargo build --tests --no-default-features --features crypto-ring: pass. Signed-off-by: Florentin Dubois <florentin.dubois@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

2026-04-29

feat(splice): operator-tunable kernel-pipe capacity Make the splice(2) kernel-pipe capacity per direction configurable via a new `splice_pipe_capacity_bytes` field on ServerConfig, instead of the hard-coded 64 KiB constant. The override flows through the same path as the existing `basic_auth_max_credential_bytes` knob: declared in `command/src/command.proto` (ServerConfig field 22), wired through `FileConfig`, the `ConfigBuilder` mapper, the runtime `Config`, and the proto round-trip via `From<&Config> for ServerConfig`. Linux-only storage at the lib layer: a `OnceLock<usize>` in `lib/src/splice.rs` populated once per worker boot from `Server::try_new_from_config` (cfg-gated), with a setter that no-ops on `0` so an explicit zero does not collapse the pipe to PAGE_SIZE. `SplicePipe::new` now applies the configured capacity via `fcntl(F_SETPIPE_SZ)` on each pipe and reads the realised value back with `fcntl(F_GETPIPE_SZ)`, storing the smaller of the two sizes in a new `capacity` field. This handles the kernel's behaviour: it rounds up to PAGE_SIZE and clamps at `/proc/sys/fs/pipe-max-size` (default 1 MiB unprivileged; CAP_SYS_RESOURCE goes higher). On `F_SETPIPE_SZ` failure the kernel keeps the previous capacity (typically 64 KiB), the failure is logged at `warn!`, and SplicePipe continues with that realised value — splice still works, just at the kernel default. `splice_in` gains a `len: usize` parameter so callers thread the realised capacity through; pipe.rs reads `splice_pipe.capacity` for both the per-call `len` and the "pipe is full" backpressure check (replaces the deleted `SPLICE_PIPE_CAPACITY` const). CHANGELOG, doc/getting_started.md (feature-flag table), and doc/configure.md (global parameters table) updated to document the new knob and its kernel-side limits. Signed-off-by: Florentin Dubois <florentin.dubois@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(http,https,mux): add X-Real-IP injection and anti-spoof elision (H1+H2) Two listener-scoped opt-in bool flags, both default false and independently combinable. Configurable on `HttpListenerConfig` / `HttpsListenerConfig` and runtime-patchable through the new `Update*ListenerConfig` partial-update verbs. - elide_x_real_ip: when true, any client-supplied `X-Real-IP` header is stripped from the request before forwarding (anti-spoofing). - send_x_real_ip: when true, a proxy-generated `X-Real-IP` header carrying the connection peer IP is appended. The IP is read from the post-PROXY-v2 `session_address`, so deployments terminating PROXY v2 surface the original client IP, not the upstream proxy's. Both H1 and H2 are covered by a single elision branch in `HttpContext::on_request_headers`, dispatched from `pkawa::handle_header` for H2 initial HEADERS frames. H2 trailer HEADERS frames take a separate path through `pkawa::handle_trailer` and would otherwise bypass the elision; `handle_trailer` now drops `x-real-ip` trailer pairs when the listener flag is set, closing that gap. Five e2e tests cover the four-flag matrix on H1 (including PROXY-v2 unwrap with original client IP) plus an H2 trailer regression placeholder. Mirrors the `strict_sni_binding` precedent for trait accessor / mux Context propagation / Update*ListenerConfig apply path. Closes #1113 Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(server,mux,tcp,cli): per-cluster per-IP connection limit with HTTP 429 + graceful TCP close Caps the number of simultaneous frontend connections one source IP may hold against a given cluster. Replaces the simpler per-source-IP variant the previous PR #1193 commits proposed (those did not survive the H2 mux unification). Closes #890, #1057. Wire shape (proto): - ServerConfig.max_connections_per_ip = 22 [optional, default 0] - ServerConfig.retry_after = 23 [optional, default 60] - Cluster.max_connections_per_ip = 13 [optional, override] - Cluster.retry_after = 14 [optional, override] - CustomHttpAnswers.answer_429 = 12 [optional, custom 429 template] - Request.set_max_connections_per_ip = 50 (uint64) - Request.query_max_connections_per_ip = 51 (QueryMaxConnectionsPerIp {}) - ResponseContent.max_connections_per_ip_limit = 14 (MaxConnectionsPerIpLimit { limit: u64 }) Override semantics: cluster `None` inherits the global default, `Some(0)` is explicit "unlimited for this cluster", `Some(n > 0)` overrides. Source IP is the parsed PROXY-protocol source when present, else `peer_addr`. Enforcement: - HTTP/HTTPS via the unified mux at protocol/mux/router.rs::connect: after cluster resolution, before backend selection. The check fires AFTER auth/redirect/SNI decisions so a 401/421/redirect frontend never trips the limit. On hit: stash the resolved Retry-After on the stream context and return BackendConnectionError::TooManyConnectionsPerIp, which the mux converts into a 429 default answer through the new set_default_answer_with_retry_after path. Covers H1 and H2 — the unified mux serves both protocols. - H2 multiplex semantics: SessionManager keeps a per-token HashSet<(cluster, ip)>. Multiple streams to the same (cluster, ip) from the same H2 connection share one slot — the limit governs distinct frontend connections, not requests. Decrement is wholesale on session close (untrack_all_cluster_ip). - TCP at tcp.rs::connect_to_backend: rejection produces a graceful FIN via SessionResult::Close (no SO_LINGER trick). Answer engine (lib/src/protocol/kawa_h1/answers.rs + lib/src/protocol/mux/answers.rs): - Answer429 variant on DefaultAnswer with retry_after: Option<u32>. - 429 template registered alongside 421/503/etc. - New %RETRY_AFTER variable with `or_elide_header = true`: when the resolved value is 0/None, the engine drops the entire `Retry-After:` line. `Retry-After: 0` would invite an immediate retry that defeats the limit, so we omit instead of rendering literal 0. - legacy_to_map flattens CustomHttpAnswers.answer_429 into the per-listener answers map. - default_answer_for_code(429, ...) builds the variant. - set_default_answer_with_retry_after threads the resolved retry value through to the rendered answer. Proxy-protocol fixes (lib/src/protocol/proxy_protocol/{expect,relay}.rs): - ExpectProxyProtocol::into_pipe and RelayProxyProtocol::into_pipe now use ProxyAddr::source() from the parsed v2 header instead of the raw TCP peer_addr. Without this fix the pipe phase records the upstream PROXY-emitter (an LB / edge proxy / health-check probe), not the originating client — which means the per-(cluster, source-IP) limit would have keyed on the LB's IP. Relay mode previously discarded the parsed addresses entirely; we now stash them on the struct. SessionManager (lib/src/server.rs): - max_connections_per_ip: u64 + retry_after: u32 fields. - connections_per_cluster_ip: HashMap<(String, IpAddr), usize> + cluster_ip_tracks: HashMap<Token, HashSet<(String, IpAddr)>>. - cluster_ip_at_limit (token-aware: a token already holding a slot is never at the limit), track_cluster_ip (idempotent within a token), untrack_all_cluster_ip (drains on session close), effective_max_connections_per_ip / effective_retry_after (override resolution), clear_cluster_ip_tracking (runtime disable). - HttpProxy / HttpsProxy / TcpSession close paths drain via untrack_all_cluster_ip on the frontend token before the slab slot is reused. ProxySession trait (lib/src/lib.rs): - New cluster_id() and session_address() default-implemented as None. - New L7Proxy::sessions() so the mux router can reach the SessionManager from a Rc<RefCell<dyn L7Proxy>>. - New BackendConnectionError::TooManyConnectionsPerIp variant with cluster_id payload. CLI (bin/src/{cli,ctl/mod,ctl/request_builder}.rs): - `sozu connection-limit set <N>` / `remove` / `show` patches the global limit at runtime. The setter is non-sticky (workers reset to the TOML-configured value on restart); operators must mirror the change in the config to make it durable. - `--answer-429 <path>` flag added to `sozu listener {http,https} update`. - New build_http_answers helper threads answer_429 through the existing CustomHttpAnswers builder. Config (command/src/config.rs): - DEFAULT_MAX_CONNECTIONS_PER_IP = 0 (disabled), DEFAULT_RETRY_AFTER = 60. - FileConfig + Config + ServerConfig From conversion threaded. - FileClusterConfig + HttpClusterConfig + TcpClusterConfig + ListenerBuilder + CustomHttpAnswers gain the new fields. Metric: connections.rejected_per_cluster_ip (counter, labelled by cluster_id, backend_id always empty — rejection happens before backend selection). Replaces the simpler `connections.rejected_per_ip` from the older PR #1193 commits — single name covers H1, H2, TCP. Tests: - e2e/src/tests/cluster_ip_limit_tests.rs covers H1 global limit (429 + Retry-After), H1 Retry-After: 0 elides the header, H1 per-cluster override (unlimited cluster coexists with capped), TCP graceful close on limit. All four pass `cargo test -p sozu-e2e cluster_ip_limit`. Validation: - cargo build --all-features --locked - cargo +nightly fmt --all -- --check - cargo clippy --all-features --all-targets --locked -- -D warnings - cargo test --workspace --locked Docs: bin/config.toml, doc/configure.md (parameters + metric), CHANGELOG.md. Co-authored implementation guidance from Codex on the H2 multiplex semantics, Retry-After: 0 elision, PROXY-protocol Relay-mode address stashing, and rolling-upgrade safety (optional proto fields, not required). Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(server): evict least-active sessions when accept queue is full Add an opt-in `evict_on_queue_full` knob (proto `ServerConfig` field 22, default `false`) that, when set, evicts the oldest 1% of non-listener sessions to make room for queued sockets when `SessionManager::check_limits` refuses a new accept. Selection uses `select_nth_unstable_by_key` (introselect, O(n) average) over `Session::last_event()` to partition the candidate slab in place, avoiding an O(n log n) sort. The cap loop in `Server::create_sessions` keeps the existing `incr!("listener.connection_capped")` counter (so dashboards stay meaningful regardless of eviction outcome), runs the eviction batch, and re-checks limits before continuing. A new `sessions.evicted` counter is emitted only when the mitigation fires. Eviction is deliberately skipped during graceful `shutting_down`: forcing sessions closed there defeats the shutdown semantics and is wasted work since the worker is winding down anyway. Default `false` because during a DDoS the active sessions are more likely to be legitimate clients than the queue contents — evicting them would serve attackers. Operators dominated by normal traffic spikes can flip it on. Closes #644, closes #916. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

2026-04-28

feat(config,auth): expose basic_auth_max_credential_bytes + warn at 33% buffer The maximum length of a base64-decoded `Authorization: Basic` payload the worker accepts was a hard-coded 4 KiB. Exposing it in the main TOML config lets operators on hardened tenants lower it (256 / 512 typical) to bound the per-failed-auth allocation tighter against hostile peers sending large tokens. Surfaces: - New `ServerConfig.basic_auth_max_credential_bytes` proto field (tag 21, optional uint64), threaded from `FileConfig.basic_auth_max_credential_bytes` through `Config` so the value rides the existing config-load and state-restore plumbing. - `lib::protocol::mux::auth` replaces the const cap with a `OnceLock<usize>` overlay over a built-in default (4096). The override is committed once on each worker at boot via `lib::server::Server::try_new_from_config` calling `set_max_decoded_credential_bytes` — a single set-once handoff with no per-request atomic. Subsequent attempts to mutate are no-ops (`OnceLock::set` rejects), and an explicit `0` is treated as "use default" so a typo cannot disable the cap by accident. New unit test pins the zero-is-noop semantics. - Config validator emits a `warn!` at boot when the operator-set cap is `>= buffer_size / 3`. At that point a single failed-auth attempt can pin ~33% of the per-frontend buffer's worth of bytes; combined with in-flight request/response framing the buffer trends toward back-pressure under load. Informational only — operators with a deliberate threat model can keep the value, but the surprise stays visible in the boot log. - `doc/configure.md` "HTTP Basic authentication" section gains a "Tuning the credential decode cap" subsection covering both the knob and the 33% warning. `CHANGELOG.md` mentions both. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

chore(proto): drop preemptive `reserved` blocks on Cluster and RequestHttpFrontend The `reserved 13 to 19` (Cluster) and `reserved 16 to 19` (RequestHttpFrontend) blocks were defensive "reserve room for future fields" — not idiomatic protobuf. The `reserved` keyword is meant to guard against tag REUSE after a field is deleted, so an old serialised message doesn't alias into a new field with an incompatible type. We didn't delete anything; we only added new fields. The next person adding a field would just pick the next free tag (13 / 16) anyway, and removing the `reserved` line would be a single-line edit if those numbers were genuinely needed. Drops the blocks; the regenerated `command.rs` shrinks accordingly. No wire-format change, no downstream impact. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

fix(cli,router,auth): clear /review's deferred Medium/Low/Nit backlog Closes the items the prior `fix(answers,auth,router)` pass deferred: * M2 — `sozu frontend {http,https} add` gains the per-frontend policy flags (`--redirect`, `--redirect-scheme`, `--redirect-template`, `--rewrite-host`, `--rewrite-path`, `--rewrite-port`, `--required-auth`, `--header <position>=<name>=<value>`). The two `frontend …` siblings now share a single `build_http_frontend_add` helper so they cannot drift; each policy field is validated up-front (range-checked, scheme/policy parsed against the proto enum) so a malformed input surfaces as a typed `CtlError::ArgsNeeded` instead of reaching the worker. * M3 — `Route::Frontend(Rc<Frontend>)` is reachable. `HttpFrontend` carries the new policy fields all the way through `RequestHttpFrontend::to_frontend`, and `Router::add_http_front` flips onto the rich path whenever any policy field is non-default (otherwise still produces the legacy `Route::ClusterId` / `Route::Deny` shapes). The dead-arm warning is no longer hypothetical — the variant is exercised end-to-end on the live routing path. * L4 — `mux::auth::canonicalize_basic_credentials` tightens the whitespace grammar to `1*SP` per RFC 7235 §2.1 / RFC 9110 §11.4. SP before and between scheme and token68 is tolerated; HTAB and zero-spaces are rejected. * L5 — `ConfigError::InvalidHeaderPosition` becomes `{ index, position }` so a multi-entry config pinpoints the bad row. `parse_header_edit` takes the array index from `entries.iter().enumerate()`. * L6 — new `template_fill_adjacent_body_variables` regression test pins the `body_size` accounting against an `[%ROUTE%ROUTE]` body so a future tweak to the inter-chunk arithmetic cannot silently break back-to-back placeholders. * N2 — `Cluster` reserves field numbers 13-19 and `RequestHttpFrontend` reserves 16-19. A future migration that accidentally aliases an old field number now fails at proto compile time. Knock-on: every test fixture and example builder constructing `HttpFrontend { … }` directly inherits the eight new `Option`/`Vec` fields; existing in-tree call sites are populated with `None` / `Vec::new()` (see `lib/src/http.rs::frontend_from_request_test`). The 4 cluster-add CLI tests added in c1e0b1a6 plus the new `template_fill_adjacent_body_variables` bring sozu-lib to 436 passing tests; sozu-command-lib stays at 66/5 ignored. Build, clippy `-D warnings`, fmt --check, log-layout regression all clean. Out of scope for this pass * The header-injection runtime path on the mux side still needs to consume `Frontend.headers_request` / `headers_response` at `mux/h1.rs::writable` and `mux/h2.rs::write_streams`. The plumbing is in place (the routing layer builds and stashes the slices) — the actual emission is the next round of work. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

fix(answers,auth,router): act on /review findings Apply the High and tractable Medium/Low/Nit items the read-only review surfaced. Build, clippy --all-targets -D warnings, fmt --check, sozu-lib (435 pass) and sozu-command-lib (66 pass) all green. High * answers.rs: stop unconditionally splicing a synthetic `Content-Length` header into rendered responses. The engine now detects an operator-supplied `Content-Length:` line during the parse pass and routes its value through the existing `ContentLength` placeholder, so the rendered response carries exactly one header with the auto-computed body size. Previously a template with a literal `Content-Length:` line ended up with two headers — RFC 9110 §8.6 / RFC 7230 §3.3.2 request-smuggling vector. * mux/auth.rs: pad both candidate and stored hash into a fixed `[u8; AUTH_COMPARE_PAD_LEN + 8]` envelope (256-byte body + 8-byte little-endian length) before `subtle::ConstantTimeEq::ct_eq`, so the per-entry compare loop iterates the full padded length even when lengths differ. `subtle`'s slice `ct_eq` short-circuits on length mismatch — the padding here defeats that leak. Closes the realm-size and matching-username-length channels (CWE-208 family). * kawa_h1/editor.rs: `HttpContext::reset()` now clears `redirect_location` and `www_authenticate` between pipelined H1 requests so a future 301/401 default-answer path that bypasses routing cannot inherit a stale Location / realm from a prior request. Backed by the existing `test_reset_clears_request_response_state` regression, extended with the two new field assertions. Medium * command.proto: `HeaderPosition` gains an explicit `HEADER_POSITION_UNSPECIFIED = 0` variant; the proto-default-encoded shape now deserialises into a typed "unset" instead of failing `HeaderPosition::try_from(0)`. The runtime drops `Header { position: Unspecified, … }` entries with a `warn!` rather than guessing a position. * mux/auth.rs: replaces the hand-rolled `to_hex` (which carried two `expect("hi/lo nibble")` panics on an auth path) with `hex::encode` from the existing `hex` workspace dep. Drops the corresponding unit test and the `_METHOD_USED` workaround that kept the otherwise- unused `Method` import alive. * kawa_h1/answers.rs: `HttpAnswers::new` propagates `TemplateError` from the bundled-default-template parse path via `?` instead of `expect(...)`. The new `default_templates_all_parse` regression test guards the invariant. * router/pattern_trie.rs: new `segment_regex_rejects_partial_matches` test pins the `\A...\z` anchoring contract — `cdn[0-9]+` matches `cdn1` and `cdn123` but rejects `cdn1xxx`, `xxxcdn1`, and `cdnabc`. Without the test the next clippy or refactor pass could silently revert anchoring. Low / Nit * request_builder.rs: validate `--https-redirect-port` against `1..=65535` before sending so a typo doesn't render `Location: https://host:70000/...` on the wire. Lifts `looks_like_authorized_hash` to module scope and adds 6 unit tests covering canonical form, missing colon, short hex, uppercase, empty username, non-alnum username. * router/mod.rs: drop the dead-defensive `from_utf8(b).unwrap_or_default()` in `RewritePart`; `pattern` is `template.as_bytes()` and the split is on the ASCII byte `$`, so `template[start..i]` lies on char boundaries and indexes safely. * mux/router.rs: gate the legacy `cluster.https_redirect` `redirect_location` stash on `proxy.kind() == ListenerType::Http` so an HTTPS listener never carries a stale URL into a downstream default-answer path. Replaces `cluster_id.expect(...)` with a `let-else { unreachable!() }` form per project style. * router/mod.rs: tighten the `log_module_context!` doc-comment to reflect the single-call-site reality (`Frontend::new`'s warn). Out of scope (deferred to follow-up) * /review M2: per-frontend `--redirect`, `--rewrite-*`, `--header` CLI flags (cluster-level flags shipped in c1e0b1a6). * /review M3: making the `Route::Frontend(Rc<Frontend>)` variant reachable from `add_http_front` — needs the Wave 1c bridge that converts `RequestHttpFrontend` policy fields into a `Frontend`. * /review L4: `Authorization` whitespace tolerance (RFC 7235 §2.1 permits SP only; current accepts SP and HT). * /review L5: `parse_header_edit` failure does not include array index — minor diagnostic polish. * /review L6: `body_size -= variable.name.len() + 1` accounting fuzz test for adjacent `%VAR%VAR` patterns. * /review N2: `reserved` declarations in `Cluster` / `RequestHttpFrontend` / listener configs. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(proto): add answer templates, redirect, rewrite, headers, auth schema Lay the wire schema for the bundled feature work that brings PR #1206..#1210 onto current main. Wave 1 of a multi-wave implementation; subsequent waves land the runtime engine, mux integration, auth helper, e2e tests, and docs. Cluster gains four fields: per-cluster `answers` (status code → template body) at field 9, `https_redirect_port` at 10, `authorized_hashes` at 11, and `www_authenticate` realm at 12. RequestHttpFrontend gains eight: `redirect` policy at 8, `required_auth` at 9, `redirect_scheme` at 10, `redirect_template` at 11, `rewrite_host`/`rewrite_path`/`rewrite_port` at 12-14, and a repeated `headers` list at 15. Listener configs gain a parallel `answers` map at 31 (HttpListenerConfig) and 43 (HttpsListenerConfig); the legacy `CustomHttpAnswers http_answers` field is preserved on the wire so existing state files round-trip — the runtime will read both for one minor. New top-level enums RedirectPolicy / RedirectScheme / HeaderPosition and a Header message land alongside. An empty `Header.val` deletes the named header (HAProxy `del-header` parity). prost stops auto-deriving Hash/Eq once a message holds a map field, so the build script gains explicit `#[derive(Hash, Eq)]` for Cluster, HttpListenerConfig, HttpsListenerConfig, UpdateHttpListenerConfig, and UpdateHttpsListenerConfig. Existing initializers grow a `..Default::default()` fallthrough so this commit is a pure additive scaffolding step — no runtime behaviour changes yet. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

2026-04-27

feat(config): expose slab_entries_per_connection knob (default 4, [2,32]) The slab capacity multiplier was a private constant SLAB_ENTRIES_PER_CONNECTION = 4 introduced by the H2 mux work to accommodate stream multiplexing (1 frontend + up to 3 backend connections per session). Operators with topologies that fan out across more than 4 backends per session had no recourse short of a recompile, and slab exhaustion presents as "accept refused" with no telemetry pointing at the slab. Adds an optional uint64 slab_entries_per_connection field to ServerConfig (proto tag 20) and a matching Config field. Effective value flows through ServerConfig::effective_slab_entries_per_connection which clamps to [MIN_SLAB_ENTRIES_PER_CONNECTION = 2, MAX_SLAB_ENTRIES_PER_CONNECTION = 32]; absent or 0 falls back to DEFAULT_SLAB_ENTRIES_PER_CONNECTION = 4 so existing deployments keep their current capacity. Documented in doc/configure.md alongside max_buffers / buffer_size. A `worker.slab.utilization_pct` operator-facing gauge is left as a follow-up (gauge wiring lives in the worker hot path; this commit is config plumbing only). PR #1209 consolidated review (MED-6). Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

refactor(command): finish audit log enrichment — ts, actor_role, connect_ts, boot_generation, build_git_sha, JSON sink Closes the deferred items from 6b21c9c4. After this commit, the only audit-log gaps left are architectural decisions that need their own designs: OTel trace_id/span_id (proto bump on Request to carry tracing context end-to-end) and Linux auditd integration (new kernel-side sink, large scope). `source_ip=` stays permanently N/A while the command channel is unix-only. New mandatory fields on every `Command(...)` line ------------------------------------------------- * `ts=<RFC3339 UTC microseconds>` — in-body timestamp. Lets operators extract a single audit line and still know when it fired without cross-referencing the outer logger prefix. Implemented with a std-only Hinnant `civil_from_days` converter — no `time` or `chrono` dep added. * `actor_role=root|system|user|unknown` — buckets the actor uid so SOC dashboards can route on it without parsing UID values per host (uid=0 → root, 1..1000 → system, 1000+ → user, missing → unknown). * `connect_ts=<RFC3339 UTC>` — wall-clock `accept(2)` time of the client connection, stamped on `ClientSession.connect_ts` via `SystemTime::now()`. Forensic windowing — "all verbs from connections that opened in the 30s before the incident". * `boot_generation=<u32>` — counter incremented at every `MAIN_UPGRADED` re-exec, persisted across the re-exec via `UpgradeData.boot_generation`. Disambiguates post-upgrade sessions from pre-upgrade ones — PIDs reset, but `(boot_generation, session_ulid)` is a durable correlation pair. * `build_git_sha=<12hex>` — short git SHA embedded by `bin/build.rs` via a `cargo:rustc-env=SOZU_BUILD_GIT_SHA=…` directive. Falls back to `unknown` outside a git tree (vendored tarballs, sysroots). `bin/build.rs` re-runs only when `.git/HEAD` or `.git/refs` move so cached cargo builds stay fast. JSON sink — `audit_logs_json_target` ------------------------------------ Mirrors every audit line as a single-line JSON object to a dedicated file. Same `O_APPEND | O_CREAT | 0o640` lifecycle as the human sink. Schema is stable; missing values are JSON `null` so SIEM pipelines can flatten without conditional fields. The `audit` block groups actor identity (uid/gid/pid/user/comm/role) and the `extras` block groups completion-time fields (elapsed_ms, fanout, error_code, reason, request_sha256). Both sinks (text + JSON) are independent — operators can set both for tail-friendly + machine-parseable. Plumbed through `FileConfig` → `Config` → `ServerConfig` (proto tag 19) so workers see the field even though only the main process writes to it. Server gains `audit_log_json_writer: Option<RefCell<File>>` opened at boot via the same `open_audit_log_file` helper as the text sink. `Server.boot_generation` + `UpgradeData.boot_generation` ------------------------------------------------------- * New `pub boot_generation: u32` on `Server`, `0` on first boot, `saturating_add(1)` at every `upgrade_main` invocation BEFORE the re-exec serialises `UpgradeData` so the new main starts at the bumped value. * `UpgradeData.boot_generation` carries it across the re-exec boundary (`#[serde(default)]` so old upgrade payloads decode as `0`). * The `MainUpgraded` audit line records the new generation in its `target=` field for the audit-trail itself. Documentation + retention policy (LISA-013) ------------------------------------------ * `doc/observability.md` documents the JSON schema with a worked example. * New retention-policy section: PCI-DSS 10.7 calls for ≥ 1 year of audit retention with the most recent 3 months immediately available. Recommended shape: `logrotate` daily compress, off-host archive after 90 days, 400-day cold retention. Sōzu does NOT rotate audit files itself — delegated to the OS-level rotator with `copytruncate` since sōzu keeps the file handle open. (`SIGHUP` re-open is a TODO.) * Sample configs (`bin/config.toml`, `os-build/config.toml`) document `audit_logs_json_target` next to `audit_logs_target`. Drift-guard ----------- * `audit_format_tests` extended with three new tests: `actor_role_buckets`, `rfc3339_utc_round_numbers` (epoch + Y2K+1day with microsecond fraction), and `build_git_sha_format` (either 12 hex chars or `unknown`). * Regex updated to anchor on the new mandatory fields (`ts`, `actor_role`, `connect_ts`, `build_git_sha`, `boot_generation`) and accept the optional extras in any order between `result=…` and `sozu_version=…`. * `TestServer` stub mirrors `Server.boot_generation` so the macro expansion compiles in tests without the full Poll / listener ceremony. Macro signature change ---------------------- `audit_log_context!` now takes `($server, $client, $request_id, $entry, $result)` instead of `($client, $request_id, $entry, $result)`. The only in-tree caller is `audit_emit` which already has `server: &mut Server` in scope; updated trivially. Test stubs got a matching `TestServer { boot_generation }` minimal struct. Followups (truly architectural — not in this commit) ---------------------------------------------------- * OTel `trace_id` / `span_id` propagation needs a `Request` proto bump to carry tracing context end-to-end from sozuctl through the command channel. Worth its own design. * Linux auditd subsystem integration would be a third sink with kernel-side persistence — new dep + integration design. * `source_ip=` stays permanently N/A while the command channel is unix-only. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

refactor(command): audit log P2+P3 — ucred/user/socket, dedicated sink, diff before→after, request sha256, cert-replace new fingerprint Follow-up to 53b51e03. Takes the audit log from "MUX-layout line with the core five fields" to an operationally-complete control-plane trail that satisfies PCI-DSS 10.5 routing and records everything a SOC analyst needs to reconstruct a control-plane incident without external context. New fields on `Command(...)` ---------------------------- * `actor_user=<username>` — resolved NSS account name (`getpwuid_r(uid)` at accept time via nix `user` feature). Distinct from `actor_uid` which stays the primary attribution key. `unknown` when lookup fails. * `socket=<path>` — command-socket path the client connected through. Propagated via `Arc<str>` on `CommandHub` and cloned onto each accepted `ClientSession`. Disambiguates multi-instance deployments that share a SIEM sink. * `request_sha256=<16hex>` — truncated (64-bit, first 16 hex chars) SHA-256 of the proto `Request` wire-encoding. Set on verbs that flow through `worker_request`; useful for replay / dedupe detection. Uses the workspace's existing `sha2` dep (same crate already hashes certificates in `command/src/certificate.rs`). Dedicated sink (`audit_logs_target`) ------------------------------------ New optional config field on `FileConfig` / `Config` / `ServerConfig` (+ proto tag 18). When set to a filesystem path (e.g. `/var/log/sozu/audit.log`), every audit line is *also* appended to that file opened `O_APPEND | O_CREAT` with mode `0o640` so granting an `audit` group tail-only access via filesystem ACL is one step away. ANSI escape sequences are stripped before writing to the dedicated sink via a single-pass `strip_ansi` helper so the file stays SIEM-parseable regardless of `log_colored`. Write failures log a warning and never block the mutation. `None` (default) keeps audit lines routed only through `log_target`. PCI-DSS 10.5 ("protect audit trails") can now be met with a simple filesystem ACL and logrotate config. Sample configs (`bin/config.toml`, `os-build/config.toml`) document the option under `[General]` next to `access_logs_target`. Smarter `target=` contents -------------------------- * **`UpdateHttp/Https/TcpListener`** — `format_patch_diff_*` now takes an optional snapshot of the pre-patch listener state and emits each patched field as `field=old→new` instead of just `field=new`. Falls back to `field=?→new` when no current listener is known (e.g. the patch arrived before the listener was registered), so the audit line never swallows a change. * **`ReplaceCertificate`** — the new cert's fingerprint is computed at audit time via `sozu_command_lib::certificate::calculate_fingerprint` and included alongside the old one: `target=certificate:<addr>:old=<fp>:new=<fp>`. Forensic win: cert rotation patterns + substituted-cert detection. Dep --- * `nix` gains the `user` feature (for `User::from_uid`). * `sha2` added as an explicit `bin/` dep (was already a workspace dep used by `command/src/certificate.rs`). Drift-guard ----------- Regex extended to cover the four new mandatory fields (`actor_user`, `socket`) and the new optional `request_sha256` extras slot. All four existing tests still green. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

refactor(command): enrich audit log — MUX layout, full ucred, fanout, injection defence Follow-up to 0c2dc869 — takes the audit log from a proof-of-concept into a compliance-grade control-plane log. Driven by parallel codex / lisa analyses (`tasks/audit-log-enhancements/*.md`). Layout ------ * Drop the trailing `\t >>>` continuation marker — the line is self-contained, nothing follows the closing paren. * Rename the keyword from `Session(...)` to `Command(...)` — the payload describes a control-plane command, not a proxy session. Retains the MUX-family bracket + tab + uppercase `AUDIT` tag layout. Taxonomy (proto EventKind additions — 10 new variants) ------------------------------------------------------ * `STATE_LOADED`, `STATE_SAVED` — `LoadState`/`SaveState` emit at task completion with `ok:<n> errors:<n>` in `target=`. * `LISTENER_ADDED`, `LISTENER_REMOVED` — `AddHttp/Https/TcpListener` and `RemoveListener`. * `SOZU_STOP_REQUESTED` — SoftStop / HardStop; `target=stop:soft|hard`. * `MAIN_UPGRADED`, `WORKER_UPGRADED` — hot-upgrade entry points. * `EVENTS_SUBSCRIBED` — SubscribeEvents subscribers. Actor identification -------------------- * `peer_cred_from_stream` captures the full SO_PEERCRED triple (uid, gid, pid), not just uid. * `peer_comm(pid)` reads `/proc/<pid>/comm` at accept time so operators can tell `sozuctl` apart from ad-hoc shells sharing a UID. * `ClientSession` gains `actor_gid`, `actor_pid`, `actor_comm`. Completion-time audit + timing ------------------------------ Worker-fanning verbs emit two audit lines — attempt-time (accepted by main state) and completion-time (applied across workers). The completion line carries `fanout=ok|partial|timeout|local_only`, `workers=<ok>/<err>/<expected>`, `elapsed_ms`, and on err paths `error_code` + `reason`. Structured error taxonomy ------------------------- New `AuditErrorCode` enum (dispatch_error, worker_failure, worker_timeout, peer_cred_unavailable, invalid_input, io_error, other). Wired at dispatch rejection, worker fan-out failures/timeouts, logging filter parse errors, and state save/load I/O errors. Log-injection defence --------------------- New `sanitize_for_audit` helper replaces ASCII control chars with `?`. Applied at render time to every attacker-influenced field (target, actor_comm, reason) so embedded tabs/newlines/ANSI escapes cannot forge additional audit lines. Drift-guard test covers the injection scenario. Dependency ---------- `rustls-webpki` 0.103.12 → 0.103.13 (RUSTSEC-2026-0104 reachable panic in CRL parsing). Out of audit codepath, flagged by CVE sweep. Followups (not in this commit) ------------------------------ * Dedicated tamper-resistant audit sink (`audit_logs_target` config + O_APPEND file) — PCI-DSS 10.5. * `socket_path=` field for multi-instance SIEM disambiguation. * `actor=<username>` via getpwuid_r cached at accept. * `request_sha256=` for dedupe / tamper-detection hints. * `ReplaceCertificate` target currently carries old fingerprint only — add new fingerprint too. * Patch-diff formatters log new values only — include old values. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(command): sozu listener {http,https,tcp} update runtime-patch verb Introduces an in-place update verb for non-bind-only listener settings, so operators can tune CVE-related H2 flood thresholds, SNI binding, disable_http11, ALPN, graceful-shutdown deadline, stream-0 WINDOW_UPDATE cap, sozu_id_header, custom HTTP answers, and timeouts under attack without cycling the listening socket. Wire protocol: - UpdateHttpListenerConfig, UpdateHttpsListenerConfig, UpdateTcpListenerConfig (+ AlpnProtocols wrapper so absent/empty is unambiguous) in command.proto, RequestType tags 47/48/49, EventKind::LISTENER_UPDATED = 18. Control plane (command/src/state.rs): - ConfigState::update_{http,https,tcp}_listener with field-mask merge. - merge_custom_http_answers preserves per-field http_answers so a patch that sets only answer_503 does NOT wipe answer_401/404/etc (regression of the earlier HttpAnswers blocker). - Server-side validation (validate_h2_flood_knobs_*, validate_alpn_*, validate_sozu_id_header) enforces H2 flood knobs >= 1, h2_stream_shrink_ratio >= 2, lifetime/header caps >= 1, ALPN values in {h2, http/1.1}, and RFC 9110-approximating token grammar on sozu_id_header. Raw protobuf clients cannot bypass via LoadState. Worker plane (lib/src/http.rs, https.rs, tcp.rs, server.rs): - *Listener::update_config + *Proxy::update_listener + Server-level notify_update_* routing. - HttpsListener rebuilds rustls ServerConfig on ALPN patch via the existing create_rustls_context (pure over (&config, resolver)) so the MioTcpListener/token/resolver are preserved. - HttpAnswers::replace_defaults rewrites listener-default templates per-field and leaves cluster_custom_answers untouched, fixing the silent-data-loss blocker that naive HttpAnswers::new swap would cause. - Defense-in-depth: worker-side update_config also runs the pub validators so a raw protobuf client or state replay cannot bypass. - ListenerError::InvalidValue maps StateError::InvalidValue through to the WorkerResponse failure path. CLI + audit (bin/src/cli.rs, ctl/request_builder.rs, command/requests.rs): - Per-protocol Update variants mirroring add/remove/activate/deactivate. - Paired --foo / --no-foo boolean flags via ArgAction::SetTrue + overrides_with; --alpn-protocols / --reset-alpn pair; answer-file paths loaded client-side. - worker_request dispatch + audit_entry_for emit EventKind::LISTENER_ UPDATED with the field-diff string on the audit log only (Event has no free-form field). Tests (e2e/src/tests/listener_update_tests.rs): - 14 e2e tests (5 passing, 9 currently #[ignore] with TODO notes) plus 26 new state.rs unit tests + replace_defaults_preserves_* in answers.rs. - Passing: test_flood_knob_validation, test_http_answers_replace_ preserves_cluster_overrides (the codex HIGH must-pass), test_not_ found, test_disable_http11_toggle, test_disable_http11_inflight_ keepalive. Ignored tests document follow-up needs on the e2e command-channel query path and timing harness. Docs: - doc/configure.md: new "Runtime patch" section with mutability-class table, CVE references, and worked examples. - CHANGELOG.md: Added + Changed entries. Known follow-ups (review notes in the commit, not blockers): - ALPN rebuild master/worker divergence rollback (M-1 codex review). - Full RFC 9110 token tokenizer for sozu_id_header. - Un-ignoring the 9 e2e tests once command-channel query + timing harness land. Verification: - cargo build --locked --all-features: clean - cargo +nightly fmt --all: clean - cargo clippy -p sozu-command-lib -p sozu-lib -p sozu --all-targets --locked: clean - cargo test -p sozu-command-lib --lib: 63/63 - cargo test -p sozu-lib --lib --test-threads=1: 382/382 - cargo test -p sozu --bin sozu: 2/2 - cargo test -p sozu-e2e listener_update --test-threads=1: 5/5 passed, 9 ignored with TODO rationale. Plan: /home/florentin/.claude/plans/i-want-you-to-precious-sun.md Cross-reviewed by codex + /review + /guidelines + /simplify. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(listener): configurable Sozu-Id correlation header name Adds a new listener knob `sozu_id_header` that lets operators rename the per-request correlation header Sozu injects into every request and response. Default stays `"Sozu-Id"`; common rebrands include `"X-Request-Trace"` or `"X-Edge-Id"`. The value is plumbed end-to-end: * `command/src/command.proto` — new field on both `HttpListenerConfig` (= 30) and `HttpsListenerConfig` (= 42), `optional string` so the default is only applied when absent. * `command/src/config.rs` — matching `Option<String>` on the builder with plumbing through `to_http` / `to_tls`. * `command/src/proto/display.rs` — `sozuctl`-style output shows the knob when set. * `lib/src/lib.rs` — `L7ListenerHandler` trait gains `fn get_sozu_id_header(&self) -> &str` with default `"Sozu-Id"` so call sites that haven't been updated keep the legacy value. * `lib/src/http.rs` + `lib/src/https.rs` — concrete listener implementations honour the config, falling back to the literal `"Sozu-Id"` when the field is `None` or empty. * `lib/src/protocol/kawa_h1/editor.rs` — `HttpContext` gains `sozu_id_header: String`, initialised from the listener at stream creation. Both the request-side and response-side writers use it instead of a hard-coded `kawa::Store::Static(b"Sozu-Id")`. * `lib/src/protocol/mux/mod.rs` — H2 stream creation reads the name from the listener and passes it to `HttpContext::new`. * `lib/src/protocol/kawa_h1/mod.rs` — H1 `Http::new` reads from the listener and passes it to `HttpContext::new` (the listener handle is already in scope, no upstream call-site change). * `doc/configure.md` — documents the knob. Tests: * `test_sozu_id_header_default_name_stored_on_context` — default path. * `test_sozu_id_header_custom_name_stored_on_context` — operator override stored verbatim. Hot-reload semantics: the value is cached on `HttpContext` per connection, so a config change takes effect on NEW connections only. Existing keep-alive connections continue to emit the old header name until they close — consistent with how sozu handles other listener-level config (connect_timeout, sticky_name, etc.) and with typical HTTP-proxy expectations. Addresses issue #1145 (§B3-t in the 2026-04-21 H2 triage plan). Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(h2): make soft_stop graceful-shutdown deadline configurable Introduces per-listener knob `h2_graceful_shutdown_deadline_seconds` (proto field on `HttpListenerConfig` and `HttpsListenerConfig`, TOML key on `[[listeners]]`). When a worker receives `soft_stop`, the existing `Mux::shutting_down()` now arms a forced-close deadline at the moment `graceful_goaway()` first transitions the connection into draining; once the budget elapses the session is torn down even with Linked/Unlinked streams still in flight. Default: 5 seconds (preserves historic intent). `0` maps to `None`, which disables the forced-close branch entirely and reverts to the old behavior of waiting indefinitely for streams to drain. Wiring: - `command.proto`: `h2_graceful_shutdown_deadline_seconds = 28` on `HttpListenerConfig`, `= 40` on `HttpsListenerConfig`. - `command/src/config.rs`, `proto/display.rs`: ListenerBuilder field + `to_http`/`to_tls` propagation + display row. Mirrors the sibling H2 knob wiring (`h2_max_rst_stream_per_window` template). - `lib/src/lib.rs`: new trait method `L7ListenerHandler::get_h2_graceful_shutdown_deadline` with a `Some(Duration::from_secs(5))` default. - `lib/src/http.rs`, `lib/src/https.rs`: HttpListener / HttpsListener implementations. Value `0` maps to `None`. - `lib/src/protocol/mux/h2.rs`: `H2DrainState` gains `started_at` (armed once by `graceful_goaway`) and `graceful_shutdown_deadline`. Peer-initiated GOAWAYs (via `handle_goaway_frame`) deliberately do NOT arm the timer — the budget applies only to the proxy's own soft-stop. Adds `ConnectionH2::graceful_shutdown_deadline_elapsed`. - `lib/src/protocol/mux/connection.rs`: `new_h2_server` / `new_h2_client` plumb the deadline; enum-level `Connection::graceful_shutdown_deadline_elapsed` (H1 returns `false` — no multiplex to drain). - `lib/src/protocol/mux/mod.rs::Mux::shutting_down`: checks `graceful_shutdown_deadline_elapsed` right after `drive_frontend_shutdown_io` and returns `true` on expiry so the server loop closes the session. - `lib/src/protocol/mux/router.rs`, `lib/src/https.rs`: plumb the deadline through `new_h2_client` / `new_h2_server` call sites. - `lib/src/protocol/mux/LIFECYCLE.md`: documents the armed timer and forced-close branch in §8.3 Session drain. - `doc/configure.md`: adds the knob to the H2 connection tuning table and TOML example. Tests (e2e/src/tests/h2_tests.rs): - `test_h2_graceful_shutdown_timeout_forces_close` — default 5 s budget fires between 3 s and 15 s after soft_stop with a held request. - `test_h2_graceful_shutdown_deadline_configurable_short` (deadline=1 s) — worker stops in under 4 s. - `test_h2_graceful_shutdown_deadline_configurable_long` (deadline=60 s) — no premature close within 10 s, then completes on release. Uses `resolve_request_timeout(60 s)` so hyper does not abort the in-flight stream inside the assertion window. Validation: `cargo build --all-features --locked`, `cargo clippy --all-targets --locked`, `cargo +nightly fmt --all -- --check`, `cargo test --workspace --locked --lib`, focused `cargo test -p sozu-e2e --locked -- --test-threads=1 test_h2_graceful` — all pass. Resolves feat/h2-mux audit Q1 (2026-04-21): the soft_stop deadline is intentional, must remain configurable, default preserved. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(h2): add flood counter for connection-level WINDOW_UPDATE (stream 0) Adds a new per-sliding-window counter in `H2FloodDetector` that tracks non-zero stream-0 WINDOW_UPDATE frames and triggers GOAWAY(ENHANCE_YOUR_CALM) when the configurable threshold is exceeded. Context: the pre-existing detector tracked RST_STREAM, PING, SETTINGS, empty DATA, and CONTINUATION floods plus a generic glitch counter, but non-zero stream-0 WINDOW_UPDATE frames were uncounted. Zero-increment stream-0 WINDOW_UPDATEs already short-circuit into GOAWAY(PROTOCOL_ERROR) per RFC 9113 §6.9, but legal non-zero increments have no per-frame cost limit and a peer could burn proxy CPU by sending millions of them. Changes: * `command/src/command.proto` — new optional field `h2_max_window_update_stream0_per_window` on both `HttpListenerConfig` (= 29) and `HttpsListenerConfig` (= 41). * `command/src/config.rs` — matching `Option<u32>` on `ListenerBuilder` with plumbing through `to_http` / `to_tls`. * `command/src/proto/display.rs` — extends `add_h2_flood_rows` with the new field so `sozuctl`-style output shows it when set. * `lib/src/protocol/mux/h2.rs`: - new constant `DEFAULT_MAX_WINDOW_UPDATE_STREAM0_PER_WINDOW = 100` (mirrors the other per-window defaults). - `H2FloodConfig` gains `max_window_update_stream0_per_window: u32` with matching default, `new()` arg, and `.max(1)` clamp. - `H2FloodDetector` gains `window_update_stream0_count: u32`, initialised to 0, halved in `maybe_reset_window`, and checked in `check_flood` with metric key `h2.flood.violation.window_update_stream0_window`. - `handle_window_update_frame` increments the counter (saturating) on every non-zero stream-0 WINDOW_UPDATE before the arithmetic, and calls `check_flood` so a burst is stopped before we pay the cost. * `lib/src/{http,https}.rs` — wire the listener-config field into `get_h2_flood_config` with defaults fallback. * `doc/configure.md` — document the knob in the flood-thresholds table and the TOML example. Tests: * `test_flood_detector_window_update_stream0_trips_at_threshold` asserts strict greater-than semantics + correct violation metadata. * `test_flood_detector_window_update_stream0_honours_default` asserts the default counter config value matches the documented constant. * `test_flood_detector_half_decay_on_window_expiry` extended to exercise the new counter alongside the existing four. Addresses Codex finding G2 from the 2026-04-21 H2 triage (~/.claude/plans/ask-h2-issues-triage-plan.md §Un-ticketed Gaps Matrix and §Implementation Outcome follow-up list). Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

docs(cluster): clarify http2 flag is a backend-capability hint cluster.http2 = true signals that the BACKEND speaks HTTP/2 (h2c or h2+TLS). It does NOT gate H2 acceptance at the frontend — frontend H2 is negotiated via TLS ALPN on the listener (alpn_protocols) and is fully independent of per-cluster configuration. Per user decision: feat/h2-mux / PR #1209 planning, 2026-04-21 (audit Q8 — clarify http2 field semantics). Files updated: - doc/configure.md: added callout box under "HTTP/2 backend connections" - command/src/command.proto: expanded Cluster.http2 doc comment - command/src/config.rs: expanded ClusterConfig.http2 doc comment - CLAUDE.md: tightened H2 config knobs bullet Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(logging): surface TLS metadata and XFF chain on access logs Extend the `ProtobufAccessLog` wire schema with five additional optional fields (tags 25–29) capturing the per-connection TLS handshake metadata and the upstream-attested forwarded chain. Follows the same pattern as `x_request_id` (c9ec90cf): plumb through `mux::Context` → `HttpContext` → `RequestRecord` → proto, populated at every access-log emit site. Fields: - `tls_version` (`&'static str`): short label from `rustls_version_label` (e.g. `TLSv1.3`). New helper alongside the existing metric-prefixed `rustls_version_str` so the log records `TLSv1.3` rather than the dotted metric key. - `tls_cipher` (`&'static str`): short label from `rustls_ciphersuite_label` (e.g. `TLS_AES_128_GCM_SHA256`). Paired with `tls_version` in `lib/src/https.rs::upgrade_handshake`. - `tls_sni` (`&str`): borrowed from `HttpContext.tls_server_name`, same pre-lowercased value the routing layer uses to enforce the SNI ↔ `:authority` binding (CWE-346 / CWE-444). - `tls_alpn` (`&'static str`): on-the-wire ALPN label (`h2`, `http/1.1`) captured alongside the existing `AlpnProtocol` match. - `xff_chain` (`&str`): verbatim `X-Forwarded-For` value snapshotted in `editor.rs::on_request_headers` *before* Sōzu appends its own peer hop — the log records the upstream-attested chain, not the rewritten header Sōzu forwards. TLS fields are connection-scoped (stamped once on `mux::Context` at handshake completion, propagated to every per-stream `HttpContext` via `Context::create_stream`); `HttpContext::reset()` intentionally preserves them across H1 keep-alive so request N+1 still carries the handshake metadata. `xff_chain` is per-request and resets. Coverage across session types: - H1 + H2 mux: populated from the shared `HttpContext`. - WSS post-upgrade pipe: `Pipe` grows `set_tls_metadata`, called from `https.rs::upgrade_mux` so the WebSocket access log inherits the handshake metadata. - Plain TCP / WS / TCP+proxy-protocol: always `None` (no TLS termination on those paths). Also: - `HttpContext` gains corresponding fields + reset-preservation test. - `doc/configure.md` documents the five new access-log fields with their wire tags and source of truth. Signed-off-by: Florentin Dubois <florentin.dubois@clever-cloud.com> Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(metrics): metrics.detail cardinality knob (foundation) Foundation for per-listener / per-cluster / per-backend metric label opt-in, mirroring HAProxy's `process|frontend|backend|server` extra-counters knob. Adding labels to the StatsD keyspace under load (e.g. per-listener bytes_in on a host with many listeners) can blow up the keyspace of any statsd aggregator; HAProxy solved this by making the detail level an explicit operator choice. This commit lands the same control surface on Sōzu. Added: - `MetricDetail` proto enum in `command/src/command.proto` (`DETAIL_PROCESS=0 | FRONTEND=1 | CLUSTER=2 | BACKEND=3`, each a superset of the previous). `ServerMetricsConfig` gains an optional `detail` field (tag 4) so workers built before this lands default to DETAIL_CLUSTER on the lib side and preserve historical behaviour. - `MetricDetailLevel` Rust enum in `command/src/config.rs` with `serde(rename_all = "lowercase")` so operators write `detail = "frontend"` in the TOML config. Doc-comment calls out the superset relationship and the HAProxy analogue. Deferred (intentional — scope hard stop per the brief): - Wiring the new macros / labels into the existing `incr!`/`gauge!` call sites. That's >50 call sites and risks either a breaking API change to the macro or a parallel `incr_listener!` family — either shape wants a dedicated follow-up MR so the macro surface stays reviewable. - Touching `lib/src/metrics/network_drain.rs` to actually honour the detail level on the wire. Depends on the macro decision. - The accept-path telemetry (commit 4378101c) already labels by listener address regardless of this knob; once the wiring lands, that path becomes opt-in via DETAIL_FRONTEND and collapses to DETAIL_PROCESS otherwise. Shipping the proto + config enum first means the follow-up MR is a pure labelling change against a stable config shape — operators can already set the knob in their TOML files today; the value is simply ignored by the current drain until wiring lands. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(http): propagate x-request-id and log it on access log Add end-to-end `x-request-id` handling. The header is the de-facto correlation key used by every modern LB (Envoy, HAProxy, most cloud load-balancers) — before this change Sōzu dropped incoming values and never synthesised one, so request flows couldn't be correlated across the proxy. Behaviour: - Incoming request with `x-request-id`: value preserved verbatim in `HttpContext.x_request_id`, forwarded unchanged to the backend, `incr!("http.x_request_id.propagated")`. - Incoming request without `x-request-id`: generate a header value from the request ULID (`self.id`), inject it into the block list, store the value in `HttpContext.x_request_id`, `incr!("http.x_request_id.generated")`. Works for both H1 and H2 because `pkawa.rs` decodes HPACK into a kawa-H1 representation and dispatches into the shared `on_headers` callback in `editor.rs`. Also surfaced on access logs: - `x_request_id: Option<&str>` added to `RequestRecord`. - `optional string x_request_id = 24;` added to `ProtobufAccessLog` (wire-compatible append). - Populated from `HttpContext.x_request_id` on H1 and H2 mux paths; always None on pure-TCP / WebSocket paths. - Reset test updated. doc/configure.md gains a `Request-ID propagation` subsection. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(command): extend EventKind with control-plane mutation variants Foundation for a control-plane audit trail. Today the SubscribeEvents bus carries only four backend-health events (BACKEND_DOWN, BACKEND_UP, NO_AVAILABLE_BACKENDS, REMOVED_BACKEND_HAS_NO_CONNECTIONS). Add 14 new EventKind variants for the mutation surface so subscribers (audit shims, SIEMs, compliance ingestion) can react to cluster / frontend / certificate / listener / configuration / worker / logging changes without polling state diffs. New variants (numeric tags 4..17, existing tags unchanged for wire compatibility): - CLUSTER_ADDED, CLUSTER_REMOVED - FRONTEND_ADDED, FRONTEND_REMOVED - CERTIFICATE_ADDED, CERTIFICATE_REMOVED, CERTIFICATE_REPLACED - LISTENER_ACTIVATED, LISTENER_DEACTIVATED - CONFIGURATION_RELOADED - WORKER_KILLED, WORKER_RELAUNCHED - LOGGING_LEVEL_CHANGED, METRICS_CONFIGURED `Display for Event` (`command/src/proto/display.rs`) gains the matching human-readable strings. `bin/Cargo.toml` enables the `nix` `socket` feature so the follow-up that wires emit sites can use `nix::sys::socket::sockopt::PeerCredentials` to capture the SO_PEERCRED actor UID off the unix socket. The feature flag has no runtime cost when unused. Deferred to follow-up (intentionally not in this commit): - Emit sites in `bin/src/command/{requests,server}.rs`. Each mutating request handler needs to push an `Event` with the matching kind and log a structured audit line. Touches ~10 handlers. - SO_PEERCRED capture at the unix-socket accept site, plumbed through `ClientSession` into the audit log line. - Per-verb `incr!("config.<verb>", ...)` counters (depend on a cardinality decision: per-actor labels need scoping rules first). - doc/configure.md update describing the new event taxonomy. Compliance regimes (PCI-DSS 10.2, ISO 27001 A.8.15, SOC 2) require an immutable audit trail of privileged mutations; this commit is the wire- format prerequisite for that work to land without breaking subscribers. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>

feat(mux): propagate per-session ULID through the protocol stack + log-context schema Introduces a stable per-connection identity that survives protocol upgrades (ExpectProxy → TLS handshake → H1/H2) and rewrites the access log-context block so operators can grep a whole TCP/TLS session (`session_id`) independently of a single HTTP exchange (`request_id`). **Log-context schema** (`command/src/logging/access_logs.rs`, `display.rs`): - `LogContext { session_id: Ulid, request_id: Option<Ulid>, cluster_id, backend_id }`. Rendered as `[<session_id> <request_id_or_-> <cluster_id_or_-> <backend_id_or_->]`. - `session_id` is minted once per accepted socket and copied across every protocol state change. - `request_id` becomes optional: present for H1 keep-alive exchanges and per H2 stream, absent for pre-upgrade events that belong to the session but not to any single request. - `RequestRecord` serialisation now emits both ids so downstream access-log consumers can correlate either axis. **`SessionTcpStream` wrapper** (`lib/src/socket.rs`, +287 lines): New `SocketHandler::session_ulid() -> Option<Ulid>` method with a default `None` for raw `mio::TcpStream`. `SessionTcpStream` is a thin `mio::TcpStream` wrapper that carries the owning `Ulid` and returns it from `session_ulid()` — used by every frontend socket path so error logs inside `SocketHandler` implementations can stamp the session prefix without threading it through every call site. **Propagation**: the session ULID flows from the outer `HttpSession` / `HttpsSession` constructors (`lib/src/{http,https}.rs`) through: - `Connection::new_h1_server(session_ulid, socket, …)`, `new_h2_{server,client}` - `ConnectionH1 { …, session_ulid: Ulid }` and the per-stream generation of `request_id` - `Pipe { …, session_id, request_id }` (pipe protocol now tracks both) - `Context::new(session_ulid, pool, listener, …)` on the mux side - ProxyProtocol `expect`/`relay`/`send` handlers pick up the ULID when the upgrade transfers the socket forward. **`log_context!` macros** (`mux/{h1,h2,mod,router}.rs`, `kawa_h1/{mod,editor}.rs`, `pipe.rs`, `rustls.rs`, `tcp.rs`): read the new `LogContext.session_id`/`request_id.unwrap_or(session_id)` fields when formatting the bracketed prefix. No behaviour change for logs that were already session-scoped; adds the second ULID slot everywhere else. **proto** (`command/src/command.proto`): access-log wire format gains a `session_id` field alongside the existing `request_id`. Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>