Proto commits in istio/ztunnel

These 37 commits are when the Protocol Buffers files have changed:

Commit:9e3c8cf
Author:John Howard
Committer:GitHub

wds: implement capacity (#1472) Ztunnel side impl of https://github.com/istio/istio/pull/55284

The documentation is generated from this commit.

Commit:00b7410
Author:John Howard
Committer:GitHub

Resync proto with istio/istio (#1434) This fell out of date

Commit:eda6461
Author:John Howard
Committer:GitHub

authorization: implement new serviceAccounts field (#1402)

Commit:c23771b
Author:Keith Mattix II
Committer:Keith Mattix II

Get to a working build again Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

Commit:4a5300c
Author:Jeremy L. Morris

merge in Keith's stash and deal with merge conflicts

Commit:9803ea4
Author:Jeremy L. Morris

ran cargo fmt and updated workloadmanager and related references

Commit:5ecaea9
Author:keithmattix

Stash Signed-off-by: keithmattix <keithmattix@microsoft.com>

Commit:cb56b5b
Author:John Howard
Committer:GitHub

Add support for `publishNotReadyAddresses` (#1231) * Add support for `publishNotReadyAddresses` * rename

Commit:6d58d57
Author:John Howard
Committer:GitHub

Add support for hostnetwork pods (#1216) This goes along with https://github.com/istio/istio/pull/52191. Most of the change is in Istiod, the only real change here is to stop indexing the workload address if it is a host network pod. In terms of user facing behavior, this means that if I call a host-network pod directly (NOT via service), then it will not associate with the workload. If this were to happen, we would effectively be associating it with an arbitrary host network pod on the same node; there is no way to distinguish these. In practice, hostnetwork pods cannot be HBONE, so this ensures that we consistently get "unknown" as the telemetry info rather then a random pod. This is encoded in the new test. There is additional misc trace level logging improvements that helped me debug/develop this.

Commit:8458d66
Author:John Howard
Committer:GitHub

Support ip_families field in service (#1161) * Support ip_families field in service * Move to service level

Commit:e2bcfbe
Author:John Howard
Committer:GitHub

Drop unused field in WDS (#1159) This is cleanup from an earlier change on Istio side. This is deadcode. workload.proto changes are copy+paste from Istio

Commit:2d2ad04
Author:Istio Automation
Committer:GitHub

Don't use trust_domain in zds (#1091) Signed-off-by: Benjamin Leggett <benjamin.leggett@solo.io> Co-authored-by: Benjamin Leggett <benjamin.leggett@solo.io>

Commit:4ff9897
Author:Ben Leggett
Committer:GitHub

Don't use trust_domain in zds (#1088) Signed-off-by: Benjamin Leggett <benjamin.leggett@solo.io>

Commit:f918a76
Author:John Howard
Committer:GitHub

Implement locality aware load balancing (#868) * Implement locality aware load balancing Fixes https://github.com/istio/ztunnel/issues/862 * Update docs * Add tests and pick better names * fixes

Commit:e570cd8
Author:Steven Landow
Committer:GitHub

proxy protocol for sandwich (#850) * PROXY protocol sandwich * move app tunnel to GatewayAddress * Revert "move app tunnel to GatewayAddress" This reverts commit 24184d0bac4c38366a38edc9f152ace890da2f10. * fix fuzz lockfile * fix test

Commit:2907fd3
Author:Yuval Kohavi
Committer:GitHub

proxy: validate ZDS provided workload service account when asserting rbac (#777) * proxy: assert rbac using workload uid instead of destination ip * pr feedback: - match workload properties after fetching by network addr - cleanup data structures * introduce ProxyRbacContext for assert_rbac, so we can use it with the new connection manager. * make fix * cleanup * add tests

Commit:bcdce3f
Author:Jeremy L. Morris
Committer:GitHub

Add waypoint to Service proto definition in WDS (#823)

Commit:bc68182
Author:Yuval Kohavi
Committer:GitHub

inpod redirection mode (#747) * inpod * make fix * git apply <(curl ...) * typo * pr comments * make gen * remove NoSnapshotPresent * due to how iptables redirect works, we need to configure the dns address (so we can set it to 127.0.0.1) * improve log msg * make gen * 1 bit for inpod mark. accept mark from env * send hello message * Apply suggestions from code review Co-authored-by: John Howard <howardjohn@google.com> * remote mocks * make non-mocked tests not run if not root * protocol.rs: - remove workload_processor_helpers sub module - remove the need for iovs_buf; Code restructured so that buffer var is not borrowed * fix clippy notes * make gen * git apply * pr comments: - move admin handler from metrics - feed metrics to inpod instead of registry - remove inpod prefix from metrics - change mark to 1337 * be explicit about what's a uid vs String * pr comments: - better error message when mark fails - minor clean-ups * - change mark in redirect script for namespaced tests - fix tests * remove pub from inpod * allow for SO_REUSEPORT note that SO_REUSEPORT only work if both old and new ztunnel have the same uid * Change log level on stream end in src/inpod/workloadmanager.rs Co-authored-by: John Howard <howardjohn@google.com> * tests for port re-use --------- Co-authored-by: John Howard <howardjohn@google.com>

Commit:a1c799e
Author:j2gg0s
Committer:GitHub

Fix typos in doc/comment by https://github.com/crate-ci/typos (#748)

Commit:1eddf6a
Author:Keith Mattix II
Committer:GitHub

Add oneway tls port to waypoint (#692) * Add port for hbone single TLS Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Remove optional proto Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Add default fields Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Make gen Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

Commit:5940467
Author:Keith Mattix II
Committer:Keith Mattix II

Remove optional proto Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

Commit:647b57a
Author:Keith Mattix II
Committer:GitHub

Add port for hbone single TLS (#682) * Add port for hbone single TLS Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> * Use port 15003 for single TLS Signed-off-by: Keith Mattix II <keithmattix@microsoft.com> --------- Signed-off-by: Keith Mattix II <keithmattix@microsoft.com>

Commit:539ff43
Author:Kevin Dorosh
Committer:GitHub

Implement on-demand hostname for Workloads (#612) * WIP implement hostname Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> WIP testing to see what it would take to get something working Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> Begin dns resolution WIP Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Refactor UID as key for endpoints in service Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Add back removed per-pod hostname code Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Implement repeated polling in async DNS Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Cargo clippy Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Use util to get system forwarder Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Improve clarity of different hostname fields on workload Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Allow custom nameservers for system fordwarder Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Make gen Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Remove async_hostname; just reuse hostname on the workload Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Move load_balance() to DemandProxyState and make async in prep for on-demand impl Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Messy POC: on-demand DNS initiated on request path Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * No need for delay in direct test now since we have on demand DNS Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * WIP: begin to filter DNS polling based on recent requests received Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * WIP: ensure we only call load_balance() once per outgoing request Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * WIP: make a second request to smoke check cache hits are working (they are) Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * WIP: flatten some of task context logic, and don't blow away last_queried time from poller Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * PR still WIP: add metrics for on-demand DNS cache hits and misses Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * PR still WIP: begin using resolver instead of forwarder Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * PR still WIP: optimize to only poll if we recevied another request during DNS TTL Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * PR still WIP: remove polling DNS optimization until initial PR is approved Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * PR still WIP: ipv6 support Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * PR still WIP: flatten some code Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * PR comments Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Refresh once per 30s if we have no dns records Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * More accurate comment Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * More accurate comment Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * More accurate comment Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * More accurate comment Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * PR comment; remove unneeded clone Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * PR comment; set_gateway_address does not need to be a method Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * PR comment; only call set_gateway_address once for waypoints Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * PR comments; move dns cfg/opts to DemandProxyState, rename some things Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Add endpoints, even for workloads that don't have an address Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * No reason to return join handle since we aren't joining several handles Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Flatten extra await Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> --------- Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io>

Commit:bc65fea
Author:Nathan Mittler
Committer:GitHub

Remove unused vip field (#583) This field was removed by https://github.com/istio/istio/pull/45764 and https://github.com/istio/ztunnel/pull/572. The control plane is no longer sending it and the field has been removed from the proto.

Commit:ea748c6
Author:Nathan Mittler
Committer:GitHub

Update API for headless services (#572) Preferring service keys rather than VIPs for referencing services from workload. Also, change requirements to allow both addresses and hostnames for addressing a headless service.

Commit:a3fd66a
Author:Kevin Dorosh
Committer:GitHub

Update `Workload` primary xds key to UID (#529) * Test repeated addresses on workload Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Move workloads to Arc Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Move workloads_by_uid to Arc Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Update UID api Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Update workload primary xds key to UID Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Load balance to all workload IPs randomly Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Don't unwrap error while collecting Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Add service builder to namespaced tests Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Don't include pod IP as suffix in example UIDs Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Update proto/workload.proto Co-authored-by: John Howard <howardjohn@google.com> --------- Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> Co-authored-by: John Howard <howardjohn@google.com>

Commit:e36680f
Author:Kevin Dorosh
Committer:GitHub

Ambient xds evolution (#486) * WIP * Finish ambient xds workload evolution based on https://docs.google.com/document/d/1V5wkeBHbLSLMzAMbwFlFZNHdZPyUEspG4lHbnB0UaCg/edit#heading=h.qpcs4a7wyicb Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Search for waypoint workload by service VIPs too Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Search all service endpoints for determining if we bypassed waypoint Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Cargo fmt /fuzz * Implement fetch_address() Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Implement find_waypoint() Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Fix misleading name for function that gets waypoint svc ip Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Better internal representation for GatewayAddress Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Implement TryFrom for GatewayAddress Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Simpler insert or create new map Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Minor refactor to XdsService -> Service Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Fix bug with stale staged endpoints cleanup Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * cargo clippy Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * First pass network address services Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Update to use network address as key for staged vips Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Update some proto comments Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * NetworkAddressKey is more generic, not just service secondary key Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * NetworkAddress for fetch Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * More to NetworkAddress Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Stop hard coding the network Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Update GatewayAddress to use network-prefixed keys for lookup Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Update workload field protocol to tunnel_protocol Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Update inbound cert provider to use configured network Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Clean up some formatting Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Stronger test assertion for load balancing Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Implement custom serde for NetworkAddress since it keys a map Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Clarify network assumptions on inbound requests Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Simplify check_waypoint function and return noisy error for unsupported feature Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Flatten indentation with more idiomatic code Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Rename internal representation of native_tunnel too Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Don't lock twice when we only need to once Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Remove some unnecessary clones when fetching by NetworkAddress Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Update some more comments with clarifying network assumptions Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Don't confuse myself, instead use better variable names Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Default network should be empty string Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Service can select workloads in other networks Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Check if request came from gateway before attempting lookup Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Don't specify values that match the default Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * DRY check_gateway Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Add UID to proto for workloads in remote networks with unknown IP Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Improve test coverage for config dump Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Update comment on services to be more complete Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Update api to reflect DNS resolution by ztunnel per workload Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Remove OpaqueEndpoint Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Update service primary xds key to namespace/hostname Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Use NetworkAddress for workload_to_vip Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Improve test coverage and fix endpoint propagation bug on service upsert Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Share more code so service endpoints is optional on local yaml test config Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Store services on the heap so mutations are shared across maps Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Dedup unneeded workload_to_vip insertions Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Better name for internal service maps so config dump is clearer Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Flatten functions so they are easier to read Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Ensure we clean up map indexing properly for services on upsert Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> * Fix test to ensure local config has unique svc hostnames Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io> --------- Signed-off-by: Kevin Dorosh <kevin.dorosh@solo.io>

Commit:8074321
Author:Ian Rudie
Committer:GitHub

sample changes to proto (#501) * sample changes to proto Signed-off-by: ilrudie <ian.rudie@solo.io> * add/updates comments in proto for clarity Signed-off-by: ilrudie <ian.rudie@solo.io> --------- Signed-off-by: ilrudie <ian.rudie@solo.io>

Commit:3aa10e6
Author:Yossi Mesika
Committer:GitHub

Get and use Cluster ID from the WDS (#439)

Commit:a4a66bc
Author:John Howard
Committer:GitHub

Add health status (#374) Replaces https://github.com/istio/istio/pull/43085

Commit:0e123ea
Author:John Howard
Committer:GitHub

Handle upstream proto renames (#324) * Handle upstream proto renames * fmt

Commit:65225bb
Author:Steven Landow
Committer:GitHub

parse proxy config (#294) * parse proxy config * status port * lint * and_then * wip * use yaml merging * cleanup err handling * more cleanup + test * abandon prost gen for proxyconfig * cleanup

Commit:f385be4
Author:John Howard
Committer:GitHub

implement L4 AuthorizationPolicy (#297) * Initial RBAC implementation This commit introduces the first iteration of L4 AuthorizationPolicy support. * Policies are read over XDS and converted to a new internal type. These are indexed by namespace * Inbound requests check against applicable policies and send 401 in the HBONE response when they fail to match * Current matching algorithm is fairly efficient (TODO: benchmarks), but not optimized (one could imagine a trie like implementation; today it is a linear search). * add tests * redundant clones

Commit:d652ecb
Author:John Howard
Committer:GitHub

xds: pass metadata, including NODE_NAME (#256)

Commit:f4cb475
Author:Zhonghu Xu
Committer:GitHub

Make waypoint_addresses vec (#147) * Update waypoint_addresses to Vec * Fix liny

Commit:40e9998
Author:John Howard
Committer:GitHub

Add support for reading from local file instead of XDS (#28) * Add support for reading from local file instead of XDS * Update for new Istiod proto * apply local comments

Commit:f8b91c9
Author:John Howard

Initial import of code