Proto commits in EventStore/EventStore-Client-Dotnet

These 54 commits are when the Protocol Buffers files have changed:

Commit:8f20744
Author:Sérgio Silveira

checkpoint

Commit:1c52e9f
Author:Sérgio Silveira

Refactor stream handling and implement new append model Removed legacy stream handling code and replaced it with more modular, extensible append models using `OneOf` and Proto-based definitions. Introduced new `Appender` and `Protocol` utilities for improved batching, schema handling, and multi-stream transactional operations. Updated metadata processing strategy for compatibility with both old and new database contracts.

Commit:7689e0c
Author:Sérgio Silveira

Move and reorganize streams and protocol files into new modules. Refactored the folder structure for streams and protocol files to improve organization and clarity. Introduced `Modules` as the main folder for streams, and migrated protocol definitions to a new hierarchical structure under `proto/kurrentdb/protocol`. Added new v2 protocol files for schema registry functionality.

Commit:faf35ef
Author:William Chong
Committer:William Chong

Rename namespaces and samples

The documentation is generated from this commit.

Commit:b770aac
Author:William Chong
Committer:William Chong

Rename namespaces and samples

The documentation is generated from this commit.

Commit:f4e0c5d
Author:YoEight

fixup

Commit:681a4bd
Author:YoEight

feat: supports new caught up and fell behind events.

Commit:08bee94
Author:Sérgio Silveira
Committer:GitHub

chore: [DEVEX-185] Rebranding (#326) * moved all files into EventStore.Client project * moved files into main project and deleted remaining empty projects dropped support for net6.0 and net7.0 added some helpers to fluentdocker playing around the idea of keeping the container on tests * Remove unused target frameworks and pass certificate in healthcheck * Tests * Rename test symbols * Test workflow * Testing something * More tests * Improve CI * Improve CI * Fixup * Publish actions * Temporarily disable Public actions * Fixup * Test warmup * [DEVEX-222] Add built-in auto-serialization (#329) * [DEVEX-222] Added configuration options for KurrentClientSettings Create method This should enable easier customisation like changing serializer settings. * [DEVEX-222] Added overloads of Append methods that take regular events They currently use dummy serialisation * [DEVEX-222] Added TryDeserialize method to ResolvedEvent Currently it's implemented in a dummy way * [DEVEX-222] Added first working, but not yet complete JSON built-in serialization It's not fully ready, as it has hardcoded schema serializer. Main question will be how much from schema registry I need to move here. * [DEVEX-222] Added autoserialization settings and removed serializer from ResolvedEvent Per @RagingKore suggestion it'll be better not to keep the reference to serializer in ResolvedEvent to keep the clear intention behind it. Now, the serializer is resolved from schema registry based on the auto-serialization settings. Made possible to register more than one serializer per content type. Currently it doesn't support more schema definition types. To be discussed how to include them later on. * [DEVEX-222] Made serialization settings to create specific serializers That will make resolution easier and give users ability to inject their own serializers implementation * [DEVEX-222] Added override of the serialization settings to Persistent Subscriptions * [DEVEX-222] Added serialization type and merged serialization into Serialization Context Previously we had DeserializationContext, but after consideration, it'll be better to merge those concepts to make easier customizations per operations (e.g. append, subscribe, etc.). * [DEVEX-222] Added message type name resolution strategy Refactored the code accordingly. It takes the full object instead of the limited number of parameters, as you may be using metadata to get parameters about clrtype. * [DEVEX-222] Refactored Event Type Mapper to not be responsible for resolution of CLR types * [DEVEX-222] Added automatic message clr type resolution * [DEVEX-222] Added metadata extensions to allow injecting type informations * [DEVEX-222] Made CLR type be resolved based on the message type metadata header * [DEVEX-222] Added the MessageTypeResolutionStrategyWrapper for caching resolved types * [DEVEX-222] Added MessageSerializationContext to pass additional information like category name * [DEVEX-222] Refactored SerializationContext into MessageSerializerWrapper to make clearer responsibilities Actually, it's just wrapping message serializer based on the client settings. * [DEVEX-222] Added message type maps registration * [DEVEX-222] Added helpers for custom type resolutions strategy Removed also obsolete SerializationSettings * [DEVEX-222] Replaced MessageSerializerWrapper usage with more generic IMessageSerializer There's no need to force all code to know about wrapper existence, it's okay to just create it during the setup. * [DEVEX-222] Made Message to be record instead of Struct * [DEVEX-222] Refactored new AppendToStreamAsync method to have simplified syntax Now they don't require all the fancy, but redundant in most cases Stream Positions, Directions, EventData etc. * [DEVEX-222] Refactored Reading events signatures to use options instead of parameters Thanks to that safe defaults are already provided * [DEVEX-222] Renamed MessageTypeResolutionStrategy to MessageTypeNamingStrategy * [DEVEX-222] Made old Read methods be wrapper of the new one, instead the other way round * [DEVEX-222] Added serialization configuration overrides Old append and read methods won't be doing default serialization by default. The new one will use auto-seriailization and allow to customize or disable it. * [DEVEX-222] Removed storing CLR type in metadata and moved it to message type name Now the FullTypeName is placed in the event type name. Not perfect but here it is. * [DEVEX-222] Added possibility to register types for category name Thanks to that they will be resolved automatically by the registered convention. Then user doesn't need to provide the string name by hand. Added MessageTypeNamingResolution context besides the MessageSerializationContext as it's orthogonal and just overlapping context. * [DEVEX-222] Added SubscribeToAllOptions and Subscription Listener This is a similar API as we already have in Java client, to simplify the handlers registration. * [DEVEX-222] Addes SubscribeToStreamOptions and applied it accordingly to subscirbe to all changes * [DEVEX-222] Renamed DeserializedEvent to Message in ResolvedEvent I think that in the longer term, this will make intention clearer * [DEVEX-222] Added persistent subscription litener and capability to customize serialization for Persistent Subscriptions * [DEVEX-222] Extended Serialization tests for Appends and Reads * [DEVEX-222] Added capability to automatically deserialize message metadata By default it'll try to resolve the TraceMetadata, it can be override with the general type or using custom resolution strategy. * [DEVEX-222] For unknown reason the reverse Enumerable.Reverse wasn't taken but Array.Reverse 🤷 * [DEVEX-222] Fixed type resolution to correclty find types by assembly * [DEVEX-222] Added tests for serialization compatibility between manual and automatic deserialization * [DEVEX-222] Added tests for appends with autoserialization * Revert "[DEVEX-222] Added tests for appends with autoserialization" This reverts commit 04e068554a0a8510af7da0b1e5b035edf513e679. I'm reverting it as old methods are wrappers for new ones, so the test scenarios are covered by the old test suite. * [DEVEX-222] Extended serialization test suite * [DEVEX-222] Added tests for auto-serialization in Catch-up subscriptions * [DEVEX-222] Added tests for auto-serialization in persistent subscriptions * [DEVEX-222] Added Expected prefix to StreamState and StreamRevision in AppendToStreamOptions * [DEVEX-222] Merged namespaces Fixed also XML docs comments * [DEVEX-222] Renamed NulloMessageSerializer to NullMesageSerializer * [DEVEX-222] Added AppendToStream methods that has expected stream revision as a parameter * [DEVEX-222] Added XML documentation for Serialization settings * [DEVEX-222] Added Unit tests * [DEVEX-222] Added tests for type resolution Used also two external assemlies - one that's loaded, - one that's never loaded. To double-check behaviour of loading types from different assemblies. * [DEVEX-222] Added SchemaRegistry tests * Revert "[DEVEX-222] Add built-in auto-serialization (#329)" (#332) This reverts commit 44607ea8381974c5fa6a903fa3b34101825d3036. * [DEVEX-222] Add built-in auto-serialization (#333) * [DEVEX-222] Added configuration options for KurrentClientSettings Create method This should enable easier customisation like changing serializer settings. * [DEVEX-222] Added overloads of Append methods that take regular events They currently use dummy serialisation * [DEVEX-222] Added TryDeserialize method to ResolvedEvent Currently it's implemented in a dummy way * [DEVEX-222] Added first working, but not yet complete JSON built-in serialization It's not fully ready, as it has hardcoded schema serializer. Main question will be how much from schema registry I need to move here. * [DEVEX-222] Added autoserialization settings and removed serializer from ResolvedEvent Per @RagingKore suggestion it'll be better not to keep the reference to serializer in ResolvedEvent to keep the clear intention behind it. Now, the serializer is resolved from schema registry based on the auto-serialization settings. Made possible to register more than one serializer per content type. Currently it doesn't support more schema definition types. To be discussed how to include them later on. * [DEVEX-222] Made serialization settings to create specific serializers That will make resolution easier and give users ability to inject their own serializers implementation * [DEVEX-222] Added override of the serialization settings to Persistent Subscriptions * [DEVEX-222] Added serialization type and merged serialization into Serialization Context Previously we had DeserializationContext, but after consideration, it'll be better to merge those concepts to make easier customizations per operations (e.g. append, subscribe, etc.). * [DEVEX-222] Added message type name resolution strategy Refactored the code accordingly. It takes the full object instead of the limited number of parameters, as you may be using metadata to get parameters about clrtype. * [DEVEX-222] Refactored Event Type Mapper to not be responsible for resolution of CLR types * [DEVEX-222] Added automatic message clr type resolution * [DEVEX-222] Added metadata extensions to allow injecting type informations * [DEVEX-222] Made CLR type be resolved based on the message type metadata header * [DEVEX-222] Added the MessageTypeResolutionStrategyWrapper for caching resolved types * [DEVEX-222] Added MessageSerializationContext to pass additional information like category name * [DEVEX-222] Refactored SerializationContext into MessageSerializerWrapper to make clearer responsibilities Actually, it's just wrapping message serializer based on the client settings. * [DEVEX-222] Added message type maps registration * [DEVEX-222] Added helpers for custom type resolutions strategy Removed also obsolete SerializationSettings * [DEVEX-222] Replaced MessageSerializerWrapper usage with more generic IMessageSerializer There's no need to force all code to know about wrapper existence, it's okay to just create it during the setup. * [DEVEX-222] Made Message to be record instead of Struct * [DEVEX-222] Refactored new AppendToStreamAsync method to have simplified syntax Now they don't require all the fancy, but redundant in most cases Stream Positions, Directions, EventData etc. * [DEVEX-222] Refactored Reading events signatures to use options instead of parameters Thanks to that safe defaults are already provided * [DEVEX-222] Renamed MessageTypeResolutionStrategy to MessageTypeNamingStrategy * [DEVEX-222] Made old Read methods be wrapper of the new one, instead the other way round * [DEVEX-222] Added serialization configuration overrides Old append and read methods won't be doing default serialization by default. The new one will use auto-seriailization and allow to customize or disable it. * [DEVEX-222] Removed storing CLR type in metadata and moved it to message type name Now the FullTypeName is placed in the event type name. Not perfect but here it is. * [DEVEX-222] Added possibility to register types for category name Thanks to that they will be resolved automatically by the registered convention. Then user doesn't need to provide the string name by hand. Added MessageTypeNamingResolution context besides the MessageSerializationContext as it's orthogonal and just overlapping context. * [DEVEX-222] Added SubscribeToAllOptions and Subscription Listener This is a similar API as we already have in Java client, to simplify the handlers registration. * [DEVEX-222] Addes SubscribeToStreamOptions and applied it accordingly to subscirbe to all changes * [DEVEX-222] Renamed DeserializedEvent to Message in ResolvedEvent I think that in the longer term, this will make intention clearer * [DEVEX-222] Added persistent subscription litener and capability to customize serialization for Persistent Subscriptions * [DEVEX-222] Extended Serialization tests for Appends and Reads * [DEVEX-222] Added capability to automatically deserialize message metadata By default it'll try to resolve the TraceMetadata, it can be override with the general type or using custom resolution strategy. * [DEVEX-222] For unknown reason the reverse Enumerable.Reverse wasn't taken but Array.Reverse 🤷 * [DEVEX-222] Fixed type resolution to correclty find types by assembly * [DEVEX-222] Added tests for serialization compatibility between manual and automatic deserialization * [DEVEX-222] Added tests for appends with autoserialization * Revert "[DEVEX-222] Added tests for appends with autoserialization" This reverts commit 04e068554a0a8510af7da0b1e5b035edf513e679. I'm reverting it as old methods are wrappers for new ones, so the test scenarios are covered by the old test suite. * [DEVEX-222] Extended serialization test suite * [DEVEX-222] Added tests for auto-serialization in Catch-up subscriptions * [DEVEX-222] Added tests for auto-serialization in persistent subscriptions * [DEVEX-222] Added Expected prefix to StreamState and StreamRevision in AppendToStreamOptions * [DEVEX-222] Merged namespaces Fixed also XML docs comments * [DEVEX-222] Renamed NulloMessageSerializer to NullMesageSerializer * [DEVEX-222] Added AppendToStream methods that has expected stream revision as a parameter * [DEVEX-222] Added XML documentation for Serialization settings * [DEVEX-222] Added Unit tests * [DEVEX-222] Added tests for type resolution Used also two external assemlies - one that's loaded, - one that's never loaded. To double-check behaviour of loading types from different assemblies. * [DEVEX-222] Added SchemaRegistry tests --------- Co-authored-by: Oskar Dudycz <oskar.dudycz@eventstore.com> * Rename namespaces (#335) * Rename namespaces * feat: add ability to manually dispatch publish workflow * chore: set tag prefix to `kurrent@` * chore: set minimum-major-minor to 1.0 * chore: change name to `Kurrent Inc` * disable minver * refactor: move ExpectedRevision to StreamState. (#337) * [DEVEX-185] Removed leftover folders that had KurrentDb instead of KurrentDB (#340) It seems that after conflicts when merging StreamRevision PR, the code before merging stayed as duplicated without capital B in KurrentDb. I checked by doing copying with override from those folders to the proper one, and there were no differences besides those stream revision changes, so it's safe to just remove them. * chore: update license and readme * [DEVEX-222] - Auto serialization feedback improvements (#338) * [DEVEX-222] Fixed the missing application of custom serialization setting in AppendToStreamAsync method * [DEVEX-222] Added stream state as a mandatory parameter in append to stream Added also override for raw event append that has options and made other methods obsolete. Introduced OperationOptions that compose default client operation settings plus credentials and deadlines. * [DEVEX-222] Ensured that metadata is serialized with auto-serialization as regular system metadata * [DEVEX-222] Adjusted samples to match the append api changes Added also factor for event data that's not requiring to provide id immediately * [DEVEX-222] Added MessageData type as an easier to use replacement for EventData, and better aligned with general usage of messages instead of just events * [DEVEX-222] Refactored setting stream metadata to use OperationOptions * [DEVEX-222] Added DeserializedData and DeserializedMessages extensions for read results That should be a middle ground without forcing people to use a regular ResolvedEvent if they don't want to * [DEVEX-222] Simplified the message serializer implementation Now serializer creates a derived serializer merging operation settings with default settings if needed. That allowed to hide internal details and there's no need to pass content type explicitly, as it's already in settings. Extended SchemaRegistry to provide wrapper methods to naming resolution context. Thanks to that MessageSerializer doesn't need to reference it internally. To make responsibilities clearer made also naming resolution context part of serializer context. * [DEVEX-222] Marked old Stream Metadata options as obsolete * [DEVEX-222] Made old Read methods obsolete and used the new ones in tests * [DEVEX-222] Made Read options nullable, to precisely know if user provided options or not * [DEVEX-222] Made the Read Options nullable and thanks to that smarter, so e.g. FromEnd assigns also backwards if it wasn't set * [DEVEX-222] Made old SetStreamMetadata and ConditionalAppend methods obsolete * [DEVEX-222] Added delete method with options * [DEVEX-222] Made old subscription methods obsolete and adjusted tests to use them * [DEVEX-222] Added Tombstone method with options and made the old obsolete * [DEVEX-222] Made all persistent subscription methods obsolete * [DEVEX-222] Added samples showcasing auto-serialization usage * [DEVEX-222] Reshaped type resolution strategy to take EventRecord instead of just raw string. That should enable current users to override their strategy if they used something more than just message type name from EventRecord. It needs to be EventRecord and not ResolvedEvent, as resolution happens before creating ResolvedEvent. * [DEVEX-222] Changed message type registration to allow multiple message type names mapping to the same CLR class * [DEVEX-222] Nested Message type mapping into settings object * [DEVEX-222] Added nested message type mapping * [DEVEX-222] Replaced clr type with clr type name in naming resolution Tha makes naming strategy focused on just mapping types back and forth and allowing to disable auto registration * chore: point workflows to kurrent-io docker images --------- Co-authored-by: William Chong <william-chong@outlook.com> Co-authored-by: Oskar Dudycz <oskar.dudycz@eventstore.com> Co-authored-by: Yo Eight <yo.eight@gmail.com> Co-authored-by: Oskar Dudycz <oskar.dudycz@gmail.com> Co-authored-by: Jason Mitchell <jasonmitchell@live.co.uk>

The documentation is generated from this commit.

Commit:288e0ac
Author:Jason Mitchell
Committer:Jason Mitchell

wip

Commit:89069e4
Author:Jason Mitchell
Committer:Jason Mitchell

wip

Commit:1bb43c1
Author:Oskar Dudycz
Committer:GitHub

[DEVEX-185] Removed leftover folders that had KurrentDb instead of KurrentDB (#340) It seems that after conflicts when merging StreamRevision PR, the code before merging stayed as duplicated without capital B in KurrentDb. I checked by doing copying with override from those folders to the proper one, and there were no differences besides those stream revision changes, so it's safe to just remove them.

The documentation is generated from this commit.

Commit:42e4d02
Author:William Chong
Committer:GitHub

Rename namespaces (#335) * Rename namespaces

Commit:d613ba3
Author:William Chong

Publish actions

Commit:ebc6e60
Author:William Chong
Committer:William Chong

Rename test symbols

Commit:2cdf1f0
Author:Sérgio Silveira

moved files into main project and deleted remaining empty projects dropped support for net6.0 and net7.0 added some helpers to fluentdocker playing around the idea of keeping the container on tests

Commit:7029ca7
Author:Sérgio Silveira

moved all files into EventStore.Client project

Commit:ed1fb24
Author:Joseph Cummings
Committer:GitHub

DEV-297 - Add tracing instrumentation of Append & Subscribe operations (#294)

Commit:5bd189a
Author:João Bragança
Committer:GitHub

Subscriptions Without Callbacks (DEV-112) (#282) * add sensible subscriptions * add sensible persistent subscriptions * obsolete old subscription / ps apis * put missing props on sever exception in net48

Commit:14ff67b
Author:William Chong

Add missing options in create projection

Commit:ea785a5
Author:Rahul Patel
Committer:Rahul Patel

live caught up control message for subscription

Commit:4e82f49
Author:Pascal van Buijtene
Committer:Pascal van Buijtene

Update persistent subscriptions to minimum viable tier

Commit:41ef71f
Author:thefringeninja
Committer:Timothy Coleman

implement server capabilities - ensure calls are disposed - rediscovery on server not available - client reusable after discovery or server caps failure Co-authored-by: Timothy Coleman <timothy.coleman@gmail.com>

Commit:9671bb4
Author:thefringeninja
Committer:thefringeninja

pipe server messages through channel first

Commit:81949fc
Author:thefringeninja
Committer:Hayley Campbell

implement bidi append

Commit:6e0a210
Author:thefringeninja
Committer:thefringeninja

implement bidi append

Commit:a5a483d
Author:Hayley Campbell
Committer:GitHub

Add filtered persistent subscriptions (#122)

Commit:28742a0
Author:Shaan Nobee
Committer:GitHub

Implement persistent subscriptions to $all: (#108) Update persistent subscriptions proto file Make StreamPosition and Position implement new interface: IPosition Convert `StreamPosition StartFrom` to `IPosition? StartFrom` in PersistentSubscriptionSettings Move SystemStreams.cs from src/EventStore.Client.Streams to src/EventStore.Client and add the $all stream Move SystemMetadata class to a separate file in the EventStore.Client.Streams project Added: Create persistent subscriptions to $all Added: Update persistent subscriptions to $all Added: Delete persistent subscriptions to $all Added: Read from persistent subscriptions to $all Delete no longer valid test: create_on_all_stream.the_completion_fails_with_invalid_stream Update existing tests to use SystemStreams.AllStream/SystemRoles.All instead of "$all" Move all persistent subscription to stream tests to SubscriptionToStream directory Move stream persistent subscription tests to EventStore.Client.SubscriptionToStream namespace Added tests: create persistent subscriptions to $all Added tests: update persistent subscriptions to $all Added tests: delete persistent subscriptions to $all Added tests: connect to persistent subscriptions to $all Added tests: ACK/NAK for persistent subscriptions to $all Add the following wrappers to the persistent subscription client: CreateToAllAsync, DeleteToAllAsync, UpdateToAllAsync, SubscribeToAllAsync Update tests to use wrappers: CreateToAllAsync, DeleteToAllAsync, UpdateToAllAsync, SubscribeToAllAsync

Commit:8e6e098
Author:thefringeninja
Committer:thefringeninja

update proto

Commit:7c206b2
Author:thefringeninja

added missing field in proto

Commit:8c25f25
Author:thefringeninja
Committer:thefringeninja

ensuring backwards compatibility

Commit:6403ee9
Author:Pieter Germishuys
Committer:Pieter Germishuys

Add restarting persistent subscriptions

Commit:fb4f661
Author:James Geall

Changed: Restructured stream name for future planned changes

Commit:f99fc41
Author:shaan1337
Committer:shaan1337

Do gossip requests over gRPC Update VNodeState enum to match server and add unused values to notAllowedStates

Commit:6bce49c
Author:pgermishuys
Committer:pgermishuys

Provide a read status which will allow the client to avoid exceptions

Commit:6ad42fe
Author:pgermishuys
Committer:pgermishuys

wip

Commit:595520e
Author:pgermishuys
Committer:pgermishuys

Introduce the ability for the consumer to decide to throw on failure This behaviour is currently only applied when performing a write.

Commit:8053016
Author:hayley-jean
Committer:hayley-jean

Add operations proto contract and implementation Add the following to client.Operations: - MergeIndexes - Shutdown - ResignNode - SetNodePriority Add RestartSubsystem to client.ProjectionManagement

Commit:2ddca7b
Author:thefringeninja
Committer:thefringeninja

reorganize solution - spin up a container per test fixture as we can no longer run the server in process - turn on nulability - split each proto into its own project - move tests to different folder

Commit:72efaba
Author:thefringeninja
Committer:thefringeninja

initial commit

Commit:f775ce5
Author:shaan1337
Committer:shaan1337

gossip over gRPC: - Add flags internal_tcp_uses_tls and external_tcp_uses_tls to cluster.proto - Set the internal/external secure/non-secure endpoints depending on the value of the flags

Commit:a24de1f
Author:Hayley Campbell
Committer:Hayley Campbell

Split out gossip and elections services in proto.

Commit:bb48f5c
Author:Hayley Campbell
Committer:Hayley Campbell

use snake case in cluster proto.

Commit:34b29bd
Author:Pieter Germishuys
Committer:Hayley Campbell

Internal gossip and elections over gRPC - Add proto definition for internal gossip and elections - Add a Cluster gRPC service and client - Add a cluster client cache to allow channel reuse - Remove ElectController and Post for the GossipController - Don't run GossipController over internal http

Commit:d556247
Author:João Bragança
Committer:GitHub

Consolidate Protos (#2291) * use shared proto * use better terminology

Commit:65e15de
Author:thefringeninja
Committer:thefringeninja

windoze

Commit:ad5213d
Author:thefringeninja
Committer:thefringeninja

use internal subscriptions model in grpc stream subscriptions

Commit:da668a5
Author:thefringeninja
Committer:thefringeninja

use int64 for time consistently

Commit:f0c6adb
Author:thefringeninja
Committer:thefringeninja

Align names of projects, packages and classes

Commit:13bb04e
Author:João Bragança
Committer:James Nugent

Respect `maxCount` when reading from `$all`

Commit:b266442
Author:thefringeninja
Committer:thefringeninja

implementation of scavenge database in grpc

Commit:290a805
Author:thefringeninja
Committer:thefringeninja

Allow Clients to Specify Which UUID Type They Want

Commit:51bb32d
Author:thefringeninja

make all plural -> singlar in proto oneof

Commit:6e1778a
Author:João Bragança
Committer:Pieter Germishuys

Protobuf Contract Improvements (#2102) * Making End explicit on Reads * Zero Allocation UUID implementation

Commit:08e007f
Author:João Bragança
Committer:Pieter Germishuys

GRPC Client and Server (#2012) - move solution to dotnet core 3.1 - async await most tests - kestrel http service - GRPC Implementation of Streams API - GRPC implementation of persistent subscriptions - GRPC implementation of projections management api - GRPC implementation of user management - Acceptance Tests for Client API