Proto commits in littlehorse-enterprises/littlehorse

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

Commit:f00a031
Author:Eduwer Camacaro

Merge branch 'master' into feat-bulk-job-proposal

Commit:e5f4632
Author:Jacob Snarr
Committer:GitHub

feat(server): add LittleHorse Native Maps (#2368)

Commit:c26cc63
Author:Eduwer Camacaro

Merge branch 'master' into feat-bulk-job-proposal

Commit:d8a0cfe
Author:Snarr

Merge branch 'master' into feat/add-maps-to-lh

Commit:2a4833b
Author:JakeRose03

Merge remote-tracking branch 'origin/master' into feat/workflow-migrations

Commit:336b621
Author:Eduwer Camacaro

search and get bulk jobs

Commit:f9f0e61
Author:JakeRose03

Merge remote-tracking branch 'origin/master' into feat/thread-run-queue

Commit:2bca440
Author:JakeRose03

feat(server): thread run queue

Commit:560a925
Author:Jacob Snarr
Committer:GitHub

feat(proto): add `InlineStructDef` type to `TypeDefinition` (#2257)

Commit:b1c5591
Author:Eduwer Camacaro

track of the last seen timestamp

Commit:e2c2123
Author:JakeRose03

fix ambigous field "dependencies" and remove unnecassary comments

Commit:4f4af68
Author:Snarr

Merge branch 'master' into feat/proto/add-inlinestructdef-to-typedef

Commit:23ceb38
Author:Eduwer Camacaro

Refactor proposal

Commit:b56f4de
Author:Snarr

feat(proto): add maps

Commit:0c650f9
Author:JakeRose03

Workflow Migrations: rpc applyWorkflowMIgrationPlan, putWorkflowMigrationPlan, getWorkflowMIgrationPlan

Commit:047ef50
Author:Eduwer Camacaro

Track progress of a bulk job

Commit:e62d0f7
Author:Eduwer Camacaro

refactors

Commit:2f235e2
Author:JakeRose03

Switch to node_migrations

Commit:b06c195
Author:Eduwer Camacaro

Create models for bulk job wfrun delete

Commit:56b7d94
Author:Eduwer Camacaro

Initial implementation

Commit:b421247
Author:JakeRose03

Merge remote-tracking branch 'origin/master' into feat/workflow-migrations

Commit:ceeb9e5
Author:JakeRose03

rpc putWorkflowMigrationPlan, applyWorkflowMigrationPlan, GetWorkflowMIgrationPlan

Commit:91d0579
Author:Eduwer Camacaro
Committer:GitHub

feat(server): Count all the NodeRun in a tenant (#2304) Extends the counted-tag proposal to include the ability to count the number of NodeRun on a single tenant.

Commit:2125cd7
Author:JakeRose03

Register workflowmigration Plan with lh server and some implementation for applyworkflowmigration

Commit:acd95bb
Author:Eduwer Camacaro
Committer:GitHub

feat(server): Add counted tags to the existing Tag architecture (#2265) Implement proposal 021 for counted tags.

Commit:6a43278
Author:Bryson G

feat(dashboard): overhaul WfSpec detail page layout and metadata Redesign the WfSpec header, thread selector, and thread details sections for clearer navigation and add CountNodeRun storage usage support.

Commit:75bff55
Author:Mijaíl Rondón
Committer:Eduwer Camacaro

ci(snapshots): publish snapshots for minor versions (#2278) Adds a new `publish-snapshots` workflow that publishes both Java and Docker SNAPSHOT artifacts on pushes to `master` and release branches (e.g., `1.1`). The snapshot version is computed from `gradle.properties` by extracting only the `major.minor` portion: - `master` (`version=1.2.0-SNAPSHOT`) → publishes `1.2-SNAPSHOT` - `1.1` branch (`version=1.1.1-SNAPSHOT`) → publishes `1.1-SNAPSHOT` Co-authored-by: Eduwer Camacaro <eduwerc@gmail.com>

Commit:2995d53
Author:Mijaíl Rondón
Committer:GitHub

ci(snapshots): publish snapshots for minor versions (#2278) Adds a new `publish-snapshots` workflow that publishes both Java and Docker SNAPSHOT artifacts on pushes to `master` and release branches (e.g., `1.1`). The snapshot version is computed from `gradle.properties` by extracting only the `major.minor` portion: - `master` (`version=1.2.0-SNAPSHOT`) → publishes `1.2-SNAPSHOT` - `1.1` branch (`version=1.1.1-SNAPSHOT`) → publishes `1.1-SNAPSHOT` Co-authored-by: Eduwer Camacaro <eduwerc@gmail.com>

Commit:2f5d1eb
Author:Snarr

feat(proto): add InlineStructDef to TypeDef defined_type

Commit:6c9a437
Author:Jacob Snarr
Committer:GitHub

feat(server): add POW operation to WfSpec DSL (#2243)

Commit:70ab88c
Author:Colt McNealy
Committer:GitHub

feat(server): implement `Quota` proposal (#2141) - Creates `rpc PutQuota`, `rpc DeleteQuota`, `rpc GetQuota`, and `rpc SearchQuota` - Adds intelligent server-side throttling with `RESOURCE_EXHAUSTED` - Adds client interceptors to gracefully wait and then retry upon `RESOURCE_EXHAUSTED` - Introduces `LHC_GRPC_RESOURCE_EXHAUSTED_RETRY` configuration

Commit:c6b83dc
Author:Jacob Snarr
Committer:GitHub

feat(server): add SizeOf unary operation for Strs, Arrays, JsonArrs (#2230)

Commit:69b3bf3
Author:Colt McNealy
Committer:GitHub

feat(server): allows building structs inside the wfspec dsl (#2224) - Adds `StructBuilder` option to `VariableAssignment` - Implements it in the Java SDK

Commit:afaac51
Author:Jacob Snarr
Committer:GitHub

feat(server): add `is_nullable` flag for StructFieldDefs (#2215)

Commit:c8bcfe7
Author:Christian Caicedo
Committer:GitHub

feat(server): GetMetricWindow and SearchWfMetricWindow RPCs (#2204) Metrics now support tags for easier searching. Added `GetMetricWindow` to fetch a metric window by ID and `SearchWfMetricWindow` to find workflow metric windows by name and optional time range.

Commit:2b49196
Author:Christian Caicedo
Committer:GitHub

feat(server): Tenant aggregated metrics (#2171) Adds tenant-level aggregated metrics alongside existing workflow/task metrics. Each punctuator now produces both scoped and global aggregations, enabling queries that return totals across all workflows or tasks.

Commit:cad74ce
Author:Jacob Snarr
Committer:GitHub

feat(server): Add native `ARRAY` type (#2179)

Commit:8c03c1a
Author:Christian Caicedo
Committer:GitHub

feat(server): Workflow metrics (#2071) Adds workflow and task metrics to the server. Metrics are collected per WfSpec and per TaskDef in 1-minute windows and exposed via a gRPC RPC to query metrics for a specific WfSpec or TaskDef.

Commit:46b0e7c
Author:Jacob Snarr
Committer:Snarr

docs(proto): add comments to all RPCs, Messages, Enums (#2140)

Commit:e53849c
Author:Jacob Snarr
Committer:GitHub

docs(proto): add comments to all RPCs, Messages, Enums (#2140)

Commit:ad15e95
Author:Jacob Snarr
Committer:GitHub

fix(server): fix and refactor Struct field masking (#2101)

Commit:b623b86
Author:Eduwer Camacaro
Committer:GitHub

feat(server): Edge condition refactor (#2080)

Commit:84da785
Author:Jacob Snarr
Committer:GitHub

feat(server): Archive Inactive ThreadRuns (#2044)

Commit:584aea5
Author:Eduwer Camacaro
Committer:GitHub

fix(server): Fix pagination for List rpc (#2027)

Commit:970f942
Author:Colt McNealy
Committer:GitHub

feat(server): allow waking up parent wfRun when child is rescued (#2074)

Commit:219efff
Author:Eduwer Camacaro

First iteration: -Add new proto structures - Remove dependency for WorkflowCondition

Commit:e8edc9d
Author:JakeRose038

migrations

Commit:6d2b291
Author:Eduwer Camacaro
Committer:GitHub

feat(server): Support using WfRunVariable as input for RunChildWfNode (#2045) This PR extends the RunChildWfNode to support using WfRunVariable as the WfSpec name when running a child workflow. This allows the child WfSpec to be determined dynamically at runtime instead of being fully specified at WfSpec compile time. Changes include: Support for resolving the child WfSpec from a VariableAssignment instead of only from a static RunChildWfNode .wfSpecName. New backward compatible field for RunChildWfNode Existing behavior using a static wfSpecName continues to work unchanged. Validation of child workflow inputs/outputs is not executed at WfSpec compile time when the spec is provided dynamically.

Commit:6b2a161
Author:Colt McNealy
Committer:Bryson G

fix(server): save timer iterator progress and isolate timers (#2042) This PR fixes two problems with the Timer Processor in the core topology. - We iterate from the beginning of the timer store in the first punctuation. When creating a new processor (eg. after a rebalance) that involves iterating over tombstones, and that has been observed to take more than 60 seconds, which increases our downtime window. - We were not isolating LHTimer's with the Storeable infrastructure but rather storing them as if they were the only thing in the core store. Thus when we put something inside the cluster-scoped store, we get problems as the timer iterator sees those things as timers. We'll also need to guard the Tenant names to prevent them from starting with a digit, as that could result stuff in the tenant-scoped stores being mistaken for a cluster-scoped timer. This PR also disables the timer topology by default.

Commit:3a559a8
Author:Colt McNealy
Committer:GitHub

fix(server): save timer iterator progress and isolate timers (#2042) This PR fixes two problems with the Timer Processor in the core topology. - We iterate from the beginning of the timer store in the first punctuation. When creating a new processor (eg. after a rebalance) that involves iterating over tombstones, and that has been observed to take more than 60 seconds, which increases our downtime window. - We were not isolating LHTimer's with the Storeable infrastructure but rather storing them as if they were the only thing in the core store. Thus when we put something inside the cluster-scoped store, we get problems as the timer iterator sees those things as timers. We'll also need to guard the Tenant names to prevent them from starting with a digit, as that could result stuff in the tenant-scoped stores being mistaken for a cluster-scoped timer. This PR also disables the timer topology by default.

Commit:542798d
Author:Snarr

Merge branch 'master' into fix/threadruns-archive

Commit:37a0e72
Author:Snarr

Merge branch 'feature/archive-threadruns' into fix/threadruns-archive

Commit:4ee1c3d
Author:Colt McNealy
Committer:GitHub

feat(server): allow different strategies in `WaitForThreadsNode` (#2040) Implements #2037 according to Proposal 13.

Commit:9d7171f
Author:Snarr

feat(proto): add inactivethreadrun proto

Commit:40701b6
Author:Snarr

remove unused proto / mock rpc getarchivedthreadrun

Commit:95ccc6c
Author:Snarr

Merge branch 'master' into fix/threadruns-archive

Commit:fd8c854
Author:Eduwer Camacaro

Merge branch 'master' into metrics

Commit:1835aec
Author:Eduwer Camacaro
Committer:GitHub

refactor(server): Remove timer topology (#1979) Introduces ServerTopologyV2, unifying core logic and timer processing into a single Kafka Streams topology behind the LHS_X_ENABLE_TIMER_STREAMS feature flag. This removes the separate timer topology, reducing operational complexity, resource usage, and inter-topic communication. Timers are stored and executed within the core pipeline using a shared state store with smart routing and repartitioning only when required.

Commit:93157e4
Author:bryson-g
Committer:GitHub

feat(sdk-java): add description to LHTaskMethod annotation (#1989)

Commit:6a0f418
Author:Christian Caicedo
Committer:GitHub

fix(server): Improve how node outputs are retrieved (#1975) This PR adds a new NODE_OUTPUT storeable that saves node outputs by node name, enabling direct and faster retrieval. Only outputs from nodes referenced in the thread spec are stored to optimize storage. Backward compatibility is preserved by falling back to the legacy backward lookup for existing workflows.

Commit:28d30f1
Author:bryson-g
Committer:GitHub

feat(server): add descriptions to TaskDefs (#1990)

Commit:dde53bd
Author:JakeRose03

add migration variables to proto, allow migration request to provide variableValues

Commit:3ee3dda
Author:JakeRose038

Init rough draft, base case migrates, still need variables and threading

Commit:3c79c6d
Author:Snarr

WIP add archivedthreadrunid model

Commit:e6277a0
Author:Snarr

feat(proto): add archivedthreadrun

Commit:c51d278
Author:Eduwer Camacaro

wip

Commit:c9a5ffa
Author:Eduwer Camacaro

Merge branch 'master' into metrics

Commit:bfd8f6e
Author:Colt McNealy
Committer:GitHub

perf(server): put getables for a single wfrun under same prefix (#1842) Implement index locality optimization for workflow runs - Grouping Getables and also and WfRunStoredInventory under a shared prefix - Introducing new key formats with backward compatibility - Old running workflows can be completed in the new format

Commit:e70a9ad
Author:Colt McNealy
Committer:Eduwer Camacaro

perf(server): allow safe deletion of massive WfRun's (#1924) - Allows deleting large WfRun's by breaking the deletion into multiple Commands to prevent blocking the StreamThread. - Introduces LHS_X_MAX_DELETES_PER_COMMAND which controls the size of chunks.

Commit:6af9e01
Author:Colt McNealy
Committer:GitHub

perf(server): allow safe deletion of massive WfRun's (#1924) - Allows deleting large WfRun's by breaking the deletion into multiple Commands to prevent blocking the StreamThread. - Introduces LHS_X_MAX_DELETES_PER_COMMAND which controls the size of chunks.

Commit:1598bc5
Author:Christian CAicedo

Merge branch 'master' into perf/index-locality

Commit:1dda917
Author:Jacob Snarr
Committer:Christian CAicedo

fix(server): default to JSON_OBJ when typeDef defined case is not set (#1905) This fixes an incompatibility issue with old clients (< 0.14.1) that expect an undefined `TypeDefinition.primitive_type` to serialize to `JSON_OBJ`.

The documentation is generated from this commit.

Commit:8e28ad8
Author:Colt McNealy
Committer:Christian CAicedo

fix(server): dodge tombstones on rehydration (#1902) See this documentation in RocksDB: https://github.com/facebook/rocksdb/wiki/Implement-Queue-Service-Using-RocksDB When doing a range scan in rocksdb, if there were keys recently deleted from that range, then you end up iterating over tombstones for all of those keys. In the case of a queue-like service, you may have many (potentially 100k to 1M+) tombstones. Assuming it takes 2 hours for a tombstone to be fully compacted down to the bottom level (which is totally reasonable in the case of significant state > 20GB), if you have 50 tasks per second per partition then you can end up with 50 * 3600 * 2 == 360,000 tombstones in the range. This is because for every `TaskRun` we end up with a new `Schedule In our Kafka Streams Processor `init()` method, we do a full prefix scan over `ScheduledTaskModel` store. In case of high per-partition load, this can take a very long time, causing the producer to get fenced which is a big orzdash. This PR solves the problem with two changes: * Storing `ScheduledTask`s in chronological order and processing them in chronological order. * Remembering the timestamp of the last-processed ScheduledTask * In the `init()` method, starting the iteration from the timestamp of the last processed `ScheduledTask` so as to skip over the tombstones in the range.

Commit:f9a038c
Author:Jacob Snarr
Committer:GitHub

fix(server): default to JSON_OBJ when typeDef defined case is not set (#1905) This fixes an incompatibility issue with old clients (< 0.14.1) that expect an undefined `TypeDefinition.primitive_type` to serialize to `JSON_OBJ`.

The documentation is generated from this commit.

Commit:6e91d7a
Author:Colt McNealy
Committer:GitHub

fix(server): dodge tombstones on rehydration (#1902) See this documentation in RocksDB: https://github.com/facebook/rocksdb/wiki/Implement-Queue-Service-Using-RocksDB When doing a range scan in rocksdb, if there were keys recently deleted from that range, then you end up iterating over tombstones for all of those keys. In the case of a queue-like service, you may have many (potentially 100k to 1M+) tombstones. Assuming it takes 2 hours for a tombstone to be fully compacted down to the bottom level (which is totally reasonable in the case of significant state > 20GB), if you have 50 tasks per second per partition then you can end up with 50 * 3600 * 2 == 360,000 tombstones in the range. This is because for every `TaskRun` we end up with a new `Schedule In our Kafka Streams Processor `init()` method, we do a full prefix scan over `ScheduledTaskModel` store. In case of high per-partition load, this can take a very long time, causing the producer to get fenced which is a big orzdash. This PR solves the problem with two changes: * Storing `ScheduledTask`s in chronological order and processing them in chronological order. * Remembering the timestamp of the last-processed ScheduledTask * In the `init()` method, starting the iteration from the timestamp of the last processed `ScheduledTask` so as to skip over the tombstones in the range.

Commit:e858f77
Author:Colt McNealy
Committer:Colt McNealy

fix(server): dodge tombstones on rehydration See PR description for longer explanation.

Commit:28e8036
Author:Jacob Snarr
Committer:GitHub

feat(server): Add `LHPath` for accessing Struct fields at runtime (#1882)

Commit:a5c1674
Author:Colt McNealy
Committer:GitHub

feat(server): introduce durable execution through `Checkpoint` in `TaskRun` (#1881) Implements #1765

Commit:a0a5c32
Author:Colt McNealy

Merge branch 'master' into feat/checkpointed-tasks

Commit:d4af363
Author:Colt McNealy
Committer:GitHub

feat(server): run child wf from inside `WfSpec` (#1808) Implements #1886

Commit:2a80dc7
Author:Snarr

docs(server): add comments deprecating jsonpath

Commit:df6d864
Author:Snarr

feat(server): Add LHPath indexing and type safety

Commit:45e800f
Author:Snarr

feat(server): Add `LHPath` to `VariableMutation`s

Commit:959c67d
Author:Snarr

feat(server): Add LHPath for VariableAssignments

Commit:b90ed87
Author:Snarr

feat(proto): add LHPath

Commit:1002a9b
Author:Colt McNealy

fix(server): forgot to add total checkpoints to scheduledTask

Commit:4ecf0d4
Author:Colt McNealy
Committer:Colt McNealy

fix(server): forgot to add PutCheckpointRequest to Command proto.

Commit:606a1d5
Author:Colt McNealy

feat(server): adds Checkpoint as a Getable

Commit:ae89b6a
Author:Colt McNealy

feat(server): adds checkpointed task proposal and protobuf

Commit:0a538bf
Author:Colt McNealy

Merge branch 'master' into child-workflows

Commit:de0c6ce
Author:Jacob Snarr
Committer:GitHub

fix(server): Remove partial InlineArrayDef support (#1878)

Commit:c856420
Author:Christian Caicedo
Committer:GitHub

feat(server): Add supporf for casting on primitive types (#1806) Introduces full support for automatic and manual type casting for Primitive Types in LittleHorse workflows , including server-side casting for variable assignments and task inputs, explicit casting methods in the Java SDK (.castToInt(), .castToDouble(), etc.).

Commit:149fe04
Author:Colt McNealy

Merge branch 'master' into child-workflows

Commit:f34cc35
Author:Colt McNealy
Committer:Colt McNealy

some progress

Commit:4643739
Author:Christian CAicedo

Merge branch 'master' into casting-implementation

Commit:7cf0475
Author:Jacob Snarr
Committer:GitHub

feat(server): add structs to `server` and `sdk-java` (#1796)

Commit:e0ee110
Author:Christian CAicedo

Merge branch 'master' into casting-implementation

Commit:e2b0f49
Author:Snarr

Merge branch 'master' into feat/server/structs

Commit:136815c
Author:Colt McNealy
Committer:GitHub

fix(server): uuid in timer key (#1838) Previously the key for a timer in the timer store was {maturation-time}/{topic}/{partitionkey}/{tenant}. The partitionKey is always the WfRunId. Therefore if a single WfRun had two timers maturing at the same time, the second one would override the first (same key). This backwards-compatible PR changes the timer keys to be {maturation-time}/{uuid}. This will also have the benefit of shortening the keys which improves rocksdb performance.

Commit:d2ca460
Author:Jacob Snarr
Committer:GitHub

feat(sdk-go): Add StructDef get, search, delete (#1833)