Proto commits in kapetan-io/querator

These 51 commits are when the Protocol Buffers files have changed:

2026-06-03

Route dead-letter production through the dead-letter queue's requestLoop (#68) * Route dead-letter production through the dead-letter queue's requestLoop Dead-letter movement wrote directly to the dead-letter queue's storage partition from the source queue's requestLoop, bypassing the dead-letter queue's own loop. This violated the single-writer-per-partition invariant (ADR-0003), left the dead-letter partition's in-memory accounting stale, and never woke consumers blocked on the dead-letter queue. Replace Logical.ProduceInternal with ProduceDeadLetter, which enqueues a new MethodProduceInternal request on the dead-letter queue's requestCh and blocks for confirmation. This restores single-writer, advances GC scheduling, and wakes blocked consumers in one move. The hand-off is synchronous (delete-after-confirm) and non-blocking on a full channel (backpressure leaves the source item in place). Add MethodProduceInternal to the handleRequest, reqToState, isHotRequest, and shutdown-drain switch sites. Expose unLeased and nextLifecycleRun on QueuePartitionStats (proto + types), sourced from in-memory QueueState, so accounting drift is observable. Add surface tests covering blocked-consumer wakeup, stats accounting, and delete-after-confirm/backpressure; all run across every storage backend. Fixes ENG-55 * Add invariant-test report for dead-letter single-writer routing Documents the invariant ledger derived from ADR-0003 and the ENG-55 blueprint, and records the mutation proofs confirming the three committed SingleWriter surface tests are genuine invariant tests: - WakesBlockedConsumer / StatsReflectDeadLetteredItem go red under a single-writer bypass (direct off-loop storage write) while integrity stays green, proving the partition mutex masks the integrity bucket but cannot satisfy the liveness/accounting consequences. - RetainsItemOnFailedMove goes red when the delete-after-confirm guard is removed. Both mutations reverted; full suite green across all backends. * Simplify dead-letter stats and source-item cleanup Quality cleanups from a simplify pass over the ENG-55 change (no behavior change): - handleStats: capture Clock.Now() once per stats snapshot instead of calling it twice per partition (Stats() and NextLifecycleRun.Sub()). - LifeCycle: collect the source item IDs during the DLQ-prep loop rather than re-walking deadItems in a second loop after the produce. * Assert dead-letter items get a fresh ID distinct from SourceID Strengthen the DLQ SourceID test: after a dead-letter move the original ID is cleared so the dead-letter queue generates a new one, so the moved item's ID must be non-empty and distinct from its SourceID. * Reword SingleWriter test comments to be self-contained Drop references to the prior implementation ("the old direct storage write", "fails on main") and to external acceptance-criteria numbers. Comments now describe what each test asserts and why it holds, so a reader needs no access to the previous code or the blueprint.

The documentation is generated from this commit.

Route dead-letter production through the dead-letter queue's requestLoop Dead-letter movement wrote directly to the dead-letter queue's storage partition from the source queue's requestLoop, bypassing the dead-letter queue's own loop. This violated the single-writer-per-partition invariant (ADR-0003), left the dead-letter partition's in-memory accounting stale, and never woke consumers blocked on the dead-letter queue. Replace Logical.ProduceInternal with ProduceDeadLetter, which enqueues a new MethodProduceInternal request on the dead-letter queue's requestCh and blocks for confirmation. This restores single-writer, advances GC scheduling, and wakes blocked consumers in one move. The hand-off is synchronous (delete-after-confirm) and non-blocking on a full channel (backpressure leaves the source item in place). Add MethodProduceInternal to the handleRequest, reqToState, isHotRequest, and shutdown-drain switch sites. Expose unLeased and nextLifecycleRun on QueuePartitionStats (proto + types), sourced from in-memory QueueState, so accounting drift is observable. Add surface tests covering blocked-consumer wakeup, stats accounting, and delete-after-confirm/backpressure; all run across every storage backend. Fixes ENG-55

The documentation is generated from this commit.

2026-05-08

Implement Phase 1: Proto and Core Type Updates - Rename env_tag to key_tag (field 5) in proto/apikeys.proto - Add api_key_tag field (field 4) to NamespaceInfo in proto/namespaces.proto - Regenerate proto/apikeys.pb.go and proto/namespaces.pb.go - Add APIKeyTag string field to types.Namespace and wire into ToProto - Update service.go to read req.KeyTag instead of req.EnvTag - Update ADR-0025 to reflect qtr- prefix replacing sk- prefix

2026-01-05

Implement Phase 3: Roles + RBAC - Add Role and RoleBinding types with storage interfaces - Add Role and RoleBinding storage implementations (Memory, BadgerDB) - Add permission constants for all operations - Add AuthBackend interface with cascading permission checks - Add role HTTP endpoints (create, update, list, delete) - Add role binding HTTP endpoints (create, list, delete) - Add role and role binding client methods - Add role validation functions - Add role and role binding error types

Implement Phase 2: Users + API Keys - Add User storage interface and implementations (Memory, BadgerDB) - Add APIKey storage interface and implementations (Memory, BadgerDB) - Add API key generation utilities (GenerateAPIKey, HashAPIKey) - Add auth cache for caching authenticated principals - Add user HTTP endpoints (create, list, delete) - Add API key HTTP endpoints (create, list, delete) - Add user and API key client methods - Add user and API key error types - Add validation functions for user and API key creation - Close users and API keys stores on shutdown

Implement Phase 1: Namespace Foundation + Minimal Auth - Add Namespace type with Name, Description, CreatedAt fields - Add Namespaces storage interface and implementations (Memory, BadgerDB) - Add namespace HTTP endpoints (create, list, delete) - Add namespace client methods - Add ErrUnauthorized and ErrForbidden error types - Add Principal and User types for auth foundation - Add context utilities for principal extraction - Add namespace field to QueueInfo - Add namespace field to proto messages - Close namespace storage on shutdown to prevent goroutine leaks - Fix QueuesInfo test assertion for RequestedPartitions

2025-12-31

Remove unimplemented retry flag from QueueClear The retry flag was defined but never implemented. Retry items and scheduled items are functionally identical (both use EnqueueAt field), so the scheduled flag already handles clearing retry items. - Remove retry field from QueueClearRequest proto message - Remove Retry field from internal ClearRequest type - Update validation to check only queue and scheduled flags - Update comments to clarify scheduled includes retry items

2025-12-11

Address PR review feedback - Remove ineffectual assignment in handleReload - Rename MethodReloadPartitions to MethodReload - Add QueueReloadRequest and QueueReloadResponse proto messages - Update all usages to use new proto types instead of QueueClearRequest

2025-12-10

Implement Phase 2: Expose Stats.Scheduled in Proto - Add scheduled field to QueuePartitionStats proto message (field 7) - Regenerate proto files with make proto - Map stat.Scheduled to proto response in service.go - Add Stats/Scheduled test that verifies: - Scheduled count shows correct number of scheduled items - Scheduled count drops to 0 after items move to ready queue

2025-12-07

Implement Phase 1: SourceID Field and Storage Support - Add SourceID field to types.Item for DLQ idempotency tracking - Add source_id field to proto/storage.proto StorageItem message - Update Item.ToProto() and Item.FromProto() to handle SourceID - Update Item.Compare() to include SourceID comparison - Memory storage: add bySourceID index for duplicate detection - BadgerDB storage: add secondary index with source: prefix - PostgreSQL storage: add source_id column with partial unique index - All storage backends skip items with duplicate SourceID (idempotent) - Add SourceID functional tests for all storage backends

2025-06-09

Implement /v1/queues.info endpoint Add support for retrieving detailed information about individual queues. ### Implementation - Add QueuesInfoRequest protobuf message with queue_name field - Implement QueuesInfo service method in service layer - Add HTTP handler and routing for /v1/queues.info endpoint - Add QueuesInfo client method for API calls - Include comprehensive tests for success and error scenarios - Support both InMemory and BadgerDB storage backends

Fix OpenAPI specification discrepancies with Go struct definitions - Fix QueueInfo field name in OpenAPI example from 'name' to 'queue_name' - Add missing response structs to protobuf: QueueProduceResponse, QueueCompleteResponse, QueueRetryResponse - Add dead_deadline field to StorageItem protobuf definition - Regenerate protobuf files to reflect schema changes - Ensure API documentation accurately matches implementation for better client compatibility

2025-06-05

Update OpenAPI specification to match current implementation

2025-06-04

Implement scheduled items - Add scheduled item storage and lifecycle management - Update protobuf definitions for scheduled items - Add configuration for MaxConcurrentRequests - Refactor storage setup and add validation - Update API to support scheduled operations - Rename defer to reply and reservation to lease for clarity

2025-06-03

Implement scheduled items feature - Add scheduled item storage and lifecycle management - Update protobuf definitions for scheduled items - Add configuration for MaxConcurrentRequests - Refactor storage setup and add validation - Update API to support scheduled operations - Rename defer to reply and reservation to lease for clarity

2025-04-21

Renamed defer to reply and reservation to lease

2025-04-15

Added scheduled ADR

2025-04-04

Renamed defer to retry

2025-04-01

ReserveTimeout lifecycle working as expected

WIP: LifeCycle partition writes moved to Logical

Implement Reserve Expired Life Cycles

2024-10-11

WIP: Refactor

2024-10-02

Removed ReserveBlocked

2024-09-20

WIP: Logical Queue Refactor

2024-09-18

WIP: Refactor QueueManager, Queue and LogicalQueue

2024-09-16

Rebase after OpenAPI updates

Partition Testing

Produce, Reserve, Complete now support multiple partitions

Updated OpenAPI and Proto definitions

Updated OpenAPI documentation

2024-09-03

WIP: Partitions refactor

2024-08-19

Removed /queue.pause from the public API

2024-08-01

Shutdown during pause testing

2024-07-21

WIP: Queues Storage

2024-07-18

Code clean up and refactor

Renamed QueueOptions to QueueInfo

2024-07-17

Added /queue.clear to the api

WIP Race detected

2024-07-13

WIP: Refactor of storage tests continues

2024-07-12

wIP: Queue Storage API

2024-07-01

Changed the name of body to bytes and now reference it as payload

2024-06-29

WIP: /queue.complete

2024-06-28

Produce might be working now

2024-06-27

Refactor code layout to match map

2024-06-26

WIP: Produce is coming along

2024-06-22

Message is dead, long live the queue item

WIP: Testing producing and consuming

2024-06-19

WIP: More QueueStorage tests

2024-06-17

WIP: Laying out the produce path

2024-06-14

Daemon and Client working

2024-06-13

Initial commit