Proto commits in FgForrest/evitaDB

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

2026-09-12

feat: restore a catalog to one of its earlier versions in one operation Adds `EvitaManagementContract#restoreCatalogToVersion`, which backs a live catalog up at a past version, unpacks the archive into a temporary catalog, loads it, and swaps it into service - as one task a client can watch. This is what evitaLab's `Restore to this version` button needs: it previously required a manual backup, a manual download, a manual restore under a different name and a manual replace, with no way to match the produced file back to the request. The WAL is deliberately excluded from the archive. Including it copies every log file wholesale and a restore replays them forward to the head of the log, landing back on the state being escaped - so the restored catalog carries no mutation history, and the javadoc says so rather than leaving it to be found. Composition notes worth keeping: - The unpacking step cannot be built when the sequence is assembled, because the archive's id, location and size are chosen by the export service while the backup runs. `PublishRestoredCatalogTask` is the step that builds it at the moment its inputs exist. - The archive is pulled back through `ExportService#fetchFile` rather than read from a computed path: the S3-backed export service has no local path at all. - `replaceCatalog` tolerates a target name no catalog holds, so one uniform flow covers replace-in-place, replace-another-catalog and create-new. It also skips the naming-convention check when overwriting, so a brand-new target name is validated before any work starts. Also fixes `SequentialTask#getStatus`, which aggregated step progress with `|=` where it meant `+=` - two steps at 100% and 50% reported 59% instead of 75%. The error stayed invisible while the only shape was two steps averaging `100|0`, where OR and addition happen to agree. Ref: #1553

The documentation is generated from this commit.

2026-09-10

docs: carry the cadence and fence depth correction into the proto The misleading cadence sentence was also in `GrpcDurabilityStatistics`, which is the copy every generated client reads - including the one evitaLab's label text was written from. Corrected there too, and the stubs regenerated rather than hand-edited. Comment-only: the descriptor bytes are unchanged, so nothing on the wire moves. Ref: #1339

2026-09-07

chore: merge origin/dev (equalized histogram bucketing) into the branch Nine commits from #1505. One conflict, in the generated `GrpcEnums.java`: `dev` added `EQUALIZED_OPTIMIZED` to `GrpcHistogramBehavior` while this branch added `GrpcHierarchyParentsBehaviour`, and the two land in the same serialized file descriptor. `GrpcEnums.proto` - the source of truth - merged cleanly and carries both, so the conflict was resolved by regenerating the Java from it rather than by hand-merging a descriptor string. Verified afterwards that the descriptor names both enums and that regenerating from every proto produced no diff outside the three files the merge already touched. Ref: #1365

refactor: deprecate EQUALIZED_OPTIMIZED in favour of EQUALIZED The keep/drop-empty axis that separated the pair has collapsed. The equalised algorithm places every boundary on a value the data actually contains, so it never emits an empty bucket and the "optimized" variant has nothing to drop. The two constants are identical, and `EQUALIZED` no longer honours the guarantee its own JavaDoc used to make - "always contains the number of buckets you asked for" - because the old implementation met that guarantee by fabricating: empty buckets at thresholds no entity holds, and, when the gaps could not take them, buckets appended past the data maximum with `getMaxValue()` moved along to cover them. Restoring the distinction honestly was considered and declined. The ceiling for any algorithm placing boundaries on values the data contains is `min(bucketCount, distinctValues)`, so a variant that spent surplus distinct values as extra cuts would still return fewer buckets on low-cardinality data and the contract would still have to read "at most". A guarantee that holds only usually is worse than an honest ceiling. Reasoning and measurements are in the ADR. The constant stays in the grammar - removing it would break clients for no gain - and the marker is published everywhere the schema can express it: `[deprecated = true]` in `GrpcEnums.proto` and a `deprecationReason` on the GraphQL enum value. OpenAPI cannot express it, since `deprecated` there applies to a whole schema and never to a single enum item, so `OpenApiEnum` is left alone and the REST surface says nothing. `since = "2026.2"` matches the reactor version. Generated gRPC sources are byte-exact protoc 3.25.8 output with the tracked licence header spliced back on. Ref: #1501

docs: align every published histogram contract description with the equalized behaviour The equalized `relativeFrequency` carries a smoothed value density normalised to the maximum of the density curve, not a share that sums to 100 - but four surfaces that *publish* that contract still described the superseded one, and none of them shares a symbol with the code that changed, so only a prose search finds them: - `HistogramDescriptor` - the GraphQL and OpenAPI field description, served verbatim to clients through schema introspection - `GrpcExtraResults.proto` and its generated Java - the gRPC field comment - `PriceHistogram`, `AttributeHistogram` and `ReferenceHistogramStatistics` - the constraint JavaDoc, all three still calling `EQUALIZED_OPTIMIZED` a distinct behaviour with empty-bucket suppression User documentation is restructured to lead with the practical case rather than the mathematics. Two problems are stated in order - a linear slider spends its track on the wrong products, and equalizing the track then empties the columns - each with measured figures from a production catalogue, and Baymard Institute's slider research cited for the first, whose own worked example (50% of the width controlling 2% of the products) closely matches ours. The client rendering rules and the statistical derivation both move into collapsed blocks beneath that, since neither is needed to use the feature. Also documents the zero-width last bucket - its threshold equals `max` whenever the largest value is numerous enough to be isolated, which is common - on the record component, in the user documentation and on the cruncher, so renderers floor the bar width instead of drawing the tallest bar as nothing. One malformed-HTML JavaDoc error is fixed. It is invisible to `mvn javadoc:javadoc` because the root POM disables doclint and the package is not exported in `module-info`, so it only surfaces under `javac -Xdoclint:all`. The ADR records the strict-comparison correction, why the original differential could not have caught it - it compared the implementation against the prototype it was transcribed from - and the contract surfaces a future change in this area has to move with the code. Ref: #1501

fix: equalized histograms bucket on the quantile function and report a kernel density `HistogramBehavior.EQUALIZED` shipped in 2026.2 with two independent defects, both visible on production data. Bucketing advanced one bucket per distinct value however many quantile targets that value had crossed, so a price shared by hundreds of products starved every bucket after it. Thresholds are now the empirical inverse CDF sampled at k/bucketCount, de-duplicated and compared in exact integer arithmetic; a value absorbing two or more ranks additionally emits a threshold at the next distinct value, closing its mass into a bucket of its own. The emitted count provably never exceeds the requested one, and returning fewer is now routine rather than exceptional. `relativeFrequency` was `occurrences / bucketWidth` sum-normalised - the right quantity (the density-quantile function) estimated from the single gap between two adjacent prices, so one reprice moved it by orders of magnitude. On a production category the bucket holding 81 products rendered at 35.15 while the one holding 338 rendered at 6.03. It is now read off one global triangular-kernel density estimate over the value axis, h = sqrt(6) * 0.9 * min(sigma_w, IQR_c / 1.34) * D^(-1/5), normalised against the curve's own maximum and evaluated at each bucket's weighted median observation. Two deliberate departures from Silverman, both because this smooths a catalogue known in full rather than inferring a latent population from a sample: the count term is the number of distinct values (replication invariance), and the robust spread caps a heavy value at min(w, (N - w) / 2) - written as a min because a hard majority switch is discontinuous and live data sits five records from that cliff. `relativeFrequency` carries the new value: no new field, no proto change, no client migration. Its meaning changes from sum-to-100 to max-to-100 for the equalized family, and the rendering contract is now documented on the record component, on the behaviour enum and in the user documentation. `padWithEmptyBuckets` and `BucketCountMode` are retired - the equalized algorithm places every boundary on a value the data contains and so never emits an empty bucket, which makes EQUALIZED_OPTIMIZED identical to EQUALIZED. Verified by differential comparison against the reference implementation over 4 000 random catalogues across six shapes and both production catalogues (every threshold, count and height identical), plus an invariant battery covering the bucket budget, target-crossing accounting, gap insensitivity, replication invariance, support fragmentation, quartile-boundary continuity and a dominance sweep. Ref: #1501

refactor: deprecate EQUALIZED_OPTIMIZED in favour of EQUALIZED The keep/drop-empty axis that separated the pair has collapsed. The equalised algorithm places every boundary on a value the data actually contains, so it never emits an empty bucket and the "optimized" variant has nothing to drop. The two constants are identical, and `EQUALIZED` no longer honours the guarantee its own JavaDoc used to make - "always contains the number of buckets you asked for" - because the old implementation met that guarantee by fabricating: empty buckets at thresholds no entity holds, and, when the gaps could not take them, buckets appended past the data maximum with `getMaxValue()` moved along to cover them. Restoring the distinction honestly was considered and declined. The ceiling for any algorithm placing boundaries on values the data contains is `min(bucketCount, distinctValues)`, so a variant that spent surplus distinct values as extra cuts would still return fewer buckets on low-cardinality data and the contract would still have to read "at most". A guarantee that holds only usually is worse than an honest ceiling. Reasoning and measurements are in the ADR. The constant stays in the grammar - removing it would break clients for no gain - and the marker is published everywhere the schema can express it: `[deprecated = true]` in `GrpcEnums.proto` and a `deprecationReason` on the GraphQL enum value. OpenAPI cannot express it, since `deprecated` there applies to a whole schema and never to a single enum item, so `OpenApiEnum` is left alone and the REST surface says nothing. `since = "2026.2"` matches the reactor version. Generated gRPC sources are byte-exact protoc 3.25.8 output with the tracked licence header spliced back on. Ref: #1501

docs: align every published histogram contract description with the equalized behaviour The equalized `relativeFrequency` carries a smoothed value density normalised to the maximum of the density curve, not a share that sums to 100 - but four surfaces that *publish* that contract still described the superseded one, and none of them shares a symbol with the code that changed, so only a prose search finds them: - `HistogramDescriptor` - the GraphQL and OpenAPI field description, served verbatim to clients through schema introspection - `GrpcExtraResults.proto` and its generated Java - the gRPC field comment - `PriceHistogram`, `AttributeHistogram` and `ReferenceHistogramStatistics` - the constraint JavaDoc, all three still calling `EQUALIZED_OPTIMIZED` a distinct behaviour with empty-bucket suppression User documentation is restructured to lead with the practical case rather than the mathematics. Two problems are stated in order - a linear slider spends its track on the wrong products, and equalizing the track then empties the columns - each with measured figures from a production catalogue, and Baymard Institute's slider research cited for the first, whose own worked example (50% of the width controlling 2% of the products) closely matches ours. The client rendering rules and the statistical derivation both move into collapsed blocks beneath that, since neither is needed to use the feature. Also documents the zero-width last bucket - its threshold equals `max` whenever the largest value is numerous enough to be isolated, which is common - on the record component, in the user documentation and on the cruncher, so renderers floor the bar width instead of drawing the tallest bar as nothing. One malformed-HTML JavaDoc error is fixed. It is invisible to `mvn javadoc:javadoc` because the root POM disables doclint and the package is not exported in `module-info`, so it only surfaces under `javac -Xdoclint:all`. The ADR records the strict-comparison correction, why the original differential could not have caught it - it compared the implementation against the prototype it was transcribed from - and the contract surfaces a future change in this area has to move with the code. Ref: #1501

fix: equalized histograms bucket on the quantile function and report a kernel density `HistogramBehavior.EQUALIZED` shipped in 2026.2 with two independent defects, both visible on production data. Bucketing advanced one bucket per distinct value however many quantile targets that value had crossed, so a price shared by hundreds of products starved every bucket after it. Thresholds are now the empirical inverse CDF sampled at k/bucketCount, de-duplicated and compared in exact integer arithmetic; a value absorbing two or more ranks additionally emits a threshold at the next distinct value, closing its mass into a bucket of its own. The emitted count provably never exceeds the requested one, and returning fewer is now routine rather than exceptional. `relativeFrequency` was `occurrences / bucketWidth` sum-normalised - the right quantity (the density-quantile function) estimated from the single gap between two adjacent prices, so one reprice moved it by orders of magnitude. On a production category the bucket holding 81 products rendered at 35.15 while the one holding 338 rendered at 6.03. It is now read off one global triangular-kernel density estimate over the value axis, h = sqrt(6) * 0.9 * min(sigma_w, IQR_c / 1.34) * D^(-1/5), normalised against the curve's own maximum and evaluated at each bucket's weighted median observation. Two deliberate departures from Silverman, both because this smooths a catalogue known in full rather than inferring a latent population from a sample: the count term is the number of distinct values (replication invariance), and the robust spread caps a heavy value at min(w, (N - w) / 2) - written as a min because a hard majority switch is discontinuous and live data sits five records from that cliff. `relativeFrequency` carries the new value: no new field, no proto change, no client migration. Its meaning changes from sum-to-100 to max-to-100 for the equalized family, and the rendering contract is now documented on the record component, on the behaviour enum and in the user documentation. `padWithEmptyBuckets` and `BucketCountMode` are retired - the equalized algorithm places every boundary on a value the data contains and so never emits an empty bucket, which makes EQUALIZED_OPTIMIZED identical to EQUALIZED. Verified by differential comparison against the reference implementation over 4 000 random catalogues across six shapes and both production catalogues (every threshold, count and height identical), plus an invariant battery covering the bucket budget, target-crossing accounting, gap insensitivity, replication invariance, support fragmentation, quartile-boundary continuity and a dominance sweep. Ref: #1501

chore: merge origin/dev (storage-part classification) into the branch Brings the nine commits of #1503 onto the branch so #1370 stays mergeable. Two files conflicted, both in the gRPC enum surface. `GrpcEnums.proto` — not a numbering collision. Both sides appended an enum to the end of the file, so each side's last enum was sharing the file's closing brace. The types are disjoint (`GrpcHierarchyParentsBehaviour` here, `GrpcStoragePartKind` / `GrpcStoragePartGroup` on dev), so no field number is contested. Kept all three and closed this branch's enum explicitly. `GrpcEnums.java` is generated but checked in, so it was regenerated from the resolved proto via `generate-sources` rather than hand-merged. That also refreshed dev's `GrpcStatistics` descriptors and emitted `GrpcStoragePartKind` / `GrpcStoragePartGroup`. Values verified disjoint: MATCHING=0 / COMPLETE=1 against STORAGE_PART_KIND_* and STORAGE_PART_GROUP_*. Verified on a clean build (not incremental): full reactor `BUILD SUCCESS`, and 641 targeted tests green including the gRPC converter suites this merge touched. Ref: #1365

fix: name the histogram group for the reference feature it actually measures Adversarial review caught the new taxonomy reproducing, one level down, the bug it was written to fix. A group called `HISTOGRAM_INDEX` documented as "everything built to answer histogram requests" promises the `attributeHistogram` and `priceHistogram` extra results - which are computed on the fly from the filter and price indexes and have NO persisted storage part at all, so they can never appear in a composition breakdown. The four parts under the group back the runtime `HistogramIndex`, implemented only by `ReducedGroupEntityIndex` and `ReferencedTypeEntityIndex`, and the feature is declared solely by `ReferenceSchemaContract.getHistogramIndexDefinitions`. An operator reading the old label would have concluded they were paying for the histograms everybody knows. Renamed to `REFERENCE_HISTOGRAM_INDEX`, with javadoc and proto comment that say what it is and what it is not. Not folded into `REFERENCE_INDEX`: it is a separately declared, separately droppable schema capability, and hiding its cost is what the fine groups exist to prevent. The wire constant moves with it, which is free - no release has shipped this message. Two further review findings: `INDEX_MANIFEST` claimed its whole group is rarely rewritten. That holds for the manifest, whose smallness is the documented reason attribute indexes were split out of it, but not for the membership bitmaps, which were evicted precisely BECAUSE they churn on every insert and delete. What puts both in one group is that they describe the index rather than the values under it. `shouldFoldEveryGroupToItsKind` asserted only that `kind()` returns something, which no compiled enum constant can fail - a test of the Java language rather than of this taxonomy. It now pins the group-to-kind table, and moving `FACET_INDEX` under `METADATA` fails it. The same tautology (`part.group().kind()` against `part.kind()`, where the latter is defined as the former) is dropped from the composition and driver tests; the converter test keeps its version, where it is not tautological because it checks the encoder actually populated the wire field. Ref: #1500

feat: add the storage-part group and kind to the gRPC statistics contract `GrpcStoragePartUsage.storagePartType` is the simple class name of a storage part - an OPEN set that grows whenever the engine gains an index structure, and one a client cannot classify: two of the engine's index parts carry no `Index` in their class name at all. Two new CLOSED enums say what a type actually holds, so a composition table groups on a contract rather than on an allowlist of class names it has to maintain from outside the engine. `GrpcStoragePartGroup` has fourteen values; `GrpcStoragePartKind` is their coarse fold. The index groups mirror the data groups (`ATTRIBUTE_DATA` against `ATTRIBUTE_INDEX`, and so on) so a schema owner can read what indexing a feature costs against what storing it costs. `kind` is a field of its own rather than something the client derives, because a generated client enum carries no behaviour to derive it with. Both fields are additive (tags 4 and 5) on a message no release has shipped. Ref: #1500

2026-09-03

feat: carry the parents behaviour to the server as a query parameter Until now `HierarchyParentsBehaviour` could only reach the server inlined in an EvitaQL string through the unsafe query endpoint: a parametrised query turns each constraint argument into a `GrpcQueryParam`, and that message had no arm able to hold the enum, so the Java driver could not send `COMPLETE` at all. `GrpcHierarchyParentsBehaviour` is appended to `GrpcEnums.proto` and bound into `GrpcQueryParam` as field 28 - append-only, nothing renumbered. `EvitaEnumConverter` gains both directions with the exhaustive switch the defensive-design rule requires, and `QueryConverter` wires them into the two sides of the parameter conversion. `MATCHING` is the zero value on purpose. An absent field is what an older client sends, and it has to read as the behaviour `hierarchyContent` has always had rather than as `COMPLETE`. In practice the arm is only ever occupied by an explicit `COMPLETE`, because `HierarchyContent#isArgumentImplicit` elides the default from `getArgumentsExcludingDefaults()`, so a default query emits no placeholder for it - but the zero value carries the guarantee for the cases where it is written out. Verified: `EvitaClientReadWriteTest#shouldQueryCompleteParentChainThroughParametrisedQuery` builds a four-node chain whose middle ancestor holds no English data and asserts the driver receives body(3) -> pointer(2) -> body(1) with nothing above the root, which is the shape the mode exists for and could not have been requested through the safe path before. `QueryConverterTest` covers the round trip of the parameter itself. Regressions green: gRPC session service 56, behaviour matrix 97, QuerySerialization 89, EntityConverter 10, the driver suites 173 with one pre-existing skip. Ref: #1365

fix: keep the resolved parent chain intact across every seam The quality gate over the query-model and engine phases found the feature defeated at the seams rather than in the walk itself: a chain resolved by the parent fetch was thrown away by the next re-wrap, could not be carried over the wire, and could not even be expressed when `entityFetchAllContent()` was written beside an explicit requirement. Eleven defects, each fixed red-to-green: - a resolved parent slot is now carried verbatim through every re-wrap. The `EntityDecorator` copy constructor inherits the re-wrapped decorator's raw slot instead of re-deriving it from the delegate, and the four `EntityCollection` decorate sites read it through the non-interpreting getter. Reading it through `getParentEntity()` cannot tell "the chain ends here" from "nobody resolved it", so it resurrected one raw ancestor past every stopAt cut, every MATCHING cut and every COMPLETE substitution; - an enrichment restates a single required locale as a filter for the ancestor body read, so `enrichEntity` no longer materializes the very ancestor the original query's locale hid; - `GrpcEntityReferenceWithParent` gained `parentEntity = 5` (append-only), so a body can sit above a bodyless pointer on the wire. The legacy `parent` field is still filled with the same ancestor reduced to primary keys; - a ring member is reported once, not twice: the fragment top of a ring carries a key the index resolves, which was indistinguishable from a key it cannot; - a sub-chain collected under a stopAt bound is no longer published for reuse, since the bound is measured from the queried entity; - an unresolvable parent key is no longer queued for body fetching; - combineWith is order-insensitive when neither side asks for bodies, keeps a bound only when both sides carry it, and containment of the bare form holds only in a COMPLETE container; - a scope with no hierarchy index resolves to the chain terminator rather than NULL, which is the same resurrection through another door; - the ancestor tail is folded into the IO counters only where the read-time walk cannot reach it, which double-counted a body sitting directly above; - `EvitaRequest#isRequiresParent` reduces every hierarchyContent match with combineWith instead of demanding a single one, so `entityFetchAllContentAnd(hierarchyContent(COMPLETE, entityFetch(...)))` expresses one requirement instead of failing the query; - the deprecated CONCEALED_ENTITY terminator keeps its identity across Java de-serialization. Two findings were deliberately left alone, with the reasoning recorded in the ADR: CacheEden carries no parent chain in either direction, so its null parent slot is the one the cache-miss path uses; and the scope-without-index miss has no constructible test seam short of reflection. `AbstractHierarchyTranslator` keeps the backticked reference to `AbstractHierarchyStatisticsComputer#createStatistics` rather than a link: that class is package-private in the producer sub-package and cannot be linked from there. Verified: the behaviour matrix 97 tests across eight nested classes, the requirement algebra 15, the decorator parent slot 12, the gRPC chain conversion 2, plus 653 tests over the fetch, proxying, hierarchy-filtering and gRPC session suites and 821 over the GraphQL and REST suites - all green, only pre-existing skips. Ref: #1365

2026-08-31

fix: validate the assembled attribute, never the half-written chain `AbstractAttributeSchemaBuilder#toInstance()` both assembled the attribute and validated it, and Lombok's `@Delegate(types = AttributeSchemaContract.class)` sat on that method in all three subclasses. Every contract getter the builder exposed therefore meant "assemble the whole attribute and apply the finished schema's rules" - including when the builder called one on itself mid-chain to compute its own mutation. A chain was then judged by the finished chain's rules while it was still being written, so `acceleratedFor(...)` followed by the `filterable()` that licenses it was refused before the licence was given. Assembly and validation are now separate. `assembleInstance()` applies the recorded mutations and caches; `toInstance()` assembles and then validates once per assembly, gated by an `updatedSchemaValidated` flag that a rebuild clears and a successful `validate` sets - so a refusal is never cached as success. The `@Delegate` moves onto `assembleInstance()`: a getter reports what is currently declared, while `toInstance()` is what hands out a validated schema. Nine call sites stop validating a half-written attribute - `withDefaultValue`, the three `non*(...)` withdrawals, the delegated getters, and four `toInstance().getName()` reads in `GlobalAttributeSchemaBuilder` that assembled and validated an entire attribute to read a name that cannot change. The 30-line `getDeclaredAcceleratorsInScope` workaround that read the axis off the mutation pipeline is removed; the delta resolves against the builder's own getter again. `AbstractAttributeSchemaBuilder` was the only builder whose `toInstance()` validated, so the other ten carry no equivalent trap. Session close now validates the catalog schema before flushing rather than inside the flush future, so a refused warm-up session no longer performs the write it is being refused for. The schema is still persisted by `EntityCollection#updateSchema` and `Catalog#terminateInternally` - a pre-existing warm-up atomicity gap this rule makes observable rather than creates, tracked as #1466. Also: accelerator validation delegates to `AttributeSchemaContract#validate()` instead of restating it; four local variables lose the pre-rename "capability" vocabulary; the stale "filterable only" accelerator rule is corrected in two descriptors and two proto doc comments (stubs regenerated); copyright years fixed on the new files. Ref: #1454

refactor: lift filter accelerators onto their own schema axis `FilterIndexCapability` was an argument of `filterable(...)`, which meant a `unique()`-only attribute had no call to hang a substring accelerator on even though a unique attribute already carries a filter index. Acceleration is now a sibling axis: whichIs -> whichIs.unique().sortable() .acceleratedFor(AttributeFilterAccelerator.SUBSTRING_SEARCH) Renames: `FilterIndexCapability` -> `AttributeFilterAccelerator` (`SUBSTRING` -> `SUBSTRING_SEARCH`), `ScopedFilterCapabilities` -> `ScopedAttributeFilterAccelerators`, `Capability.SUBSTRING_FILTERABLE` -> `SUBSTRING_ACCELERATED`, and their gRPC, GraphQL and REST mirrors. `SetAttributeSchemaFilterableMutation` returns to its released shape and a new `SetAttributeSchemaAcceleratedMutation` carries the axis. The rule relaxes from "filterable" to "has a filter index" - filterable *or* unique - and moves off the mutation, which sees intermediate state. Mutation combination reorders a filterability change after an accelerator change, so per-mutation validation made declaration order significant. It is now enforced only on assembled schemas, in two places: `AbstractAttributeSchemaBuilder#validate` for the Java API, and a new `AttributeSchemaContract#validate` reached from `CatalogSchema#validate` for mutations arriving over gRPC, REST and GraphQL, which previously bypassed validation entirely. The axis never shipped: `release_2026-2` has no reference to it. So no new backward-compatible serializers were written, the duplicate `SetAttributeSchemaFilterableMutationSerializer_2026_2` registration is deleted, and the retired proto field number is freed rather than reserved. Restating `filterable()` no longer withdraws accelerators; only `nonAcceleratedFor(...)` does. Verified: 3134 schema+serialization tests green, 42 gRPC converter tests green, 5090 test classes from a full run with no failure in this surface. Ref: #1454 Claude-Session: https://claude.ai/code/session_01422J5ZVdhw98gbF4Epgnqs

feat: add SUBSTRING filter index capability to attribute schemas Introduces `filterable(FilterIndexCapability.SUBSTRING)` and `filterableInScope(ScopedFilterCapabilities...)` (per the `unique(AttributeUniquenessType)` precedent) across all schema layers: API contracts, DTOs, builders, mutations, gRPC, GraphQL, REST, Kryo/WAL serialization with `*_2026_2` backward-compatible readers, and the `Capability.SUBSTRING_FILTERABLE` usage-statistics row. The capability is declared only for now - the trigram index it enables arrives in follow-up increments on this branch. Guard rails and non-obvious decisions carried by this change: - Adding a capability to an attribute of a non-empty collection is refused at schema time (building the index over already-stored entities is not supported yet; lifting this later is backward compatible, the reverse is not). The same reasoning refuses SUBSTRING on reference attributes. - `Catalog.verifyEntitySchemaMutationsApplicable` is an advisory-narrow preflight for the global-attribute cascade: it surfaces only the emptiness refusal and deliberately swallows everything else - the three conditions making that safe are documented at the swallow site. - Cascade behaviour must be tested by applying the catalog-level `SetAttributeSchemaFilterableMutation`, not by driving a builder - the builder path short-circuits the cascade and produced a green test that tested nothing. - The legacy 13-arg entity / 14-arg global `AttributeSchema` constructors are restored and arity-checked: a convenience constructor at those exact arities once silently rebound `@SerializableCreator` call sites (boolean into the Scope[] slot). The legacy forms now occupy those arities so a future collision fails loudly instead. - `ModifyAttributeSchemaNameMutation` leaves the original attribute in place (the rebuild filters on the NEW name), so per-name capability diffing in the refusal is correct and a capability-carrying "rename" on a populated collection is correctly refused; pinning tests fail loudly if rename semantics ever change. - The two capability-vs-type checks are switch EXPRESSIONS on purpose: only the expression form makes a future `FilterIndexCapability` constant a compile error there. Ref: #1454

2026-08-24

fix: reattach observedSince field comments to the right field Field reordering during the schema-capability-usage merge left the observedSince-describing comment attached to the measured field instead (both are contiguous, so buf attaches the whole block to whichever field follows). buf lint flagged observedSince as undocumented in GrpcBrowsedIndex, GrpcIndexDetail and GrpcSchemaCapabilityUsage, breaking the dev CI lint gate. Comment text unchanged, just reordered to sit above its own field; no wire format change. Ref: #1430

2026-08-23

fix: stop the tracking switch from throwing on ranked browse and on older servers Two defects in the `usageStatisticsTracking` switch, both of which threw rather than reporting wrongly, and both found by adversarial review. **Ranked browse over an unmeasured catalog threw.** `cutPage` substituted the frozen `rankedValue` for both counts whenever the holder was absent. Under `ENTITY_COUNT` that value is the entity count, which stays real with the counters off - so the row reported a cardinality as query traffic and tripped `BrowsedIndex`'s own premise check. Both counts are now `0` whenever the holder is absent, whatever the ordering. **Responses from an older server were rejected.** `measured` was a bare proto3 `bool`, which defaults to `false`, so a server predating the field decoded as "not measured" while still carrying real counts - a self-contradictory row that the record constructors reject with an exception. Such a server had no switch and therefore always measured, so the field is now `google.protobuf.BoolValue` and its absence decodes as `true`. Only an explicit `false` means counting was switched off. Both regression tests are calibrated: each was re-run against the reinstated defect and confirmed to fail. Ref: #1429

feat: gate index and schema-capability usage statistics behind a server switch `server.usageStatisticsTracking` (default true) turns both usage-counting surfaces off at once. With it off no `IndexActivity` holder is allocated for any index, and neither the query nor the write path resolves a capability holder. The prior JMH gate measured read-path throughput and found no regression, which left two costs unaddressed: the 56 bytes per index charged unconditionally by `EntityIndex.getBaseHeapSizeInBytes` (a large catalog holds hundreds of thousands of indexes), and the write path, which was never benchmarked at all. On the read path the filter is re-translated once per candidate index set, so `recordRequestedCapability` mints and hashes a `SchemaCapabilityKey` N times per logical query - the accumulator dedup happens on the resolved holder, after the lookup. The knob lives on `ServerOptions` rather than `ObservabilityOptions`: the observability module depends on the engine and never the reverse, and these counters feed the gRPC management API rather than the Prometheus endpoint. `EntityIndex.activity` becomes `@Nullable` and the allocation is skipped, rather than sharing a no-op holder - a shared holder reclaims nothing per index and hands callers an object that claims to have been observed. Reporting: `BrowsedIndex`, `IndexDetail` and `SchemaCapabilityUsageStatistics` gain a `measured` component, mirrored on the wire. `alignWith` keeps running with the switch off, so every declared capability still has a row - it reports `measured=false` instead of zeros. Rendering an unmeasured row as a zero would say "nothing uses this flag, drop it" about a flag nobody was counting, so the three record constructors reject that combination outright. Ref: #1429

2026-08-21

refactor: rename the usage carrier and fix two capability recording sites `SchemaCapabilityUsageSnapshot` borrowed `Snapshot`, a word this codebase already spends on the transactional MVCC memento mechanism. A weakly-consistent read of live counters is not one, so it becomes `SchemaCapabilityUsageStatistics`, following the `...Statistics` vocabulary the surrounding package already uses. The rename carries through the gRPC mirror, its enums and the tests. Two recording sites were filing rows nobody can act on: - `FacetHavingTranslator` recorded FACETED against every scope the query named, although its own assertion only demands that *one* of them declare the flag. The surplus rows landed in scopes where no facet index exists and no write can ever file a matching maintenance count, which reads as "nothing maintains this" about a flag that is simply not declared there. Now recorded per scope. - `ReferenceHistogramStatisticsTranslator` reached the entity schema through an `Optional` and silently recorded nothing when it was empty, and filed the whole requested scope set without asking whether a histogram is maintained there at all. It now takes the schema from the planner and gates each scope on `maintainsHistogramIn(...)`; a count-only histogram is left without a row. Ref: #1429

2026-08-19

feat: expose schema-capability usage as a diagnostic surface listCapabilityUsage reports one row per capability flag, element and scope - requested against updated counts with their stamps and the observation window - through the same plumbing browseIndexes takes: EntityCollectionContract and CatalogContract answer for their own registries, EvitaManagementContract dispatches by entity type (null selects the catalog's globally-unique attributes), and the gRPC surface crosses with an additive message, rpc and enums. The ElementKind and Capability vocabulary moved from the engine's key to the public snapshot record so the key, the surface and the wire speak one enum that cannot drift. Response is schema-bounded, so it is a plain ordered list - no paging, no criteria. Ref: #1429

2026-08-18

refactor: order the browse by a key and a direction, not a flat enum The six-value ordering enum spelled out a product type the query language already models as OrderDirection: two counters times two directions, plus a direction baked into the entity-count name. IndexBrowseOrdering is reshaped into four keys - MAP_ORDER, ENTITY_COUNT, QUERY_COUNT, UPDATE_COUNT - and IndexBrowseCriteria carries the direction beside it. MAP_ORDER accepts ASC alone and rejects DESC at construction rather than ignoring it - a silently honoured-looking direction would hand the client a page cut from the wrong end of nothing. ENTITY_COUNT ascending becomes expressible for free on the same frozen-candidate machinery, and the deep- page bound stays fail-safe by naming the exempt key. The gRPC enum reshapes in place to values 0-3 with the existing GrpcOrderDirection joining the request; the whole surface is in no release tag, which is what made this reshape free - after a release it would have cost a deprecation cycle.

feat: order the index browse by usage counters The browse could rank indexes by what they cost (entity count) but not by what they earn. Four orderings join it - query and update count, each in both directions - with the ascending variants being the drop-candidate hunt: least-queried first surfaces the indexes paying maintenance for nothing. The correctness rule every ranked path now follows: a comparator never reads a live activity holder. A PriorityQueue does not reheapify when an element's priority mutates, and repeated comparisons of one pair could disagree, so each candidate freezes its ranked reading exactly once during the walk, the comparator sees only the frozen value plus the EntityIndexKey tiebreaker, and the row reports the very value that placed it - no row can contradict its own position. The other activity readings stay fresh reads, which only the ranked one cannot afford. The collection browse serves all five ranked orderings through one walk and one bounded-heap page cut; the catalog browse renders its at-most-one-row- per-scope up front and plain-sorts, and stops collapsing the counter orderings - a catalog index is chosen and maintained like any other, only the entity-count ordering keeps its documented degeneracy there. Pages under a moving sort key are best-effort top-N views and say so; exhaustive enumeration belongs to MAP_ORDER. The deep-page bound now covers every ranked ordering and fails safe - the check names the exempt MAP_ORDER, so a future value is bounded by default. MAX_SIZE_ORDERED_WINDOW is renamed to MAX_ORDERED_WINDOW accordingly; the introducing commit is in no release tag, so nothing released compiles against the old name. The gRPC enum gains four additive values (3-6), and the criteria round-trip is parameterized over the whole enum so a future value cannot ship without a wire mapping.

fix: report an unknown observation window as absent, not as the epoch Decoding an old server's silence to the epoch avoided the crash but fabricated a decades-long observation window - "never queried in the last week" would read true against it when the truth is unknown, which corrupts exactly the sentences the reading exists to keep honest. "Now" would lie in the opposite direction with a zero-length window; every substituted instant fabricates one. observedSince is therefore nullable on both records, null meaning one thing only: a remote server too old to report it. A current server always sets it, so unlike the two stamps there is no "not yet" case. observedSinceIfKnown() joins the other two IfKnown accessors, and a client computing a rate or a "never in N days" sentence must skip an unknown window rather than invent one. The absence also travels onward on re-encode instead of being replaced.

feat: carry observedSince over gRPC GrpcBrowsedIndex gains tag 13 and GrpcIndexDetail tag 9, additive only. The field is message-typed like the two stamps but, unlike them, always present - an index is observed from the moment it exists, so there is no "not yet" for absence to express. The epoch placeholder the previous change decoded in the read direction is gone with the wire now carrying the real value.

2026-08-16

feat: report how often each index is queried against how often it is updated The per-index statistics surface could say what an index costs — its heap footprint, entity count and cardinality — but not what it earns. Every index now also reports how many executed query plans chose it, how many entity mutations acquired it for modification, and when each last happened, on both the browse row and the drill-down, and over gRPC. The counters live in a separate IndexActivity holder rather than in the index, because a commit that dirties an index replaces it instead of mutating it: a plain field would reset on exactly the indexes worth measuring, silently and while still looking like it worked. The holder is passed by reference through all six constructors that rebuild an index, the way primaryKey travels and unlike version; fresh creation and reload-from-disk allocate a new one, which is what makes the readings "since catalog load" by construction. Semantics, stated on BrowsedIndex and IndexActivity: the query side counts chosen, not consulted, so a candidate index that loses the cost comparison counts nothing. The update side counts effort including effort a rollback undoes, deliberately inverting IndexPopulation's rule, which is right for state counters and wrong for effort counters. Counters are non-transactional and are never persisted. Two exclusions are documented rather than fixed: index maintenance dispatched through EntityCollection#applyIndexMutations never reaches the executor, so updateCount reads as a floor; and the verification debug modes build the preferred plan twice and execute the losers, so exact query counts hold only without them. The gRPC wire is additive only — GrpcBrowsedIndex gains tags 9-12 and GrpcIndexDetail tags 5-8, with the two stamps message-typed so "never" is expressible as absence rather than as an epoch.

2026-08-14

Merge branch 'dev' into 1339-catalog-entity-collection-statistics-for-the-management-api Reconciles 191 commits of `dev` with the statistics work. Eight files conflicted; the substantive ones: - `CatalogContract` loses `duplicateTo`. Dev removed it and this branch was merely carrying it unchanged from the merge base, so the removal wins; dev keeps the operation as a two-argument method on `Catalog` itself, taking the target folder token alongside the name. - `ObsoleteFileMaintainer` keeps both sides. This branch's `getMaintainedFileVersions` and dev's version pinning, release and retention floor landed at the same offset but are independent of one another. - `SessionRegistry` keeps this branch's `reportTransactionResolution` and takes dev's reworded contract for `createCatalogConsumerControl`. - `UnusableCatalog` keeps the component-selected `getStatistics` over dev's flat one, and stops measuring its own folder - see below. Dev's catalog-folder decoupling binds a catalog to an opaque `CatalogFolderId` and establishes that **the engine must never hold a path derived from a catalog's identity**. `UnusableCatalog#measureStorageSize` did exactly that: it listed the catalog directory to decompose an unopenable catalog's disk into bootstrap, WAL and unaccounted bytes - the reading `STORAGE_SIZE` exists in order to survive a corrupted catalog with, since how much of what is holding the disk is what tells an operator "restore it" apart from "shorten WAL retention". Neither rule is dropped to settle it. The measurement moves behind the folder SPI: `CatalogFolderOperations` gains `catalogFolderFootprint(CatalogFolderId, String)`, the decomposing counterpart of its existing `catalogFolderSize`, implemented in the storage layer where resolving a token to a directory is legal. The engine keeps the capability and never joins a token with the storage root. The alternatives were to report `STORAGE_SIZE` as unavailable for an unusable catalog, which discards the one component deliberately built to survive one, or to report only the existing scalar total with every other class left at zero - which reads as measured-and-empty rather than as not-measured, the exact confusion the component model answers with a status and a reason instead. Ref: #1339

feat: component-selected catalog and collection statistics, measured one index at a time The management API answered exactly one statistics question: a flat `CatalogStatistics` computed in full, for every catalog, on every call. It is now a component model - a client names the parts it wants and the engine computes only those - across two calls, one describing a catalog and one describing a single entity collection. **Components, not a detail ladder.** A nested `BASIC < STORAGE < FULL` ladder would force a once-and-for-all ruling on which statistic sits at which level, and make a client wanting one expensive number drag everything cheaper along with it. Components are independently *selectable* but deliberately not independently *computed*: `STORAGE_SIZE`, `FRAGMENTATION` and `HISTORY` all need file lengths, so one directory listing per request serves all three - not merely cheaper, but the only way the three cannot describe different moments. A requested component that cannot be computed answers with a status and a reason, never with zeroes - zeroes during a bulk load read as "idle and healthy", the exact inverse of the truth, and a corrupted catalog rendering as an empty one is precisely what an operator opened the screen to diagnose. **Two levels, and the line between them is drawn by cost.** The catalog response is the one that gets polled, so it reports aggregates only and never a per-collection breakdown - its size must not grow with the number of collections. Where a component cannot be aggregated cheaply, its catalog-level form reports something different and cheaper rather than summing the collection-level one. **Heap footprints are exact walks verified against JOL, not formulas.** A formula is unfalsifiable at scale: it stays plausible while being wrong. Every index, container and leaf structure now prices itself, and the arithmetic is checked per class against a real JOL walk. That is what made the ~90 %-of-live-heap agreement checkable, and what surfaced the defect below. **A cached map view was being retained on every index.** A `HashMap` keeps the `keySet`/`values`/`entrySet` view it hands out, so a walk on a construction or flush path costs sixteen retained bytes on every index in the catalog rather than nothing - fourteen such views per entity index, seventeen once flushed, ~117 MB on a production catalog, now zero. Both transactional map decorators override `forEach` to delegate to the backing map, and the walks use it. The price index's two *query* accessors keep handing out a cached view deliberately: they are called repeatedly against the same index by the price translators, which is exactly what the JDK's caching is for, and converting them would trade one retained view for a fresh walk per query. **The exact heap figure is reached by naming one index.** A browse page is selected by map order or entity count, so the caller does not choose what lands on it - a 20-row page can cost 200 ms nobody asked for. `browseIndexes` lists cheaply, `getIndexDetail` measures the one index the caller named; the cost is bounded by the catalog's single worst index and paying it is a choice. An index is addressed by its integer primary key rather than by the browse row's discriminator, which is a rendering - injective, so rows never collide, but it prints representative values through `toString` and cannot be turned back into an index key. Both calls serve a catalog's own indexes and an entity collection's through one surface, selected by a nullable `entityType`. Separate catalog-only twins would have duplicated the criteria, the row type and the detail record, leaving a UI two code paths for what an operator reads as one table; the engine's asymmetry between the owners - a collection walks hundreds of thousands of indexes, a catalog at most one per scope - is a cost difference, not a shape difference. What a catalog index lacks is stated by absence rather than a stand-in: `indexType` is null and `entityType` with it, and `entityCount` is absent rather than filled with the summed unique-value count, which is not an entity count at all - a globally-unique index maps values to records of any collection, so an entity carrying three such attributes would be counted three times. On the wire each absence is an unset wrapper, never a sentinel: `""`, `INDEX_TYPE_UNSPECIFIED` and `0` are all values a converter reading without a presence check would silently accept. Two dead selectors are removed rather than kept for a future producer. `NOT_SUPPORTED` never had one, and the retention argument was backwards - adding an enum value is the wire-compatible direction, so it can return the day something declines, whereas keeping it makes every client branch on an outcome no server sends. `CatalogStatisticsComponent#isCatalogLevel` answered true for every component once `MEMORY_FOOTPRINT` was withdrawn, so its gate rejected only the empty selection while every reader had to decode it. The vacated proto number is `reserved` rather than renumbered: renumbering is free in principle, but it is the one option under which a client built against a pre-release proto asks for the old number and is silently handed a different value. One `EntityIndexType` in `evita_api` replaces the engine enum plus its API mirror. The mirror's only divergence was a value deprecated since 2024.12; with that retired the two were identical, and two identical enums joined by a mapping can only drift. Retiring that value - `REFERENCED_HIERARCHY_NODE` - is a storage-format change rather than a rename, because an entity index storage part persists its type through `Enum#name()` and `Enum.valueOf` would have failed the load of any catalog written before 2024.12. The fold lives in a dedicated `EntityIndexTypeSerializer` registered against the single Kryo registration all four storage-part serializer vintages read the type through, so it covers every format that can carry the retired name rather than requiring somebody to prove that only one vintage can. The gRPC surface follows the Java naming. Nothing here has shipped on `dev`, so these messages were renamed and retyped freely as the surface settled - the strictly-additive rule applies to messages that have actually left the branch, and nothing that did was touched. Note for whoever bisects through this: renaming a protobuf message leaves the old generated `.class` behind in `target/classes`, because protoc writes to `src/main/java` and an incremental compile never deletes it. The installed jar then carries classes that exist in no source file, and dependent tests fail with `NoSuchMethodError` naming symbols that are gone. A `mvn clean install` is required, not an incremental one. Ref: #1339

2026-08-05

fix: stop a read-only browse from mutating the index map, and make its identity injective Two defects a code-quality pass found in code committed earlier tonight. PersistentTransactionalMap.sealed() is not a getter. While the state is still a thawed HashMap it builds the immutable map and writes it back into `state`, which is right on the commit path - the next transactional touch then finds it already sealed - and wrong from a read. EntityCollection.browseIndexes called it with no session and no lock, so a management call was mutating the collection's index map. That cost two things. Each browse sealed the map and the next non-transactional write thawed it again, an O(N) copy, so a reader alternating with a bulk load turned every write into a full rebuild - the map's own javadoc rules this out on the grounds that warm-up writes all precede transactional life, which an API callable at any time falsifies, and createCopyWithNewPersistenceService already skips sealing for exactly this reason. Worse, the frozen map is built by iterating the live HashMap, so a concurrent warm-up write landing during that iteration was discarded when the result was published, leaving `indexes` and `indexesByPrimaryKey` describing different sets. A statistics call must not be able to lose an index. Adds a non-publishing snapshot() and uses it from the read path. Live catalogs are unaffected - the state is already immutable and both methods return it as is. A concurrent warm-up writer can still make the reader fail with ConcurrentModificationException; that is a loud, reader-local failure instead of silent corruption of a map every caller shares, and it is documented as the deliberate trade. The browsed-index discriminator was not injective. It was rendered with RepresentativeReferenceKey's toString, which is value-based - but value- based is necessary and not sufficient for an identity. That rendering joins the representative values with an unescaped ", " and prints an absent one as the literal NULL, so ["a", "b, c"] and ["a, b", "c"] come out identical, as do [null] and ["NULL"]. Two genuinely distinct indexes then reported one identity, which is the defect the discriminator was introduced to remove, reintroduced one level down in the rendering. Each part is now length-prefixed, an encoding no value can forge, and toString keeps its readable shape for logging. The size-ordered page also re-read entityCount when cutting the page instead of carrying the value it had ordered by. Freezing the map freezes its structure, not the mutable indexes inside it, so a row could be ordered by one count and reported with another - a page that reads as out of order. It now carries the count from the walk, which also drops a lookup and a bitmap read per row. Two documentation defects went with them: a comment describing the wrong consequence of an offset overflow (the hazard is wrapping to a small positive offset and serving an unrelated window, and the size ordering cannot reach it at all now that its window is capped), and a claim that a per-index memory estimate is reported for the selected page when none is reported anywhere. The proto's memory-footprint component now also says it is undelivered and why, which the Java enum already said and the file client implementers actually read did not. Tests: a projection unit class covering both discriminator collisions, heap eviction, the page-cut clamp and empty input without booting an engine; a descriptor-population case per index kind; and the scope partition test now archives an entity first - with nothing archived it asserted `unfiltered == unfiltered + 0` and held however broken the filter was. Verified: 30/30 modules, buf lint clean, functional suite 6802 tests, 0 failures, 0 errors, 27 skipped. Ref: #1339

fix: make a browsed index identifiable and bound the size-ordered walk Three defects a Codex review over the branch found, all real. Deep paging in BY_ENTITY_COUNT_DESC was unbounded. Capping pageSize does not bound that ordering: its heap retains everything up to the *end* of the requested page, so an arbitrarily large pageNumber retained every matching index and then sorted all of them, only to hand back an empty page. One cheap-looking request could therefore force a full sort and a proportional allocation on a collection with hundreds of thousands of indexes - the opposite of the guarantee the bounded heap exists to give. The retention window is now capped at MAX_SIZE_ORDERED_WINDOW, checked in long arithmetic because the product overflows int well before it reaches the limit. The cap is attached to that ordering alone: MAP_ORDER counts as it walks and materialises only the window, so it costs O(pageSize) at any depth and stays unlimited, which is also the answer for a client that genuinely wants to page through everything. A browsed index could not be identified. RepresentativeReferenceKey carries the representative attribute values that tell two targets of one reference apart, and they participate in its equality and ordering - so two distinct indexes can agree on both reference name and target primary key, which were the only parts the descriptor rendered. A client deduplicating across pages would silently lose one of them. The descriptor now carries the full discriminator, and the two existing fields are documented as projections that are not unique between them. The same mistake was in this branch's own test helper, which keyed page identity on exactly that pair; the fixture has no representative attributes, which is why it passed. Note the identity leans on the discriminator's toString being value-based. Both permitted implementations are, and there is now a comment saying so where someone might change it. A unique index's covered-record count can read low. recordIds is an eager cache that drops a record on the first of its values removed, and one record owns several values in a single index when an attribute is localized *and* unique globally, since that pairing has one locale-less key. This is not substituted with a computed figure: the same bitmap backs getRecordIdsFormula, so it is what the engine queries the index through, and a separately-derived number would describe an index the engine does not have. It is reported as the engine's own membership view and documented as such, with distinctValueCount named as the reading to use when the question is how much the index holds. Verified: 30/30 modules, buf lint clean, functional suite 6793 tests, 0 failures, 0 errors, 27 skipped. Ref: #1339

feat: browse a collection's entity indexes, paginated and filtered INDEX_SUMMARY answers how many indexes of each kind a collection holds; this answers which ones. A count of forty thousand REFERENCED_ENTITY indexes tells an operator something is wrong but not which reference caused it, and nothing until now could say. Adds BrowseEntityCollectionIndexes to the management API, reachable embedded and over gRPC: pick a page, order it, and filter by index kind, scope or reference name. Filters are conjunctive across categories and disjunctive within one; an empty category does not filter. Every call walks the whole index map - O(indexes), unavoidably, since there is no per-kind index of the indexes and building one would duplicate every key while still costing a full pass to order. Paging bounds the answer, not the work, so this is documented as a drill-down and never something to poll. What it does avoid: iteration runs over keySet(), whose ChampKeyIterator allocates nothing per element, and every filter reads off the key alone, so a rejected index is never fetched. Under MAP_ORDER only the requested window is materialised. Ordering by entity count uses a bounded heap rather than a full sort, and breaks ties by index kind, then scope, then discriminator - EntityIndexKey's own total order. The tiebreaker is load-bearing rather than cosmetic: index sizes are heavily tied in practice, so ordering on the count alone would let successive pages re-cut a re-permuted tie block, showing some indexes twice and others never. There is deliberately no ordering by estimated memory. Entity count is an O(1) bitmap cardinality, whereas a memory estimate must traverse an index - ordering by it would mean estimating every index in the collection on every call, destroying the property that makes this surface affordable. The estimate belongs to the page that was selected, never to selecting it. pageSize is capped and an over-large request is refused, not clamped: a clamped page is indistinguishable from a complete one, so a client paging until it sees a short page would stop early believing it had seen everything. GrpcTaskStatusesRequest needs no such cap because task counts are small; index counts are not, and both factors of the bounded heap are client-chosen. A reference name the schema does not declare is likewise an error rather than an empty page, so a typo cannot read as "this reference has no indexes". The walk runs over a sealed view of the index map, so the match count and the page contents cannot come from two different states - in WARMING_UP the map is otherwise forwarded mutably by reference, where a concurrent bulk load could move the paging offset mid-walk. Each page reports the catalog version it was read at, so a client can tell that two pages describe two different index sets. That comparison only discriminates once the catalog is alive: the version advances per committed transaction and a warming-up catalog runs none, so during a bulk load it stays put while the index set churns hardest. The caveat is recorded on IndexBrowseResult and on the proto field, and both halves are pinned by tests - the version static while the match count grows during warm-up, and advancing after goLiveAndClose. Sealing is unaffected; warm-up costs cross-page comparison, never within-page coherence. Verified: 30/30 modules, buf lint clean, functional suite 6792 tests, 0 failures, 0 errors, 27 skipped (6772 before, plus the 20 added here). Ref: #1339

feat: report catalog index cardinality at the catalog level INDEX_CARDINALITY gains a catalog-level form describing the catalog index's global unique indexes - one entry per globally-unique attribute, per locale in use, per scope. Every reading is an O(1) counter maintained by the index's backing tree rather than a walk, so the cost grows with neither entity count nor collection count. The collection-level form is unchanged and stays expensive; the two halves deliberately report different indexes, and neither aggregates the other. The catalog-level form is admitted to the instance-wide all-catalogs call. That call multiplies every listing it returns by the catalog count, so the component was weighed on payload as much as on compute time. It was admitted because barring it would not remove the cost - only force a client into one call per catalog for the same bytes - and because selection is opt-in, so a client that cannot afford it simply does not name it. That is what separates it from MEMORY_FOOTPRINT, where the collection-level walk is genuinely expensive and no cheap catalog-level form exists to fall back on: there a bar prevents work, here it would only relocate it. Visibility differs from the collection-level index counts by design. Those are a plain int[] handed forward at each catalog version boundary and can only report published state, while the readings taken here go through the index tree's TransactionalReference and consult the calling thread's transactional layer when one is bound. Remote callers are unaffected - an API handler thread carries no such layer. Ref: #1339

feat: report per-index cardinality and maintain index counts incrementally Adds the INDEX_CARDINALITY component (distinct values and records covered per attribute index) and replaces the index-map walks behind INDEX_SUMMARY with counters kept per index type and scope. Both changes exist because a production collection reaches hundreds of thousands of per-referenced-entity indexes: - IndexCardinalityProjection never walks the index map. It constructs the keys of the schema-bounded kinds (GLOBAL, REFERENCED_ENTITY_TYPE, REFERENCED_GROUP_ENTITY_TYPE) and looks each up individually, so its cost is proportional to the schema rather than the data. The data-bounded kinds are reported as a count only - describing them would make the response size a function of the catalog's contents. - IndexPopulation holds one counter per (index type, scope) pair, so reporting the breakdown is a fixed number of array reads. The counts move on the two paths that publish a change, never at the call sites: the transactional path derives them at commit from the delta the index merge already computes, and the non-transactional bulk-load path moves them inline. Rollback correctness is therefore structural - a discarded transaction never reaches the commit that would have moved anything. The transactional/non-transactional split keys off whether a transaction is bound to the thread, not off whether the index map already has a diff layer: computeIfAbsent is the inherited Map default, so its mapping function runs before the put that creates that layer, and the first index of every transaction would otherwise be counted twice. AttributeIndexType is a new API enum naming which structure a reading came from; it deliberately carries only UNIQUE, FILTER and SORT. It is distinct from the identically-named persistence discriminator in AttributeIndexStoragePart, which is baked into storage-part keys and lives in a module the API cannot reference. Ref: #1339

feat: report deferred-durability fence readings through the DURABILITY component The checkpoint coordinator already computed cadence, fence depth, files forced and force duration on every completed checkpoint, but drained them straight into the observability event with getAndSet(EMPTY) - nothing survived for a statistics read. It now retains the four figures as one object through a single volatile write taken from the same locals as the event, so a reader can never pair a cadence from one checkpoint with a fence depth from the next. Fence depth is deliberately not derived from the commit pipeline watermarks: it is how long the oldest unforced change waited for the device, while durabilityLag() is a version count. A handful of large transactions and a flood of small ones produce the same version lag and very different fence depths, so neither figure substitutes for the other. When checkpointing runs inline - checkpointIntervalInMillis at zero, or sync writes off - there is no fence to describe and the component declines with FEATURE_DISABLED rather than reporting zeroes, which would read as "durability is instant and free" when the truth is the exact inverse. Scoped by the DURABILITY enum javadoc rather than the issue's list: the WAL byte size, file count and oldest timestamp it also mentions already ship in HistoryStatistics. With this component delivered no catalog-level component is declined any more, so the end-to-end declined-component proof moves to the collection level on MEMORY_FOOTPRINT, and CATALOG_LEVEL_NOT_SUPPORTED becomes an empty set - phrased so an unhandled component still fails the assertion instead of vacuously passing it. Ref: #1339

feat: stamp collection storage headers with a last-modified time EntityCollectionFileHeader gains lastModifiedMillis, stamped in createEntityCollectionHeader - the single place a header is built, which both the flush path and compaction reach, so the stamp cannot drift out of step with the contents it describes. CollectionHeaderInfo and GrpcCollectionHeaderInfo carry it to clients. It answers "when did anything last change here", which the monotonic version() cannot: a version says how many times, never when. Deliberately not File.lastModified() of the data store file, which was the zero-format-change alternative: that survives no restore - a restored catalog would report the restore as its last write - and compaction moves it without any logical change. The catalog-level half of this needed no work. HistoryStatistics already reports newestTimestamp as the wall-clock time of newestCatalogVersion, read from a bootstrap record that is never trimmed, so only the per-collection figure was missing and the persisted format change is confined to the collection header. This is a persisted format change. serialVersionUID moves from -2149051526452828365 to 7284410593068317745 - the class had not been bumped since 2024.12, so that single orphaned value covers every catalog written by 2026.1 and 2026.2 alike - and EntityCollectionHeaderSerializer_2026_2 is registered against it in CatalogHeaderKryoConfigurer, the only configurer that registers this class. The field is strictly appended, so that reader consumes the old layout exactly and stops before it. Absence is preserved rather than defaulted. A header written before this release carries no timestamp at all; it is reconstructed as NOT_STAMPED, surfaces as null on the API record, and is left unset on the wire, so a client renders "unknown" instead of 1970. A catalog upgraded from an earlier release reports that for every collection until each is next flushed. equals ignores the new field, following activeRecordShare, which the record has always excluded: both are measurements taken while writing rather than part of what the header addresses, and folding a wall clock into equality would make two headers describing identical data unequal. EntityCollectionHeaderBackwardCompatibilityTest renders a pre-2026.3 record and reads it back through the composed catalog kryo rather than the serializer alone, so it exercises the real registration and version routing. Removing the reader registration makes it fail with StoredVersionNotSupportedException naming the orphaned UID - the exact failure an operator would hit opening a 2026.2 catalog, and total rather than partial because the header is read on catalog open. Its sibling test round-trips the current format, so "absent" in the first is the old layout speaking rather than a field nobody persists. Ref: #1339

feat: report catalog write activity through the ACTIVITY component Delivers CatalogStatisticsComponent.ACTIVITY at every layer: the ActivityStatistics record, the GrpcActivityStatistics message on field 13 of the catalog snapshot, both converter directions, and the counters themselves on TransactionManager - which survives catalog generation switches and therefore does not reset under write load. Transactions, mutations and WAL bytes are sampled at one point: the appending stage's point of no return, where both the mutation count and the appended length are in hand, so the three cannot straddle two transactions. Counting at trunk incorporation instead would also count every transaction replayed from the write-ahead log at startup, so a restart would report a burst of write activity that never happened. Rolled-back and conflicted transactions are separate counters: neither ever reaches the log, so neither has a sample to contribute. Rates are exponentially weighted with read-time idle decay, mirroring the waste accumulation rate, so a catalog written hard and then left alone converges on zero instead of reporting that load for the life of the process. The raw counters ship alongside them with countingSince, since they are process-scoped and cannot be read without the instant they were zeroed. Two defects in already-committed code on this branch go with it: describeCommitPipeline read the four watermarks in forward pipeline order while claiming that ordering prevented negative deltas - it is reading the trailing watermark first that bounds them - and ACTIVITY and COMMIT_PIPELINE read those watermarks independently, so pipelineDepth could contradict the watermarks it is by definition the span of, within a single response. They are now read once per request. Ref: #1339

feat: report data-store fragmentation and separate the catalog's own store Adds the FRAGMENTATION component at both levels: waste accounting in OffsetIndex, an EWMA waste-accumulation rate with read-time idle decay, and a compaction forecast evaluated beside the compaction predicate so the engine never re-derives it. Three components - FRAGMENTATION, STORAGE_SIZE and VOLATILE_STATE - now report the catalog's own data store apart from the aggregate it is folded into. That slice is not derivable by subtracting the collections: the aggregate covers only collections whose persistence service is open, and cross-call consistency is not guaranteed. The fragmentation slice carries the eligibility flag as well as the bytes, so a raised compactionEligibleNow is attributable to a store. CollectionFragmentation and CollectionVolatileState become DataStoreFragmentation and DataStoreVolatileState - one record describes any data store, mirroring the SPI where CompactionForecast.plus folds a forecast into its own type. StorageSizeStatistics deliberately keeps two flat longs instead of a nested record: awaitingDeletionBytes and unaccountedBytes are not attributable to any single store, so nesting would ship unfillable fields. Fragmentation and footprint now share one directory listing, removing 1 + C redundant stat calls per request and, more importantly, the possibility of reporting two different lengths of the same file within one response. Fixes IndexSummaryStatistics.totalIndexCount, which hard-coded a single catalog index. A CatalogIndex exists per Scope - LIVE always, ARCHIVED lazily - so an archived entity left the total one short. Ref: #1339

2026-08-04

feat: expose the formula plan and flatten the GraphQL telemetry profile Stage D of the query telemetry campaign, covering issue items 8 and 5. Item 8 - GraphQL publishes the profile as a typed, pre-order flat list of steps carrying `level` and `stepsCount`, rather than a recursive object. In GraphQL the client picks the selection depth, so a nested `steps` field would force a selection set as deep as the deepest query ever profiled and silently truncate anything below it. The `hierarchy` extra result already makes the same trade for the same reason, and a flat list is the shape a flame chart consumes. REST keeps the nested tree. Item 5 - `queryTelemetry(PLAN)` additionally returns the formula plan the planner built: every index-selection alternative it costed *including the ones it rejected*, plus the plan that actually ran. The renderer computes nothing. `Formula#getMemoizedResult()` is the new primitive that makes that possible - it reports whether a result is available for free, never producing one. A forcing renderer would execute the plans the engine had decided to skip, so asking for a profile would change the query instead of observing it. Nodes that never ran report no `actualCost` and no `resultCount`, which is the honest answer: absent is not zero. `QueryTelemetryContent` follows the `ConstraintWithDefaults` pattern used by `priceHistogram`/`HistogramBehavior` - `TIMINGS` is the default, is always stored, and is omitted only from the EvitaQL string form, so `queryTelemetry()` and `queryTelemetry(TIMINGS)` are the same constraint and both print as the former. The level is single-valued rather than a set, because `queryTelemetry(TIMINGS, PLAN)` would be self-contradictory. Two things worth knowing: - `QueryTelemetrySerializer` used to write nothing, which was correct while the constraint was stateless. Once it gained an argument that became silent data loss - a `queryTelemetry(PLAN)` arriving through the Java driver or replayed from a traffic recording would have been downgraded to the default with no error anywhere. A constraint gaining its first argument needs its Kryo serializer revisited. - The non-forcing guarantee has a dependency: `AbstractFormula`'s default `getCostInternal()` computes every inner formula, and the visitor does read `getCost()` on a short-circuited AND. It stays safe only because `AndFormula` overrides the cost path with one that short-circuits at the same index, and `NotFormula` handles its case explicitly. A future formula type that skips children while inheriting the default would break it. Breaking changes, all safe today because nothing consumes these surfaces yet, and all deliberate rather than compatible workarounds layered over shapes that had to change regardless: - REST require shape: `"queryTelemetry": true` -> `"queryTelemetry": "TIMINGS"` or `"PLAN"`. A single-argument constraint publishes unwrapped. - GraphQL: a bare `queryTelemetry` field selection is no longer valid, and the field returns a list. Ref: #1341

feat: add component-selected statistics procedures to the gRPC contract Introduces the wire surface for the component-selected catalog and entity collection statistics. Nothing references the new messages yet - the Java side follows in a separate change, so this one can be reviewed as the contract it is. - GrpcEnums.proto: GrpcCatalogStatisticsComponent (all 15 components plus an explicit COMPONENT_UNSPECIFIED that is rejected rather than ignored), GrpcComponentAvailability and GrpcEntityIndexKind. Values are prefixed because proto3 scopes enum values to the package rather than the enum, so bare names such as GLOBAL or HISTORY would eventually collide. - GrpcStatistics.proto (new): the twenty component sub-messages and the two snapshot containers. Kept out of the management API file following the existing GrpcTrafficRecording / GrpcEvitaTrafficRecordingAPI split. Named GrpcStatistics rather than GrpcCatalogStatistics because protoc emits a file-named holder class into the same Java package, which would clash with the existing legacy message of that name. - GrpcEvitaManagementAPI.proto: the three request/response pairs and the procedures GetCatalogStatisticsSnapshot, GetAllCatalogStatisticsSnapshots and GetEntityCollectionStatisticsSnapshot. The deprecated GetCatalogStatistics comment now names all three as its replacement. Absence is meaningful on the wire: a component that was not requested is absent and has no status entry, a delivered component is present even when all its fields are zero, and a requested component that could not be computed is absent with a status carrying the reason. Verified with tools/lint-proto.sh (clean) and generate-sources followed by compile on the shared module. Ref: #1339

feat: deliver record counts and storage size, retire the legacy statistics shape Completes the migration to the component-selected statistics model. RECORD_COUNTS and STORAGE_SIZE are now answered at both levels, which was the precondition for removing the old shape: the four numbers its consumers read (totalRecords, indexCount, sizeOnDiskInBytes, per-collection rows) all come from components now, so nothing is blanked on the way out. RECORD_COUNTS keeps totalRecords at its historical meaning - the number of entity body storage parts - and derives the live/archived split from the cardinality of the global index of each scope. The three are deliberately not reconciled: a body part in neither global index counts towards totalRecords alone, and that gap is worth seeing rather than hiding. STORAGE_SIZE reports the same measured total as before with nothing attributed to a storage class yet, so the whole of it reads as unaccountedBytes - honestly "measured, not yet attributed". Later stages move bytes out of that remainder, which is the signal the field exists to carry. UnusableCatalog delivers STORAGE_SIZE rather than reporting CATALOG_UNUSABLE: file lengths are readable whether or not a catalog's contents parse, and how much disk a corrupted catalog holds is exactly what an operator needs from it. The legacy flat shape now exists only as the GrpcCatalogStatistics message, assembled from the component model in EvitaDataTypesConverter. Every Java carrier of it is gone - io.evitadb.api.CatalogStatistics, both contract getStatistics() methods, and EvitaManagementContract#getCatalogStatistics() with both its implementations. Re-typing that contract method instead would have forced the driver to rebuild CatalogIdentity from a wire message carrying no goingLive, fabricating a field inside a component the model declares always-delivered; its honest replacement arrives with the proto work. The Java driver therefore has no statistics call until then, while non-Java clients keep the RPC unchanged - which is what the wire-compatibility rule protects. That RPC is marked deprecated in the proto and in Java. Its comment explains what the flat shape cannot express rather than naming a replacement message, since none exists on the wire yet; it should name one once it does. CollectionStorageSize gained sizeOnDiskInBytes and unaccountedBytes and lost activeRecordShare, making it the exact per-collection analogue of StorageSizeStatistics, with the same by-construction invariant. The ratio was duplicated in CollectionFragmentation, which is the component whose job it is. Both records now document how to read awaitingDeletionBytes: superseded files occur in both modes, but only under time travel do they survive the whole history window, where the lever is WAL retention rather than compaction. Ref: #1339

feat: give query telemetry typed, actionable metrics per step Telemetry could say *where* a query spent its time but never *why*: the only numeric planner signal that escaped the engine was an estimated cost embedded in an English sentence. Steps now carry typed measurements alongside the prose. Engine: * `QueryTelemetry.StepMetric` - a closed enum indexing a `long[]` allocated on the *first* `recordMetric` call and never before. A `Map<String, Number>` would have allocated and boxed on a path that must stay free, and would have let each recording site invent its own key. `Long.MIN_VALUE` marks a metric as unset, so "not measured here" stays distinct from a measured `0` - which four of these can legitimately be. `getMetric` returns `OptionalLong`, so reading one allocates nothing on the embedded path. * `QueryPlan#recordQueryMetrics` attaches the eight query level numbers the JFR `FinishedEvent` already computes onto the still-open `OVERALL` root - none of which previously reached the client debugging the one slow query it cared about. The guard is telemetry's own rather than the event's: the two are switched independently, and folding them together would drop the metrics whenever JFR happened to be off. * Both costs report `Long.MAX_VALUE` for "not known" - the estimate on arithmetic overflow, the real one when the formula was never computed - and that is mapped to an unrecorded metric rather than a nine-quintillion cost. Eight metrics ship, not the ten originally sketched. `LOOPS` has no writer, and a permanently-null field is the same defect the REST schema correction just fixed; `RECORDS_FOUND` and `ACTUAL_CARDINALITY` are one value under two names. The enum only ever grows by appending, so both can be added when something actually records them. External APIs: * Published as a nested `metrics` object rather than flattened onto every node. Only the root carries metrics, so flattening would repeat eight nulls on every node of a forty-node tree. * gRPC fields are `optional` - proto3 implicit presence cannot distinguish a measured `0` from an unmeasured metric, and `prefetched` is a real `bool` rather than the `1`/`0` the engine packs internally. * Metrics round-trip through `ResponseConverter`, unlike `selfTime`. Self-time is server-derived and any client can recompute it; metrics are measured, so dropping them would make embedded and remote disagree about the same query. The "metrics live on the root only" contract holds for a non-local reason: a nested query's planning context is seeded with the step that spawned it, not the tree root, so `getTelemetryRoot()` there returns an inner node. What saves it is that nested queries are planned through `planNestedQuery` and never reach `QueryPlan#execute`. That is spread across three classes, so it is pinned by a test that runs a real nested query and walks the whole subtree, not just the root's direct children. Tests also cover the REST and GraphQL response shape end to end for the first time - `queryTelemetry` had no wire-level coverage in either API, so the `selfTime` and `formattedSpentTime` fields added earlier in this campaign were shipping unasserted too. Ref: #1341

feat: turn query telemetry from a phase timer into an actionable query profile Engine: * `pushStep`/`popStep` now take a `Supplier<String>` instead of an eagerly built `String`. Every annotated site previously paid for string concatenation even with telemetry switched off; the hottest of them — `ReferencedEntityFetcher` (once per reference name per page) and `QueryPlanner`'s per-candidate-index `toStringWithCosts` — now cost nothing unless the profile is actually collected. The eager overloads are removed rather than kept alongside, so no call site can regress to them. * `QueryTelemetry#annotate(String)` appends an argument to an already open step. `SortResolutionStrategies` has to record its tally on the *current* step, and the alternative — relaxing the one-shot assert in `finish(String...)` — would throw on the `NestedContextSorter` path, which pushes `EXECUTION_SORT_AND_SLICE` with a description already set. Appending independently of `finish()` keeps that assert meaningful. External APIs: * Telemetry nodes carry `selfTime` (`spentTime` less the time accounted for by direct children) plus pre-formatted `formattedSpentTime` and `formattedSelfTime`. Self-time is derived at the API boundary, not in the engine, so the zero-cost-when-off guarantee is untouched. * BREAKING: the REST/GraphQL `spentTime` property is typed `Long` instead of `String`. The schema declared a string for what has always been serialized as a number; this corrects the contract rather than working around it, and `formattedSpentTime` is what clients wanting the human-readable form should read instead. * gRPC gains `GrpcQueryTelemetry.selfTime` (field 7). It is server-derived; the Java driver rebuilds telemetry through the explicit constructor and does not read it back. Documentation: * Requirement JavaDoc corrected — `start` is a monotonic reading with no epoch and must never be rendered as a date, the overhead is per phase, and the claim that the tree exposes the formula evaluation hierarchy was untrue. * The user-facing telemetry page documents `startedAt`, `selfTime` and the `formatted*` fields, drops the GraphQL `format` argument that no longer exists, and warns that tree shape varies per query and that collecting a profile perturbs what it measures. * JavaDoc debt paid down across the touched classes, including members that predate this change. `QueryPlanningContextTelemetryTest` pins the laziness guarantee: with telemetry off, no supplier is ever invoked. Ref: #1341

2026-08-02

fix: reset sinceIndex when sinceVersion is clamped down in reverse paging ChangeCaptureConverter.toChangeCaptureRequest (reverse paging) clamped an explicit sinceVersion above the requested catalog version down to it, but carried an explicit sinceIndex over unchanged. That index was computed by the client against its own, now-discarded version and no longer identifies a valid position in the clamped-to version - the same defect class this PR's first commit fixed elsewhere. toChangeCaptureRequestForward already guards the analogous forward case via its own versionClamped flag. The fix narrows the reset to only the genuine clamp-down case (an explicit sinceVersion exceeding the bound) rather than reusing the forward helper's condition verbatim, which would also reset the index whenever sinceVersion is merely absent - breaking the already-covered and intentional behavior of honouring an explicit sinceIndex alongside an omitted sinceVersion. Also documents the reset in the sinceIndex proto field comment, which described the direction-based default but not that a clamp discards an explicit value. Ref: #1349

2026-08-01

feat: add forward-chronological mutation-history RPCs and stream Adds getMutationsHistoryForward/getMutationsHistoryReversed to EvitaSessionContract (deprecating the direction-ambiguous getMutationsHistory), and their gRPC counterparts GetMutationsHistoryForward/GetMutationsHistoryPageForward, completing the CDC mutation-history surface alongside the existing reverse-chronological RPCs. - reads forward history via the WAL's live-safe reader (getCommittedLiveMutationStream), not the unsafe one meant only for a WAL that is no longer being actively written - forward paged timeFrame.to is an inclusive stop bound, matching the documented contract; resolves the true last-eligible version via MaterializedVersionBlock.endVersion() rather than startVersion(), which under-selected whenever a checkpoint batches more than one catalog version (applied the same correction to the reverse handler's equivalent resolution, for consistency) - a timeFrame.from moment in the future now yields an empty page instead of clamping down to the newest known block; a from in the past is left checkpoint-granular like the reverse RPC already is, since there is no lag-free way to tell "nothing committed after this" apart from "committed after this but not yet checkpointed" - a forward anchor clamped up to the floor resets sinceIndex to 0 instead of carrying over an index meant for a different version - the forward streaming RPC now closes its mutation stream on normal completion via try-with-resources, matching the paged handlers Ref: #1349

fix: paginate gRPC mutation-history reads without losing or duplicating records GetMutationsHistoryPage groups the change-capture stream by (version, index) via the new ChangeCatalogCaptureRecords helper, so a page never splits an entity/schema mutation from the local-mutation captures it produced. hasNext is computed from a one-record lookahead instead of leaving the last page ambiguous, and the resolved sinceVersion is echoed back to the client so a multi-page traversal can pin its anchor across pages instead of re-resolving "current version" on every call - which previously let concurrent commits shift the anchor and skip or duplicate records between pages. Ref: #1349

2026-07-31

docs: state the evitaDB release version on every proto deprecation Extends the proto documentation convention with the same `since` discipline `@Deprecated(since = "X")` already follows on the Java side (see .claude/rules/deprecation-policy.md): every `[deprecated = true]` field or `option deprecated = true` message must state the evitaDB release (`YYYY.MAJOR`) it became deprecated in, since proto has no structured attribute to hold it. Backfills all 40 existing deprecations across the gRPC surface with their actual version, found by walking each element's git history to the commit that introduced it and resolving the first release tag that contains that commit - the same method tools/audit-deprecated-since.sh uses for Java. Also fixes a bare `[deprecated = true]` on GrpcCreateReflectedReferenceSchemaMutation.faceted that carried no reason at all, unlike its sibling messages in the same file. Ref: #1350

fix: correct gRPC mutation-history CDC bugs and null-filter placement Fixes three defects in the gRPC mutation-history/CDC surface: - GetMutationsHistoryPageRequest ignored `sinceIndex` when set without `sinceVersion`, silently collapsing the anchor to index 0 - the slot reserved for the transaction header - instead of the direction-appropriate default. - The inverse ChangeCatalogCapture converter dropped the `infrastructureMutation` body arm entirely, losing that mutation kind on round-trip. - GetTransactionOverview crashed when a requested catalog version was unknown to history, because the null placeholder for unknown versions was passed straight into a @Nonnull converter. Null-filtering for unknown catalog versions is moved to its true origin, DefaultCatalogPersistenceService, instead of the gRPC service layer - CatalogContract and EvitaSession are pure delegates, and a pre-existing long-running test already calls Catalog#getCatalogVersionDescriptors directly, bypassing any session. Filtering at the persistence layer fixes every caller uniformly rather than relying on each one to remember to filter. Also de-caveats GrpcEvitaSessionAPI.proto and GrpcChangeCapture.proto now that the underlying bugs are fixed, and drops issue-tracker self-references from proto/test comments - issue numbers rot once closed and mean nothing to non-Java client generators reading the same comment. Ref: #1349

docs: rewrite gRPC proto documentation and add buf lint enforcement Documentation-quality audit of evitaDB's .proto surface (issue #1350): rewrites ~387 weak/tautological field, message, enum, and RPC comments across 16 .proto files so the comments are accurate, complete, and usable as the primary API reference for non-Java gRPC clients. Establishes and applies consistent conventions for wrapper-type (google.protobuf.*Value) nullability, units, the two paging models in use (1-indexed pageNumber vs. 0-indexed offset/skip), oneof exclusivity, and deprecation phrasing. Also documents GrpcEvitaAPI.proto, which the issue's own "worst offenders" table flagged but never assigned to a chunk. - Adds .claude/rules/proto-documentation.md, the house convention all of the above follows (wrapper nullability, units, paging models, oneof exclusivity, deprecation phrasing, no Javadoc markup in proto comments since it leaks into other-language codegen, TOBEDONE #issue markers instead of bare TODO). - Regenerates the 294 affected Java files under evita_external_api_grpc/shared/.../generated via protobuf-maven-plugin (protoc 3.25.8); comment-only diffs, verified via per-file field/message/enum/rpc count parity against the pre-edit baseline. - Fixes the matching hand-written doc on EntitySchemaDescriptor.java#withHierarchy to the same corrected wording as GrpcEntitySchema.proto. - Adds buf lint (evita_external_api_grpc/shared/buf.yaml, COMMENTS ruleset only - STANDARD's naming-convention rules would force renaming ~1000 established camelCase fields that mirror the generated Java accessors) plus tools/lint-proto.sh, a Docker-based wrapper needing no local buf install. Wires it into ci-dev.yml as the first step, before JDK setup/build, and adds evita*/**/*.proto to the workflow's path filter so a proto-only push actually triggers the pipeline. - Adds a Claude Code PostToolUse hook (.claude/hooks/lint-proto-on-edit.sh, .claude/settings.json) that runs the same buf lint immediately after Claude edits or writes a .proto file, surfacing violations mid-turn instead of only at CI time. Six real (non-documentation) bugs surfaced while tracing semantics for accurate docs were left unfixed and posted separately to issue #1349, since they aren't in this issue's scope. Ref: #1350

2026-07-30

feat: report telemetry start as an offset and anchor the tree with startedAt `QueryTelemetry.start` carried the server's raw `System.nanoTime()` reading - a monotonic counter with no defined epoch. Over the wire that value is unusable: it cannot be rendered as a time, cannot be compared to anything the client holds, and only becomes meaningful once the client subtracts the root step's start itself. Every remote client had to know that quirk and do the subtraction by hand. The external APIs now normalize `start` to the number of nanoseconds elapsed since the root step began, so the root reports `0` and every node is directly plottable on a timeline. Two conversion points cover all three remote surfaces: `QueryTelemetryDto` (REST and GraphQL) and `GrpcQueryTelemetryBuilder` (gRPC, and therefore `EvitaClient`). The engine object is deliberately left alone. `start` is `final` and the root is handed to the client in `QueryPlan#fabricateExtraResults` before `finalizeTelemetry()` runs, so it cannot be rewritten in place; and for an embedded caller the raw reading is genuinely useful, because it shares the caller's clock. `QueryTelemetry#start` now documents both meanings. Normalizing alone would have removed the last trace of wall-clock information from the payload, so the root step is additionally stamped with `startedAt` - the instant the query began, captured once per query through the new `QueryTelemetry#root(...)` factory. The wall-clock position of any node is `startedAt` plus that node's `start` offset. It travels as `GrpcOffsetDateTime` over gRPC and as an ISO-8601 string over REST and GraphQL, whose `ObjectMapper` has no JavaTimeModule registered and would fail on a raw `OffsetDateTime`. Compatibility: any client treating `start` as absolute was already wrong, and a correct one computed `node.start - root.start`, which is exactly the new value. A new driver against an old server reads raw `nanoTime` as an offset and gets nonsense, but the field was never usable remotely, so nothing that previously worked breaks. One capability is genuinely lost. `start` was the only cross-query timing signal the payload carried, so comparing two queries' starts to prove they overlapped no longer works. That premise is recorded in the `@Disabled` reason of `CatalogGraphQLAsyncQueriesFunctionalTest`, whose overlap assertion relied on it.

2026-07-27

feat: expose engine settings and capabilities via GetEngineSettings management RPC Clients had no reliable way to read the engine-wide default conflict resolution. The only path was getConfiguration, which returns the raw YAML configuration and is refused outright while the engine runs in read-only mode, so a client could not resolve the effective conflict resolution for an entity type: the catalog and entity schemas already travel over the wire, but the engine default that forms the base of the precedence walk did not. Add a GetEngineSettings management RPC returning a curated EngineSettings record. It carries no sensitive values, is therefore unrestricted, and stays readable in read-only mode - the engine implementation deliberately skips the writability assertion that getConfiguration performs. Alongside the conflict resolution it reports the capabilities a client may need to rely on: whether time travel, change data capture, traffic recording and the query cache are enabled. The enabled external APIs with their URLs and the read-only flag are deliberately absent - they already ship in the server status. The record is flat rather than mirroring the configuration sections. Only a small fraction of the configuration is client-actionable - the vast majority are internal tuning knobs a client can neither act on nor benefit from - and which section a value happens to live in is an accident of the server's own configuration history that the caller should not have to know. Two closely related timeouts already sit in different sections today. The wire types were already in place: GrpcConflictResolution and its enums live in GrpcEnums.proto and ConflictResolutionConverter already round-trips them for catalog and entity schemas. The client refuses a response whose conflict resolution is absent rather than letting it decode to the zero enum value, which would read as "no conflict detection at all". No fallback is attempted when an older server answers UNIMPLEMENTED - such a server predates configurable conflict resolution entirely, and silently substituting a default would mask a version mismatch. getConfiguration and its read-only gate are left untouched.

2026-07-17

feat: on-demand export of buffered traffic recording Adds an on-demand export of the currently buffered traffic-recording window, independent of the existing start/stop streaming recorder task. Callers trigger ExportTrafficRecording over the gRPC traffic-recording API and download the resulting ZIP through the file-fetch API; the export takes a consistent one-shot snapshot without starting, stopping or interrupting live recording. Along the way this fixes a family of concurrency and wrap-around correctness bugs in the disk ring buffer: - replaces OS FileChannel region locks with an in-JVM span lock, so the writer no longer crashes with OverlappingFileLockException when a reader holds an overlapping span, and two readers no longer silently drop each other - distinguishes slot validity from session identity: an evicted-and-reused slot passed the validity check, so an export could splice a foreign session's bytes into the archive - makes ringBufferHead/ringBufferTail volatile - fixes three RingBufferInputStream wrap bugs (single-byte off-by-one, bulk read no-advance, skip without seek) - stops lockAndWrite swallowing IOException - fixes isSessionLocationStillInValidArea for a fully-packed buffer (head == tail) - fixes a partial export file leak on failure, a block leak on MemoryNotAvailableException, and a duplicate onNext in stopTrafficRecording - reconciles the filter-predicate and sampling JavaDoc with the actual semantics - removes a stale malformed-FQN TrafficRecorder SPI registration file Adds a JMH suite covering the write, read, flush and index paths plus an export-interference benchmark, and cuts 656-712 B/op from recordQuery by avoiding a redundant label merge. Ref: #1282

2026-07-14

feat: propagate granular conflict resolution through storage, WAL and external APIs Extends the schema & config surface (b23002324) with the remaining serialization and API layers for issue #503: - Storage and WAL Kryo serializers now persist the per-schema conflictResolutionOverride and the entity/catalog-level ConflictResolution. serialVersionUIDs are bumped once with _2026_1/_2026_2 backward-compatible readers so pre-#503 catalogs and unreplayed WAL still load. - gRPC (protos + converters), GraphQL and REST expose the new schema properties and the Set*/Modify* conflict-resolution mutations at full parity. - Fix a data-safety defect: CatalogSchema is registered in both SchemaKryoConfigurer and CatalogHeaderKryoConfigurer; the reader-less header registration overrode the good one (Kryo registers by class, last-write-wins), orphaning every pre-#503 catalog on open with StoredVersionNotSupportedException. The header registration now carries the _2026_1 reader too. - Deduplicate the identical conflict-resolution parse/serialize helpers into ConflictResolutionMutationConverterSupport. - Add the kryo-bwc-audit skill to mechanically verify serialVersionUID bumps and backward-compatible reader coverage across all configurers before a release. Adds unit and functional coverage across every touched layer, including a byte-fixture backward-compatibility regression for the CatalogSchema fix. Ref: #503

2026-07-07

feat: atomic partial rollback for gRPC entity mutations (1:1 with embedded) A single caught EntityUpsertMutation / EntityRemoveMutation failure inside a gRPC transaction previously poisoned the whole transaction — the commit was rejected with a RollbackException — unlike the embedded engine, where the per-entity savepoint reverts only the failing entity and the surviving mutations still commit. Make the gRPC driver behave identically to embedded: - Add SessionFlags.TRANSACTION_CONTROLLED_EXTERNALLY; EvitaSession pins the base nesting level to 1 for such sessions so every RPC mutation runs nested and never self-poisons the transaction (TransactionException still does). EvitaService sets the flag for remote read-write sessions. - Add a `rollback` field to GrpcCloseRequest / GrpcCloseWithProgressRequest; EvitaSessionService honors it by marking the session rollback-only before close, so the commit-vs-discard decision belongs to the client. - Mirror the embedded nest machinery in EvitaClientSession and route updateCatalog through session.execute(...): the transaction is marked rollback-only only when an exception escapes the root frame uncaught, and that decision is sent to the server at close time. Add gRPC driver tests covering caught-and-continue (survivors commit, no orphan facet/price index residue), uncaught escape (whole transaction rolls back), direct read-write session poisoning, and the close-with-progress rollback path. Ref: #760

2026-05-27

Merge pull request #1192 from FgForrest/1161-histograms-add-range-source-attribute-support-and-multi-histogram-schema feat: range source-attribute support and multi-histogram schema for reference histograms

feat: range source attribute support and multi-histogram schema for reference histograms Allow reference histograms to bucket over NumberRange-typed source attributes, distributing each range across every bucket its endpoints overlap, and support multiple named histograms per reference/scope with per-histogram `assignedWhen` partition selectors. Covers the full stack: annotations, schema DTOs/builders, mutations, engine indexing and the range-aware bucket sweep, gRPC/GraphQL/REST surfaces, and Kryo/WAL serializers. Tests add an oracle-based per-bucket assertion for range histograms that independently re-derives expected occurrences from the seeded ranges, replacing the prior tautological `sum == overallCount` check. Ref: #1161

2026-05-26

feat: add FETCHING_REFERENCE_BODIES query telemetry phase Introduces a single aggregate telemetry phase that wraps the orchestration of loading all referenced entities for a result page (per-reference predicate setup, referenced primary key collection, deduplication, recursive dispatch and the nested storage reads exposed as FETCHING_REFERENCES children), pushed inside ReferencedEntityFetcher around the prefetch call. Also fixes a telemetry nesting bug where the reference-index step was popped from the nested query context instead of the execution context, which flattened the FETCHING subtree and skewed per-step timings. Includes the gRPC enum value (proto + EvitaEnumConverter) and the regenerated gRPC stubs.

2026-05-15

fix: backport CDC gRPC streaming fixes and heartbeat protocol from dev Brings release_2025-7's CDC subscriber stack in line with dev so K8s rolling deploys are safe in both directions (new-client/old-server and old-client/ new-server) and so long-lived streams survive Armeria timeout extension and catalog replacement without leaking or deadlocking. Wire-neutral hardening (no proto bump): - Switch all CDC/streaming response timeouts to TimeoutMode.SET_FROM_NOW so a silent stream unblocks within streamingTimeout of the last event instead of accumulating an unbounded EXTEND deadline. - Re-key EvitaClient.activePublishers via a sealed CapturePublisherKey (SystemCaptureKey / CatalogBoundCaptureKey) so two catalogs sharing the same filter criteria no longer collide. - Once-only stream finalization on the server side via AtomicBoolean + markStreamDead, with non-blocking cancel handlers (whenComplete instead of Future.get) and deferred subscription.cancel() via CompletableFuture.runAsync to avoid event-loop blocking and catalog-replacement deadlock. - Client-side serverSideClosed flag prevents re-cancelling a stream the server has already terminated; delegate.close() runs on subscription.executorService to break the catalog-replacement deadlock when close() fires on the gRPC event loop. Heartbeat protocol (proto-additive, backward compatible): - Add GrpcHeartBeat message, HEARTBEAT response-type enum value, and heartBeat field on both register-capture responses; new HeartBeat record and HeartBeatSensor interface for the client API. - Server-side AbstractChangeCaptureSubscriber drives a periodic heartbeat task and emits ack/heartbeat/change frames through emitOnNext with proper setOnCancelHandler / setOnCloseHandler wiring on the service-method thread. - Replace inline subscribers in EvitaService and EvitaSessionService with the new subscriber/* classes. Subscription-id and heartbeat extraction are deliberately split into two abstract methods (extractSubscriptionId / extractHeartBeat) instead of one combined method that synthesizes a sentinel HeartBeat for legacy servers. A synthesized heartbeat with lastObservedVersion=0 would clobber any persisted catalog-version checkpoint maintained by a HeartBeatSensor on first connect to a legacy server. The split lets the subscription id flow through while suppressing the spurious heartbeat dispatch. Catalog-side ChangeCatalogCaptureSubscriber omits the SemVer clientVersion parameter present on dev because release_2025-7's ChangeCaptureConverter does not have the SemVer-aware toGrpcChangeCatalogCapture overload; the single-arg variant is used instead. Out of scope (dev-only features intentionally not backported): HostSystemEvent capture criteria, BackupCatalogWithProgress streaming RPCs, GrpcCatalogVersionAtResponse reshape, SemVer-aware converter overload, application-level lastKnownVersion persistence.

2026-05-08

feat: coalesce GraphQL/REST schema refresh on catalog schema changes Introduces a new `HostSystemEvent.CatalogSchemaUpdated` variant that is emitted exactly once per session close (WARMING_UP) or per transaction commit (ALIVE), regardless of how many `ModifyCatalogSchemaMutation`s were applied. GraphQL and REST refreshing observers no longer rebuild their schemas on every per-mutation engine event; instead they react to the coalesced host event, eliminating the historical refresh storm. Key changes: - New `HostSystemEvent.CatalogSchemaUpdated` host event with gRPC, GraphQL descriptor, and REST exposure. - `Evita#replaceCatalogReference` and `EvitaSession#executeTerminationSteps` emit the coalesced event; `ExpandedEngineState#replaceCatalogReference` returns a boolean indicating whether the schema actually advanced. - `SystemGraphQLRefreshingObserver` / `SystemRestRefreshingObserver` rebuild only on the host event, not on per-mutation captures. - WAL-replay no longer re-emits a schema-update event (the catalog is not yet in the live view, so live observers cannot have seen the pre-replay schema — preventing a double rebuild). - Tests trimmed to coalescing-specific contracts; full dispatch coverage is now owned by the GraphQL/REST functional subscription tests. Ref: #1153

2026-05-07

feat: introduce HostSystemEvent on system CDC stream Adds HostSystemEvent — a host-local, advisory, non-replicable event class on the system CDC stream — and surfaces it through the existing ChangeSystemCaptureCriteria pattern. First variant CatalogInstalledIntoLiveView is emitted when a real Catalog replaces an UnusableCatalog placeholder in the live view, fixing the gap where auto-upgraded catalogs were internally usable but externally invisible (GraphQL/REST endpoints not registered, Lab UI stuck in BEING_ACTIVATED) until server restart. API model: - new SystemCaptureArea enum (ENGINE, INFRASTRUCTURE) - new ChangeSystemCaptureCriteria with builder - ChangeSystemCaptureRequest gains criteria[] and engineArea() / infrastructureArea() builder methods - new sealed HostSystemEvent + CatalogInstalledIntoLiveView record - SystemCaptureBody marker implemented by EngineMutation and HostSystemEvent; ChangeSystemCapture#body widened accordingly - default-criteria divergence vs catalog stream: system stream defaults to ENGINE only; INFRASTRUCTURE must be opted in Engine: - Evita#replaceCatalogReference emits CatalogInstalledIntoLiveView via SystemChangeObserver on first install (UnusableCatalog → real) - SystemChangeObserver gains processHostEvent path bypassing the recent-events cache (live-tail only) - ChangeSystemCaptureSharedPublisher honors criteria filter and enforces live-tail-only delivery for INFRASTRUCTURE - SystemAreaPredicate family added to MutationPredicateFactory External APIs: - SystemGraphQLRefreshingObserver / SystemRestRefreshingObserver opt into infrastructureArea() and react to CatalogInstalledIntoLiveView by registering the catalog endpoint without restart - gRPC: GrpcChangeSystemCapture grows HostSystemEvent oneof branch, GrpcSystemCaptureArea + GrpcChangeSystemCaptureCriteria carrier - REST/GraphQL descriptors and serializers updated end-to-end - Lab UI subscribers will opt in via the new criteria Tests cover boot-time auto-upgrade reproduction, ordering with the preceding UpgradeCatalogFormatMutation, default vs explicit criteria behavior, and live-tail-only semantics for late subscribers. Ref: #1151

2026-05-05

feat: support fetching full price range for master product variants Introduce priceForSaleMin/priceForSaleMax fields on entity output that expose the lowest and highest sellable prices across a master product's variants. New PriceRangeForSale and PriceRangeForSaleWithAccompanyingPrices contracts are surfaced through the API, GraphQL, REST and gRPC layers, with matching documentation, examples and tests. Ref: #1086

2026-04-28

feat: enforce WAL-first discipline for engine state to prevent version drift Introduce explicit catalog lifecycle states (MISSING, OUT_OF_DATE, BEING_UPGRADED) and the engine mutations that drive transitions between them, so every change to engine state goes through the WAL before being applied. This eliminates VersionWAL drift between in-memory engine state and the persisted log, and surfaces previously silent inconsistencies as explicit, recoverable errors. Key changes: - Add MarkCatalogMissingMutation/UpgradeCatalogFormatMutation along with their operators, converters (gRPC, GraphQL, REST) and serializers. - Add CatalogMissingException, CatalogRequiresUpgradeException and CatalogBeingUpgradedException with mappings in JsonApiExceptionHandler. - Bump engine state to EngineStateSerializer_2026_1 with backward-compat dispatch via the existing _2025_6 serializer; persist CatalogInventoryDivergence and UnprocessedTransactionRecord in WAL. - Wire DefaultUpgradeExecutor / UpgradeExecutor SPI into EngineTransactionManager so format upgrades replay through the WAL. - Reorganize WAL exceptions under spi/store/engine/exception and remove the now-obsolete EngineMutationLogCorruptedException and the EngineTransactionMutationWithWalFileReference duplicate. - Add functional and unit tests covering boot-time WAL divergence, forward replay, mutation operators, converters, and exception handling. Ref: #1137

2026-04-23

Merge remote-tracking branch 'origin/dev' into 8-compute-dynamic-set-of-attribute-histogram-for-references # Conflicts: # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/GrpcEvitaSessionAPI.java # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/GrpcExtraResults.java # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/GrpcExtraResultsOrBuilder.java

fix: return real catalog and schema version for fresh read-only session EvitaClientSession#getCatalogVersion returned 0 on a brand-new EvitaClient that had only opened a read-only session, because the shared EvitaEntitySchemaCache had not yet been populated by any server round-trip. Zero is also a legitimate version for a warming-up catalog, so the numeric value alone cannot distinguish the two. The cache now tracks an explicit `initialized` flag that is flipped the first time a server response updates the catalog/schema versions. getCatalogVersion forces a fetchCatalogSchema round-trip when the cache is still uninitialized. GrpcCatalogSchemaResponse is extended with catalogVersion and catalogSchemaVersion so a single getCatalogSchema call populates both versions at once; the server-side EvitaSessionService fills them in. Also adds a regression test in EvitaClientReadOnlyTest that opens a fresh client and verifies the first getCatalogVersion call returns the actual server value rather than zero.

fix: return real catalog and schema version for fresh read-only session EvitaClientSession#getCatalogVersion returned 0 on a brand-new EvitaClient that had only opened a read-only session, because the shared EvitaEntitySchemaCache had not yet been populated by any server round-trip. Zero is also a legitimate version for a warming-up catalog, so the numeric value alone cannot distinguish the two. The cache now tracks an explicit `initialized` flag that is flipped the first time a server response updates the catalog/schema versions. getCatalogVersion forces a fetchCatalogSchema round-trip when the cache is still uninitialized. GrpcCatalogSchemaResponse is extended with catalogVersion and catalogSchemaVersion so a single getCatalogSchema call populates both versions at once; the server-side EvitaSessionService fills them in. Also adds a regression test in EvitaClientReadOnlyTest that opens a fresh client and verifies the first getCatalogVersion call returns the actual server value rather than zero.

2026-04-22

feat: expose name variants on HistogramIndexDefinition Extract a NamedContract super-interface from NamedSchemaContract that carries only name / name-variant methods (no description), and have HistogramIndexDefinition implement it. Variants are always server-generated via NamingConvention.generate(name) — never accepted from client input — so the mutation wire format (ScopedHistogramIndexDefinition) stays unchanged. Add ReferenceSchemaContract.getHistogramIndexDefinitionByName(scope, name, convention) returning Optional<HistogramIndexDefinition>, backed by a per-scope variant index in ReferenceSchema analogous to the reference name index on EntitySchema. Propagate the new field through the outbound surfaces: gRPC proto GrpcScopedHistogramIndexDefinition gains an output-only nameVariants field, GraphQL ReferenceSchemasBucketedDataFetcher emits variants, REST SchemaJsonSerializer serializes them, and the Kryo EntitySchemaSerializer persists them. No backward-compat serializer — the histogram feature is within-release. Ref: #8

2026-04-17

refactor: unify facet and reference summary via adapter pattern Introduce FacetSummaryAdapter, ReferenceSummaryAdapter, and ReferenceSummaryResultAdapter to share the producer pipeline between facet and reference summaries. Adds a new GraphQL FacetSummaryDataFetcher and a Functions utility, plus a backward-compatibility test for the facet summary surface. Ref: #8

2026-04-16

feat: add support for dynamic reference histograms in extra results Refs: #8

2026-04-10

Merge branch '8-compute-dynamic-set-of-attribute-histogram-for-references' into 8-compute-dynamic-set-of-attribute-histogram-for-references-extra-results # Conflicts: # evita_api/src/main/java/io/evitadb/api/requestResponse/schema/ReferenceSchemaContract.java # evita_api/src/main/java/io/evitadb/api/requestResponse/schema/ReferenceSchemaEditor.java # evita_external_api/evita_external_api_core/src/main/java/io/evitadb/externalApi/api/catalog/schemaApi/model/mutation/reference/CreateReferenceSchemaMutationDescriptor.java # evita_external_api/evita_external_api_core/src/main/java/io/evitadb/externalApi/api/catalog/schemaApi/model/mutation/reference/CreateReflectedReferenceSchemaMutationDescriptor.java # evita_query/src/main/java/io/evitadb/api/query/QueryConstraints.java

feat: ReferenceSummary extra result implementation with backward-compatibility to FacetSummary in APIs Refs: #8

Merge branch '8-compute-dynamic-set-of-attribute-histogram-for-references' into 109-distributed-database-support # Conflicts: # .gitignore # CLAUDE.md # evita_api/src/main/java/io/evitadb/api/requestResponse/schema/builder/ReferenceSchemaBuilder.java # evita_api/src/main/java/io/evitadb/api/requestResponse/schema/builder/ReflectedReferenceSchemaBuilder.java # evita_engine/src/main/java/io/evitadb/core/catalog/Catalog.java # evita_engine/src/main/java/io/evitadb/core/query/algebra/prefetch/PrefetchFormulaVisitor.java # evita_engine/src/main/java/io/evitadb/core/query/algebra/reference/ReferencedEntityIndexPrimaryKeyTranslatingFormula.java # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/GrpcEvitaDataTypes.java # evita_test/evita_functional_tests/src/test/java/io/evitadb/api/file/FileForFetchTest.java # evita_test/evita_functional_tests/src/test/java/io/evitadb/core/catalog/CatalogTest.java

2026-03-27

feat: implement conditional bucket indexing with histogram index infrastructure Introduce HistogramIndex and supporting types (SimpleHistogramIndex, LocalizedHistogramIndex, HistogramCapableEntityIndex) for dynamic attribute histogram computation on references. Refactor expression trigger system to support both facet and histogram triggers through shared AbstractExpressionIndexTrigger base. Reorganize index mutation packages (index/mutation/index → index/mutation/local) and expression trigger packages for cleaner separation of concerns. Add cross-entity trigger support via CrossEntityTriggerIndex and LocalTriggerIndex. Generalize ReevaluateExpressionMutation to handle both facet and histogram expression re-evaluation. Extract common expression operator logic into AbstractBinaryOperator and AbstractUnaryOperator base classes. Ref: #8

2026-03-23

feat: add bucketed histogram schema support for reference attributes Implement full-stack schema support for bucketed histogram indexing on reference attributes. This includes new HistogramIndexDefinition DTO, SetReferenceSchemaBucketedMutation, and updates across all 8 layers: contracts, DTOs, builders, mutations, external APIs (gRPC/GraphQL/REST), Kryo serializers, and WAL serializers. Ref: #8

2026-03-05

feat: add per-scope faceted partially configuration with expression support for references Introduce ScopedFacetedPartially record to allow configuring faceted behavior per scope with optional Expression filters. This enables dynamic attribute histogram computation for references by specifying which facet values participate in histogram aggregation. Key changes: - Add ScopedFacetedPartially to ReferenceSchemaContract and DTOs - Extract AbstractReferenceSchemaBuilder for shared builder logic - Update SetReferenceSchemaFacetedMutation to support expressions - Add support across all external APIs (gRPC, GraphQL, REST) - Add Kryo/WAL serializers with backward compatibility (2026_1) - Update skill documentation with converter serialization patterns - Restore backward compatibility for @Reference annotation Ref: #8

2026-02-23

feat: add per-scope reference indexed components configuration Introduces ScopedReferenceIndexedComponents to allow independent configuration of which reference components (entity, group, or both) are indexed in each scope. Updates reference schema contracts, builders, mutations, and all external API layers (gRPC, GraphQL, REST). Adds backward-compatible serializers for storage and WAL. Ref: #1088

2026-02-13

Merge remote-tracking branch 'refs/remotes/origin/1062-add-aggregated-crc32-checksum-to-wal-log' into 109-distributed-database-support

Merge branch 'dev' into 1062-add-aggregated-crc32-checksum-to-wal-log # Conflicts: # evita_store/evita_store_server/src/main/java/io/evitadb/store/wal/supplier/TransactionMutationWithLocation.java # evita_test/evita_functional_tests/src/test/java/io/evitadb/core/session/task/SessionKillerTest.java # evita_test/evita_functional_tests/src/test/java/io/evitadb/store/wal/CatalogWriteAheadLogIntegrationTest.java # evita_test/evita_functional_tests/src/test/java/io/evitadb/store/wal/EngineWriteAheadLogTest.java # evita_test/evita_functional_tests/src/test/java/io/evitadb/utils/StringUtilsTest.java

2026-02-11

refactor: relocate `TransactionMutation` to `mutation.infrastructure` and enhance traffic recording metadata Moved `TransactionMutation` to a new `mutation.infrastructure` package to align with package structure conventions. Added new metadata fields to traffic recording containers, such as `sessionRecordsCount`, `finishedWithError`, and `queryDescription`, and updated JavaDocs for accuracy. Fixed typos and enhanced `equals()` methods in relevant classes.

2026-02-05

refactor: Introduce streaming support for catalog backup methods - Added `backupCatalogWithProgress` and `fullBackupCatalogWithProgress` methods for streaming backup progress updates. - Implemented `BackupProgressObserver` for real-time progress handling. - Updated gRPC descriptors and service implementations to support streaming. - Modified synchronous backup methods to utilize the newly implemented streaming capabilities. - Adjusted error handling for better resilience against interruptions and server-side errors.

2026-01-26

refactor: replace EvitaInternalError with specific exceptions and enhance histogram logic Replaced `EvitaInternalError` with more specific exception types (`EvitaInvalidUsageException`, `GenericEvitaInternalError`, etc.) for better error handling and clarity. Improved `relativeFrequency` normalization logic for equalized histograms and updated associated Javadoc, tests, and documentation. Refs: #762 Signed-off-by: Jan Novotný <novotnaci@gmail.com>

2026-01-25

feat: add relative frequency calculations for histogram buckets Introduced `relativeFrequency` field for histogram buckets to enhance UI visualization. Implemented calculation support for both standard and equalized histograms in data crunchers, API descriptors, and serialization layers. Refs: #762 Signed-off-by: Jan Novotný <novotnaci@gmail.com>

2026-01-23

feat: Introduce alternative calculation for histograms There is a more UX-friendly version of histogram calculations documented here: https://www.howdoi.me/blog/slider-scale.html, which is also discussed in more detail in the article https://baymard.com/blog/slider-interfaces ... we are discussing its support in the team. Refs: #762 Signed-off-by: Jan Novotný <novotnaci@gmail.com>

2026-01-09

Merge branch 'refs/heads/dev' into 109-distributed-database-support

fix: Subscription should be kept alive The subscriptions gets closed after certain time of inactivity (can be configured), but it would be much better if the server regurarly sent heartbeet messages in regular intervals over the connection so that it never time out. Also we found out that when subscription is recreated from close method of the previous subscriber it doesn't work probably due to the fact that it's handled in a shared thread which received signal of subscription drop. We should invoke delegate subscribe methods asynchronously in separate executor to avoid this problem. Refs: #1057 Signed-off-by: Jan Novotný <novotnaci@gmail.com>

2025-12-26

feat: automatic backup to S3 Distinguishing files that will be managed by outside logic (backup retention) rather than internal export service retention logic. Refs: #1000

2025-12-18

feat: automatic backup to S3 Added checksum calculation. Refs: #1000

2025-12-10

feat: added timestamp field to all change capture events Timestamp comes handy when only local mutation is result of the predicate filter.

fix: fixed date range CDC localization

2025-11-19

Merge branch 'dev' into 977-support-filtering-schema-captures-by-containername # Conflicts: # evita_engine/src/main/java/io/evitadb/core/Catalog.java # evita_external_api/evita_external_api_core/src/main/java/io/evitadb/externalApi/api/catalog/schemaApi/model/mutation/catalog/CreateEntitySchemaMutationDescriptor.java # evita_external_api/evita_external_api_core/src/main/java/io/evitadb/externalApi/api/catalog/schemaApi/model/mutation/catalog/ModifyEntitySchemaMutationDescriptor.java # evita_external_api/evita_external_api_graphql/src/main/java/io/evitadb/externalApi/graphql/api/catalog/schemaApi/builder/EntitySchemaSchemaBuilder.java # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/EvitaSessionServiceGrpc.java # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/GrpcEvitaSessionAPI.java # evita_functional_tests/src/test/java/io/evitadb/api/EvitaTransactionalFunctionalTest.java # evita_functional_tests/src/test/java/io/evitadb/externalApi/api/catalog/schemaApi/resolver/mutation/DelegatingLocalCatalogSchemaMutationConverterTest.java # evita_functional_tests/src/test/java/io/evitadb/externalApi/api/catalog/schemaApi/resolver/mutation/LocalCatalogSchemaMutationInputAggregateConverterTest.java # evita_functional_tests/src/test/java/io/evitadb/externalApi/api/catalog/schemaApi/resolver/mutation/engine/ModifyCatalogSchemaMutationConverterTest.java # evita_functional_tests/src/test/java/io/evitadb/externalApi/graphql/api/catalog/schemaApi/CatalogGraphQLUpdateCatalogSchemaQueryFunctionalTest.java # evita_store/evita_store_server/src/main/java/io/evitadb/store/catalog/DefaultCatalogPersistenceService.java

2025-10-31

doc: documented entity mutations Signed-off-by: Jan Novotný <novotnaci@gmail.com>

2025-10-21

fix: It's not possible to store detached builder instances on client side Issue is documented and reproducible by tests: - io.evitadb.api.proxy.EntityEditorProxyingFunctionalTest#shouldSetReferenceGroupAsNewlyCreatedEntity - io.evitadb.api.proxy.EntityEditorProxyingFunctionalTest#shouldSetReferenceGroupByIdAndUpdateIt The root cause is that the client works with locally assigned primary keys for reference keys, which are rewritten by primary keys assigned on the server side (final primary keys). This information is not propagated to the client though (there is no way actually). We need to be able to propagate this information via. gRPC API to the client so that it could replace its former internal PKs with real PKs from the server side and continue working (and sending updates back to the server) with the correct primary keys. In order to do that we need to apply backward incompatible change on `EvitaSessionContract` level - instead of using `EntityReference` records in the methods, we need to stick to common interface `EntityReferenceContract`. This may break some clients though. Refs: #979 Signed-off-by: Jan Novotný <novotnaci@gmail.com>

2025-10-20

feat: Support filtering schema captures by containerName Team has agreed that it makes sense to filter schema mutations by containerName in the similar fashion as data captures. Because of evitaLab view we also need to create new method for acquiring transactional mutations by the set of catalogVersions. Refs: #977 Signed-off-by: Jan Novotný <novotnaci@gmail.com>

2025-10-17

feat: Support filtering schema captures by containerName Team has agreed that it makes sense to filter schema mutations by containerName in the similar fashion as data captures. Refs: #977 Signed-off-by: Jan Novotný <novotnaci@gmail.com>

2025-10-02

feat: Support for multiple references to the same entity with different attribute sets Added missing internal PK in gRPC communication. Refs: #906 Signed-off-by: Jan Novotný <novotnaci@gmail.com>

2025-09-23

Merge branch 'dev' into 906-support-for-multiple-references-to-the-same-entity-with-different-attribute-sets

feat: support for filtering stream by date and time range

Merge branch 'dev' into 906-support-for-multiple-references-to-the-same-entity-with-different-attribute-sets # Conflicts: # evita_functional_tests/src/test/java/io/evitadb/core/cdc/CatalogChangeObserverTest.java

2025-09-19

fix: added missing transaction mutation body

2025-09-16

Merge branch 'dev' into 906-support-for-multiple-references-to-the-same-entity-with-different-attribute-sets # Conflicts: # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/GrpcEntityReference.java # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/GrpcReference.java # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/GrpcReferenceAttributeMutation.java # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/GrpcReferenceOrBuilder.java # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/GrpcRemoveReferenceGroupMutation.java # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/GrpcRemoveReferenceMutation.java # evita_external_api/evita_external_api_grpc/shared/src/main/java/io/evitadb/externalApi/grpc/generated/GrpcSetReferenceGroupMutation.java

fix: fixed mutation listing Corrected CDC index calculation and handling in gRPC. Disabling session kills in `run-server.sh` so that we can test gRPC API more easily.