Proto commits in apache/tika

These 31 commits are when the Protocol Buffers files have changed:

2026-06-26

TIKA-4764 and TIKA-4763 merge conflicts (#2910)

TIKA-4763 fix e2e

TIKA-476x - fix merge conflicts, simplify tika.proto

2026-06-06

TIKA-4663 - make markdown the default content handler in tika-app, tika-server, and the async CLI (#2877) * TIKA-4663 - make markdown the default content handler in tika-app, tika-server, and the async CLI

2026-05-01

TIKA-4722: Add parse_context_json field to FetchAndParseRequest for per-request ParseContext configuration (#2797) - Replace handler_type field with generic parse_context_json string field (field 5) - parse_context_json accepts a JSON object mapping parse-context component names to their configs Example: {"basic-content-handler-factory": {"type": "HTML"}} - TikaGrpcServerImpl iterates the JSON fields and calls parseContext.setJsonConfig() for each - Allows overriding any registered parse-context component per request, not just content handler type - e2e test uses parse_context_json to request HTML and TEXT output and verifies the difference Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

TIKA-4722: Add parse_context_json field to FetchAndParseRequest for per-request ParseContext configuration - Replace handler_type field with generic parse_context_json string field (field 5) - parse_context_json accepts a JSON object mapping parse-context component names to their configs Example: {"basic-content-handler-factory": {"type": "HTML"}} - TikaGrpcServerImpl iterates the JSON fields and calls parseContext.setJsonConfig() for each - Allows overriding any registered parse-context component per request, not just content handler type - e2e test uses parse_context_json to request HTML and TEXT output and verifies the difference Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

2026-03-02

TIKA-4606: Upgrade Apache Ignite from 2.x to 3.x (fresh) (#2654) * TIKA-4606: Upgrade Apache Ignite from 2.x to 3.x - Replace Ignite 2.x embedded server with Ignite 3.x IgniteServer API - Replace Ignite 2.x cache API with Ignite 3.x KeyValueView/Table API - Remove H2 dependency (Ignite 3.x uses built-in Calcite SQL engine) - Remove ignite-spring, ignite-indexing, ignite-core 2.x dependencies - Add ignite-api, ignite-client, ignite-runner, ignite-sql-engine, ignite-table 3.x dependencies - Use text block for HOCON configuration in IgniteStoreServer - Add IgniteConfigStoreConfig with tableName/replicas/partitions (replacing cacheName/cacheMode) - Add IgniteStoreServer lifecycle to TikaGrpcServerImpl (start/shutdown) - Add TikaGrpcServerImpl.shutdown() and wire into TikaGrpcServer.stop() - Add emitter_id field to FetchAndParseRequest proto - Add NO_EMIT null check in EmitHandler to skip emission when no emitter specified - Add Ignite 3.x convergence dependencies to tika-parent (asm, picocli, snakeyaml, javax.validation) - Update tika-grpc shade plugin to use Ignite 3.x artifacts - Update all Ignite config JSON files from 2.x format to 3.x format Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * TIKA-4606: Add e2e tests for Ignite 3.x - Update e2e Ignite config files from 2.x (cacheName/cacheMode) to 3.x (tableName/replicas/partitions) - Add tika-config-ignite-local.json for local development testing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * TIKA-4606: Fix Ignite 3.x unit tests - add distribution zone, fix module opens, pin jetbrains annotations - Add java.base/java.lang.reflect to --add-opens (required by Ignite ProxyMarshaller) - Pin org.jetbrains:annotations:26.0.2-1 in dependencyManagement to fix convergence - Use startAsync().get() to expose real startup errors (works around IGNITE-22721 bug) - Create single-replica distribution zone for single-node test cluster - Fix any other test failures Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * TIKA-4606: Fix tika-grpc convergence, smoke test and ServiceLoader for Ignite 3.x - Add ServicesResourceTransformer to shade plugin to merge META-INF/services (required for Ignite 3.x ConfigurationModule ServiceLoader discovery in fat jar) - Pin Ignite 3.x transitive deps in tika-parent: micronaut 3.10.4, snakeyaml 2.5, swagger-annotations 2.2.38, kotlin-stdlib 2.2.0, jakarta.inject-api 2.0.1, jetbrains annotations 26.0.2-1 (fixes DependencyConvergence/RequireUpperBoundDeps enforcer failures when tika-grpc pulls in tika-pipes-config-store-ignite) - Fix smoke test config: use my_tika_cache (underscores) instead of my-tika-cache (hyphens are invalid SQL identifiers in Ignite 3.x) - Smoke test verified: tika-grpc starts with ignite config, gRPC health check returns SERVING on port 50052 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * TIKA-4606: Remove unnecessary NO_EMIT guard in EmitHandler.emit() The shouldEmit() method already handles the PASSBACK_ALL strategy (returns false, data is returned to client upstream). Adding an early return for NO_EMIT inside emit() was unnecessary and returned PARSE_SUCCESS while dropping parsed data — worse than the existing EmitterNotFoundException path which correctly signals misconfiguration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * TIKA-4606: Fix run-dev.sh warnings and dev config issues - Fix exec-maven-plugin param: additionalJvmArgs -> jvmArgs (unknown param warning in exec-maven-plugin 3.6.3) - Add missing --add-opens for java.lang.reflect and jdk.management required by Ignite 3.x in dev mode - Remove non-existent tika-pipes-ignite plugin root from dev-tika-config.json (Ignite integration is a config-store, not a pf4j plugin) - Fix tableName: tika-config-cache -> tika_config_cache (hyphens invalid as SQL identifiers in Ignite 3.x) - Use replicas=1 for single-node dev setup Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * TIKA-4606: Use java.io.tmpdir as default Ignite work directory /var/cache/tika/ignite-work requires root access and is unsuitable as a default for development or non-containerized use. Fall back to a subdirectory of the system temp dir, which is always user-writable. The ignite.work.dir system property still overrides the default. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * TIKA-4606: Fix run-dev.sh and IgniteStoreServer for dev mode - Pass Thread.currentThread().getContextClassLoader() as serviceLoaderClassLoader to IgniteServer.builder() so exec:java's classloader is used for ServiceLoader discovery (fixes 'No configuration modules were loaded' in dev mode) - Expand MAVEN_OPTS in run-dev.sh with all --add-opens required by Ignite 3.x (java.io, java.lang.reflect, java.lang.invoke, java.util.concurrent, etc.) since exec:java does not support jvmArgs for module opens - Remove dead <jvmArgs> block from pom.xml dev profile (unsupported by exec:java) - RocksDB NoClassDefFoundError on shutdown is harmless (classloader unloading races with RocksDB shutdown hooks during JVM exit) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * remove .java-version --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

2026-02-27

TIKA-4606: Upgrade Apache Ignite from 2.x to 3.x - Replace Ignite 2.x embedded server with Ignite 3.x IgniteServer API - Replace Ignite 2.x cache API with Ignite 3.x KeyValueView/Table API - Remove H2 dependency (Ignite 3.x uses built-in Calcite SQL engine) - Remove ignite-spring, ignite-indexing, ignite-core 2.x dependencies - Add ignite-api, ignite-client, ignite-runner, ignite-sql-engine, ignite-table 3.x dependencies - Use text block for HOCON configuration in IgniteStoreServer - Add IgniteConfigStoreConfig with tableName/replicas/partitions (replacing cacheName/cacheMode) - Add IgniteStoreServer lifecycle to TikaGrpcServerImpl (start/shutdown) - Add TikaGrpcServerImpl.shutdown() and wire into TikaGrpcServer.stop() - Add emitter_id field to FetchAndParseRequest proto - Add NO_EMIT null check in EmitHandler to skip emission when no emitter specified - Add Ignite 3.x convergence dependencies to tika-parent (asm, picocli, snakeyaml, javax.validation) - Update tika-grpc shade plugin to use Ignite 3.x artifacts - Update all Ignite config JSON files from 2.x format to 3.x format Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

2026-02-26

TIKA-4606: Upgrade Apache Ignite from 2.x to 3.x - Upgraded ignite-core 2.16.0 to ignite-runner 3.1.0 - Migrated configuration from IgniteConfiguration API to HOCON-based config files - Updated API usage from IgniteCache to new KeyValueView API - Fixed ExtensionConfigDTO to use Ignite 3.x Mapper annotations - Simplified IgniteStoreServer to synchronous embedded mode - Fixed EmitHandler: added null check for NO_EMIT scenario to prevent NPE - Updated gRPC proto: added emitter_id field to FetchAndParseRequest - Updated TikaGrpcServerImpl: proper lifecycle management for IgniteStoreServer - Added JVM --add-opens flags for Java 17+ compatibility - Updated unit tests (IgniteConfigStoreTest) for Ignite 3.x embedded server pattern Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

2025-12-30

TIKA-4606: Upgrade Apache Ignite from 2.x to 3.x - Upgraded ignite-core 2.16.0 -> ignite-runner 3.1.0 - Migrated from IgniteConfiguration to hocon-based config - Updated IgniteConfigStore to use new KeyValueView API - Fixed IgniteStoreServer for embedded mode - Updated ExtensionConfigDTO to use Ignite 3 Mapper - Added required JVM --add-opens flags for Java 17+ - Fixed EmitHandler NPE for NO_EMIT scenario - Added emitter_id to FetchAndParseRequest proto - Integrated e2e tests into parent build - Added local server mode for CI (no Docker required) - Fixed gRPC channel resource leak in tests - All 11 unit tests passing, e2e test passing

2025-12-29

TIKA-4595: Dynamic Fetcher/Emitter Management with ConfigStore Support (#2489) * TIKA-4595: Add dynamic fetcher/emitter management API to PipesClient - Added SAVE_FETCHER, DELETE_FETCHER, LIST_FETCHERS, GET_FETCHER commands - Added SAVE_EMITTER, DELETE_EMITTER, LIST_EMITTERS, GET_EMITTER commands - Implemented PipesClient public API methods for runtime configuration - Implemented PipesServer command handlers - Added deleteComponent() and getComponentConfig() to AbstractComponentManager - Added wrapper methods to FetcherManager and EmitterManager - Added remove() method to ConfigStore interface and implementations - All tests passing * Fix gRPC saveFetcher to propagate to PipesClient's forked PipesServer - saveFetcher now calls both fetcherManager.saveFetcher() and pipesClient.saveFetcher() - This ensures fetchers are available in the forked PipesServer process - Implemented deleteFetcher to call both managers as well - Fixes FetcherNotFoundException when using dynamic fetchers via gRPC The issue was that fetchers saved via gRPC were only stored in the gRPC server's FetcherManager, but when pipesClient.process() forks a new PipesServer process, that process has its own FetcherManager and doesn't have access to the dynamically created fetchers. Now both are updated. * Add file-based ConfigStore for shared config between gRPC and PipesServer - Created FileBasedConfigStore that persists to JSON file - Created FileBasedConfigStoreFactory with @Extension annotation - Updated PipesServer.initializeResources() to create and use ConfigStore - Both gRPC server and forked PipesServer can now share fetcher configs via file This enables dynamic fetcher management across JVM processes: 1. gRPC saves fetcher → writes to config file 2. PipesServer starts → reads from same file 3. Both JVMs share the same fetcher configuration * Handle 'file' ConfigStore type as built-in (not plugin) - Added direct handling for 'file' type in ConfigStoreFactory.createConfigStore() - File-based store is in core, not a plugin, so needs special handling - Avoids ClassNotFoundException when trying to load 'file' as a class name - Also added remove() method to IgniteConfigStore for interface compliance * Make ExtensionConfig Serializable for socket communication ExtensionConfig is sent over sockets between PipesClient and PipesServer, so it needs to implement Serializable. Records can implement Serializable and all fields (String, String, String) are already serializable. Fixes NotSerializableException when calling saveFetcher via gRPC. * Add embedded Ignite server architecture - Created IgniteStoreServer class that runs as embedded server - TikaGrpcServer starts Ignite server on startup (if ignite ConfigStore configured) - IgniteConfigStore now acts as client-only (clientMode=true) - No external Ignite dependency needed in Docker/Kubernetes - Server runs in background daemon thread within tika-grpc process - Clients (gRPC + forked PipesServer) connect to embedded server Architecture: ┌─────────────────────────────────┐ │ tika-grpc Process │ │ ┌──────────────────────────┐ │ │ │ IgniteStoreServer │ │ │ │ (server mode, daemon) │ │ │ └────────▲─────────────────┘ │ │ │ │ │ ┌────────┴─────────────────┐ │ │ │ IgniteConfigStore │ │ │ │ (client mode) │ │ │ └──────────────────────────┘ │ └─────────────────────────────────┘ ▲ │ (client connection) │ ┌────────┴─────────────────┐ │ PipesServer (forked) │ │ IgniteConfigStore │ │ (client mode) │ └──────────────────────────┘ * Configure Ignite work directory to /tmp - Set workDirectory to /tmp/ignite-work in IgniteStoreServer - Set workDirectory to /tmp/ignite-work in IgniteConfigStore - Avoids 'Work directory does not exist and cannot be created: /work' error - Uses system property ignite.work.dir if set, defaults to /tmp/ignite-work - Ensures Ignite can write to work directory in Docker containers * Use /var/cache/tika/ignite-work for Ignite work directory - Changed from /tmp/ignite-work to /var/cache/tika/ignite-work - Aligns with Tika's standard cache location - /var/cache/tika is already used for plugins and other Tika cache data * Use plugin classloader for Ignite server startup - Find Ignite plugin's classloader from plugin manager - Load IgniteStoreServer and CacheMode using plugin classloader - Fixes NoClassDefFoundError for H2 classes - Ensures all Ignite dependencies (including H2) are available - Plugin classloader has all dependencies from lib/ directory * Enable peer class loading in Ignite client - Set setPeerClassLoadingEnabled(true) in IgniteConfigStore - Must match server configuration - Fixes: Remote node has peer class loading enabled flag different from local - Both server and client now have peerClassLoading=true * Disable peer class loading to fix classloader conflicts - Set setPeerClassLoadingEnabled(false) on both server and client - Fixes ClassCastException due to class loaded by different classloaders - Server uses plugin classloader, client uses app classloader - Peer class loading causes the same class to be in both, creating conflicts - We don't need peer class loading for our use case * Move Ignite dependencies directly into tika-grpc - Made tika-pipes-ignite a required (non-optional) dependency of tika-grpc - Added ignite.version and h2.version properties - Removed reflection-based classloader lookup - Direct instantiation of IgniteStoreServer - Avoids all PF4J plugin classloader issues - Ignite classes now on main classpath * Disable Ignite Object Input Filter autoconfiguration - Set IGNITE_ENABLE_OBJECT_INPUT_FILTER_AUTOCONFIGURATION=false - Fixes: Failed to autoconfigure Ignite Object Input Filter - Ignite was conflicting with existing serialization filter - Apply in both IgniteStoreServer and IgniteConfigStore * Add Ignite filter autoconfiguration disable to client too * Add Ignite as built-in ConfigStore type - Handle 'ignite' type directly in ConfigStoreFactory - Load IgniteConfigStoreFactory via reflection - Works in forked PipesServer without plugin system - Matches pattern used for 'file' type - Fixes: ClassNotFoundException: ignite in forked process * Add PipesIterator management API with save, get, and delete operations * Fix Ignite tests to use temp directory and server mode * Fix TikaGrpcServerTest - update assertion for delete operation now that it's implemented

2025-12-28

Add PipesIterator management API with save, get, and delete operations

2025-09-28

TIKA-4334: replace pipes and server with newer apis

2024-08-21

TIKA-4272: completely refactor how fetchers work by integrating pf4j

2024-08-16

TIKA-4272: start of some fixes for tika grpc so that it works in tika-docker. plugins were stepping on each other's toes with classpath before.

2024-07-12

add a go package

2024-07-10

Tika grpc mv take3 (#1854) * TIKA-4181 - grpc server and client * latest updates - wip * code formatting * fix the issues with deps and such * add bidi streaming * clean up wording * make it as a stale connection expiring store * clean * delete dead code * add closeable * add closeable * add closeable * fixed issues related to proto lint * example docker file * example docker file and install docker file * fix the rm * more info * --platform linux/arm64 * --platform linux/arm64 * add work around comment for future * add comment * fix issues with files in wrong dir * add quotes * add a few more cleanups * add logging add reflection api for grpc so you can use grpc curl * fix issues and add tests for get, delete * push latest fixes * add start of a test scenario * add a fully functional http test case * add mtls as an option * add mtls in the example * add some robustness and add an exec * Fix issues with the fetch metadata * TIKA-4252: fix metadata issue * TIKA-4252: fix issue with config param serialization. * TIKA-4252: add error path * TIKA-4252: add protection against null metadata * TIKA-4252: fix merge conflicts from main * TIKA-4252: add json schema methods * TIKA-4252: fix broken tests, useless method * TIKA-4252: remove stupid sleep from test * TIKA-4252: fix checkstyle issue * TIKA-4252: log violations to console * TIKA-4252: log violations to console * TIKA-4252: exclude generated sources * TIKA-4252: if tika config is read-only, use a tmp file for tika server so that it can be modified * TIKA-4252: if tika config is read-only, use a tmp file for tika server so that it can be modified * add a health check * stop sending "metadata json" instead send a copy of the fetch configuration json that can add or override the main fetcher config * fix merge conflicts * fix issue with additional headers * move tika-grpc where it belongs inside the tika-server, not in the pipes project. * make params align with tika server and make them optional * merge TIKA-4181 and mv tika-grpc to higher level module * expand bounds in rtf parser test to deal with locally installed tesseract and exiftool --------- Co-authored-by: Nicholas DiPiazza <ndipiazza@apache.org>

The documentation is generated from this commit.

2024-07-09

TIKA-4275 -- move grpc server out of tika-pipes

The documentation is generated from this commit.

2024-06-26

move tika-grpc where it belongs inside the tika-server, not in the pipes project.

2024-06-25

TIKA-4181 - Tika Pipes Grpc Server (#1702) * TIKA-4181 - grpc server and client * latest updates - wip * code formatting * fix the issues with deps and such * add bidi streaming * clean up wording * make it as a stale connection expiring store * clean * delete dead code * add closeable * add closeable * add closeable * fixed issues related to proto lint * example docker file * example docker file and install docker file * fix the rm * more info * --platform linux/arm64 * --platform linux/arm64 * add work around comment for future * add comment * fix issues with files in wrong dir * add quotes * add a few more cleanups * add logging add reflection api for grpc so you can use grpc curl * fix issues and add tests for get, delete * push latest fixes * add start of a test scenario * add a fully functional http test case * add mtls as an option * add mtls in the example * add some robustness and add an exec * Fix issues with the fetch metadata * TIKA-4252: fix metadata issue * TIKA-4252: fix issue with config param serialization. * TIKA-4252: add error path * TIKA-4252: add protection against null metadata * TIKA-4252: fix merge conflicts from main * TIKA-4252: add json schema methods * TIKA-4252: fix broken tests, useless method * TIKA-4252: remove stupid sleep from test * TIKA-4252: fix checkstyle issue * TIKA-4252: log violations to console * TIKA-4252: log violations to console * TIKA-4252: exclude generated sources * TIKA-4252: if tika config is read-only, use a tmp file for tika server so that it can be modified * TIKA-4252: if tika config is read-only, use a tmp file for tika server so that it can be modified * add a health check * stop sending "metadata json" instead send a copy of the fetch configuration json that can add or override the main fetcher config * fix merge conflicts * fix issue with additional headers --------- Co-authored-by: Nicholas DiPiazza <ndipiazza@apache.org>

stop sending "metadata json" instead send a copy of the fetch configuration json that can add or override the main fetcher config

2024-06-19

TIKA-4252: add json schema methods

2024-06-18

TIKA-4252: add error path

2024-05-08

Fix issues with the fetch metadata

2024-04-22

add a fully functional http test case

2024-04-10

push latest fixes

2024-03-31

fixed issues related to proto lint

2024-03-29

add bidi streaming

code formatting

2024-03-02

latest updates - wip

2024-02-11

TIKA-4181 - grpc server and client