Proto commits in apache/buildstream

These 59 commits are when the Protocol Buffers files have changed:

Commit:2ae33fc
Author:Sander Striker
Committer:Sander Striker

speculative actions: ACTION overlay for cross-subaction output chaining Add ACTION overlay type to track inter-subaction output dependencies. When a compile subaction produces main.o and the link subaction consumes it, an ACTION overlay records this relationship so that priming can chain the adapted output through to the link action's input tree. Proto: ACTION = 2 in OverlayType, new source_action_digest field (3) to identify the producing subaction by its base action digest. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Commit:1178cb3
Author:Sander Striker

_protos: Sync remote_execution.proto with buildbox Synchronize the Remote Execution API proto with the version in buildbox. This adds the subactions field to ActionResult (field 99) for tracking nested executions (e.g. compiler invocations via recc). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Commit:50ba325
Author:Sander Striker
Committer:Sander Striker

speculative actions: Add proto definitions, generator, and instantiator Proto definitions: - speculative_actions.proto: SpeculativeActions, SpeculativeAction, Overlay messages for storing build action overlays - ActionResult.subactions (field 99): repeated Digest for nested execution action digests recorded by recc/trexe - Artifact.speculative_actions: Digest field for SA proto storage Generator (generator.py): - Analyzes completed builds to extract subaction digests from Actions - Builds digest cache mapping file hashes to source elements (SOURCE priority > ARTIFACT priority) - Creates overlays linking each input file to its origin element/path - Generates artifact_overlays for downstream dependency tracing Instantiator (instantiator.py): - Fetches base actions from CAS and resolves SOURCE/ARTIFACT overlays - Replaces file digests in action input trees recursively - Stores adapted actions back to CAS - Optimization: skips overlays for unchanged dependencies Config: - scheduler.speculative-actions flag (default false) in userconfig.yaml Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Commit:b52382a
Author:Jürg Billeter
Committer:Jürg Billeter

artifact.proto: Add marked directories to the build sandbox state

The documentation is generated from this commit.

Commit:50c5273
Author:Jürg Billeter
Committer:Jürg Billeter

artifact.proto: Add a field for the build sandbox state

Commit:b9613a0
Author:Jürg Billeter
Committer:Abderrahim Kitouni

artifact.proto: Add a field for the build sandbox state

Commit:8963d03
Author:Jürg Billeter
Committer:Jürg Billeter

wip: Store sandbox state in Artifact proto Use that to configure the sandbox state for `bst shell --use-buildtree`.

Commit:ea41a51
Author:Jürg Billeter
Committer:Jürg Billeter

_protos: Update local_cas.proto from buildbox

Commit:d7c73b8
Author:Jürg Billeter
Committer:Jürg Billeter

_protos: Update local_cas.proto from buildbox

Commit:ad4909f
Author:Jürg Billeter
Committer:Jürg Billeter

_protos: Update protos from remote-apis

Commit:f095829
Author:Jürg Billeter
Committer:Jürg Billeter

_protos: Update local_cas.proto from buildbox

Commit:048b683
Author:Seppo Yli-Olli
Committer:Seppo Yli-Olli

Add new changes for LocalCAS protocol with protobuf 3.20.1

Commit:6d16e3a
Author:Tristan van Berkom
Committer:Tristan van Berkom

Cleanup license headers for initial apache release. This patch removes the copyright statements from license headers, and adds license headers to files which are missing them.

Commit:be74e10
Author:Tristan van Berkom
Committer:Tristan van Berkom

_protos/buildstream/v2/artifact.proto: Adding buildroot to the artifact proto And regenerating the grpc code for the new artifact field.

Commit:6485d17
Author:Jürg Billeter
Committer:Jürg Billeter

_protos: Update local_cas.proto from buildbox-common

Commit:4968565
Author:Jürg Billeter
Committer:Jürg Billeter

_protos: Update protos from remote-apis

Commit:07252f0
Author:Abderrahim Kitouni
Committer:Abderrahim Kitouni

casserver.py: remove ReferenceStorage This is for bst1 only, and is now useless since bst-artifact-server was removed

Commit:548b23d
Author:Tristan van Berkom
Committer:Tristan van Berkom

_protos: Regenerated protos for artifact proto update Added low_diversity_meta and high_diversity_meta artifact.proto. These two fields represent detatched metadata files in which we can store artifact metadata which does not pertain directly to the artifact's cache key and as such should not stored directly on the proto itself. The low diversity data is meant to maximize on deduplication of data in the CAS store, while data which is expected to diverge more should be stored in the high diversity metadata. Also added the strict key to the proto. The strict key is interesting to have when instantiating an Element from a downloaded artifact, and also provides some context as to whether the downloaded artifact's strong key matches it's strict key.

Commit:4a65174
Author:Tristan van Berkom
Committer:Tristan van Berkom

_protos: Regenerated protos for artifact proto update Added sandbox_data to artifact.proto

Commit:74bb018
Author:Sander Striker
Committer:Tristan Van Berkom

Add Remote Asset API proto

Commit:08c0fad
Author:Jürg Billeter
Committer:bst-marge-bot

_protos: Drop BuildStream Artifact and Source services Replaced by Remote Asset API Fetch and Push services.

Commit:353a6ac
Author:Sander Striker
Committer:bst-marge-bot

Add Remote Asset API proto

Commit:17ff68f
Author:Jürg Billeter
Committer:bst-marge-bot

_protos: Update local_cas.proto from buildbox-common

Commit:fee2a26
Author:Jürg Billeter
Committer:Jürg Billeter

_protos: Update remote_execution.proto

Commit:68945db
Author:Jürg Billeter
Committer:Jürg Billeter

_protos: Update local_cas.proto

Commit:1ce31d4
Author:Jürg Billeter
Committer:Jürg Billeter

_protos: Update remote_execution.proto Move output_node_properties from Action to Command.

Commit:0379e3b
Author:Sander Striker
Committer:Sander Striker

Add Remote Asset API proto

Commit:03870cc
Author:Jürg Billeter
Committer:Jürg Billeter

artifact.proto: Add sources field This will be used for incremental builds.

Commit:59b26c9
Author:Darius Makovsky
Committer:Jürg Billeter

local_cas: Update proto

Commit:bd614b3
Author:Darius Makovsky
Committer:Jürg Billeter

remote_execution: Update proto

Commit:f3fe105
Author:Darius Makovsky
Committer:Darius Makovsky

local_cas: Update proto

Commit:80df8e2
Author:Darius Makovsky
Committer:Darius Makovsky

remote_execution: Update proto

Commit:26e79bf
Author:Jürg Billeter
Committer:bst-marge-bot

_protos: Update local_cas.proto

Commit:9f36c46
Author:Jürg Billeter
Committer:Jürg Billeter

_protos: Import local_cas.proto This imports local_cas.proto from buildbox-common [1] and the modules generated with ./setup.py build_grpc. [1] https://gitlab.com/BuildGrid/buildbox/buildbox-common

Commit:20a636d
Author:James Ennis
Committer:James Ennis

artifact.proto: Add the project name to the build dependency information We also need to store the project name of dependencies in the proto in order to reconstruct the graph. This also means that we must include the project name of the dependencies in the cache key, otherwise the proto and the cache keys may fall out of sync.

Commit:d493682
Author:Raoul Hidalgo Charman
Committer:bst-marge-bot

capabilities: Expand to include source service Part of #1038

Commit:c20eac1
Author:Raoul Hidalgo Charman
Committer:bst-marge-bot

Add new source proto Part of #1038

Commit:5490050
Author:Raoul Hidalgo Charman
Committer:Raoul Hidalgo Charman

artifact.proto: Remove ArtifactStatus method Not needed now that capabilities service is used. Part of #915

Commit:52158e4
Author:Raoul Hidalgo Charman
Committer:Raoul Hidalgo Charman

buildstream.proto: Add capabilities service Part of #915

Commit:eda3d28
Author:Raoul Hidalgo Charman
Committer:bst-marge-bot

_protos: Add new status method to artifact service And update relavant grpc files. Part of #1025

Commit:070d053
Author:Chandan Singh
Committer:Chandan Singh

Move source from 'buildstream' to 'src/buildstream' This was discussed in #1008. Fixes #1009.

Commit:f3413e6
Author:Tristan Van Berkom
Committer:Tristan Van Berkom

Change namespace from buildstream to buildstream2 This patch changes the buildstream namespace to buildstream2, so that BuildStream 2 can be parallel installable with BuildStream 1. * Moved buildstream/ directory to buildstream2/ * Updated toplevel files to reflect this change: - .pylintrc - setup.py - setup.cfg - tox.ini - MANIFEST.in * pluginutils/runcli.py module to use relative imports, this is now a part of BuildStream so it should not import "from buildstream2" * Updated plugins to derive from buildstream2 plugins (and to literalinclude their YAML files for documentation from the new ../buildstream2/plugins/.. path). * Updated imports in test cases to import from buildstream2 * _project.py, _plugincontext.py: Use the "buildstream2.plugins" namespace for plugin loading. * _protos: Regenerated the protos, these generated python files import from the buildstream module instead of using relative imports, so they needed an update. * .coveragerc: Update for new buildstream2 namespace * doc/Makefile: Update to build docs with new buildstream2 namespace * doc/bst2html.py: Import from new buildstream2 namespace * doc/source/conf.py: Update to refer to new buildstream2 namespace * doc/source/sample_plugin/setup.py: Update to refer to new buildstream2 namespace * element.py, source.py, sandbox/sandbox.py, utils.py: Updated documentation links to refer to symbols in new buildstream2 namespace

Commit:08fdcf0
Author:Raoul Hidalgo Charman
Committer:Raoul Hidalgo Charman

Add Artifact proto Part of #965

Commit:0f2bc37
Author:Martin Blanchard
Committer:Jim MacArthur

_protos: Import google.rpc.Code proto Imports code.proto from https://github.com/googleapis/googleapis and corresponding modules generated with ./setup.py build_grpc. https://gitlab.com/BuildStream/buildstream/issues/668

Commit:3db490a
Author:Tristan Maat
Committer:knownexus

Add Jürg's files

Commit:73fe9af
Author:Jürg Billeter
Committer:Jürg Billeter

Import protos for the Bazel Remote Execution API This imports protos from https://github.com/bazelbuild/remote-apis/ and the modules generated with ./setup.py build_grpc.

Commit:383e058
Author:Jürg Billeter
Committer:Jürg Billeter

Import Google proto dependencies for the Remote Execution API This imports protos from https://github.com/googleapis/googleapis/ and the modules generated with ./setup.py build_grpc.

Commit:df9a494
Author:Jürg Billeter
Committer:Jürg Billeter

remote_execution.proto: Add proposed symlink support https://github.com/bazelbuild/remote-apis/pull/18/

Commit:8a0fe27
Author:Jürg Billeter
Committer:Jürg Billeter

Add proto for BuildStream reference storage service

Commit:ab900b4
Author:Jürg Billeter
Committer:Jürg Billeter

Import Google proto dependencies for the Remote Execution API This imports protos from https://github.com/googleapis/googleapis/ and the modules generated with ./setup.py build_grpc.

Commit:0375278
Author:Jürg Billeter
Committer:Jürg Billeter

Import protos for the Bazel Remote Execution API This imports protos from https://github.com/bazelbuild/remote-apis/ and the modules generated with ./setup.py build_grpc.

Commit:d18a6fa
Author:Jürg Billeter
Committer:Jürg Billeter

remote_execution.proto: Add proposed symlink support

Commit:77a9861
Author:Jürg Billeter
Committer:Jürg Billeter

Add proto for BuildStream artifact cache service

Commit:79cc4c6
Author:Jürg Billeter
Committer:Jürg Billeter

Add proto for BuildStream artifact cache service

Commit:899258d
Author:Jürg Billeter
Committer:Jürg Billeter

remote_execution.proto: Add proposed symlink support

Commit:9486427
Author:Jürg Billeter
Committer:Jürg Billeter

Import protos for the Remote Execution API https://docs.google.com/document/d/1AaGk7fOPByEvpAbqeXIyE8HX_A3_axxNnvroblTZ_6s/ This imports protos from https://github.com/googleapis/googleapis/ and the modules generated with ./setup.py build_grpc.

Commit:9c2b790
Author:Jürg Billeter
Committer:Jürg Billeter

remote_execution.proto: Add proposed symlink support

Commit:0cf71ef
Author:Jürg Billeter
Committer:Jürg Billeter

Import protos for the Remote Execution API https://docs.google.com/document/d/1AaGk7fOPByEvpAbqeXIyE8HX_A3_axxNnvroblTZ_6s/ This imports protos from https://github.com/googleapis/googleapis/ and the modules generated with ./setup.py build_grpc.

Commit:470f357
Author:Jürg Billeter
Committer:Jürg Billeter

Add proto for BuildStream artifact cache service