These 22 commits are when the Protocol Buffers files have changed:
| Commit: | fa0b7c5 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Added OIDC identity support (#298) * refactor(sdk): rename devenv to project environment and update user environment naming * feat: added keycloak support for developing authentication * refactor: added namespace support for artifacts * feat(makefile): added namespace option * chore: clean up code * feat: first shot at issuer authorization for services * feat(cli): added service-to-service authentication * fix(lint) * fix(keycloak) * fix(github): update command args * feat: improve logging for archives * patch(worker): create parent directory before creating archive on pull * fix(format) * fix: update curl.se lock digests * patch(make): create parent directory for archive * feat(sdk): update go to use same auth as rust * feat(linux_debian): update docker digest * feat(linux-vorpal): upgrade to latest versions and add patches * chore(vorpal): update lockfile
The documentation is generated from this commit.
| Commit: | 051c7e5 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Support TLS connections (#273) * docs: update agents instruction files * feat: support tls for all distributed services * fix(cargo): update lock file * refactor: changes --update to --unlock for easier understanding * feat(github): move intel macos jobs to large to use same version as arm
| Commit: | 1dae3b2 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Update dependencies and toolchain (#259) * feat: update all rust dependencies and toolchains * fix(lint) * fix(cargo): update hash for intel darwin * chore: update hash for source * chore: update hash for source * chore: update hash for source * chore(rustc): updated digest * chore: update digests * fix: ignore local sources/artifacts in Vorpal.lock; docs: add AGENTS.md and update README * fix(lock) * feat: improve locking mechanisms * fix: support multiple artifacts in lockfile * fix(agent): properly process locked sources if they dont exist * fix(agent): keep project configurations out of lock file * feat: remove artifacts from lockfile * chore: cleanup * chore: remove all locking logic from sdk * chore: remove all locking logic from cli make command * feat(agent): remove internal memory caching * feat(agent): mulitple platform support for sources * feat(sdk): updated go sdk to reflect rust * fix(lint) * fix(generate) * fix: properly setup packages * feat: simplify lockfile structure * feat: update rust toolchain versions * docs(AGENTS) * feat: update rust toolchain versions in go sdk * feat: added lock file artifacts and terraform testing infrastructure * feat: updated lockfile for x86-64 darwin * feat: added aarch64-linux lockfile updates * feat: updated lockfile for x86-64 linux * feat: added internal traffic communication * chore: clean up * chore: clean up * chore: clean up * chore: clean up code * feat(sdk): updates to version 0.4.0-alpha
| Commit: | 5d0ac58 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Added devenv and userenv artifacts (#239) * feat(sdk): added devenv and userenv artifacts * fix(lint) * feat(cli): improves command naming conventions * feat(cli): moves prune command to system * fix(lint) * fix(cli): properly set subscriber for logging * fix(sdk): fix devenv in go * feat(cli): improved logging for registry * feat(cli): improved logging for worker * feat(cli): improved logging for agent * feat(cli): improved logging for worker * feat(cli): improved logging for agent & worker
| Commit: | e62f7e9 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Artifact secrets support (#234) * feat: implements secrets support * feat(cli): added lint rules * fix(cli): remove recently added lint rules * feat(makefile): added args for lima * feat(sdk): updated digest for curl-cacert * chore: updated Vorpal.lock * feat: updated lockfile for X8664_DARWIN * chore: updated Vorpal.lock
| Commit: | d377b87 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Make required values in SDKs (#219) * feat: make required values * fix(go): properly pass systems * feat: update hashes for go sdk * feat: added support for storing artifact aliases * feat: added lockfile support and generation * fix: lint * fix: lint * fix: missing directories * feat: added lock entries for AARCH64_LINUX * chore(deps): update rust crate toml to v0.8.21 (#213) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update rust crate tokio-util to v0.7.15 (#211) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * feat: updated lockfile for X8664_DARWIN * feat: added X8664_LINUX lockfile updates * fix(lint) * feat(sdk): set version 0.2.0-alpha --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
| Commit: | 1072878 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Added initial templates (#209) * feat(template): added go example * refactor: massive code reorganization * chore(cleanup) * chore(cleanup) * fix(lint) * chore(cleanup) * feat(Cargo): set publish to false for non-public crates * feat(cargo): updates versions * refactor: re-organizes folder structure * feat(cli): added init command with basic go and rust templates * feat(template): added rust shell in example * feat(template): included go.sum to template * refator(sdk): replace language specific shell with generic * docs(README)
| Commit: | 0d6ba2d | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Updates to support templates (#208) * feat(sdk): properly passes context for artifacts to agent * fix(cli): properly stop config if failure in output * feat: updated actions log messages
| Commit: | 470a0ea | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Implement Go SDK support (#145) * refactor: move rust SDK into own directory to promote multi-language monorepo support * feat: improve code for configuration * refactor: move all artifacts to top level * feat: begins implementing go sdk * fix: set proper go env for tools artifacts * fix(go): update hash for X8664Linux target Updated the hash value for the X8664Linux target in the Go artifact configuration to ensure correct artifact identification and retrieval. * fix: clear go cache after build * fix(protoc_gen_go): update hash for X8664Linux target Updated the hash value for the X8664Linux target in the protoc_gen_go configuration to ensure correct artifact sourcing. * refactor: move all config logic into seperate modules * refactor: move all config logic into seperate modules * feat: add support for protoc-gen-go-grpc - Replaced `artifact` function with `package` in `main.rs` to include new package functionality. - Updated `vorpal.rs` to include `protoc_gen_go_grpc` in the shell artifacts. - Modified Makefile to generate gRPC code using `protoc-gen-go-grpc`. - Added `protoc_gen_go_grpc` module in Rust SDK for artifact management. - Enhanced `ConfigContext` to handle new artifact source processing and caching logic. - Improved source path handling and error messaging in `context.rs`. * feat: implements AddArtifactSource function for context * chore: update lockfile * feat: added service support to go sdk * refactor: move steps into module * feat: added steps and shell artifact to go sdk * feat: added generic AddArtifact logic to go sdk * feat: support manifest storage on filesystem and migrate all artifacts out of SDK * refactor: migrate to builder pattern for artifact configurations * fix: various bugs * refactor: move all crates to subfolder * refactor: remove async from build function * refactor: name new non-async * refactor: move all toolchain aritfacts to rust sdk * chore: lint * refactor: migrate notary and add builders for rust in sdk * fix: properly passes PATH with bwrap step * feat: improve logic in get_path for cli * feat: move to retry logic for connecting to config * feat: added agent service for sdk to use * feat: improved logging and cli parameters * fix: lint * fix: format * feat: updated linx vorpal to include more tools * fix: lint * fix: only use default runners * fix: update linux vorpal artifact * fix: gopls, goimports and proto-gen-go-grpc * feat: added all platforms to CI * feat(sdk): minor improvements to types * feat(sdk): added missing source hashes * fix(ci): disable ubuntu-latest-arm64 * feat: consistent schema generation betweek go and rust sdks for shell * feat: consistent hashing working across go and rust sdks * feat: consistent hashing across go and rust sdks * feat: hash consistency on linux for both go and rust sdks * fix(sdk-go): recursion in fetch artifacts logic * docs(README) * fix(README) * fix(README) * chore(cleanup) * feat(ci): added check stage and additional build caching for all steps
| Commit: | 8c40bc9 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Configuration language toolchains support (#105) * feat: added rust language support in cli * feat: add support for config toolchains * feat: added language support for rust configurations * refactor: remove protoc installation comment and update artifact names - Removed the TODO comment for installing protoc in the GitHub Actions workflow. - Updated the artifact name from 'vorpal-shell' to 'vorpal-dev' in the main.rs file. - Changed the artifact_bin_paths from a vector to an array in the rust.rs file. * feat: properly sets permissions and unpacks sources at build time * feat: code improvements for building toolchains * refactor(cli): simplify get_order function signature Remove unnecessary lifetime annotation in the get_order function signature for improved code readability and maintainability. * feat: add support for AWS SSO and Rustls - Added dependencies for `aws-sdk-sso` and `aws-sdk-ssooidc` in `Cargo.lock`. - Updated `aws-config` in `Cargo.toml` to include features `rt-tokio`, `rustls`, and `sso`. - Introduced new packages such as `ring`, `rustls`, `rustls-native-certs`, `rustls-pemfile`, `rustls-webpki`, `hyper-rustls`, and others to support Rustls. - Updated existing dependencies and checksums to accommodate new features. - Enhanced security and networking capabilities with additional packages like `openssl-probe`, `schannel`, and `security-framework`. * feat: move sanitize for artifacts after pack * fix: move sanitize back * fix(rust): correct directory name for update hashes The directory name for storing update hashes was incorrect. Changed from "update-hash" to "update-hashes" to ensure consistency and correctness in the file structure. * fix(cli): remove unnecessary newline in main.rs feat(sdk): add symbolic link to /tmp in rust toolchain configuration fix(worker): ensure return after error logging in artifact service implementation * refactor(rust): remove unnecessary directory creation and symlink Removed the creation of the 'update-hashes' directory and the symlink to '/tmp' in the rust toolchain configuration. Added 'auto_self_update' setting to 'settings.toml'. * feat: support downloading remote sources * fix: lint * fix(protoc): update hash for X8664Linux target Updated the hash value for the X8664Linux target in the protoc toolchain configuration to ensure correct artifact identification. * fix(toolchain): update hash for X8664Linux in cargo.rs Updated the hash value for the X8664Linux target in the cargo.rs file to ensure correct artifact retrieval. * fix(toolchain): update hash for X8664Linux in clippy.rs Updated the hash value for the X8664Linux target in the clippy.rs configuration file to ensure correct artifact retrieval. * fix(toolchain): update hash for X8664Linux in rust_analyzer.rs Updated the hash value for the X8664Linux target in the rust_analyzer configuration to ensure correct artifact retrieval. * feat: implemented ArtifactSourceId support * feat: implement exists endpoint * refactor(rust_analyzer): improve import organization Reorganize imports in rust_analyzer.rs for better readability and maintainability. No functional changes made. * feat(paths): add cache directory setup in setup_paths function Added logic to create a cache directory if it does not exist in the setup_paths function. This ensures that the cache directory is available for use, preventing potential errors related to missing directories. * feat: checks before pushing sources to the registry * feat: update hashes for x86_64 * fix: caching check for sources * feat: update README and replace architecture diagram - Updated the README section title from "Design" to "Infrastructure". - Replaced the architecture diagram image from PNG to SVG format. - Deleted the old PNG diagram file and added the new SVG diagram file. * fix(readme): update image link to correct file name in diagram section * refactor(logging): update log messages for consistency Updated log messages to use a consistent format with '->' for better readability and uniformity across the codebase. Removed redundant log messages to streamline the logging process. * chore: clean up toml files
| Commit: | 050e95d | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Added shell artifact in SDK (#104) * feat: added simple shell implementation using artifacts * fix(workflow): update shell path to use github.workspace Updated the shell path in the vorpal.yaml workflow file to use `${{ github.workspace }}/script/dev.sh` instead of `./script/dev.sh` for both the build and deploy jobs. This ensures the correct path is used for executing scripts within the GitHub Actions environment. * fix(workflow): correct shell path for script execution Updated the shell path from `${{ github.workspace }}/script/dev.sh` to `$GITHUB_WORKSPACE/script/dev.sh` in the vorpal.yaml workflow file to ensure proper script execution. * refactor(workflow): update script execution in vorpal.yaml - Replace inline shell execution with direct script calls for consistency. - Modify steps to run pre-bake script and specific make commands separately. - Ensure clarity and maintainability by explicitly running each command. * fix(rust): correct syntax for component_paths and remove unnecessary commas fix(shell): remove redundant reference operator in get_artifact_envkey function call * feat: simplify logging output for artifacts * refactor: rename linux_headers to linux This commit refactors the code by renaming the variable and function from `linux_headers` to `linux`. The changes are applied across multiple files including `mod.rs`, `script.rs`, and `source.rs`. This update ensures consistency in naming conventions and improves code readability. * fix(cli): log config lines when not listening Previously, lines were not logged when the config was not listening. This change ensures that all lines are logged for better debugging. fix(worker): use to_string for artifact messages Replaced `format!` with `to_string()` for static artifact messages to improve code clarity and performance. * refactor(logging): update log format to include brackets around artifact name Updated the logging format in the build function to include brackets around the artifact name for better readability and consistency. This change affects various log messages related to build cache, pulling, unpacking, preparing, packing, and pushing sources. * refactor(shell): remove deactivate script and integrate functionality into activate The deactivate script has been removed, and its functionality is now integrated into the activate script. The exit-shell function within activate handles restoring and unsetting variables. This change simplifies the script management by reducing the number of files and ensuring all related operations are contained within a single script. * feat: update development setup and documentation - Replace `use_vorpal` with `use_dev` in .envrc to include new PATH additions and script execution. - Update README.md to reflect new development setup instructions, including macOS and Linux requirements, setup steps, and commands. - Modify makefile to separate development tasks into "without Vorpal" and "with Vorpal" sections. - Add new targets in makefile for building and running Vorpal configurations and shells. - Improve documentation for setting up and using the development environment with `direnv` and `dev.sh` script. * chore: clean up from pr review * refactor(readme): update examples and descriptions for clarity - Simplified the introductory sentence for building and shipping software. - Revised the overview section for better readability and understanding. - Updated example application name from "vorpal" to "example-app" in Rust, Go, and JavaScript code snippets.
| Commit: | b48c9a3 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Implement simple registry (#97) * feat: adds system to hash generation to avoid cross-platform conflicts * feat: added basic registry service for artifacts * refactor: simplify message handling and allow clippy lint - Refactored message handling by replacing `if let None = message` with `if message.is_none()` for clarity and conciseness. - Added `#[allow(clippy::too_many_arguments)]` attribute to suppress clippy lint warning for functions with many arguments. - Removed unnecessary `return` statement after logging an error in `ArtifactService` implementation. * refactor(artifact): merge mod.rs into artifact.rs and remove build.rs - Renamed `mod.rs` to `artifact.rs` for better file organization. - Removed `build.rs` as it is no longer needed. - Cleaned up unused module import in `artifact.rs`. * feat: fix environment variables passed to command * feat: update vorpal workflow to configure apparmor for bwrap - Added configuration for apparmor to allow unconfined execution of bwrap on Ubuntu runners. - Created a new apparmor profile for bwrap and included site-specific additions. - Restarted apparmor service to apply the new profile changes. * fix: add sudo to apparmor configuration command in workflow script The apparmor configuration command in the workflow script now uses sudo to ensure proper permissions when writing to /etc/apparmor.d/bwrap. * fix(workflow): update apparmor configuration process Refactor the apparmor configuration setup in the vorpal workflow. Instead of using 'sudo cat' to write directly to '/etc/apparmor.d/bwrap', the content is first written to a temporary file 'bwrap' and then moved to the target directory with 'sudo mv'. This change ensures proper file permissions and avoids potential issues with direct file writing. * feat: added s3 backend support for registry * refactor(cli, registry): simplify S3 bucket validation and default handling - Combined the S3 backend check and bucket parameter validation into a single conditional statement in `main.rs`. - Replaced `unwrap_or_else` with `unwrap_or_default` for cleaner default handling of the S3 bucket name in `service.rs`. * feat(workflow): add AWS environment variables for S3 registry backend Added AWS_ACCESS_KEY_ID, AWS_DEFAULT_REGION, and AWS_SECRET_ACCESS_KEY as environment variables to the workflow step that starts Vorpal with the S3 registry backend. This change ensures that the necessary AWS credentials are available for accessing the S3 bucket. * feat(README) * fix(paths): prevent setting timestamps on symlinks This change adds a check to ensure that file timestamps are not set on symbolic links. This prevents potential issues with symlinks by only setting timestamps on regular files. * fix: properly set dates on paths that are symlinks
| Commit: | c79e493 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Implement Linux sandboxes (#72) * feat: improve build graph logic and check command * feat: added more styling to check command * refactor(cli): modularize and clean up code - Move configuration checking logic to a new `config` module. - Move logging functions to a new `log` module. - Remove redundant `render_package` function from `main.rs`. - Update `main.rs` to use new `config` and `log` modules. - Simplify `load_config_build` function in `nickel.rs`. - Add new logging functions in `worker.rs` for better output formatting. - Clean up and refactor `build` function in `worker.rs` for clarity. * feat: begin working on sandbox artifacts for shell * refactor(workflow): rename dev cache job to sandbox and update script references - Renamed the `cache-dev` job to `sandbox` in the GitHub Actions workflow file. - Updated cache keys and paths to use `sandbox` instead of `dev`. - Replaced references to `dev.sh` with `sandbox.sh` in the workflow file. - Deleted `dev.sh` script and replaced it with `sandbox.sh` which includes additional dependency checks and installations. - Removed the `install-vorpal` script as it is no longer needed. * feat: added support for packer and vagrant * feat: restructure scripts * refactor(install): rename sandbox_path to prefix_path and optimize make commands - Updated variable name from SANDBOX_PATH to PREFIX_PATH for clarity and consistency. - Enhanced make commands to utilize all available processors with -j"$(nproc)" for faster builds. - Adjusted PATH handling in gcc.sh to be commented out for potential future use. - Modified nickel.sh to reflect the new variable name and ensure correct binary installation paths. * feat: more work on cross-platform sandbox tools * feat(setup): add bubblewrap to debian setup script Added bubblewrap to the list of packages installed by the debian setup script to enhance security and sandboxing capabilities. * feat: added zstd to sandbox * refactor: move env to .env directory for development only * feat: test dev and sandbox scripts on macos * feat: added lockfile support for sandbox multi-platform sandboxes * feat: improve nickel logic * feat: validate vorpal building vorpal on macos * refactor(sandbox): update sandbox path variables and add compression - Renamed `SANDBOX_PATH` to `SANDBOX_STORE_PATH` for clarity. - Renamed `SANDBOX_PATH_PACKAGE` to `SANDBOX_STORE_PATH_PACKAGE` for consistency. - Updated all references to the new variable names. - Added a step to compress the sandbox package directory using `zstd`. * fix(script): update script paths and improve sandbox setup - Change SCRIPT_PATH in dev.sh to point to the correct directory. - Add VORPAL_PATH to directories array in sandbox.sh. - Ensure directories are created only if they do not exist and set correct ownership. - Update messages to include specific hashes for sandbox, bash, binutils, coreutils, and zstd to avoid confusion. - Add execution of sandbox.sh in dev.sh. * feat: update README and scripts for improved setup and build process - Updated README.md to include detailed requirements for macOS and Linux. - Added instructions for installing native tools on macOS and Linux. - Modified setup steps to include new commands for generating keys, starting worker, and checking build. - Updated makefile to retain the dist directory after creating the tarball. - Enhanced script/dev.sh to export PATH for environment setup. - Fixed script/sandbox.sh to correctly exit with status 0 if the sandbox package directory exists. * chore: remove unnecessary log statement in darwin package build function The log statement "Building package" has been removed from the build function in the darwin package module to clean up the code. * refactor(darwin): remove unused tracing::info import in mod.rs The tracing::info import was not being used in the darwin module and has been removed to clean up the code. * feat: better support anyhow result handling * feat: get linux sandboxes working * feat: build linux sandbox package * feat: get builds working in CI again * refactor(workflow): rename sandbox job to package-sandbox - Removed the `sandbox` job from the workflow. - Added a new job `package-sandbox` with the same steps as the removed `sandbox` job. - Updated the `test` job to depend on the new `package-sandbox` job instead of the removed `sandbox` job. * fix(sandbox): remove include directory recursively Changed the removal command for the include directory to use `rm -rf` instead of `rm -f` to ensure that the directory and its contents are deleted recursively. This prevents potential issues with non-empty directories. * fix(workflow): conditionally run debian script based on runner OS - Add condition to skip running `debian.sh` script on `macos-latest` runner in both `dev` and `sandbox` jobs. * fix(workflow): correct conditional syntax for runner check - Updated the conditional syntax for checking the runner type from `${{ matrix.runner }} != 'macos-latest'` to `matrix.runner != 'macos-latest'` in the vorpal.yaml workflow file. - Ensured consistency in the conditional checks for both the 'dev' and 'sandbox' steps. * feat: add nproc package to debian.sh script - Added nproc package to the list of packages installed in the debian.sh script. - Updated both the 'case "$1"' sections to include nproc. - This change ensures that the nproc utility is available for use. * fix(script): add debug step to list files in /tmp before copying amber binary Added a step to list all files in the /tmp directory with detailed information before copying the amber binary to the environment path. This helps in debugging and ensuring the correct files are being handled. * fix: update debian.sh to include coreutils and remove nproc - Added coreutils to the list of packages to be installed in debian.sh - Removed nproc from the list of packages to be installed in debian.sh - Updated binutils.sh to calculate source hash and set correct permissions for history.list - Changed sandbox_stdenv_hash for MacOS systems in build.rs * fix: unify CPU_COUNT assignment and simplify amber.sh script - Removed OS-specific conditional copying in amber.sh script and unified the copy command. - Modified CPU_COUNT assignment in bash.sh, binutils.sh, coreutils.sh, and zstd.sh scripts to handle both Darwin and Linux systems consistently. - Ensured CPU_COUNT is set with the appropriate flags for parallel execution in make commands. * fix(amber.sh): update amber binary copy logic for different OS - Added a check to list all files in /tmp directory. - Modified the script to copy the amber binary differently based on the OS. - For Darwin OS, the script copies the binary from a specific directory. - For other OS types, the script copies the binary directly from /tmp. - Removed the redundant tar.gz file removal command. * feat: add support for copying amber binary for x86_64 Linux - Added a conditional check to handle copying the amber binary for x86_64 architecture on Linux systems. - Ensures the correct binary is copied to the environment path for this specific architecture and OS combination. * chore(binutils.sh): enable verbose mode in bash script Enabled the verbose mode (-x) in the binutils.sh script to provide more detailed output during execution. This change helps in debugging by showing each command before it is executed. * fix(sandbox): remove unnecessary chmod command in binutils.sh The chmod command for setting permissions on history.list in the binutils script has been removed as it was deemed unnecessary. This change simplifies the script and removes redundant operations. * fix: remove parallel build option from make commands The parallel build option (-j) has been removed from the make commands in the bash.sh, coreutils.sh, and zstd.sh scripts. This change ensures that the build process uses the default number of jobs, which can help avoid potential issues related to parallel builds. * refactor(workflow): merge package-sandbox job into sandbox job - Added a new `sandbox` job that depends on the `dev` job. - Configured the `sandbox` job to run on macOS and Ubuntu runners. - Included steps to restore the environment, run scripts, and upload artifacts. - Removed the `package-sandbox` job and its steps. - Updated the `test` job to depend on the `sandbox` job instead of the `package-sandbox` job. * chore(sandbox): clean sandbox directories before creating new ones - Added commands to remove existing sandbox directories before creating new ones in bash.sh, binutils.sh, coreutils.sh, gcc.sh, glibc.sh, linux-headers.sh, and zstd.sh scripts. - Ensures a clean state by removing the sandbox directory if it exists, preventing potential conflicts or leftover files from previous runs. * fix(makefile): remove redundant clean step from build target The 'clean' step has been removed from the 'build' target in the makefile to streamline the build process. The 'build' target now directly depends on the 'check' target, ensuring that linting is performed before building without unnecessary cleaning. * refactor(workflows): rename and update sandbox job to package-sandbox - Renamed the `sandbox` job to `package-sandbox` in the GitHub Actions workflow file. - Updated the job steps to reflect the new naming convention. - Removed redundant CPU count determination for macOS in various sandbox scripts (`bash.sh`, `binutils.sh`, `coreutils.sh`, `zstd.sh`). - Ensured consistency in CPU count determination for Linux systems across the scripts. * fix(workflow): update job dependency from 'sandbox' to 'package-sandbox' in vorpal.yaml * feat: add architecture-specific SHA256 hash support - Added architecture detection using `uname -m` and converted to lowercase. - Mapped `arm64` to `aarch64` for consistency. - Updated scripts to use architecture-specific SHA256 hash files. - Renamed existing SHA256 hash files to include architecture prefix. - Added new SHA256 hash files for `x86_64-linux` architecture. * fix(sandbox): update hash file path to include architecture - Modified the path to the hash file to include the architecture (${ARCH}) for both Darwin and Linux systems. - Ensures that the correct hash file is read based on the architecture and operating system. * chore: remove binutils package for Darwin - Removed "binutils" from the list of packages for Darwin in `sandbox.sh`. - Deleted the SHA256 checksum file for "binutils" in `aarch64-darwin`. * refactor: rename package-sandbox job to sandbox and update scripts - Renamed the `package-sandbox` job to `sandbox` in the GitHub Actions workflow. - Updated the `sandbox` job to use a fixed version `0.1.0-rc.0` for the sandbox package. - Removed the use of SHA256 hashes for package verification and replaced it with version-based verification. - Added a new script `zstd.sh` in `script/common` for installing zstd. - Updated `debian.sh` to remove an extra newline. - Modified `dev.sh` to remove `coreutils.sh` and update script paths. - Updated `amber.sh`, `nickel.sh`, and `protoc.sh` scripts to accept a sandbox package path as an argument. - Deleted `coreutils.sh`, `zstd.sh`, and `hash_path.sh` scripts from `script/dev`. - Updated `sandbox.sh` to remove hash-based verification and use version-based verification. - Updated various sandbox package scripts (`bash.sh`, `binutils.sh`, `coreutils.sh`, `gcc.sh`, `glibc.sh`, `linux-headers.sh`) to use version-based verification and accept a sandbox package path as an argument. - Removed SHA256 hash files from `script/sandbox/sha256sum`. - Updated symbolic links for `zstd.sh` in `script/dev` and `script/sandbox` to point to `../common/zstd.sh`. * chore(workflow): remove redundant sha256sum version check The sha256sum version check has been removed from the GitHub Actions workflow as it was deemed unnecessary. This change simplifies the workflow by eliminating an extraneous step. * feat(gcc.sh): disable multilib support in GCC build script - Added --disable-multilib flag to the GCC configuration step. - This change ensures that multilib support is disabled during the GCC build process. * fix: properly support files in different folders * feat: add sandbox support and native build option - Updated `print_source_cache` function to include `package_name` parameter and format the output. - Modified `build` function in `worker.rs` to handle sandbox and non-sandbox builds. - Added `sandbox` field to `Package` struct in `schema/src/lib.rs`. - Updated `contract.ncl` to include `sandbox` configuration. - Implemented conditional logic for sandbox environment setup in `run` function. - Added `native` module for non-sandbox builds. - Updated `darwin` and `linux` modules to handle optional `stdenv_dir_path`. - Created `native` module to support native build commands without sandboxing. * refactor(config): reorder sandbox property in contract.ncl Reordered the sandbox property to maintain logical grouping and consistency within the configuration file. No functional changes were made. * feat: support multiple named sources * feat: support building sandbox with vorpal * refactor(workflow): remove unnecessary dependency on sandbox job in test job * refactor(ci): remove redundant version checks in vorpal.yaml Removed unnecessary version checks for cargo, rustc, nickel, and protoc in the GitHub Actions workflow file to streamline the CI process. feat(worker): add async signature decoding Introduced an async function `decode_signature` to handle the decoding of source data signatures in a non-blocking manner. Updated the `run` function to utilize this new async decoding method, improving error handling and code readability. * fix: replace hex crate * feat: support macos sandbox build * feat: implements sandbox from packages and fixes odd length issue * feat: split sandbox into multiple packages * feat: added zlib to sandbox * fix: update sandbox cleanup paths and add rsync for include directory - Updated the cleanup function parameters to use `sandbox_package_dir_path` and `sandbox_source_dir_path` instead of `package_dir_path` and `source_dir_path`. - Corrected the variable name from `sandbox_script_file_path` to `sandbox_script_package_file_path` in the `run` function. - Added rsync command to copy contents from `$output/include/` to `$output/usr/include/` and removed the original `$output/include` directory in `sandbox.ncl`. * chore(ci): enable ubuntu-latest runner in GitHub Actions workflow * feat: added x86_64 arch configs * refactor(worker): remove unused cleanup function and redundant imports - Removed the `cleanup` function which was commented out and no longer in use. - Removed the redundant import of `Path` and `PathBuf` from `std::path`. - Removed the redundant import of `remove_dir_all` and `remove_file` from `tokio::fs`. * fix(build): remove pipefail option from sandbox script The `pipefail` option has been removed from the sandbox script commands to ensure compatibility with environments where this option is not supported. This change helps to prevent potential script execution failures. * fix(build): remove pipefail option from sandbox script The `pipefail` option has been removed from the sandbox script to prevent unintended failures in the pipeline. This change ensures that the script will not fail if any command in a pipeline fails, allowing for more robust error handling. * refactor: replace pushd/popd with cd in various scripts Replaced instances of pushd/popd with cd in the following files: - Vagrantfile - config/packages/@vorpal/package/bash.ncl - config/packages/@vorpal/package/binutils.ncl - config/packages/@vorpal/package/coreutils.ncl - config/packages/@vorpal/package/gcc.ncl - config/packages/@vorpal/package/glibc.ncl - config/packages/@vorpal/package/linux-headers.ncl - config/packages/@vorpal/package/m4.ncl - config/packages/@vorpal/package/perl.ncl - config/packages/@vorpal/package/texinfo.ncl - config/packages/@vorpal/package/zlib.ncl - config/packages/@vorpal/package/zstd.ncl This change simplifies the directory navigation commands and improves readability. * chore(ci): update vorpal workflow to install dependencies on ubuntu - Added step to install necessary packages on 'ubuntu-latest' runner - Removed unnecessary blank line after 'vorpal keys generate' step * fix(ci): add sudo to apt-get commands in vorpal.yaml Updated the CI workflow to include 'sudo' for apt-get update and install commands to ensure proper permissions on 'ubuntu-latest' runner. * fix(ci): update zlib package to zlib1g-dev in vorpal.yaml workflow * chore(workflow): remove ubuntu-latest-arm64 runner from vorpal.yaml This commit removes the ubuntu-latest-arm64 runner from the GitHub Actions workflow configuration file vorpal.yaml. The changes affect both job definitions in the workflow. * chore: update dev scripts - Removed amber.sh script and its reference from dev.sh - Added zstd to the list of scripts in dev.sh - Updated zstd.sh to include CPU_COUNT for macOS and removed unnecessary pushd/popd commands * fix(gcc.ncl): update build directory path in gcc.ncl Updated the build directory path in the gcc.ncl configuration file to use a relative path instead of an absolute path. This change ensures that the script navigates to the correct build directory. * feat(ci): add bubblewrap to GitHub Actions workflow Added bubblewrap to the list of packages installed in the vorpal.yaml GitHub Actions workflow to ensure all necessary dependencies are available. * chore: update vorpal workflow to modify AppArmor settings - Add command to set 'kernel.apparmor_restrict_unprivileged_userns' to 0 in /etc/sysctl.d/60-apparmor-namespace.conf - Ensure the new setting is applied during the workflow execution * fix(workflow): update vorpal.yaml to change bubblewrap permissions - Removed the command to modify AppArmor settings for unprivileged user namespaces. - Added a command to change ownership permissions for /usr/bin/bubblewrap. * fix(workflow): correct file permissions for bubblewrap in vorpal.yaml Changed the command from `chown` to `chmod` to set the correct file permissions for `/usr/bin/bubblewrap` in the GitHub Actions workflow configuration. * fix(workflow): update vorpal.yaml to dynamically find and set permissions for bubblewrap - Added logic to dynamically find the path of bubblewrap using `which -a bwrap | head -n 1` - Updated the script to set the correct permissions for the dynamically found bubblewrap path - Added a command to print the bubblewrap version for verification * fix(linux): change lib64 binding to use --ro-bind-try Modified the binding for lib64 directory to use --ro-bind-try instead of --ro-bind to handle cases where the directory might not exist. Removed redundant binding for /nix/store. * feat: add support for x86_64-macos and improve script execution - Added support for x86_64-macos in the cross-platform configuration. - Changed default shell from /bin/sh to /bin/bash for better compatibility. - Enhanced script execution by adding 'set -euxo pipefail' for stricter error handling. * refactor: simplify build and sandbox paths * feat: add debug commands to sandbox script generation Added `ls -alh` and `cat` commands to the sandbox script generation process for debugging purposes. This change allows for listing the contents of the output directory and displaying the contents of the generated sandbox.sh script. * chore(workflow): update bwrap configuration in vorpal.yaml Refactor the bwrap setup by adding a custom AppArmor profile for bwrap in the workflow. This change replaces the previous method of setting the suid bit on the bwrap binary with a more secure AppArmor configuration. * chore: restart apparmor service after updating rules Added a step to restart the apparmor service in the GitHub Actions workflow after updating custom rules to ensure changes take effect. * fix(workflow): add permission change for bwrap in vorpal.yaml Add a command to change the user permissions for the bwrap binary in the vorpal GitHub Actions workflow. This ensures that the bwrap command can be executed with the necessary privileges during the workflow execution. * refactor: update workflow and script paths - Removed conditional execution of debian.sh for non-macos runners in vorpal.yaml. - Added execution of debian.sh in dev.sh for Linux systems. - Renamed and moved debian.sh to dev/debian.sh. - Updated package installation in debian.sh with additional dependencies: build-essential, make, rsync, zlib1g-dev. - Corrected indentation in fusion-13.pkrvars.hcl. * refactor(ci): streamline ubuntu setup in vorpal workflow Simplified the Ubuntu setup process by removing redundant package installations and apparmor configuration. Updated the checkout action to v4 and ensured necessary repositories are added before installing apparmor. * refactor: update vorpal workflow and makefile for improved compatibility - Removed redundant AppArmor installation steps in vorpal.yaml and added commands to disable and remove AppArmor configurations. - Introduced OS_TYPE variable in makefile to generalize build-packer and validate-packer targets for different operating systems. - Updated paths in makefile to use OS_TYPE variable, allowing for more flexible Packer builds. * fix: update package system mapping for linux targets The package system mapping for "aarch64-linux" and "x86_64-linux" targets has been updated to use `packages_system_macos` instead of `packages_system_linux`. This change ensures consistency across different architectures and platforms. * fix(build): remove 'x' option from bash scripts The 'x' option in the 'set' command has been removed from both sandbox and build scripts to prevent unnecessary command tracing output. This change ensures cleaner logs and focuses on essential error handling and pipefail behavior. * refactor(sandbox.ncl): remove unused conditional script for linux targets The conditional script for linux targets was removed as it was not being utilized in the current configuration. This change simplifies the code by eliminating unnecessary conditions. * fix(linux): change bind options to --ro-bind-try Updated the bind options from --ro-bind to --ro-bind-try for several paths including /bin, /etc, /lib, /libexec, /sbin, /share, and /usr. This change ensures that the binding process is more flexible and can handle cases where the source paths may not exist, improving the robustness of the build process. * feat(workflow): add tmate session for debugging Added mxschmitt/action-tmate@v3 to the workflow steps to enable tmate sessions for debugging purposes. The session access is limited to the actor for security reasons. * refactor(workflow): relocate tmate action to run on failure The tmate action has been moved from the initial steps to execute only when a failure occurs. This change optimizes the workflow by limiting the use of tmate to scenarios where debugging is necessary, ensuring resources are used efficiently. * fix(workflow): add timeout to tmate action Added a timeout of 10 minutes to the tmate action in the GitHub workflow to prevent indefinite execution. fix(config): correct package system mapping for Linux Updated the package system mapping for "aarch64-linux" and "x86_64-linux" to use `packages_system_linux` instead of `packages_system_macos`. Added conditional script execution for Linux targets. * refactor: simplify script execution for sandbox * feat(worker): update package serialization to exclude specific fields - Added `serde_json::Value` to handle JSON manipulation. - Modified package serialization to exclude "packages" and "sandbox" fields from the JSON object before converting it to a string. fix(sandbox): update script to set locale - Updated the sandbox script to set `LC_ALL` to "C" before executing the script. chore(vorpal): update source excludes - Added "config" to the list of source excludes in `vorpal.ncl`. * fix(worker): remove unnecessary package field from serialization The "packages" field is no longer removed from the package JSON serialization, as it was deemed unnecessary. feat(bash): enable static linking in configuration Added the --enable-static-link option to the bash package configuration to support static linking. fix(sandbox): ensure proper command execution Wrapped the command execution in quotes and set the CC environment variable to "gcc" for consistent behavior. chore(makefile): clean distribution directory Added a command to remove the distribution directory during the clean process to ensure a fresh build environment. * refactor(config): remove static linking from bash configuration The static linking option (--enable-static-link) has been removed from the bash configuration script. This change simplifies the build process by only specifying the installation prefix. * refactor: move all sanitize logic to nickel configs * refactor: work on sdk poc for rust * refactor: move all schemas to protoc * feat: logic for config communication with cli * feat: updates for macos builds * feat: more work to configuration * feat: configs for binutils-native-stage-01, gcc-native-stage-01 and linux-headers on linux * feat: stable config for gcc * feat: finish initial toolchain packages * feat: added support for updating rpath in binaries * feat: added rpath update for current package * feat: added gzip, grep and gawk * feat: implement grpc config service communication for cli commands * feat: added more packages * refactor: build directly to store and use lock files * feat: begin building tools in sandbox by default (stopped at perl) * refactor(config): rename variables for clarity in add_default_script function Renamed `script_arch` to `glibc_arch` and `script_glibc` to `glibc_script` for better clarity and consistency in the `add_default_script` function. This change improves code readability by making variable names more descriptive of their purpose. * feat: start using isolated sandbox * feat: reproducible file timestamps * feat: update Dockerfile and Vagrantfile for improved setup - Updated Dockerfile to use Debian sid-slim and removed unnecessary packages. - Modified Vagrantfile to streamline setup functions and update PATH. - Refactored Rust build scripts to simplify package configuration and sandbox paths. - Added new scripts for finding library dependencies and setting up Docker. - Enhanced Makefile with new build-docker target for creating and exporting Docker images. - Updated package.proto to include symlink option in PackageSandboxPath. - Improved Debian setup script to install Docker and related tools. - Adjusted worker build logic to handle symlink paths and clean up environment variables. * feat: change root to new /vorpal path * feat: implements cross-toolchain on x86_64 * feat: add support for additional packages in cross toolchain - Updated CPPFLAGS and C_INCLUDE_PATH to include c++/14 directory. - Added build steps for m4, ncurses, bash, coreutils, diffutils, file, findutils, gawk, grep, gzip, make, patch, sed, tar, and xz. - Introduced stage 02 build steps for binutils and gcc. - Updated package sources to use ftpmirror.gnu.org for more reliable downloads. - Added new package sources for bash, coreutils, diffutils, file, findutils, gawk, grep, gzip, m4, make, ncurses, patch, sed, tar, and xz with their respective hashes and URIs. * feat: successfully built cross toolchain * feat: successfully built cross toolchain temporary tools * feat: successfully builds cross-toolchain on x86_64 * feat: successfully build vorpal with toolchain * fix: set proper name for cache package in rust * chore: clean up * feat: update Dockerfile and scripts for architecture support and streamline installation - Updated Dockerfile to handle architecture detection and adjust linux-headers installation accordingly. - Modified cargo_hash in main.rs for package configuration. - Refactored cross_toolchain.rs to dynamically set rootfs_path_dirs based on target architecture. - Adjusted package sandbox paths and script logic for aarch64 and x86_64 architectures. - Simplified debian.sh script by using get.docker.com for Docker installation and removed manual keyring and source list setup. - Ensured user is added to the Docker group after installation. * feat: update README with new images - Added a new image 'vorpal-purpose.jpg' to the README for better illustration of the tool's purpose. - Updated the existing image reference from 'vorpal.png' to 'vorpal-arch.png' to reflect the renamed file. - Renamed 'vorpal.png' to 'vorpal-arch.png' to improve clarity on the image's content. * feat: updated ci for new config params * chore(ci): comment out macOS runner in GitHub Actions workflow The macOS runner has been commented out in the vorpal.yaml workflow file to streamline the CI process. The workflow will now only run on the ubuntu-latest runner. * feat: update automation for tasks * refactor: simplify build steps in vorpal workflow The build steps in the vorpal GitHub Actions workflow have been simplified by removing the redundant script execution. The `make dist-rootfs` command is now executed directly, eliminating the need for the initial `./script/dev.sh` prebake step. * feat: support native builds and using packages for sandboxes * refactor: rename Package to Artifact * refactor: update config to use all new artifact references * refactor: updates all other places with artifact * feat: migrate all cross-toolchain sources to artifacts * chore: remove debug output * refactor: simplify artifact loading and building process - Refactored `load_artifacts` function to improve readability and maintainability. - Updated `build_artifact` function to streamline artifact creation, removing redundant code. - Replaced `Artifact` struct usage with direct function parameters for better clarity. - Enhanced error handling with `bail!` for unsupported systems. - Consolidated artifact source fetching and extraction logic. - Improved script formatting using `formatdoc` for consistency. - Removed commented-out code and unused variables for cleaner codebase. * feat(schema): updates schemas for artifact steps * feat(worker): updates to run steps in sequence * feat: finalize linux-vorpal package * feat: support source hashing in configuration * feat: support end-to-end build with vorpal * feat: updated ci * fix(workflow): correct artifact path and remove unnecessary dependency - Updated the artifact path in the upload-artifact step to remove the trailing slash for consistency. - Removed the unnecessary dependency on 'package-rootfs' in the test job to streamline the workflow. * fix: lint * debug: added output for artifact * refactor: remove rootfs build and clean steps The rootfs build and clean steps have been removed from the Makefile. This includes the removal of the `build-rootfs` and `clean-rootfs` targets, as well as the `dist-rootfs` target. The workflow file has been updated to create the `dist` directory before downloading artifacts, and the artifact path is now specified. * fix(workflow): update artifact extraction process - Remove redundant directory creation step before downloading artifact. - Correct artifact path in download-artifact action. - Adjust tar extraction command to match the updated artifact naming convention. * fix(Dockerfile): add architecture check for x86_64 and set to amd64 Added a condition to check if the architecture is x86_64 and set it to amd64. This ensures compatibility with the expected architecture naming conventions. * feat: support darwin native builds * refactor(artifact): remove unused `UnknownSystem` variant from `ArtifactSystem` enum The `UnknownSystem` variant in the `ArtifactSystem` enum was removed as it was not being used in the codebase. This change simplifies the enum and reduces unnecessary code. * chore: update packages * fix: macos builds * fix: bug for artifacts population on build steps
| Commit: | abe9451 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Decouple configuration language (#56) * feat: first implementation supporting nickel configurations * refactor(config): move package graph and map logic to separate module - Extracted package graph and map building functions from `main.rs` to a new `config.rs` module. - Implemented `build_structures` function to create `PackageStructures` containing the graph and map. - Moved topological sort, add_to_map, and add_to_graph functions to `config.rs`. - Updated `main.rs` to use the new `config` module for building and sorting packages. - Improved code organization and separation of concerns. * feat: impl build logic with worker grpc * feat: update worker from cli changes * fix: debug issues after migration to nickel * chore: remove dockerfile for vorpal-agent which was replaced by cli * chore(deps): update docker.io/library/debian:12.6-slim docker digest to 5f7d566 (#44) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update docker.io/library/debian:12.6-slim docker digest to 5f7d566 (#44) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore(deps): update docker.io/library/debian:12.6-slim docker digest to 5f7d566 (#44) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * chore: update Dockerfile and justfile for streamlined build process - Removed HEAD and parent conflict markers from Dockerfile - Simplified Dockerfile by removing redundant sections and comments - Deleted Dockerfile.sandbox - Updated justfile with new build, check, format, and update commands - Added specific build, check, format, and update commands for cargo and nix - Improved test commands in justfile - Cleaned up unnecessary files and directories in clean command * fix: rename dockerfile * fix: dockerfile * fix: tests * feat: implement dynamic system in config * feat: added support for excludes and includes options * chore: fixes lint and cleans repository * feat: support sandbox image and worker configuration * feat: support locking sandbox image for complete reproducibility * refactor(github): rename job and update image name in docker workflow - Renamed job from `sandbox` to `package` in `.github/workflows/docker.yaml`. - Updated `IMAGE_NAME` to `${{ github.repository }}-sandbox`. - Adjusted image labels and subject names accordingly. chore(github): streamline nix workflow - Removed `format`, `check`, and `lint` jobs from `.github/workflows/nix.yaml`. - Updated `package` job to run `check-nix` before packaging. refactor(cli): update worker parameter type - Changed `worker` parameter type from `&String` to `&str` in `cli/src/worker.rs`. - Updated `StoreServiceClient::connect` and `PackageServiceClient::connect` calls to use `to_owned()`. chore(lock): update sandbox image digest - Updated sandbox image digest in `vorpal.lock.ncl` to `8ef5ca1e9057e7508f28f0fc4db1b8555ba7a9cdbc5acccb1c86542e934d2027`. * chore: update dependencies and hashes - Update `quote` crate from version 1.0.36 to 1.0.37 and update its checksum in Cargo.lock - Update `system-configuration` crate from version 0.6.0 to 0.6.1 and update its checksum in Cargo.lock - Update `nixpkgs` in flake.lock with new `lastModified`, `narHash`, and `rev` - Update `rust-overlay` in flake.lock with new `lastModified`, `narHash`, and `rev` - Update `cargoHash` in flake.nix to match the new dependencies * feat: check for sandbox image before running build * fix: update cargoHash for buildRustPackage The cargoHash for the buildRustPackage has been updated from "sha256-0tvfd1uD/SbuNCu0b3mE8y4wszNvKRx3tUVm/dZvCIY=" to "sha256-QpfL3JQ+Amzr5TXjmceefY4NZtW0p/C6g9twpWYlN3s=". This change ensures the integrity and correctness of the package build process. * refactor: reorganize project structure and update import paths - Remove `.vorpal` from `.gitignore` - Add symbolic link for `@vorpal` package in `.vorpal/packages` - Update `NICKEL_IMPORT_PATH` in `flake.nix` to include `./.vorpal/packages` - Move `language.ncl` and `schema.ncl` from `config` to `nickel/packages/@vorpal` - Update import paths in `language.ncl` and `vorpal.ncl` to use `@vorpal` namespace * feat: update design chart * feat: update vorpal.png with new design elements --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
| Commit: | c8f2934 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Build and push sandbox image (#47) * feat(workflow): add sandbox image build job - Introduced a new job `build-image-sandbox` to build and push a sandbox version of the Docker image. - Added environment variable `IMAGE_SANDBOX_NAME` for the sandbox image name. - Configured steps for setting up QEMU, Buildx, and Docker login. - Included metadata action for tagging the sandbox image. - Added build-push-action to build and push the sandbox image. - Configured attestation for build provenance for the sandbox image. * refactor(workflow): rename jobs in docker.yaml for clarity Renamed the job `build-image` to `build` and `build-image-sandbox` to `build-sandbox` in the GitHub Actions workflow file `.github/workflows/docker.yaml` to improve clarity and consistency. * chore(workflow): specify Dockerfile for sandbox build Added the `file: Dockerfile.sandbox` line to the Docker build step in the GitHub Actions workflow to ensure the correct Dockerfile is used for sandbox builds. * refactor(proto): update build_image field to optional and reorder fields - Changed `build_image` field in `PackageBuildRequest` from repeated to optional and reordered it. - Updated `stream.rs` to handle the optional `build_image` field without default value. - Modified `build.rs` to use a default container image if `build_image` is not provided in the request. * refactor: remove docker-compose and update justfile for direct docker usage - Deleted `docker-compose.yml` file. - Updated `justfile` to change image tags from `altf4llc/vorpal` to `vorpal`. - Added `start-docker` and `stop-docker` recipes to `justfile` for managing Docker containers directly. - Removed `up` recipe from `justfile`. * refactor: update clean and logs commands, add --rm to start-docker - Updated `clean` command to remove `.buildx` and `./target` directories. - Removed `down` command and integrated its functionality into `clean` as `stop-docker`. - Modified `logs` command to accept a service parameter and tail the last 100 lines of logs for the specified service. - Added `--rm` flag to `start-docker` command to automatically remove the container when it exits.
| Commit: | c7be95a | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Linux build sandboxes (#40) * feat: first implementation for supporting building on linux systems * refactor: update package preparation and build processes - Added new dependencies and imports for better error handling and logging. - Refactored `setup_agent.sh` to include additional packages. - Enhanced `source.rs` to improve error handling and logging. - Updated `stream.rs` to streamline package preparation and build processes. - Improved `build.rs` to handle package building with better error handling and logging. - Refactored `prepare.rs` to enhance package preparation with better error handling and logging. - Updated `fetch.rs` to include logging for package size. - Renamed `compress_tar_gz` and `unpack_tar_gz` to `compress_gzip` and `unpack_gzip` respectively in `archives.rs`. - Removed unused function `get_package_source_path` from `paths.rs`. * feat: move archives to zstd * feat: remove unpack and re-pack for PackagePrepare rpc endpoint * chore: update dependencies and flake.lock - Updated `cc` crate from version 1.0.102 to 1.0.104 - Updated `hyper` crate from version 1.3.1 to 1.4.0 - Updated `hyper-util` crate from version 0.1.5 to 0.1.6 - Updated `pest`, `pest_derive`, `pest_generator`, and `pest_meta` crates from version 2.7.10 to 2.7.11 - Updated `rustls-webpki` crate from version 0.102.4 to 0.102.5 - Updated `serde_json` crate from version 1.0.118 to 1.0.120 - Updated `flake.lock` with new revisions and hashes for `flake-parts`, `nixpkgs`, `nixpkgs-lib`, and `rust-overlay` - Added `cargo update` command to `update` recipe in `justfile` * feat: initial container implementation * feat: more work to container implementation * refactor(service): remove redundant signature logging and messaging Removed unnecessary logging and messaging of source signatures in the `prepare` function of `stream.rs` and `run` function of `prepare.rs`. This change simplifies the code by eliminating redundant operations related to signature verification messages. * fix: lint * feat: add Docker support and update dependencies - Add .dockerignore file to exclude unnecessary files from Docker build context. - Update .gitignore to include /volume directory. - Update Cargo.lock with new versions of serde, serde_derive, and syn. - Enhance Dockerfile to include multi-stage build for Rust application. - Add docker-compose.yml for container orchestration. - Update flake.lock and flake.nix to remove process-compose-flake and add dockerTools. - Update justfile with new build and clean commands, and add Docker-related commands. - Remove obsolete setup_agent.sh script. - Modify notary module to separate agent and worker key checks. - Update agent and worker services to use new notary key checks. - Enhance store module to include sandbox path checks. - Update paths module to include sandbox path and use it for temporary files. * feat: uses zstd for compression * fix: resolve problem properly packing archives * chore(Dockerfile): update package installation and formatting - Change the case of the `AS` keyword for consistency. - Reorder and uncomment necessary packages for installation. - Remove commented out packages that are not required. - Add `automake`, `help2man`, `make`, `rsync`, and `texinfo` to the list of installed packages. - Ensure apt-get clean and removal of apt lists after installation. * feat(Dockerfile): add bison, g++, and gawk to the build dependencies - Added bison to the list of installed packages. - Uncommented and added g++ to the list of installed packages. - Added gawk to the list of installed packages. * fix: remove canoncialize from get_paths for symlinks, etc * feat: add start-worker command to justfile - Introduced `start-worker` command to run the worker service. - Ensured it builds before running the service. * chore: remove nomad from flake.nix - Removed `nomad` from the list of inherited packages. - Updated `nativeBuildInputs` to exclude `nomad`. * fix(docker-compose): simplify worker service command syntax * chore(Dockerfile): comment out unused dependencies and optimize build steps - Commented out unused dependencies in the apt-get install command to reduce the image size and improve build performance. - Added `cargo check` step before `cargo build` to catch any compilation issues early. - Removed redundant newline between `cargo build` and `cargo test` steps. * chore(deps): update dependencies - Bump async-compression from 0.4.11 to 0.4.12 - Bump http-body from 1.0.0 to 1.0.1 - Bump bytes from 1.6.0 to 1.6.1 - Bump cc from 1.1.0 to 1.1.6 - Bump clap from 4.5.9 to 4.5.10 - Bump clap_builder from 4.5.9 to 4.5.10 - Bump deflate64 from 0.1.8 to 0.1.9 - Bump jobserver from 0.1.31 to 0.1.32 - Bump mio from 0.8.11 to 1.0.1 - Bump object from 0.36.1 to 0.36.2 - Bump openssl from 0.10.64 to 0.10.66 - Bump openssl-sys from 0.9.102 to 0.9.103 - Bump redox_syscall from 0.5.2 to 0.5.3 - Bump rustls from 0.23.11 to 0.23.12 - Bump rustls-webpki from 0.102.5 to 0.102.6 - Bump security-framework from 2.11.0 to 2.11.1 - Bump security-framework-sys from 2.11.0 to 2.11.1 - Bump serde_with from 3.8.3 to 3.9.0 - Bump syn from 2.0.70 to 2.0.72 - Bump thiserror from 1.0.61 to 1.0.63 - Bump thiserror-impl from 1.0.61 to 1.0.63 - Bump tokio from 1.38.0 to 1.39.1 - Bump tokio-macros from 2.3.0 to 2.4.0 - Bump tonic from 0.12.0 to 0.12.1 - Bump tonic-build from 0.12.0 to 0.12.1 Update flake.lock: - Update nixpkgs to rev 68c9ed8bbed9dfce253cc91560bf9043297ef2fe - Update rust-overlay to rev a6afdaab4a47d6ecf647a74968e92a51c4a18e5a Update flake.nix: - Update cargoSha256 to "sha256-5etEhsumYUtYMDzkQi+VpQkjt5GwvjNhClAzHfFflTk=" * fix: lint * chore: update dependencies and remove bash shebangs - Update `anstream` from 0.6.14 to 0.6.15 - Update `anstyle` from 1.0.7 to 1.0.8 - Update `anstyle-parse` from 0.2.4 to 0.2.5 - Update `anstyle-query` from 1.1.0 to 1.1.1 - Update `anstyle-wincon` from 3.0.3 to 3.0.4 - Update `clap` from 4.5.10 to 4.5.11 - Update `clap_builder` from 4.5.10 to 4.5.11 - Update `clap_derive` from 4.5.8 to 4.5.11 - Update `clap_lex` from 0.7.1 to 0.7.2 - Update `colorchoice` from 1.0.1 to 1.0.2 - Update `is_terminal_polyfill` from 1.70.0 to 1.70.1 - Update `nixpkgs` and `rust-overlay` in `flake.lock` - Remove bash shebangs and `set -euxo pipefail` from `justfile` - Modify `start-agent` and `start-worker` to use `sudo` with the built binary * fix: update cargoSha256 hash for Rust package The cargoSha256 hash for the Rust package has been updated from "sha256-5etEhsumYUtYMDzkQi+VpQkjt5GwvjNhClAzHfFflTk=" to "sha256-aOET7RYbm5puxtJieYYYaaayDfxa7AHiM1CZqbCOLJU=". This change ensures the integrity and correctness of the package dependencies. * feat: added docker support in ci * chore: update Docker GitHub Actions to latest versions - Update docker/login-action from v3 to v3 - Update docker/metadata-action from v5 to v5 - Update docker/build-push-action from v6 to v6 * fix(workflow): correct syntax for conditional push in docker.yaml Corrected the syntax for conditional push in the docker.yaml workflow file by wrapping the condition in `${{ }}` for both `push` and `push-to-registry` fields. * fix(workflow): conditionally run attest-build-provenance action - Add condition to run attest-build-provenance action only if the event is not a pull request. - Update push-to-registry parameter to always be true.
| Commit: | 51ec435 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Implements store paths for builds (#34) * refactor: simplify services and build steps * refactor: improve caching for services and structure * chore(deps): update dependencies in Cargo.lock and flake.nix - Update `sync_wrapper` to version `0.1.2` in Cargo.lock - Update `httparse` to version `1.9.4` in Cargo.lock - Add `hyper-rustls` version `0.27.2` to Cargo.lock - Update `spin` to version `0.5.2` in Cargo.lock - Update `miniz_oxide` to version `0.7.4` in Cargo.lock - Update `reqwest` to version `0.12.5` in Cargo.lock - Add `ring` version `0.17.8` to Cargo.lock - Add `rustls` version `0.23.10` to Cargo.lock - Add `rustls-webpki` version `0.102.4` to Cargo.lock - Add `spin` version `0.9.8` to Cargo.lock - Add `sync_wrapper` version `1.0.1` to Cargo.lock - Add `tokio-rustls` version `0.26.0` to Cargo.lock - Add `untrusted` version `0.9.0` to Cargo.lock - Update `zstd-sys` to version `2.0.11+zstd.1.5.6` in Cargo.lock - Update `cargoSha256` in flake.nix to `sha256-kDe3EEc2InW39X+VXyUUwNYPY+mlUh5A3KOpc3ITqNc=` * feat: adding package outputs to sandbox PATH environment variable * feat(proto): add environment map to ConfigPackageBuild and PackageBuildRequest - Updated `config.proto` to include `environment` map in `ConfigPackageBuild`. - Updated `package.proto` to include `build_environment` map in `PackageBuildRequest`. feat(clean): add clean-cache target to justfile - Added `clean-cache` target to `justfile` to remove store cache. refactor(agent): update build package handling in ConfigService - Modified `ConfigService` implementation to handle `build_environment` and populate `build_packages`. refactor(worker): enhance logging and environment handling in PackageService - Updated `PackageService` to log sandbox paths and build packages. - Added handling for `build_environment` in sandbox command. * feat(proto): add sandbox option to config and package build messages - Added `sandbox` field to `ConfigPackageBuild` message in `config.proto`. - Added `build_sandbox` field to `PackageBuildRequest` message in `package.proto`. - Updated `PackagePrepareResponse` and `PackageBuildResponse` messages to use `log_output` as the first field. refactor(service): enhance package handling and sandbox execution - Updated `service.rs` to handle new `sandbox` and `build_sandbox` fields. - Improved package output handling by including `ConfigPackageOutput` in responses. - Enhanced sandbox execution by adding support for Gzip and Xz decoders. - Refined environment variable handling for sandbox execution. fix(worker): correct source hash mismatch error message - Updated error message for source hash mismatch in `service.rs`. refactor(archives): improve tar archive handling - Replaced `Archive` with `ArchiveBuilder` for better control over tar archive extraction. - Removed unnecessary logging in `compress_tar_gz` function. * feat: added support for zip sources * fix: lint * fix: update cargoSha256 for vorpal package in flake.nix The cargoSha256 hash for the vorpal package has been updated to ensure the integrity and correctness of the build process. * refactor: migrate all logs to send as bytes * refactor: update source preparation and file copying logic - Refactored `source_prepare` function to use a temporary source path for unpacking archives and copying files. - Added `source_hash` parameter to `source_prepare` function. - Moved `copy_files` function from `service/worker/service.rs` to `store/paths.rs`. - Updated imports in `service/worker/service.rs` to use consolidated imports. - Updated calls to `copy_files` to use the new location in `paths.rs`. - Improved logging messages for better clarity during source preparation and file operations. * fix(package): remove unnecessary references in function calls Removed unnecessary references in the function calls to `get_package_source_path` and `compress_tar_gz` to align with the expected parameter types. This change improves code readability and correctness. * chore: update flake.lock dependencies - Remove flake-utils and systems dependencies - Update nixpkgs to revision d603719ec6e294f034936c0d0dc06f689d91b6c3 - Update rust-overlay to revision c9a793a5278f711a59fe77b9bf54b215667022c6
| Commit: | 645801d | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Implement streamed responses (#31) * feat: add futures dependency and refactor services - Added `futures` dependency to `Cargo.toml` and `Cargo.lock`. - Updated README.md to correct service start commands. - Renamed `build.proto` to `command.proto` and updated package and service definitions. - Modified `package.proto` to support streaming responses. - Updated `build.rs` to compile new proto files. - Refactored example Rust project to use `tokio-stream` and updated dependencies. - Updated `vorpal.rs` to use `CommandServiceClient` and handle streaming responses. - Modified `flake.nix` to update build inputs and process commands. - Cleaned up `justfile` to remove redundant clean commands. - Refactored `main.rs` to use `clap` for command-line parsing and added subcommands for keys and services. - Removed `command/mod.rs` and integrated its functionality into `main.rs`. - Updated `lib.rs` to include new proto modules and removed `command` module. - Renamed `notary::init` to `notary::check_keys`. - Refactored `run_build.rs` and `run_prepare.rs` to support streaming responses and improved error handling. - Updated `service.rs` to use new streaming response types. - Refactored `proxy` service to use `CommandServiceServer` and updated package handling. - Improved logging and error handling throughout the codebase. * refactor(vorpal): simplify hash assignment in package source Simplified the assignment of the hash value in the PackageSource struct by removing unnecessary line breaks. This change improves code readability without altering functionality. * fix: improve code quality and enforce stricter linting - Updated `justfile` to enforce stricter linting by adding `-D warnings` to `cargo clippy`. - Simplified boolean field initialization in `run_build.rs` by removing redundant field name. - Replaced `format!` with `to_string` for `package_log` in `mod.rs` to improve code readability and performance. * chore: update dependencies - Update `backtrace` from 0.3.72 to 0.3.73 - Update `clap` and `clap_builder` from 4.5.6 to 4.5.7 - Add new package `displaydoc` version 0.2.4 - Update `git2` from 0.18.3 to 0.19.0 - Update `http-body-util` from 0.1.1 to 0.1.2 - Update `httparse` from 1.8.0 to 1.9.3 - Add new packages `icu_collections`, `icu_locid`, `icu_locid_transform`, `icu_locid_transform_data`, `icu_normalizer`, `icu_normalizer_data`, `icu_properties`, `icu_properties_data`, `icu_provider`, `icu_provider_macros` all version 1.5.0 - Update `idna` from 0.5.0 to 1.0.0 - Update `libgit2-sys` from 0.16.2+1.7.2 to 0.17.0+1.8.1 - Add new package `litemap` version 0.7.3 - Update `memchr` from 2.7.2 to 2.7.4 - Update `object` from 0.35.0 to 0.36.0 - Update `redox_syscall` from 0.5.1 to 0.5.2 - Add new package `stable_deref_trait` version 1.2.0 - Add new package `synstructure` version 0.13.1 - Replace `tinyvec` and `tinyvec_macros` with `tinystr` version 0.7.6 - Update `url` from 2.5.0 to 2.5.1 - Add new packages `utf16_iter` version 1.0.5 and `utf8_iter` version 1.0.4 - Add new packages `write16` version 1.0.0 and `writeable` version 0.5.5 - Add new packages `yoke` version 0.7.4 and `yoke-derive` version 0.7.4 - Add new packages `zerofrom` version 0.1.4 and `zerofrom-derive` version 0.1.4 - Add new package `zerovec` version 0.10.2 and `zerovec-derive` version 0.10.2 Update `flake.lock`: - Update `nixpkgs` to `e9ee548d90ff586a6471b4ae80ae9cfcbceb3420` - Update `process-compose-flake` to `9344fac44edced4c686721686a6ad904d067c546` - Update `rust-overlay` to `7c2d603cb67c974ef8c5cfee1150060dbb299e04` * chore(dependencies): update multiple dependencies - Update `backtrace` from 0.3.72 to 0.3.73 - Add new package `cfb` version 0.7.3 - Update `clap` and `clap_builder` from 4.5.6 to 4.5.7 - Add new package `displaydoc` version 0.2.4 - Update `git2` from 0.18.3 to 0.19.0 - Update `http-body-util` from 0.1.1 to 0.1.2 - Update `httparse` from 1.8.0 to 1.9.3 - Add new packages related to `icu` (icu_collections, icu_locid, icu_locid_transform, icu_locid_transform_data, icu_normalizer, icu_normalizer_data, icu_properties, icu_properties_data, icu_provider, icu_provider_macros) all version 1.5.0 - Update `idna` from 0.5.0 to 1.0.0 - Update `infer` from 0.3.7 to 0.16.0 - Update `libgit2-sys` from 0.16.2+1.7.2 to 0.17.0+1.8.1 - Add new package `litemap` version 0.7.3 - Update `memchr` from 2.7.2 to 2.7.4 - Update `object` from 0.35.0 to 0.36.0 - Update `process-stream` from 0.3.3 to 0.4.1 - Update `redox_syscall` from 0.5.1 to 0.5.2 - Update `regex` from 1.10.4 to 1.10.5 - Update `regex-automata` from 0.4.6 to 0.4.7 - Update `regex-syntax` from 0.8.3 to 0.8.4 - Add new package `stable_deref_trait` version 1.2.0 - Add new package `synstructure` version 0.13.1 - Replace `tinyvec` with `tinystr` version 0.7.6 - Update `url` from 2.5.0 to 2.5.1 - Add new packages `utf16_iter` version 1.0.5 and `utf8_iter` version 1.0.4 - Add new package `write16` version 1.0.0 - Add new package `writeable` version 0.5.5 - Add new packages `yoke` version 0.7.4 and `yoke-derive` version 0.7.4 - Add new packages `zerofrom` version 0.1.4 and `zerofrom-derive` version 0.1.4 - Add new packages `zerovec` version 0.10.2 and `zerovec-derive` version 0.10.2 * fix: update cargoSha256 hash for vorpal package Updated the cargoSha256 hash for the vorpal package in flake.nix to ensure the correct dependencies are fetched and built.
| Commit: | fb0705b | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Implements basic local and remote sources support (#17) * feat: support local directories * feat(example): add support for local compressed file in vorpal example - Updated `example/rust/vorpal.rs` to include a new example for building from a local compressed file. - Changed the name of the existing example from "example-source" to "example-local-directory". - Modified `src/service/proxy/package/mod.rs` to simplify the call to `store::unpack_source` by removing unnecessary conversion to `PathBuf`. * refactor(example): rename package and remove local compressed file example Renamed the package from "example-local-directory" to "example-rust" for better clarity. Removed the example of a local compressed file package to streamline the code and reduce redundancy. * feat: add new dependencies and update package preparation and build process - Added new dependencies in Cargo.toml and Cargo.lock including `futures-util`, `git2`, `reqwest`, `tokio-stream`, `url`, and others. - Updated `PackageService` to handle streaming `PrepareRequest`. - Modified `prepare` function to handle different source types (Git, HTTP, Local) and prepare the source accordingly. - Implemented `prepare_source` function to sign and send source data in chunks. - Updated `build` function to handle new source preparation and build process. - Renamed `vorpal.rs` to `build.rs` and updated its content to reflect new build process. - Updated `flake.nix` to include new dependencies and build inputs. - Refactored store module functions to use consistent naming and paths. - Added new functions in notary module for signing data. - Updated service modules to use new store path functions and handle new build process. - Improved error handling and logging throughout the codebase. * refactor(service): simplify error handling in build preparation Simplified the error handling in the build preparation module by using the `map_err` method for more concise and readable code. Removed redundant error logging statements. This change improves code maintainability and readability. * feat: add rust-overlay and cargo-udeps to flake configuration - Added `rust-overlay` input to `flake.nix` and `flake.lock`. - Included `cargo-udeps` in the list of inherited packages. - Updated `devShells` to include `cargo-udeps` in `nativeBuildInputs`. - Configured `rust-overlay` to follow `nixpkgs`. - Added `cargo-nightly` application to `process-compose` configuration. * refactor: rename build.rs to vorpal.rs and update references - Renamed `build.rs` to `vorpal.rs` in `example/rust` directory. - Updated `Cargo.toml` to reflect the new binary name and path. - Modified `flake.nix` to include `clippy` and `rustfmt` in `nativeBuildInputs`. - Replaced `.clone()` with dereferencing in `src/command/mod.rs`. - Simplified function calls by removing unnecessary references in `src/notary/mod.rs`, `src/service/build/build/mod.rs`, `src/service/build/prepare/mod.rs`, `src/service/proxy/package/mod.rs`, and `src/store/mod.rs`. - Improved code readability by using shorthand syntax for increment operations and string concatenations. * refactor: rename modules and update build script - Renamed `build/mod.rs` to `run_build.rs` - Renamed `prepare/mod.rs` to `run_prepare.rs` - Renamed `build/sandbox_default.rs` to `sandbox_default.rs` - Updated `service.rs` to use new module names - Simplified build script generation in `run_build.rs` - Removed redundant error logging in `run_build.rs` - Changed function parameters from `String` to `&str` in `proxy/package/mod.rs` and `store/mod.rs` - Added unit tests for store directory paths in `store/mod.rs` - Updated `flake.nix` to include clippy checks and tests in the build phase and removed clippy from devShells. * refactor: update flake.nix and run_prepare.rs for improved build process - Updated `checkPhase` in `flake.nix` to include `cargo fmt --check --verbose` and modified `cargo test` command to use `--locked --all-features --all-targets`. - Added `rustfmt` to `nativeBuildInputs` in `flake.nix`. - Removed `rustfmt` from `devShells.default.nativeBuildInputs` in `flake.nix`. - Removed `cargo-nightly` app configuration from `apps` in `flake.nix`. - Simplified the creation of `PrepareResponse` in `run_prepare.rs`. * chore: update dependencies - Bump `addr2line` from 0.21.0 to 0.22.0 - Bump `anstyle-query` from 1.0.3 to 1.1.0 - Bump `hyper` from 0.14.28 to 0.14.29 - Bump `backtrace` from 0.3.71 to 0.3.72 - Bump `cc` from 1.0.98 to 1.0.99 - Bump `clap` from 4.5.4 to 4.5.6 - Bump `clap_builder` from 4.5.2 to 4.5.6 - Bump `clap_derive` from 4.5.4 to 4.5.5 - Bump `clap_lex` from 0.7.0 to 0.7.1 - Bump `gimli` from 0.28.1 to 0.29.0 - Bump `object` from 0.32.2 to 0.35.0 - Bump `proc-macro2` from 1.0.84 to 1.0.85 - Bump `tar` from 0.4.40 to 0.4.41 - Bump `tokio` from 1.37.0 to 1.38.0 - Bump `tokio-macros` from 2.2.0 to 2.3.0 - Bump `utf8parse` from 0.2.1 to 0.2.2 Remove `cargo-udeps` from `flake.nix` and update `cargoSha256`.
| Commit: | 04dfd59 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Add proxy service (#9) * chore: update flake.lock dependencies - Updated `flake-parts` to revision `2a55567fcf15b1b1c7ed712a2c6fadaec7412ea8` with new `narHash` `sha256-iKzJcpdXih14qYVcZ9QC9XuZYnPc6T8YImb6dX166kw=`. - Updated `nixpkgs` to revision `57610d2f8f0937f39dbd72251e9614b1561942d8` with new `narHash` `sha256-yZKhxVIKd2lsbOqYd5iDoUIwsRZFqE87smE2Vzf6Ck0=`. - Updated `nixpkgs-lib` tarball URL to `https://github.com/NixOS/nixpkgs/archive/eb9ceca17df2ea50a250b6b27f7bf6ab0186f198.tar.gz` with new `narHash` `sha256-lIbdfCsf8LMFloheeE6N31+BMIeixqyQWbSr2vk79EQ=`. * feat(cli): add CLI service with package request and response - Introduced `cli.proto` defining `CliService` with `Package` RPC. - Added `PackageRequest` and `PackageResponse` messages. - Updated `build.rs` to compile new proto file. - Created example Rust project with `Cargo.toml`, `.gitignore`, and `main.rs`. - Implemented `vorpal.rs` to demonstrate CLI service usage. - Updated `flake.nix` and `flake.lock` to include `process-compose-flake`. - Refactored command module to include `Package` and `Service` commands. - Moved builder module to `service/build` and added `service/proxy`. - Implemented proxy service to handle package requests. * refactor(proto): rename cli service to build service Renamed the `cli.proto` file to `build.proto` and updated the package name from `vorpal.cli.v0` to `vorpal.build.v0`. Updated all references to `CliService` to `BuildService` in the codebase, including the build script, example client, library module, and proxy service. This change reflects the new naming convention and improves clarity in the service's purpose. * feat(api): update build and package proto definitions - Added `build_deps` and `install_deps` fields to `PackageRequest` in `build.proto`. - Removed `Package` message from `package.proto`. - Updated `BuildRequest` in `package.proto` to include `build_deps` and `install_deps`. - Modified `build.rs` to configure and compile proto files. - Updated `Cargo.lock` with new dependencies. - Enhanced `vorpal.rs` to handle new `PackageRequest` structure. - Removed `Package` command from CLI in `mod.rs`. - Added logging for build dependencies in `build/mod.rs`. - Refactored `package/mod.rs` to use updated `PackageRequest` and handle dependencies. * refactor: remove unused dependencies and update build phase messages - Removed unused dependencies from Cargo.lock: - form_urlencoded - idna - tinyvec - tinyvec_macros - unicode-bidi - unicode-normalization - url - Updated build phase messages in vorpal.rs: - Changed "hello, world!" to "foo" in the first build phase - Changed "hello, world!" to "bar" in the second build phase - Removed debug print statements for `foo` and `bar` source IDs in vorpal.rs * refactor(example): simplify build and install phases in vorpal.rs - Consolidate build_phase commands into single-line strings for both 'foo' and 'bar' packages. - Update install_phase commands to single-line strings, removing unnecessary mkdir commands for 'foo' and 'bar' packages. * feat(example): add new package request for 'baz' with dependencies - Introduced a new package request for 'baz' with build and install phases. - Updated the 'bar' package request to store the result in a variable. - Modified the 'bar' install phase to simplify the copy command. - Added 'baz' package request with 'bar' as an install dependency. - Included additional ignore paths for 'baz' package request. * chore: add renovate configuration file for dependency management
| Commit: | c0b89b6 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Implement macOS sandbox (#7) * refactor(proto): remove Status and Retrieve RPCs from PackageService - Removed `Status` and `Retrieve` RPCs from `PackageService` in `package.proto`. - Updated `BuildRequest` message to remove `ignore_paths` field and adjust field numbers. - Removed `StatusRequest`, `StatusResponse`, `RetrieveRequest`, and `RetrieveResponse` messages from `package.proto`. refactor(service): update service implementation to match proto changes - Removed `Status` and `Retrieve` RPC implementations from `service.rs`. - Updated `Build` RPC implementation to handle new `BuildRequest` structure. - Refactored variable names for clarity and consistency. - Added TODO comments for future implementation steps. refactor(database): add find_source_by_id function - Added `find_source_by_id` function to `database/mod.rs` to query source by ID. - Renamed `find_source` to `find_source_by_uri` for clarity. * refactor(builder): modularize build and prepare logic - Moved build logic to `src/builder/build/mod.rs` - Moved prepare logic to `src/builder/prepare/mod.rs` - Updated `src/builder/service.rs` to use the new modularized functions - Added `mod build` and `mod prepare` to `src/builder/mod.rs` for module inclusion - Simplified `impl PackageService for Packager` by delegating to the new modularized functions * feat: add new dependencies and update build process - Added new dependencies: `bstr`, `crossbeam-deque`, `crossbeam-epoch`, `crossbeam-utils`, `globset`, `globwalk`, `ignore`, `pest`, `pest_derive`, `pest_generator`, `pest_meta`, `ryu`, `serde_json`, `signal-hook-registry`, `tera`, `ucd-trie`, `unic-char-property`, `unic-char-range`, `unic-common`, `unic-segment`, `unic-ucd-segment`, `unic-ucd-version`, `tempfile`. - Updated `Cargo.lock` and `Cargo.toml` to include new dependencies. - Modified `package.proto` to change `source_id` type from `string` to `int32` and renamed `BuildResponse` field `data` to `package_data`. - Updated `flake.nix` with new `cargoSha256`. - Enhanced build process in `build/mod.rs` to include new steps for handling temporary directories, scripts, and sandbox execution. - Added `sandbox_default.rs` for sandbox profile template. - Updated `prepare/mod.rs` to handle new source directory structure and unpacking logic. - Modified database schema and functions to store and retrieve source by `hash` and `name` instead of `uri`. - Refactored `store/mod.rs` to include new utility functions for handling paths, file hashes, and source unpacking. - Updated `notary/mod.rs` to use async file operations and reduced key size to 2048 bits. * chore: add TODO comment to improve profile with more granular permissions in sandbox_default.rs * feat(api, builder): add compression flag to BuildResponse - Updated `package.proto` to include `is_compressed` field in `BuildResponse` message. - Modified `run` function in `mod.rs` to set `is_compressed` flag based on the presence of tar file. - Added OS type check to ensure the build runs only on macOS. - Adjusted build output path logic to handle both compressed and uncompressed package data. - Improved logging to display the correct build output path.
| Commit: | d7e12b8 | |
|---|---|---|
| Author: | Erik Reinert | |
| Committer: | GitHub | |
Implement in Rust (#4) * refactor: implement timebox rust poc * refactor: implement in rust including prepare stage * chore: update project description in flake.nix * feat: hash verification between client and server * feat: add file permission settings and start commands - Added `_start` command to `justfile` for running profiles with `nix`. - Introduced `start` and `start-build` commands in `justfile`. - Modified `prepare` function in `package/mod.rs` to set directory and tar file permissions. - Updated `service/mod.rs` to set permissions for source tar and directory. - Implemented `set_files_permissions` function in `store/mod.rs` to handle file permissions. - Ensured file and directory permissions are set to read-only or executable as needed. * feat: add RSA encryption, database integration, and package signature verification - Added `ahash`, `base64ct`, `byteorder`, `const-oid`, `der`, `fallible-iterator`, `fallible-streaming-iterator`, `hashlink`, `lazy_static`, `libm`, `libsqlite3-sys`, `num-bigint-dig`, `num-integer`, `num-iter`, `num-traits`, `pem-rfc7468`, `pkcs1`, `pkcs8`, `pkg-config`, `rsa`, `rusqlite`, `signature`, `smallvec`, `spin`, `spki`, `subtle`, `vcpkg`, `zerocopy`, `zerocopy-derive`, `zeroize` dependencies. - Updated `Cargo.toml` to include `rand`, `rsa`, and `rusqlite` dependencies. - Modified `package.proto` to include `source_signature` in `PrepareRequest`. - Updated `flake.nix` to include `openssl` and `pkg-config`. - Enhanced `main.rs` to initialize directories, generate keys, and set up the database. - Created `database` module for SQLite operations. - Created `notary` module for RSA key generation and retrieval. - Updated `package` module to sign source tar and send signature in `PrepareRequest`. - Enhanced `service` module to verify source signature and store source information in the database. - Updated `store` module to manage directory paths and key file paths.