Proto commits in crawlab-team/crawlab

These 42 commits are when the Protocol Buffers files have changed:

Commit:f441265
Author:Marvin Zhang

feat(sync): add gRPC file synchronization service and integrate end-to-end - add proto/services/sync_service.proto and generate Go pb + grpc bindings - implement SyncServiceServer (streaming file scan + download) with: - request deduplication, in-memory cache (TTL), chunked streaming - concurrent-safe broadcast to waiters and server-side logging - register SyncSvr in gRPC server and expose sync client in GrpcClient: - add syncClient field, registration and safe getters with reconnection-aware timeouts - integrate gRPC sync into runner: - split syncFiles into syncFilesHTTP (legacy) and syncFilesGRPC - Runner now chooses implementation via config flag and performs streaming scan/download - controller improvements: - add semaphore-based rate limiting for sync scan requests with in-flight counters and logs - misc: - add utils.IsSyncGrpcEnabled() config helper - improve HTTP sync error diagnostics (Content-Type validation, response previews) - update/regenerate many protobuf and gRPC generated files (protoc/protoc-gen-go / protoc-gen-go-grpc version bumps)

Commit:8c2c23d
Author:Marvin Zhang

feat: Update gRPC service definitions and implement CheckProcess method - Downgraded protoc-gen-go-grpc and protoc versions for compatibility. - Added CheckProcess method to TaskService with corresponding request and response types. - Updated Subscribe and Connect methods to use new generic client stream types. - Refactored server and client implementations for Subscribe and Connect methods. - Ensured backward compatibility by maintaining existing method signatures where applicable. - Added necessary handler for CheckProcess in the service descriptor.

Commit:c0e230e
Author:Marvin Zhang

refactor: rename PING code to HEARTBEAT in node service and update related proto files

Commit:65aeb3e
Author:Marvin Zhang

feat: add PING mechanism for connection health checks; update proto and generated files - Introduced PING code in TaskServiceConnectCode enum for health checks. - Updated Runner to use proper PING messages instead of fake log messages for connection health checks. - Modified TaskServiceServer to handle PING requests and acknowledge them. - Adjusted generated gRPC files to reflect changes in proto definitions and ensure compatibility.

Commit:8bd3ef0
Author:Marvin Zhang

feat: add goroutine count metric to the MetricService and update related files

Commit:17f8917
Author:Marvin Zhang

chore: update dependencies and enhance gRPC services - Updated various dependencies in go.mod and go.sum files, including cloud.google.com/go/compute/metadata to v0.5.2, google.golang.org/grpc to v1.69.2, and google.golang.org/protobuf to v1.36.1. - Refactored gRPC service definitions to use the latest protoc-gen-go and protoc versions, ensuring compatibility with the latest gRPC-Go features. - Introduced a new logger utility in core/utils/logger.go to improve logging capabilities across the application. - Added a README.md for gRPC setup and compilation guidance, enhancing developer experience. - Improved the Python installation script to handle version listing more effectively and ensure user-specific environment setup.

Commit:8842621
Author:Marvin Zhang

feat: updated dependency management

Commit:e77d4cd
Author:Marvin Zhang

feat: updated dependency management

Commit:79c1d5d
Author:Marvin Zhang

feat: updated dependency config setup

Commit:53643da
Author:Marvin Zhang

feat: updated dependency config setup

Commit:0f034e9
Author:Marvin Zhang

feat: added dependency config setup (wip)

Commit:c5c08df
Author:Marvin Zhang

feat: added dependency config setup (wip)

Commit:25580b4
Author:Marvin Zhang

feat: prepare for dependency lang install/setup

Commit:89f93aa
Author:Marvin Zhang

feat: prepare for dependency lang install/setup

Commit:0f34a54
Author:Marvin Zhang

feat: optimized dependency logic

Commit:a0989d3
Author:Marvin Zhang

feat: optimized dependency logic

Commit:48f3ffa
Author:Marvin Zhang

feat: updated dependency handler logic

Commit:10bb511
Author:Marvin Zhang

feat: updated dependency handler logic

Commit:4f6ac6f
Author:Marvin Zhang

feat: optimizing dependency services including grpc, api

Commit:fbf8e5f
Author:Marvin Zhang

feat: optimizing dependency services including grpc, api

Commit:71f0a21
Author:Marvin Zhang
Committer:Marvin Zhang

refactor: fixed dependency errors

Commit:ca5bbe6
Author:Marvin Zhang
Committer:Marvin Zhang

refactor: fixed dependency errors

Commit:fa14330
Author:Marvin Zhang
Committer:Marvin Zhang

refactor: updated grpc services

Commit:789f71f
Author:Marvin Zhang
Committer:Marvin Zhang

refactor: updated task related grpc proto

Commit:8a5f51d
Author:Marvin Zhang
Committer:Marvin Zhang

refactor: updated grpc services

Commit:f0f0aa9
Author:Marvin Zhang
Committer:Marvin Zhang

refactor: updated task related grpc proto

Commit:bbe43fa
Author:Marvin Zhang

refactor: Update SendNotification function to handle old and new settings triggers

The documentation is generated from this commit.

Commit:8c79d51
Author:Marvin Zhang

refactor: Update SendNotification function to handle old and new settings triggers

The documentation is generated from this commit.

Commit:7eb8c08
Author:Marvin Zhang

feat: added metric service

Commit:9084680
Author:Marvin Zhang

feat: added metric service

Commit:840100d
Author:Marvin Zhang

fix: unable to sync directories to work nodes

Commit:023ba27
Author:Marvin Zhang

fix: unable to sync directories to work nodes

Commit:7bdce1a
Author:Marvin Zhang

feat: added metrics service v2

Commit:8d3eba1
Author:Marvin Zhang

feat: added metrics service v2

Commit:9c620fa
Author:Marvin Zhang

fix: get all spider issue

Commit:11a0134
Author:Marvin Zhang

fix: get all spider issue

Commit:35bc3a5
Author:Marvin Zhang

feat: updated grpc for dependencies service

Commit:9727139
Author:Marvin Zhang

feat: updated grpc for dependencies service

Commit:e5b20af
Author:Marvin Zhang

feat: updated grpc proto

Commit:3e2c071
Author:Marvin Zhang

feat: updated grpc proto

Commit:0b67fd9
Author:Marvin Zhang

feat: added modules

Commit:c4d795f
Author:Marvin Zhang

feat: added modules