Proto commits in scaleoutsystems/fedn

These 46 commits are when the Protocol Buffers files have changed:

2025-05-09

Feature/SK-1261 | Split Learning in FEDn (#776)

The documentation is generated from this commit.

2025-05-07

Fixed spelling errors

MVP

2025-05-05

WIP

2025-04-28

merging master (updating grpc files)

The documentation is generated from this commit.

2025-04-25

Feature/SK-1566 | Added Telemetry Store/DTO/Route/GRPC (#897)

2025-04-24

merging in current master

2025-04-23

Added telemetry

2025-04-14

Feature/SK-1546 | Added attributes GRPC calls #882

2025-04-11

resolving merge conflicts

2025-04-07

Feature/SK-1520 | Added SendModelMetric gRPC call (#866)

2025-03-06

merge in master

2025-03-05

send cpu and memory utilisation via send heart beat (#837)

2024-12-20

fixing grpc

2024-12-19

resolving merge conflict from master

2024-12-18

initial splitlearning implementation

2024-12-04

Feature/SK-1246 | Change proto enum WORKER to CLIENT (#768)

Bugfix/SK-1241 | Fix missing ROLE and TYPE in client status messages (#766)

2024-10-31

Feature/SK-946 | Add functionality for user defined server-functions (#666)

2024-10-23

Feature/SK-1081 | Use stores in Combiner + ModelPredict (#718)

2024-06-25

Feature/SK-751 | Clients should have unique IDs (#638)

2024-05-07

Feature/SK-736 | Ruff linter (#552) * revert --fix etc * move setup.py configs * reset repo change * move init for pyproject * include setup.py changes * remove generated * fix minor ruff errors * ruff fix * remove setup.py * update test build path * cli path update * update cli path * update cli path * fix cli path * change dockerfile path * change compose paths * update compose paths * change path * change pash * add editable * add virtualenv to pyproject * fix pyproject.toml * change project structure * trigger CI * trigger CI * fix paths compose * fix compose paths * setuptools for -e . in dockers * revert setuptools enforce * enforce setuptools>65 in dockerfile

2024-03-01

Feature/SK-544 | Server side changes for c++ client (#534)

2024-02-15

Feature/SK-669 | Store session id on status (event) and validation objects (#528) * client status log stores session id * validations stores session id

2024-02-09

Bugfix/SK-675 | Model staging not in sync, TaskStream (#523)

status timestamp type change: string => timestamp (#527)

validation timestamp type change: string => timestamp (#526)

2024-01-25

Make choice of aggregator dynamic at session level and additional aggregators (FedOpt) (#498) * Removed unused base class * work in progress, fedadam * Fix race condition in docker-compose template * Working fedopt, sgd as server side optimizer * Working fedopt, sgd as server side optimizer * Simple notebook demonstrating use of API to run an experiment and to plot validation results * Make it possible to configure the aggregator per session * code checks * Set initial model_id in session config * fedadam working for pytorch * Rename numpyarrayhelper to numpyhelper * Updated helper interface with numerics primitives * PyTorch models now use list of numpy ndarray as format * kerashelper and pytorchhelper consolidated into one numpyhelper * Cleaned a bit in examples and added documentation * removed non working healthcheck * codechecks * add back inference entrypoint * Update integration tests * codechecks * Fix imports * Removed unused arguments to combine_models * Refactor helper module and update unit tests * Refactor helper module * codecheck * Improve aggrgator interface * codecheck * Removed addition to fedn.yaml * Updated docstrings * Changed RoundControl to RoundHandler to avoid confusion with the global Controller * Clean up notebook * Moved notebook into pytorch example folder * Added notebooks in torch example folder to gitignore * Updated docstrings --------- Co-authored-by: Andreas Hellander <andreas.hellander@gmail.com>

2023-12-11

Feature/SK-604 | Refactor common (#491) --------- Co-authored-by: Andreas Hellander <andreas.hellander@gmail.com> Co-authored-by: Fredrik Wrede <fredrik@scaleoutsystems.com>

2023-12-07

Feature/SK-592 | Refactor gRPC server (#490)

2023-10-25

Feature/SK-505 | Flush model update queues at new session + Buffer size config (#476) * model update queues gets flushed in the beginning of a new session * Changed confusing log message * buffer_size now configurable, solves sk-520 * exclude isort for protobuf files * Deleted commented code * Added response status message --------- Co-authored-by: Andreas Hellander <andreas.hellander@gmail.com> Co-authored-by: Fredrik Wrede <fredrik@scaleoutsystems.com>

2023-08-02

Feature/SK-505 | Flush model update queues at new session + Buffer size config (#476) * model update queues gets flushed in the beginning of a new session * Changed confusing log message * buffer_size now configurable, solves sk-520 * exclude isort for protobuf files * Deleted commented code * Added response status message --------- Co-authored-by: Andreas Hellander <andreas.hellander@gmail.com> Co-authored-by: Fredrik Wrede <fredrik@scaleoutsystems.com>

2023-05-30

Refactor core for a more streamlined API for aggregators/controllers (#459) * Add instructions to connect examples to distributed deployment * Update README.md * Update README.md * Update README.md * Formatting and sorting imports in Python files * fix * Fix sorting * inference rest * add inference to validation channel * fixing formatting * formatting * merge option for entripoints * add task type * clean up requirement.txt file/dir * Make default compose a development compose * Add dev mount to pytorch example * Status type for inference * improve entrypoint * fix sorting * Remove license * add stuff to test * add python version to matrix * fix * remove version from name * Increase sleep for reducers and clients * Improve logs * Always print logs * Fix PyTorch example data mount path in compose file Fix PyTorch example data mount path in override compose file. * mets many python versions * quotes * Fix CI sleep time * Add Python versiong * don't fail fast * remove python 3.10 * remove python 3.10 * fix numpy for py 3.7 * Inference CI * minor * fix * fix * fix * fix * fix * fix * fix * reduce CI time * fix conflict * Initial implementation toggle ssl for REST service * Removed unused reducer inference interface mockup * Removed geoip2 dependency * Dockerfile update, install developer tools * Draft implementation * Remove mocked inference endpoint in restservice * Develop (#418) * validate user-specified name (#415) * Delete old Docker-related files (#412) Co-authored-by: Marco Capuccini <marco@scaleoutsystems.com> * fix code-checks * insecure mode in ci (http) * secure option to package download and checksum * work in progress * fix flake8 warning * Remove Talisman * bugfix, combiner now correctly uses secure flag in connector * Revert accidetal change to compose file * sort import * Changed combiner ssl default config to False * Fixed code checks * Code checks * Add docstings in connecy.py * Add docstings in certificatemanager * Docstrings * Changed some parameter names in reducer CLI * Default no-ssl for REST, ssl for gRPC * Fix code check * Harmoize option names between combiner and reducer * Add help text for combiner options * Make --secure option flag * Works to disable secure grpc * Added back use of copy * Remove possibility to generate cert for reducer * Default to insecure gRPC setting * Fix code scanning alerts * Initial refactor * Initial refactor reducer * Introduce base class for controller * More refactoring and cleaning * refactored look-aside loadbalancer * Refactored load-balancer * Fixed code checks * latest * work in progress * Fixed code checks * Update control page * added metadata field to modelupdaterequest * Client passes on metadata dict with model update * Latest * Latest * latest * Refactor aggregation * Fix * Add docstring for load_model_update * Extract model update metadata and make available in aggregator * Added some docstrings * More docstrings * Renamed aggregator files and base class * suppress LOG status messages in stdout * Introduce policy for when to trigger aggregation at combiner * Latest * Added files * Fixes * Fixed broken congig file generation. * Added option to parse client name from config file * Flattened client config file, generalized so that all settings can be passed in the file * Fixed file generation * Latest * Updated config template * Removed mongotracing in control, will refactor to have all tracing data in one collection * Refactored combiner job submit * Remove psutil tracing * Refactor tracer * cleaning * get latest round refactored * Enable early termination by default * Removed unused round_config object * Remove printout of sensitive information * Remove old control, make new version default * Remove unused code * Changed default name for fedn network in config template * Cleaning, docstrings * bugfix * Variable name changes * Removed old combine models implementation * bugfix * Add a hook to validate the model update before putting it on the aggregation queue * Validate metadata on model 'update * Validate metadata on model 'update * incremental weighted average in new style aggregator * small cleaning in control form * Added instructions in controller form, rearranged menu items * latest * Resolve merge conflicts * Added back accidentally removed file * Conflict resolution * Remove unused readme file * More merging * latest * Fixed round_config regression * Controller polls db instead of combiners * More api docs * Add infer_instruct * Cleaning * Added training metadata for keras example * work in progress db cleanup * Refactor * More refactoring in db backend * Remove 'control' setting from reducer config file * Flatten combiner config * Flatten combiner config * Flatten combiner config * Harmonize CLI option names * Refactor helpers * Refactor helpers * Refactor helpers * Refactor helpers * Refactor helpers * Plugin arch for helpers * Updated UI config * Raise exception if misconfigured helper * Added tracing of sessions in the db * Update version to 0.5-dev * Updated torch version * Updated torch version * bugfix * Skip osx tests * latest * change helper name * fix formatting and syntax * fix formatting and syntax errors * update ci new db * fix round_id key and equal weight to reduce models * save helper for metrics and metadata * improve readability and add test for fedavg * update doc strings for client and combiner * Resolve conflict * formatting * add id to logging * extra logging and doc strings --------- Co-authored-by: mcapuccini <marco@scaleoutsystems.com> Co-authored-by: Andreas Hellander <andreas.hellander@gmail.com> Co-authored-by: Fredrik Wrede <fredrik@scaleoutsystems.com>

2021-01-29

updates to develop

2021-01-22

pie chart

2020-09-30

Updated protocol

2020-09-07

Basic control flow working

2020-09-02

- Major library refactor restructure.

2020-08-30

- Introduced message for connection status checking of combiner. Ensuring combiner requirements are fullfilled on the fly. - Client - new connector to check where to be assigned. - Combiner - new connector to announc ready to be tasked. - Reducer takes over most of discoverys responsibilities. The benefits of this is - Client adhoc reassignment - Client failover tolerance - Combiner logic for client is robust and can handle client dropouts.

2020-08-29

- Protocol includes new control interface for Combiner.

2020-08-22

Started specifying new interface for Combiner Control through gRPC.

2020-08-04

Streaming Models - Models can now be requested from combiner. Combiner holds candidate models received (in memory atm) and will upon validation register global model in backend persistent storage at checkpoints (currently s3 is default).

Streaming Models - Models can now be requested from combiner. Combiner holds candidate models received (in memory atm) and will upon validation register global model in backend persistent storage at checkpoints (currently s3 is default).

2020-07-24

Streaming Models - Models can now be requested from combiner. Combiner holds candidate models received (in memory atm) and will upon validation register global model in backend persistent storage at checkpoints (currently s3 is default).

Streaming Models - Models can now be requested from combiner. Combiner holds candidate models received (in memory atm) and will upon validation register global model in backend persistent storage at checkpoints (currently s3 is default).

2020-07-14

- Taking the repo public. -- Adding components for FEDn