Proto commits in ORNL/DataFed

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

2026-04-01

[DAPS-1920] - release: v5.1.0 * [DAPS-1663] - feature: core, LogContext to dbGetRaw, Correlation_ID to dbMaintenance, metricThread and task_worker (#1885) * [DAPS-1890] - fix: test, mock_core server build (#1890) * [DAPS-1887] - refactor: facility fuse, remove dead code. (#1888) * [DAPS-1857] - feature: python client, tests, support schema functions, to hit feature parity with web ser… (#1859) * [DAPS-1855] - feature, core, add schema factory to decouple schema logic (#1891) * [DAPS-1896] - feature: foxx, schema format and type added to schema create API (#1897) * [DAPS-1893] - feature: common, proto3 add fields for schema type, format, and metadata format (#1894) * [DAPS-1830-1] - refactor: core foxx refactored json schema integration 1 (#1892) * [DAPS-1857-2] - feature: python client schema support (#1895) * [DAPS-1830-2] - refactor: core foxx refactored json schema integration (#1899) * [DAPS-1902] - feature: core, common, get schema api client into compliance with API spec file. (#1903) * [DAPS-1906-1] - feature: add core unit testing to CI (#1907) * [DAPS-1830-3] - refactor: core foxx refactored json schema integration 3 1827 (#1898) * [DAPS-1910] - upgrade: playwright 1.51.1 version. (#1911) * [DAPS-1914] - fix: web, bug in schema id name version mismatch (#1915) * [DAPS-1913] - refactor: foxx ci test scripts consolidate database name and default to different database for tests (#1917) * [DAPS-1916] - tests: add integration test for schema client handler to cmake (#1918) * [DAPS-1912] - refactor: register linkml storage engine with schema handler (#1912) * [DAPS-1919] - feature: python, web support linkml schema (#1921) * [DAPS-1923] - update: version numbers bumped. (#1923) Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> Co-authored-by: Austin Hampton <amh107@latech.edu> Co-authored-by: Blake Nedved <blakeanedved@gmail.com> Co-authored-by: Polina Shpilker <infinite.loopholes@gmail.com> Co-authored-by: JoshuaSBrown <brownjs@ornl.gov>

The documentation is generated from this commit.

2026-03-31

Stage release (#1920) * refactor: core server files refactored to support re-use in the mock implementation * refactor: changed mock core server to work with proto3 and reusue abstractions from core server. * fix: prevent defaults being set to undefined, and interpret numbers a… (#1861) * fix: prevent defaults being set to undefined, and interpret numbers and enums as strings. * chore: Auto-format JavaScript files with Prettier * fix: version numbers from proto3 messages follow camel case. (#1868) * docs: fix jsdoc error. * [DAPS-1862] - fix allocation change failure (#1864) * [DAPS-1663] Adding LogContext to dbGetRaw, Correlation_ID to dbMaintenance, metricThread and task_worker (#1885) Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> Co-authored-by: Joshua S Brown <brownjs@ornl.gov> * fix: mock_core server build (#1890) * [DAPS-1887] - refactor: facility fuse, remove dead code. (#1888) * [DAPS-1857] - feature: python client, tests, support schema functions, to hit feature parity with web ser… (#1859) * [DAPS-1855] - feature, core, add schema factory to decouple schema logic (#1891) * [DAPS-1896] - feature: foxx, schema format and type added to schema create API (#1897) * [DAPS-1893] - feature: common, proto3 add fields for schema type, format, and metadata format (#1894) * [DAPS-1830-1] - core foxx refactored json schema integration 1 (#1892) * [DAPS-1857-2] python client schema support (#1895) Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * [DAPS-1830-2] - core foxx refactored json schema integration (#1899) * [DAPS-1902] - core, common, feat: get schema api client into compliance with API spec file. (#1903) * [DAPS-1906-1] - feature: add core unit testing to CI (#1907) Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * [DAPS-1830-3] - core foxx refactored json schema integration 3 1827 (#1898) * [DAPS-1910] - upgrade: playwright 1.51.1 version. (#1911) * [DAPS-1914] - bug, web schema id name version mismatch (#1915) * [DAPS-1913] - refactor: foxx ci test scripts consolidate database name and default to different database for tests (#1917) * [DAPS-1916] - tests add integration test for schema client handler to cmake (#1918) Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> --------- Co-authored-by: JoshuaSBrown <brownjs@ornl.gov> Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> Co-authored-by: Austin Hampton <44103380+megatnt1122@users.noreply.github.com> Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>

The documentation is generated from this commit.

2026-03-23

[DAPS-1893] - feature: common, proto3 add fields for schema type, format, and metadata format (#1894)

The documentation is generated from this commit.

refactor: simplify proto message attributes.

The documentation is generated from this commit.

feat: common, proto3, add support for schema_type, metadata_format, schema_format

2026-02-25

[DAPS-1874] - release: v5.0.0 * [DAPS-1835] - refactor: common, rework protobuf files following proto3 design principles, replace monolithic SDMS.proto/SDMS_Anon.proto/SDMS_Auth.proto with 186 individual proto3 message files and envelope-based message routing * [DAPS-1837] - refactor: common, update common library to use new proto3 envelope for serialization * [DAPS-1845] - refactor: repo, repository server refactored to be compatible with proto3 envelope * [DAPS-1848] - refactor: authz, upgrade Authz module to use new proto3 envelope for serialization * [DAPS-1847] - refactor: python_client, python client updated to support proto3 * [DAPS-1850] - refactor: web, update web server to use proto3 files * [DAPS-1825] - refactor: core, repo, authz, gcs, upgrade crypto and openssl dependencies * [DAPS-1839] - refactor: ci, split combined build/retag CI pipeline files into separate job files * [DAPS-1774] - test: core, python_client, foxx, web, add query end-to-end tests * [DAPS-1786] - test: web, add test for hitting password reset * [DAPS-1522] - refactor: foxx, comprehensive logging improvements across all routers * [DAPS-1788] - refactor: foxx, authz_router logging improvements * [DAPS-1789] - refactor: foxx, data_router logging improvements * [DAPS-1795] - refactor: foxx, group_router logging improvements * [DAPS-1796] - refactor: foxx, note_router logging improvements * [DAPS-1797] - refactor: foxx, metrics_router logging improvements * [DAPS-1801] - refactor: foxx, topic_router logging improvements * [DAPS-1813] - refactor: foxx, config_router logging improvements * [DAPS-1814] - refactor: foxx, admin_router logging improvements * [DAPS-1817] - refactor: foxx, acl_router logging improvements * [DAPS-1818] - refactor: foxx, coll_router logging improvements * [DAPS-1820] - refactor: foxx, query_router logging improvements * [DAPS-1821] - refactor: foxx, schema_router logging improvements * [DAPS-1822] - refactor: foxx, proj_router logging improvements * [DAPS-1823] - refactor: foxx, repo_router logging improvements * [DAPS-1838] - refactor: foxx, general logging cleanup * [DAPS-1862] - fix: core, fix record allocation change failure * [DAPS-1872] - fix: core, web, foxx, fix tests running until timeout, proto3 compatibility bugs, log bugs, removed proj search * [DAPS-1824] - fix: gcs, web, fix mapped collection consent flow * [DAPS-1839] - fix: ci, CI image retag triggered where a build is needed * [DAPS-1833] - fix: ci, end to end tests failing in CI * [DAPS-1805] - refactor: foxx, refactor user tokens expiring route * [DAPS-1808] - fix: foxx, user router logging fix * [DAPS-1803] - refactor: foxx, fix user update logs leaking sensitive information * [DAPS-1777] - fix: foxx, fix user_router regression in missing response * [DAPS-1775] - fix: core, foxx, add missing {}, foxx query_router add params object schema to routes * [DAPS-1277] - fix: common, core, PROXY_BASIC_ZMQ and PROXY_CUSTOM correctly defined * [DAPS-1790] - fix: common, core, repo, xix zmq assertion failure during EXCEPT call due to calling zmq_msg with invalid state after closing it * [DAPS-1791] - fix: python_client, fix requirements.txt being moved to a folder named requirements.txt during cmake configure script * [DAPS-1861] - fix: web, rrevent defaults being set to undefined, and interpret numbers and enums as strings * [DAPS-1868] - fix: web, version numbers from proto3 messages follow camelCase * [DAPS-1877] - fix: web, fix user router /update Contributors Co-authored-by: Joshua S Brown joshbro42867@yahoo.com Co-authored-by: Blake Nedved nedvedba@ornl.gov Co-authored-by: Blake Nedved blakeanedved@gmail.com Co-authored-by: nedvedba 145805866+nedvedba@users.noreply.github.com Co-authored-by: Austin Hampton amh107@latech.edu Co-authored-by: Austin Hampton 44103380+megatnt1122@users.noreply.github.com

[DAPS-1872] - fix: QA, tests running until timeout, proto3 compatibility bugs, log bugs, removed proj search

fix: revert proto enum names to what they were previously.

2026-02-24

fix: add attribute to query update to distinguish partial replace from full document replace.

2026-02-18

[DAPS-1770] - release: v4.0.0 (#1772) * [DAPS-1770] - release: v4.0.0 * [DAPS-1585] - update: dependencies, upgrade ssl dependency. 3.2.5 (#1646) * [DAPS-1605] - fix: scripts, install_foxx.sh by splitting ssl_args (#1623) * [DAPS-1651] - refactor: scripts, compose, univify treatment of env variables in compose env generator (#1656) (#1658) * [DAPS-1675] - feature: foxx, adding the logger functions for future PR's (#1675) * [DAPS-1659] - refactor: scripts, remove dependencies install scripts (#1660) * [DAPS-1670] - feature: common, core, repo, python_client, web, allow passing repo types in protobuf messages (#1670) * [DAPS-1671] - feature: foxx, add repository and execution strategy types (#1672) * [DAPS-1661] - refactor: compose, scripts, remove remaining occurences of zeromq system secret. (#1661) * [DAPS-1522] - refactor: foxx, user router logging improvements, remove non helpful logs from tasks.js (#1629) * [DAPS-1691] - refactor: foxx, adjust validation.js swap g_lib with error_code require rem… (#1691) * [DAPS-1692] - tests: ci, End-to-end web tests, fix flaky test (#1693) * [DAPS-1694] - refactor: foxx, move permissions functions from support.js to lib/permissions (#1695) * [DAPS-1685] - feature: compose, enable arangodb ssl (#1687) * [DAPS-1700] - fix: ci, limit arangodb job output to last 3 hours. (#1701) * [DAPS-1676] - feature: foxx, arango add factory for repositories for metadata and globus (#1697) * [DAPS-1718] - feature: web, core, python client, Protobuf ExecutionMethod enum, add RepoAllocationCreateResponse (#1719) * [DAPS-1713] - refactor: core, web, python client, protobuf, allow optional fields when creating repo to support metadat… (#1714) * [DAPS-1715] - refactor: core, make path, pub_key, address, endpoint optional in repoCreateRequest (#1716) * [DAPS-1705] - feature: foxx, integrate metadata globus factory repo router create (#1706) * [DAPS-1688] - update: dependencies, core, repo, authz, gcs, Crypto libssl switched to version 3 globus_sdk version pinned (#1689) * [DAPS-1729] - fix: ci, downstream datafed dependencies pipelines are building the container image from incorrect sha (#1732) * [DAPS-1711] - refactor: foxx standardize repo response schema (#1712) * [DAPS-1725] - refactor: remove confusing apache conf file. (#1728) * [DAPS-1707] - update: dependencies, web, update web dependencies before install (#1709) * [DAPS-1522] - refactor: foxx, task router logging improvements (#1648) * [DAPS-1522] - refactor: foxx, query router logging improvements (#1627) * [DAPS-1735] - bug: foxx, remove duplicate user_router test (#1736) * [DAPS-1731] - reature: scripts, compose, add scripts to generate globus credentials for web service (#1731) * [DAPS-1725] - refactor: tests, mock core server centralized (#1726) * [DAPS-1741] - update: scripts, native client id in intialize_globus_endpoint and globus_clea… (#1741) * [DAPS-1745] - fix: scripts, account for nested client credentials. (#1746) * [DAPS-1725-2] - fix; tests, centralized mock core service libraries fixed (part 2) (#1747) * [DAPS-1742] - refactor script replace os.path.join with urllib.parse.urljoin (#1744) * [DAPS-1749] - refactor: cmake, set cmake policy to silence noisy warning. (#1750) * [DAPS-1522] - refactor: foxx, feature tag router logging improvements (#1734) * [DAPS-1378] - fix: web, mapping of multiple globus accounts. (#1753) * [DAPS-1756] - fix: scripts, foxx, add retries and connection check to install_foxx.sh script. (#1757) * [DAPS-1522] - refactor: foxx, Version Router Logging Improvements (#1758) * [DAPS-1737] - refactor: compose, cleanup arango ssl env variables (#1765) * [DAPS-1766] - fix: ci, python client provisioning job * [DAPS-1663] - feature: core Service, adding Correlation ID to Logging (#1704) Co-authored-by: Aaron Perez <perezam@ornl.gov> Co-authored-by: AronPerez <aperez0295@gmail.com> Co-authored-by: Blake Nedved <nedvedba@ornl.gov> Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> Co-authored-by: nedvedba <145805866+nedvedba@users.noreply.github.com> Co-authored-by: Austin Hampton <amh107@latech.edu> Co-authored-by: Austin Hampton <44103380+megatnt1122@users.noreply.github.com> Co-authored-by: Blake Nedved <blakeanedved@gmail.com> Co-authored-by: Polina Shpilker <infinite.loopholes@gmail.com> * [DAPS-1777] - Foxx, fix: user_router fix regression in missing response. (#1778) [DAPS-1786] - Web tests, refactor: add test for hitting password reset. (#1787) * [DAPS-1774] - Core, Python, Database, Foxx, Test add query end to end tests (#1779) * [DAPS-1775] - fix: core, foxx, add missing {}, foxx query_router add params object schema to routes. (#1781) * [DAPS-1777] - fix: foxx, user_router fix regression in missing response. (#1778) * [DAPS-1786] - refactor: web tests, add test for hitting password reset. (#1787) * [DAPS-1277] - fix: mock, core, common, PROXY_BASIC_ZMQ and PROXY_CUSTOM correctly defined * [DAPS-1790] - fix: common, core, repo, zmq assertion failure during EXCEPT call due to callin zmq_msg with invalid state after closing it. * [DAPS-1791] - fix: build, python client, requirements.txt was being moved to a folder named requirements.txt during cmake configure script. * chore: Auto-format JavaScript files with Prettier * [DAPS-1774] - continuation of fix (#1798) * [DAPS-1774] - continuation of fix (#1798) * release: version 4.0.1 (#1807) * refactor: only print subset of user properties. (#1804) * refactor: only print subset of user properties. * chore: Auto-format JavaScript files with Prettier * [DAPS-1806] - refactor: foxx, user tokens expiring route. (#1806) --------- Co-authored-by: Aaron Perez <perezam@ornl.gov> Co-authored-by: AronPerez <aperez0295@gmail.com> Co-authored-by: Blake Nedved <nedvedba@ornl.gov> Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> Co-authored-by: nedvedba <145805866+nedvedba@users.noreply.github.com> Co-authored-by: Austin Hampton <amh107@latech.edu> Co-authored-by: Austin Hampton <44103380+megatnt1122@users.noreply.github.com> Co-authored-by: Blake Nedved <blakeanedved@gmail.com> Co-authored-by: Polina Shpilker <infinite.loopholes@gmail.com>

2026-02-11

[DAPS-1837] - refactor common library to use proto3 (#1842) * [DAPS-1837] - refactor: core server to support proto3 (#1844) * [DAPS-1845] - refactor repo server to use proto3 (#1846) * [DAPS-1847] - refactor: python package to be compatible with proto3 envelope (#1849) * [DAPS-1848] - refactor: update authz files to use proto3. (#1851) * [DAPS-1850] - refactor web server proto3 (#1852)

2026-02-09

refactor: remove all proto files.

2026-02-06

[1835-DAPS] - refactor: proto proto3 with envelope part 2 (#1841)

2026-02-05

refactor: add missing enum files.

refactor: enum reverted to original syntax to be more consistent and introduce fewer changes.

2026-02-04

refactor: Small change to align needed changes in error messages to be consistent and the addition of files to CMake Variable

2026-02-02

[DAPS-1836] - feat: add protobuf3 replacement files. (#1836)

2026-01-29

refactor: change sdms to SDMS in package name to be consistent with what was there before, update INCLUDE directive in CMakeLists.txt from proto to proto3

refactor: make envelope more consistent with existing implementation.

feat: add protobuf3 replacement files.

2025-11-17

[DAPS-1770] - release: v4.0.0 * [DAPS-1585] - update: dependencies, upgrade ssl dependency. 3.2.5 (#1646) * [DAPS-1605] - fix: scripts, install_foxx.sh by splitting ssl_args (#1623) * [DAPS-1651] - refactor: scripts, compose, univify treatment of env variables in compose env generator (#1656) (#1658) * [DAPS-1675] - feature: foxx, adding the logger functions for future PR's (#1675) * [DAPS-1659] - refactor: scripts, remove dependencies install scripts (#1660) * [DAPS-1670] - feature: common, core, repo, python_client, web, allow passing repo types in protobuf messages (#1670) * [DAPS-1671] - feature: foxx, add repository and execution strategy types (#1672) * [DAPS-1661] - refactor: compose, scripts, remove remaining occurences of zeromq system secret. (#1661) * [DAPS-1522] - refactor: foxx, user router logging improvements, remove non helpful logs from tasks.js (#1629) * [DAPS-1691] - refactor: foxx, adjust validation.js swap g_lib with error_code require rem… (#1691) * [DAPS-1692] - tests: ci, End-to-end web tests, fix flaky test (#1693) * [DAPS-1694] - refactor: foxx, move permissions functions from support.js to lib/permissions (#1695) * [DAPS-1685] - feature: compose, enable arangodb ssl (#1687) * [DAPS-1700] - fix: ci, limit arangodb job output to last 3 hours. (#1701) * [DAPS-1676] - feature: foxx, arango add factory for repositories for metadata and globus (#1697) * [DAPS-1718] - feature: web, core, python client, Protobuf ExecutionMethod enum, add RepoAllocationCreateResponse (#1719) * [DAPS-1713] - refactor: core, web, python client, protobuf, allow optional fields when creating repo to support metadat… (#1714) * [DAPS-1715] - refactor: core, make path, pub_key, address, endpoint optional in repoCreateRequest (#1716) * [DAPS-1705] - feature: foxx, integrate metadata globus factory repo router create (#1706) * [DAPS-1688] - update: dependencies, core, repo, authz, gcs, Crypto libssl switched to version 3 globus_sdk version pinned (#1689) * [DAPS-1729] - fix: ci, downstream datafed dependencies pipelines are building the container image from incorrect sha (#1732) * [DAPS-1711] - refactor: foxx standardize repo response schema (#1712) * [DAPS-1725] - refactor: remove confusing apache conf file. (#1728) * [DAPS-1707] - update: dependencies, web, update web dependencies before install (#1709) * [DAPS-1522] - refactor: foxx, task router logging improvements (#1648) * [DAPS-1522] - refactor: foxx, query router logging improvements (#1627) * [DAPS-1735] - bug: foxx, remove duplicate user_router test (#1736) * [DAPS-1731] - reature: scripts, compose, add scripts to generate globus credentials for web service (#1731) * [DAPS-1725] - refactor: tests, mock core server centralized (#1726) * [DAPS-1741] - update: scripts, native client id in intialize_globus_endpoint and globus_clea… (#1741) * [DAPS-1745] - fix: scripts, account for nested client credentials. (#1746) * [DAPS-1725-2] - fix; tests, centralized mock core service libraries fixed (part 2) (#1747) * [DAPS-1742] - refactor script replace os.path.join with urllib.parse.urljoin (#1744) * [DAPS-1749] - refactor: cmake, set cmake policy to silence noisy warning. (#1750) * [DAPS-1522] - refactor: foxx, feature tag router logging improvements (#1734) * [DAPS-1378] - fix: web, mapping of multiple globus accounts. (#1753) * [DAPS-1756] - fix: scripts, foxx, add retries and connection check to install_foxx.sh script. (#1757) * [DAPS-1522] - refactor: foxx, Version Router Logging Improvements (#1758) * [DAPS-1737] - refactor: compose, cleanup arango ssl env variables (#1765) * [DAPS-1766] - fix: ci, python client provisioning job * [DAPS-1663] - feature: core Service, adding Correlation ID to Logging (#1704) Co-authored-by: Aaron Perez <perezam@ornl.gov> Co-authored-by: AronPerez <aperez0295@gmail.com> Co-authored-by: Blake Nedved <nedvedba@ornl.gov> Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> Co-authored-by: nedvedba <145805866+nedvedba@users.noreply.github.com> Co-authored-by: Austin Hampton <amh107@latech.edu> Co-authored-by: Austin Hampton <44103380+megatnt1122@users.noreply.github.com> Co-authored-by: Blake Nedved <blakeanedved@gmail.com> Co-authored-by: Polina Shpilker <infinite.loopholes@gmail.com>

2025-10-23

[DAPS-1713] - Core, Web, Python Client, refactor: protobuf, allow optional fields when creating repo to support metadat… (#1714)

[DAPS-1718] - Web, Core, Python Client - Protobuf ExecutionMethod enum, add RepoAllocationCreateResponse (#1719)

2025-10-03

release: 3.14.0 (#1674)

2025-10-02

1669 feature support core service protobuf message with repo types (#1670)

2025-08-28

[DAPS-1517] Update c++ to support type fields

2025-08-27

fix: make secret optional before removing completely. (#1617)

2025-08-18

[DAPS-1517] Update c++ to support type fields

2025-06-12

[Release] v3.13.01 Resolve Master Branch Diff (#1485) * Address formatting of jquery file * add more descriptive posix path error message * Remove redundant password file creation command * Add eslint plugin for jsdoc to github action * Update esling.config.js to error if invalid jsdoc * Add missing eslint jsdoc plugin * Remove non existant rule * Fix jsdoc issues in data_router * Fix JSDOc issues in process.js fiel * Fix JSDOc issues in repo_router.js * Address JSDoc in tasks.js * Address JSDoc linting issues * Switch to ignores statement * Address more eslint jsdoc items * Be more selective about eslint rules * Remove requrie description because seems to be faulty * Address finicky jsdoc linter * Fix js formatting * Feature DLT 1109 foxx new token data (#1194) * Stub logic for updating related edge in router function for token/set * Include logic for conditionally updating token data and pushing data to edge. * Change token_key to remove illegal characters * Fix object attribute call * Formally add query params to function. Extract query params near start of action. Check both exist before taking action on edge. Update comments, notes, TODOs * Stub testing for user_router * Fix naming of user_router test so it will be picked up with sane specification. Add test to CMakeLists * Add some fixtures for creating related user to allow for integration testing * Fix manifest json. Hard code user collection check. Set time variable. * Use correct path for router to user endpoints. Extract some query params for validation. * Remove intentionally failing test. Break now working test into logical sections. Update comments. * Add test for new features * Add further validation of no unexpected side-effects * Remove extraneous user document checks and comments * More testing for branches in introduced logic. Additional check on newly inserted globus_coll * assert -> expect * Work around multiple access and ordering issue by utilizing more test user fixtures. * Rearrange key for token edge, add suggested fields to documents * Pull user_doc from existing logic. Use object destructuring. Add error condition. Introduce protobuf enum to backend. Add parser for other_token_data string. Update document creation functions * Extract fixture build call to separate script. Add script as fixture in cmake * Remove some extraneous logic, remove password file overwrite. * Remove unused zeromq flag * Adjust testing to include enum, expect error on invalid input combination, typo, naming * Adjust scope string in user_router test. Add validations to parseOtherTokenData. Add tests for positive and error cases of parseOtherTokenData. * Change chai expects in support.test.js to use callable. Update perms on fixture setup script. Use URL encoded pipe character in user_router.test.js * Group OR in user_router.js * Adjust user_router.test.js integration tests to access deconstructed other_token_data * Add baseline jsdoc for parse method. Remove some unwanted data from globus_token doc insert * Add GLOBUS_DEFAULT to AccessTokenType enum. Add default to user_router endpoint param. Update logic to switch based on enum. Test new change by specifying default type and other_token_data. Fix name of test. * Move query parameter validations earlier in function. Decrease nesting of switch logic. * Cleanup some whitespace changes, add desired comments on assumptions. * Formatting * Fix naming for support test so they will run. * Update CHANGELOG.md with changes in minor feature. Update comments on user_router.js for saving a new globus_coll * Add comment about more fields in support.js::parseOtherTokenData * Fix issue for formatter * Add expiration check * Commit for CI * Formatting --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> * Add ci jobs to improve security posture * Change to unit stage for testing * Remove sast job * Attempt to fix semgrep sast job failure * Debug jobs * Add test stage for sast template job * Add missing test stage * Add dependency scanning * Split sast into separate test stage * Remove commented out line * Remove extra comment that is unused * Debugging log failure * Address jsdoc and typo in description * Remove outdated comments * Make comments JSDoc compliant * Address formatting inconsistency * Account for multiple images returned in log * Update .gitlab/common.yml Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Debugging log container matching * Fix typo in gitlab anchor * Debug escape anchor * Debug * Debug anchor 2 * Debug anchor 3 * Debug anchor 4 * Debug anchor 5 * Debug 6 * Debug 7 * Debug 8 * Fix names of containers from gcs-authz to gcs * Cleanup add comment back in * Update common.yml * Remove Exports * Update data_router.js Remove nested docs. * Update repo_router.js * Update task_router.js * Update eslint.config.js * Update eslint.config.js * fix jsdoc comment in posix path * Address JSDOc complaints * Address JSDoc errors * Address linter * Address linter errors * Fix order of parameters * Apply formatting * Add missing argument form strategy method call * Apply prettier * Update posix_path.js * Address ai suggestion about better error message * Debugging log failure * Account for multiple images returned in log * Update .gitlab/common.yml Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Debugging log container matching * Fix typo in gitlab anchor * Debug escape anchor * Debug * Debug anchor 2 * Debug anchor 3 * Debug anchor 4 * Debug anchor 5 * Debug 6 * Debug 7 * Debug 8 * Fix names of containers from gcs-authz to gcs * Cleanup add comment back in * Update common.yml * Remove Exports * Cleanup comments and apply formatting * Update nlhoman json version * Update protobuf versions * Add tests for authz router * Specify ubuntu 20.04 explicitly * Revert dependency versions * Fix problems with path consistent method * Cleanup javascript * Format javascript * Feature DLT 1127 relay new token data (#1217) * Add additional params to DatabaseAPI::userSetAccessToken to pass message with addtional info on to database API. * Add convenience method with old type signature of DatabaseAPI::userSetAccessToken for compatibility * Conditionally add other token options param when calling dbGetRaw. * Pass token type along to Database API. * Pass token type from message. * Extract logic for building URL from dbPost and dbGetRaw into separate function for easier testing and debugging * Extract url building logic from dbGet * Add some comments for moved code, address a TODO, add new TODO for AccessToken Message type field * Address default on token type, formatting * Add new AccessTokenType entries for a sentinel value and Globus default in SDMS.proto, use as desired in DatabaseAPI.cpp * Change query param to match backend changes * Restore formatting of SDMS.proto * Make built URL const where appropriate in DatabaseAPI.cpp and DatabaseAPI.hpp * Change default AccessTokenType for UserSetAccessTokenRequest in SDMS_Auth.proto. * Remove logging of potentially senitive data * Adjust AccessTokenType ACCESS_SENTINEL to have value of uint8 max. --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> * [DLT-1110] Update files with prettier [DLT-1110] Add controller unit tests [DLT-1110] Remove debug [DLT-1110] Correct import statements, endpoint list [DLT-1110] Refactor out into MVC [DLT-1110] Add debug [DLT-1110] Functioning modal, needs refactoring [DLT-1110] Refactor dlg start transfer using OOO programming. Should be MVC cased on what I'm seeing but we'll see * [DLT-1110] Split branches * [DLT-1110] Update model, make logic private, remove controller form transfer request when update or get * [DLT-1110] Add commnets to logic * [DLT-1110] Correct HTML escape, francy tree init * [DLT-1110] Pull out template HTML * [DLT-1110] Fix update bug * [DLT-1110] Dependency injection * [DLT-1110] Update if statements * [DLT-1110] Update style * Revert "[DLT-1110] Mapped Collection Endpoint Browse (1/4)" * Address eslint * Address remaining eslint items * Fix bug on task_router.js abort function (#1234) Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> * Address Aaron feedback * applied potential fix to deprecation warning * Reorder createRecord * Apply formatting * Throw error if OpenStack error code returned from API * Add dependency between logs * Add dependencies on end to end jobs as well * Job rule is not supported * Remove always run log * add end to end signal job * Fix tag of signal * Feature DLT 1120 retrieve transfer token (#1228) * Add conditional switch and stub some logic for handling other resources in callbacks * Fix order of logging. Add redirect * Debug logging for web service. Attempt to set token. * Remove incorrect method call to data, use data attribute. * Add some TODOs and logging * Give setAccessToken ability to accept additional params to pass to protobuf * Add additional params to DatabaseAPI::userSetAccessToken to pass message with addtional info on to database API. * Add convenience method with old type signature of DatabaseAPI::userSetAccessToken for compatibility * Conditionally add other token options param when calling dbGetRaw. * Pass token type along to Database API. * Pass token type from message. * Stub logic for updating related edge in router function for token/set * Include logic for conditionally updating token data and pushing data to edge. * Change token_key to remove illegal characters * Fix object attribute call * Prettier formatting for datafed-ws.js * Roll back changes to DatabaseAPI.hpp and DatabaseAPI.cpp that are covered in #1127 * Refactor new inclusions in datafed-ws.js to use as much existing code as possible, only diverge where necessary. * datafed-ws.js change scopes to scope * datafed-ws.js log user ID array * datafed-ws.js clarify uid assignment at ui/authn endpoint, add note on error cases, remove incorrect comment about uid, add appropriate error if collection_id not present * datafed-ws.js Extract AccessTokenType enum, add comment about fetching from protobuf; remove unnecessary commented code * datafed-ws.js Extract transfer set logic when calling setAccessToken to improve readability of router. * datafed-ws.js Create function for resolving token type; create function to handle logic in building optional_data for setAccessToken; reduce unneccessary nesting in ui/authn endpoint * datafed-ws.js Fix spacing, fix typo in variable name * datafed-ws.js Fix bug referring to request session * datafed-ws.js Nesting was necessary. * datafed-ws.js Address some TODOs, set token type with more context * datafed-ws.js Remove some additional extraneous logging * datafed-ws.js Formatting * Add comment about token set * datafed-ws.js Update deprecated substring method, Fix order for error case when new user receives transfer token * TokenHandler.js Refactor token handling logic out of web server main. datafed-ws.js User new OAuthTokenHandler class to handle token logic. * TokenHandler.js Implement token validator; move OAuthTransferToken def to top. TokenHandler.test.js Minimal testing of token handler * TokenHandler.test.js Add tests for case when resource server is auth * TokenHandler.test.js Add testing for Globus transfer resource server * TokenHandler.js Implement validation for existence of required keys. TokenHandler.test.js Adjust other_tokens fixtures to be nested according to incoming data; formatting. * datafed-ws.js formatting * TokenHandler.js Update error messages, remove unnecessary code in getTokenType, returns in resolveTokenType * datafed-ws.js Add error handling around token_handler construction; add errors and handling for setAccessToken, JSDoc. * datafed-ws.js Remove hard coded collection_id. * datafed-ws.js Remove thrown errors in nested functions; single redirect. --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> * Bug daps 1243 datafed web logging (#1251) * datafed-ws.js Reverse order of LogLevel object properties * datafed-ws.js Throw error so stack trace is adequately populated * Revert line number * TokenHandler.js Fix linting errors. --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> * [DLT-1110] Mapped Collection Endpoint Browse (1/4) (#1240) * [DLT-1110] Update style [DLT-1110] Update if statements [DLT-1110] Dependency injection [DLT-1110] Fix update bug [DLT-1110] Pull out template HTML [DLT-1110] Correct HTML escape, francy tree init [DLT-1110] Add commnets to logic [DLT-1110] Update model, make logic private, remove controller form transfer request when update or get [DLT-1110] Split branches [DLT-1110] Update files with prettier [DLT-1110] Add controller unit tests [DLT-1110] Remove debug [DLT-1110] Correct import statements, endpoint list [DLT-1110] Refactor out into MVC [DLT-1110] Add debug [DLT-1110] Functioning modal, needs refactoring [DLT-1110] Refactor dlg start transfer using OOO programming. Should be MVC cased on what I'm seeing but we'll see * [DLT-1110] Mapped Collection Endpoint Browse Tests (2/4) (#1207) * [DLT-1110] Add tests * [DLT-1110] Update chai with mocha, attempt usage of mock-import * [DLT-1110] Add mocha config, correct prettier * [DLT-1110] Update tests, correct logic * [DLT-1110] Update tests, add fixtures, update setup, lock in packages * [DLT-1110] Update test * [DLT-1110] Config * [DLT-1110] Upgrade node v14 -> v23 * [DLT-1110] Upgrade node v23 -> v18, better jump * [DLT-1110] Correct ver .sh file * Revert "[DLT-1110] Correct ver .sh file" This reverts commit 5c78c59ea79c031452b4fca0179204e0e485b310. * Revert "[DLT-1110] Upgrade node v23 -> v18, better jump" This reverts commit 9d66847ca2ead4bca6f331c93ba206ff9c7695a6. * Revert "[DLT-1110] Upgrade node v14 -> v23" This reverts commit fb8b93d2137035500c138f0ac5f84f0faf35e958. * Revert "[DLT-1110] Config" This reverts commit 82715f07140b039eaf086c768b8ea281ade3da3d. * Revert "[DLT-1110] Mapped Collection Endpoint Browse Tests (2/4) (#1207)" This reverts commit 22545fa1927a0793d49e8172536a74ad0729e8d7. * added updated version t0 requirements.txt * Fix edge case in GridFTP Authz where '/' is used * cpp-py-formatter * Add changelog comment * Address prettier formatting * Add method for testing config, fix bug in global config, fix redundant log statements * Make error detection clearer in bool * Add mock core server * Add integration and, liveness test for mock * Address mock integration setup and consolidate * Apply formatting * Remove unused timeout variable from mock * Address JSDoc complaints * Apply prettier * Fix formatting and eslint prettier issues * Address codacy feedback * Add changelog comment * Clean up grammar in comment * Add note in Config.h * Apply clang formatting * Address codacy issues * Add env var * Update Arangodb packaged version to using 3.12.4 * Fix confusion in key uses in end to end tests * Add change log comment * Add changelog comment * corrected imported member * Add compose entries for foxx tests * Update the schema documentation to improve the usability. * Add example schema with a reference * Add notebook for linting jupyter * Using working directory option * Update .github/workflows/jupyternotebook-lint.yml Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Update python notebooks * Attempt to fix jupyternote book linting errors * Format with black * Specify line length with black to be consistent with flake8 * Add a Changelog comment * Change line length to 88 * Add black formatters * Make formatting consistent across project * Fix linter * Temporarily keep line length in black to 79 and address in separate issue * Change line length to 88 for now : * updated changelog * Fix bug in CI scripts associated with repo and gcs image build * Merge hotfixes into devel (#1326) * [DAPS-DEPS] Update cpp-py-formatter to 0.3.0 * [DAPS-DEPS] Update cpp-py-formatter to 0.3.1 * [DAPS-DEPS] Update ver, add push * updated versions for release * Update project.rst * Update format-check.yml * Pull in version numbers appropriately * Avoid build cache * Aggressive prune * Trigger rebuild * generate_repo_config.sh Accept env variable for datafed server from repo service instead of hard coding * Allow core config thread options to be read in from env * Add changelog comment * Adjust deployment version number * Fix Version month * Update CHANGELOG.md Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> --------- Co-authored-by: Aaron Perez <perezam@ornl.gov> Co-authored-by: Blake Nedved <nedvedba@ornl.gov> Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> Co-authored-by: JoshuaSBrown <brownjs@ornl.gov> Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> Co-authored-by: nedvedba <145805866+nedvedba@users.noreply.github.com> Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Feature DAPS 1215 foxx UserGetAccessTokenRequest mapped collection support (#1284) * Begin prototyping required code for mapped collection token retrieval. * SDMS_Auth.proto Add optional fields to UserGetAccessTokenRequest and UserAccessTokenReply to support finding correct tokens * user_router.js Add baseline query params and checks. DatabaseAPI.cpp & DatabaseAPI.hpp Use updated names and types. ClientWorker.cpp Update comments. * user_router.js Add logic for finding token based on collection; add token match and missing match cases * Return necessary data from Database to refresh tokens. Mock data on dir/list to test out changes. * user_router.js Add check for existence of Globus collection * user_router.js Remove existence check, move to filter method; Add notes on covered cases; add notes on code * datafed-ws.js Temporary handling of need consent response. * user_router.js Add queryParam validation for new params via joilib. ClientWorker.cpp Add refresh token try/catch to send through flow if refresh fails * datafed-ws.js Use correct converted casing for protobuf messages; redirect to error when needs_consent * datafed-ws.js No redirect, show error directly in list. * datafed-ws.js Early return to prevent further requests * support.js Update AccessTokenType enum. user_token.js New class introduced for handling logic for validating params when getting tokens and building GetAccessToken responses. user_router.js Replace logic with calls to new lib * user_token.js Add jsdoc definition for return type and format function. user_router.js Remove unnecessary comment. * user_token.js More jsdoc definition; Add validation for collection token document required fields. user_token.test.js Add preliminary unit tests for static methods. * CMakeLists.txt Add new unit tests to build. user_token.test.js Modify naming to match convention. * CMakeLists.txt Fix variable get. user_token.js Fix export. * user_fixture.js Add new user for get/token tests. user_router.test.js Add simple tests for get/token endpoint. user_token.js Fix naming for token document scopes field. user_token.test.js Changes for scope field bugfix. * user_router.js Formatting; Throw error if more than one token matches, add note about token type. * ClientWorker.cpp Address some TODOs and extraneous comments. DatabaseAPI.cpp Remove TODOs. SDMS_Auth.proto Remove unused field. * user_token.js Update formatUserToken to always return object values. DatabaseAPI.cpp Add note on possibly missing field. user_token.test.js Update tests to check for new expected values. * user_router.js Clean up some comments for clarity. datafed-ws.js Formatting. * ClientWorker.cpp Rewrite comment regarding mapped token refresh. * user_token.js Formatting. * CHANGELOG.md update for feature * user_router.js Re-introduce accidentally removed `globus_collection` object. * CMakeLists.txt Remove comment about necessity * Update token lookup logic to use `byExample`, add test case for missing Globus collection. Add globus_coll fixture. * SDMS_Auth.proto Change UserAccessTokenReply needs_consent to optional field. * user_router.js Update token/get to default to start consent flow when collection tokens are present. user_router.test.js Update test from error to needs_consent. * Formatting changes. --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> * [DLT-1110] Implement Consent Required Action (3/4 & 4/4) (#1242) * [DLT-1110] Refactor browse into component and use CSS * [DAPS-1110] Update * [DAPS-1110] Update web server * [DAPS-1110] add refactors * [DAPS-1110] Prettier * [DAPS-1110] Refactor tests * [DAPS-1110] Remove .. in ep ls, remove logs * [DAPS-1110] Add sinon, remove dead code * [DAPS-1110] Address transfer start button bug * [DAPS-1110] eslint, update api.test.js to reflect usage * [DAPS-1110] format * [DAPS-1110] Correct bug * JSON parse error fix (#1328) * DatabaseAPI.cpp Pull in json dependency, use json object to serialize request payload * Replace json serialization in more locations where parseSearchRequest is being called. * Add comments around locations that will need json serialization * Convert missed body append to serialization format * typo * Refactoring DatabaseAPI.cpp to incorporate json serialization * More serialization work * Pull in changes from 1214 for dataPut * Bring in changes from 1214 for dataGet * DatabaseAPI.cpp Fix error with missing curly brace. Some formatting that is likely wrong. Serialize up to original point. Need to validate. * Add missing scope resolution * More typos * More typos. Scoping * Remove non-existant vars * declare body * Formatting * Formatting and verification of updates * Replace most places * Formatting * Remove redeclaration * DatabaseAPI.cpp Remove some comments. * DatabaseAPI.cpp Add missing push_back * Adding braces * Prevent Double Escaping string, by using nlohmann json parse to read in value. * remove escapeJSON calls, redundant with nlohmann json dump * Allow parsing for metric data to be run in parallel for now, output messages when old serialization is used. * Ensure ASCII setting. * Upgrade debug to warning. --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> Co-authored-by: Austin Hampton <amh107@latech.edu> Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> * Feature - Mapped Collection Token Support (#1334) * Begin prototyping required code for mapped collection token retrieval. * SDMS_Auth.proto Add optional fields to UserGetAccessTokenRequest and UserAccessTokenReply to support finding correct tokens * user_router.js Add baseline query params and checks. DatabaseAPI.cpp & DatabaseAPI.hpp Use updated names and types. ClientWorker.cpp Update comments. * user_router.js Add logic for finding token based on collection; add token match and missing match cases * Return necessary data from Database to refresh tokens. Mock data on dir/list to test out changes. * user_router.js Add check for existence of Globus collection * user_router.js Remove existence check, move to filter method; Add notes on covered cases; add notes on code * datafed-ws.js Temporary handling of need consent response. * user_router.js Add queryParam validation for new params via joilib. ClientWorker.cpp Add refresh token try/catch to send through flow if refresh fails * datafed-ws.js Use correct converted casing for protobuf messages; redirect to error when needs_consent * datafed-ws.js No redirect, show error directly in list. * datafed-ws.js Early return to prevent further requests * support.js Update AccessTokenType enum. user_token.js New class introduced for handling logic for validating params when getting tokens and building GetAccessToken responses. user_router.js Replace logic with calls to new lib * user_token.js Add jsdoc definition for return type and format function. user_router.js Remove unnecessary comment. * user_token.js More jsdoc definition; Add validation for collection token document required fields. user_token.test.js Add preliminary unit tests for static methods. * CMakeLists.txt Add new unit tests to build. user_token.test.js Modify naming to match convention. * CMakeLists.txt Fix variable get. user_token.js Fix export. * user_fixture.js Add new user for get/token tests. user_router.test.js Add simple tests for get/token endpoint. user_token.js Fix naming for token document scopes field. user_token.test.js Changes for scope field bugfix. * user_router.js Formatting; Throw error if more than one token matches, add note about token type. * ClientWorker.cpp Address some TODOs and extraneous comments. DatabaseAPI.cpp Remove TODOs. SDMS_Auth.proto Remove unused field. * user_token.js Update formatUserToken to always return object values. DatabaseAPI.cpp Add note on possibly missing field. user_token.test.js Update tests to check for new expected values. * user_router.js Clean up some comments for clarity. datafed-ws.js Formatting. * ClientWorker.cpp Rewrite comment regarding mapped token refresh. * user_token.js Formatting. * CHANGELOG.md update for feature * user_router.js Re-introduce accidentally removed `globus_collection` object. * CMakeLists.txt Remove comment about necessity * Add collection_type and collection_id to dat/put task state * support.js Update getAccessToken method to work based on collection information, JSDoc. tasks.js Include context required in call to getAccessToken, add comment on refresh. TaskWorker.cpp Identify where changes will be reflected. * Update token lookup logic to use `byExample`, add test case for missing Globus collection. Add globus_coll fixture. * SDMS_Auth.proto Change UserAccessTokenReply needs_consent to optional field. * user_router.js Update token/get to default to start consent flow when collection tokens are present. user_router.test.js Update test from error to needs_consent. * Formatting changes. * SDMS_Auth.proto Add new fields to DataGetRequest * Begin refactor of token decision logic. * Address bugs in fetching and mapping data, make unit tests pass. * globus_token.js Match other models for get * tasks.js Pull in new logic for determining tokens, replace tokens; Needs to be verified. user_token.js Add new format function for compatibility with transfer doc replacement. * DatabaseAPI.cpp Sane serialization of json body for taskInitDataPut. TaskWorker.cpp Comments around work items. * DatabaseAPI.cpp Sane serialization of json body for taskInitDataGet. * ClientWorker.cpp Stub out required changes for consent flow when creating tasks * ClientWorker.cpp More thorough consent flow for procDataPutRequest, stub on needs_consent * tasks.js Throw error when initializing data put * tasks.js Format error throwing. ClientWorker.cpp comment out potentially unnecessary code. * user_token.js Add exists method to abstract check. tasks.js User exists method when checking token. * user_token.js JSDoc typing * tasks.js Add more required fields to taskRunDataPut for refresh in TaskWorker. * tasks.js Add collection_id to params in taskRunDataPut for refresh in TaskWorker. TaskWorker.cpp Refresh conditionals * DatabaseAPI.cpp Update merged devel changes to address some leftover code. * DatabaseAPI.cpp Add collection specification to get request. * DatabaseAPI.cpp whitespace. ClientWorker.cpp Clean up unused code. user_router.js Switch to checking token existence through standard API. * user_token.js Add comment about type return. data_router.js Do validations at router level for user collection token relationship. tasks.js Remove validation from logic level; make dataGet mirror dataPut; Updates to naming and params structure. * ClientWorker.cpp Revert to minimize diff. TaskWorker.cpp Remove unnecessary TODO; move set client for refresh in transfer tasks. * models/ JSDocs, formatting, make token fetches follow defined model. user_token.js JSDocs and typing. * user_token.js Cover case where collection does not exist. * Small changes for UI detection of collection type * fix epview call for UI changes, add list view change to react appropriately to needing consent according to current code * extract ep view logic to function for either path to utilize * small changes and notes for UI updates * UI search based on path, not ID. camelCasing * TaskWorker.cpp small fix to only enforce requirement of additional refresh fields when token is transfer token. * add params for dat/get on UI * task.js Fix bug where client is incorrectly referenced. * Add user model tests. Extract DataFedOAuthToken.js from user_token.js for reusability. * Add globus collection model tests. * Add globus token tests. Add fixtures to support globus token testing. Make all model properties static for iteration purposes. Utilize constructors for typing. * Add user token tests. Make non-existent user throw an error when building user tokens. * Change error to be not found * Formatting changes * Remove unused changes in support.js getAccessToken * Changelog update, update comments, remove log. --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> * Add small fix for storeCollectionId (#1341) * Add small fix for storeCollectionId * move todo comment --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> * Fix bug with playwright (#1343) * Fix bug with playwright install for end to end web test --------- Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * [DAPS-1330] Address Download Extension Transfer (#1340) * [DAPS-1337] Refactor GET and Put mode * [DAPS-1337] Clear consent issue * [DAPS-1337] Correct bug * [DAPS-1337] Add Dir mode path logic update * Parse md json strings (#1347) * Parse md json strings * parse execQuery params * move to direct only * make params an object * remove console log --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> * [DAPS-1349] Provenance Capture Update Bug (#1350) * [DAPS-1349] Return logic to original check * [DAPS-1349] version * [Release] February 2025 (#1352) * updated versions for release * Trigger rebuild * Adjust deployment version number * Fix Version month * Update Version.cmake * removed leading zero --------- Co-authored-by: JoshuaSBrown <brownjs@ornl.gov> Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> * Add .venv/ to gitignore (#1335) * [DAPS-1388] Add GCP Support (#1392) * [DAPS-1388] Add endpoint model, Add endpoint model, unify seperate collection logic, endpoint validation check, Add endpoint query validations, next is token * [DAPS-1388] Add tests for model * [DAPS-1388] PR comments * Update web/static/components/transfer/transfer-endpoint-manager.js Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> --------- Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> * Add missing python module (#1404) * Set reasonable default that will help with filling in repo form for core t… (#1400) * Set reasonable default that will help filling in repo form for core to repo communication in compose file * Update scripts/compose_generate_env.sh Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Set appropriate default when building compose image --------- Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Add documentation describing how to test the foxx microservices unit … (#1393) * Add documentation describing how to test the foxx microservices unit tests * Update testing.rst * Note on database version, variable definition and usage when running arangodb version described, and steps to run unit test numbered. * Fix ci build (#1390) * updated requirements * updated requirements * updated protobuf submodule * update unit test ci version * updated protobuf version * updated setuptools version * fixed condition * changed where packages are installed * used python3.11 to create the virtual environment instead of latest * reverted protobuf version * testing specificying version * removed pyvenv in ci job for consistency * installed python3.11 in dependencies docker file * added missing -y flag * removed unification of install scripts * added missing python3.11 install * updated ci job to install python3.11 * changed installed python version to 3.9 since protobuf does not officially support 3.11 * changed installed python version to 3.9 since protobuf does not officially support 3.11 * missed version * updated includes for newer gcc versions * updated versions * updated nlohmann json * updated json schema version * changed library search path * hopefully fixed json dep issues * updating function signature to match newer version of base class * updated protobuf * updated protobuf and cmake * downgraded cmake * fixed protobuf version * fixed protobuf version * 1398 pin python version (#1405) * Pin python version * fixed incorrect path * Switch to using .tar file install of libsodium (#1414) * 1413 libsodium build refactor (#1415) * Switch to using .tar file install of libsodium * Switch bad option in wget command from -C to -P * Update scripts/dependency_install_functions.sh * Update scripts/dependency_install_functions.sh * Update dependency_install_functions.sh Libsodium folder version number. * Update dependency_install_functions.sh Make paths explicit. * cleaned up comments * Update scripts/dependency_install_functions.sh Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> * Update cmake/JSONSchema.cmake Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> * Update scripts/dependency_install_functions.sh Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> * Update cmake/JSONSchema.cmake Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> * add changes from review * Changed install scripts to be consistent with new python installation method (#1417) * changed install scripts to be consistent * added sudo check * add apt sources check --------- Co-authored-by: Blake Nedved <nedvedba@ornl.gov> Co-authored-by: nedvedba <145805866+nedvedba@users.noreply.github.com> * Fix regression (#1428) * Fix docker logs output in end-to-end tests (#1426) * Improve serialization randomization (#1427) * Increase serialization randomization pause heuristic - 6 seconds is not long enough to register running pipelines. * improve serialization * Upgrade formatter (#1440) * Update action for CPP and Python formatter * 1437 fix zmq inproc bind order (#1438) * Reorder zmq bind and connect calls to prevent undefined behavior when using INPROC * Make sure zmq messages are closed when exception is encountered * Make correction to socket name * Initialize cmd call * Add missing include * Allow looping over containers if there is more than one. (#1446) * 1445 docker logs multiple containers (#1447) * Allow looping over containers if there is more than one. * Fix multipline comment * Fix conflicting dependency install flags (#1444) * [DAPS-1408] Provenance Visual Management Lint (1/2) (#1431) * [DAPS-14xx] Move files over and move out svg * [DAPS-1408] Provenance Visual Management (2/2) (#1419) * [DAPS-14xx] Base * [DAPS-14xx] Add styling and tooltips * [DAPS-14xx] Add node and label customization * [DAPS-14xx] Add editor modal on right-click. Draggable model * [DAPS-14xx] Revert * [DAPS-14xx] Seperate files, update styles * [DAPS-14xx] simplify feat * [DAPS-14xx] Expand mocha tests, prettier * [DAPS-14xx] Update checkout version * [DAPS-14xx] Add todo * [DAPS-14xx] Update tests, consts, fix tooltip * [DAPS-14xx] Base: [DAPS-14xx] Address collapse, expand, hide * [DAPS-14xx] THEME, address comments * [DAPS-14xx] Zoom * [DAPS-14xx] JSDOC * Update web/static/jquery-ui-dark/datafed.css Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> * Update web/static/jquery-ui-dark/datafed.css Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> * Update web/static/jquery-ui-dark/datafed.css Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> --------- Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> * [DAPS-1395] Address Memory Leaks in Core Server (#1424) * Add missing virtual destructors * Fix ZeroMQ INPROC and TCP context * Add memory tests to CMakeLists.txt as an option * Add protobuf library shutdown to unit tests for memory cleanup * [TASK] Update tests and fixture (#1454) * [DAPS-1331] Collection ID Updated on restart (#1468) * [DAPS-1331] Modified the script to only delete and recreate guest collections when the base path actually changes * [DAPS-1331] Add base path logic * [DAPS-1331] Update guest collection file * [DAPS-1331] Fix base path caching logic in should_recreate_guest_collection function to recreate by default when no cached path is available * [DAPS-1331] Add tests * [DAPS-1331] Revert Gitlab CI * [DAPS-RELEASE] * [TASK] Remove icons from labels (#1489) --------- Co-authored-by: Anthony <64566330+t-ramz@users.noreply.github.com> Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> Co-authored-by: Aaron Perez <perezam@ornl.gov> Co-authored-by: AronPerez <aperez0295@gmail.com> Co-authored-by: Blake Nedved <nedvedba@ornl.gov> Co-authored-by: par-hermes <par-hermes@automation.bot> Co-authored-by: nedvedba <145805866+nedvedba@users.noreply.github.com> Co-authored-by: Austin Hampton <amh107@latech.edu> Co-authored-by: Theo Beers <32523293+theodore-s-beers@users.noreply.github.com>

2025-03-11

Merge devel into Release branch (#1339) * Merge hotfixes into devel (#1326) * [DAPS-DEPS] Update cpp-py-formatter to 0.3.0 * [DAPS-DEPS] Update cpp-py-formatter to 0.3.1 * [DAPS-DEPS] Update ver, add push * updated versions for release * Update project.rst * Update format-check.yml * Pull in version numbers appropriately * Avoid build cache * Aggressive prune * Trigger rebuild * generate_repo_config.sh Accept env variable for datafed server from repo service instead of hard coding * Allow core config thread options to be read in from env * Add changelog comment * Adjust deployment version number * Fix Version month * Update CHANGELOG.md Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> --------- Co-authored-by: Aaron Perez <perezam@ornl.gov> Co-authored-by: Blake Nedved <nedvedba@ornl.gov> Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> Co-authored-by: JoshuaSBrown <brownjs@ornl.gov> Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> Co-authored-by: nedvedba <145805866+nedvedba@users.noreply.github.com> Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> * Feature DAPS 1215 foxx UserGetAccessTokenRequest mapped collection support (#1284) * Begin prototyping required code for mapped collection token retrieval. * SDMS_Auth.proto Add optional fields to UserGetAccessTokenRequest and UserAccessTokenReply to support finding correct tokens * user_router.js Add baseline query params and checks. DatabaseAPI.cpp & DatabaseAPI.hpp Use updated names and types. ClientWorker.cpp Update comments. * user_router.js Add logic for finding token based on collection; add token match and missing match cases * Return necessary data from Database to refresh tokens. Mock data on dir/list to test out changes. * user_router.js Add check for existence of Globus collection * user_router.js Remove existence check, move to filter method; Add notes on covered cases; add notes on code * datafed-ws.js Temporary handling of need consent response. * user_router.js Add queryParam validation for new params via joilib. ClientWorker.cpp Add refresh token try/catch to send through flow if refresh fails * datafed-ws.js Use correct converted casing for protobuf messages; redirect to error when needs_consent * datafed-ws.js No redirect, show error directly in list. * datafed-ws.js Early return to prevent further requests * support.js Update AccessTokenType enum. user_token.js New class introduced for handling logic for validating params when getting tokens and building GetAccessToken responses. user_router.js Replace logic with calls to new lib * user_token.js Add jsdoc definition for return type and format function. user_router.js Remove unnecessary comment. * user_token.js More jsdoc definition; Add validation for collection token document required fields. user_token.test.js Add preliminary unit tests for static methods. * CMakeLists.txt Add new unit tests to build. user_token.test.js Modify naming to match convention. * CMakeLists.txt Fix variable get. user_token.js Fix export. * user_fixture.js Add new user for get/token tests. user_router.test.js Add simple tests for get/token endpoint. user_token.js Fix naming for token document scopes field. user_token.test.js Changes for scope field bugfix. * user_router.js Formatting; Throw error if more than one token matches, add note about token type. * ClientWorker.cpp Address some TODOs and extraneous comments. DatabaseAPI.cpp Remove TODOs. SDMS_Auth.proto Remove unused field. * user_token.js Update formatUserToken to always return object values. DatabaseAPI.cpp Add note on possibly missing field. user_token.test.js Update tests to check for new expected values. * user_router.js Clean up some comments for clarity. datafed-ws.js Formatting. * ClientWorker.cpp Rewrite comment regarding mapped token refresh. * user_token.js Formatting. * CHANGELOG.md update for feature * user_router.js Re-introduce accidentally removed `globus_collection` object. * CMakeLists.txt Remove comment about necessity * Update token lookup logic to use `byExample`, add test case for missing Globus collection. Add globus_coll fixture. * SDMS_Auth.proto Change UserAccessTokenReply needs_consent to optional field. * user_router.js Update token/get to default to start consent flow when collection tokens are present. user_router.test.js Update test from error to needs_consent. * Formatting changes. --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> * [DLT-1110] Implement Consent Required Action (3/4 & 4/4) (#1242) * [DLT-1110] Refactor browse into component and use CSS * [DAPS-1110] Update * [DAPS-1110] Update web server * [DAPS-1110] add refactors * [DAPS-1110] Prettier * [DAPS-1110] Refactor tests * [DAPS-1110] Remove .. in ep ls, remove logs * [DAPS-1110] Add sinon, remove dead code * [DAPS-1110] Address transfer start button bug * [DAPS-1110] eslint, update api.test.js to reflect usage * [DAPS-1110] format * [DAPS-1110] Correct bug * JSON parse error fix (#1328) * DatabaseAPI.cpp Pull in json dependency, use json object to serialize request payload * Replace json serialization in more locations where parseSearchRequest is being called. * Add comments around locations that will need json serialization * Convert missed body append to serialization format * typo * Refactoring DatabaseAPI.cpp to incorporate json serialization * More serialization work * Pull in changes from 1214 for dataPut * Bring in changes from 1214 for dataGet * DatabaseAPI.cpp Fix error with missing curly brace. Some formatting that is likely wrong. Serialize up to original point. Need to validate. * Add missing scope resolution * More typos * More typos. Scoping * Remove non-existant vars * declare body * Formatting * Formatting and verification of updates * Replace most places * Formatting * Remove redeclaration * DatabaseAPI.cpp Remove some comments. * DatabaseAPI.cpp Add missing push_back * Adding braces * Prevent Double Escaping string, by using nlohmann json parse to read in value. * remove escapeJSON calls, redundant with nlohmann json dump * Allow parsing for metric data to be run in parallel for now, output messages when old serialization is used. * Ensure ASCII setting. * Upgrade debug to warning. --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> Co-authored-by: Austin Hampton <amh107@latech.edu> Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> * Feature - Mapped Collection Token Support (#1334) * Begin prototyping required code for mapped collection token retrieval. * SDMS_Auth.proto Add optional fields to UserGetAccessTokenRequest and UserAccessTokenReply to support finding correct tokens * user_router.js Add baseline query params and checks. DatabaseAPI.cpp & DatabaseAPI.hpp Use updated names and types. ClientWorker.cpp Update comments. * user_router.js Add logic for finding token based on collection; add token match and missing match cases * Return necessary data from Database to refresh tokens. Mock data on dir/list to test out changes. * user_router.js Add check for existence of Globus collection * user_router.js Remove existence check, move to filter method; Add notes on covered cases; add notes on code * datafed-ws.js Temporary handling of need consent response. * user_router.js Add queryParam validation for new params via joilib. ClientWorker.cpp Add refresh token try/catch to send through flow if refresh fails * datafed-ws.js Use correct converted casing for protobuf messages; redirect to error when needs_consent * datafed-ws.js No redirect, show error directly in list. * datafed-ws.js Early return to prevent further requests * support.js Update AccessTokenType enum. user_token.js New class introduced for handling logic for validating params when getting tokens and building GetAccessToken responses. user_router.js Replace logic with calls to new lib * user_token.js Add jsdoc definition for return type and format function. user_router.js Remove unnecessary comment. * user_token.js More jsdoc definition; Add validation for collection token document required fields. user_token.test.js Add preliminary unit tests for static methods. * CMakeLists.txt Add new unit tests to build. user_token.test.js Modify naming to match convention. * CMakeLists.txt Fix variable get. user_token.js Fix export. * user_fixture.js Add new user for get/token tests. user_router.test.js Add simple tests for get/token endpoint. user_token.js Fix naming for token document scopes field. user_token.test.js Changes for scope field bugfix. * user_router.js Formatting; Throw error if more than one token matches, add note about token type. * ClientWorker.cpp Address some TODOs and extraneous comments. DatabaseAPI.cpp Remove TODOs. SDMS_Auth.proto Remove unused field. * user_token.js Update formatUserToken to always return object values. DatabaseAPI.cpp Add note on possibly missing field. user_token.test.js Update tests to check for new expected values. * user_router.js Clean up some comments for clarity. datafed-ws.js Formatting. * ClientWorker.cpp Rewrite comment regarding mapped token refresh. * user_token.js Formatting. * CHANGELOG.md update for feature * user_router.js Re-introduce accidentally removed `globus_collection` object. * CMakeLists.txt Remove comment about necessity * Add collection_type and collection_id to dat/put task state * support.js Update getAccessToken method to work based on collection information, JSDoc. tasks.js Include context required in call to getAccessToken, add comment on refresh. TaskWorker.cpp Identify where changes will be reflected. * Update token lookup logic to use `byExample`, add test case for missing Globus collection. Add globus_coll fixture. * SDMS_Auth.proto Change UserAccessTokenReply needs_consent to optional field. * user_router.js Update token/get to default to start consent flow when collection tokens are present. user_router.test.js Update test from error to needs_consent. * Formatting changes. * SDMS_Auth.proto Add new fields to DataGetRequest * Begin refactor of token decision logic. * Address bugs in fetching and mapping data, make unit tests pass. * globus_token.js Match other models for get * tasks.js Pull in new logic for determining tokens, replace tokens; Needs to be verified. user_token.js Add new format function for compatibility with transfer doc replacement. * DatabaseAPI.cpp Sane serialization of json body for taskInitDataPut. TaskWorker.cpp Comments around work items. * DatabaseAPI.cpp Sane serialization of json body for taskInitDataGet. * ClientWorker.cpp Stub out required changes for consent flow when creating tasks * ClientWorker.cpp More thorough consent flow for procDataPutRequest, stub on needs_consent * tasks.js Throw error when initializing data put * tasks.js Format error throwing. ClientWorker.cpp comment out potentially unnecessary code. * user_token.js Add exists method to abstract check. tasks.js User exists method when checking token. * user_token.js JSDoc typing * tasks.js Add more required fields to taskRunDataPut for refresh in TaskWorker. * tasks.js Add collection_id to params in taskRunDataPut for refresh in TaskWorker. TaskWorker.cpp Refresh conditionals * DatabaseAPI.cpp Update merged devel changes to address some leftover code. * DatabaseAPI.cpp Add collection specification to get request. * DatabaseAPI.cpp whitespace. ClientWorker.cpp Clean up unused code. user_router.js Switch to checking token existence through standard API. * user_token.js Add comment about type return. data_router.js Do validations at router level for user collection token relationship. tasks.js Remove validation from logic level; make dataGet mirror dataPut; Updates to naming and params structure. * ClientWorker.cpp Revert to minimize diff. TaskWorker.cpp Remove unnecessary TODO; move set client for refresh in transfer tasks. * models/ JSDocs, formatting, make token fetches follow defined model. user_token.js JSDocs and typing. * user_token.js Cover case where collection does not exist. * Small changes for UI detection of collection type * fix epview call for UI changes, add list view change to react appropriately to needing consent according to current code * extract ep view logic to function for either path to utilize * small changes and notes for UI updates * UI search based on path, not ID. camelCasing * TaskWorker.cpp small fix to only enforce requirement of additional refresh fields when token is transfer token. * add params for dat/get on UI * task.js Fix bug where client is incorrectly referenced. * Add user model tests. Extract DataFedOAuthToken.js from user_token.js for reusability. * Add globus collection model tests. * Add globus token tests. Add fixtures to support globus token testing. Make all model properties static for iteration purposes. Utilize constructors for typing. * Add user token tests. Make non-existent user throw an error when building user tokens. * Change error to be not found * Formatting changes * Remove unused changes in support.js getAccessToken * Changelog update, update comments, remove log. --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> * Add small fix for storeCollectionId (#1341) * Add small fix for storeCollectionId * move todo comment --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> --------- Co-authored-by: Anthony <64566330+t-ramz@users.noreply.github.com> Co-authored-by: Aaron Perez <perezam@ornl.gov> Co-authored-by: Joshua S Brown <joshbro42867@yahoo.com> Co-authored-by: JoshuaSBrown <brownjs@ornl.gov> Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> Co-authored-by: Aaron Perez <aperez0295@gmail.com> Co-authored-by: Austin Hampton <amh107@latech.edu>

2025-03-06

Feature - Mapped Collection Token Support (#1334) * Begin prototyping required code for mapped collection token retrieval. * SDMS_Auth.proto Add optional fields to UserGetAccessTokenRequest and UserAccessTokenReply to support finding correct tokens * user_router.js Add baseline query params and checks. DatabaseAPI.cpp & DatabaseAPI.hpp Use updated names and types. ClientWorker.cpp Update comments. * user_router.js Add logic for finding token based on collection; add token match and missing match cases * Return necessary data from Database to refresh tokens. Mock data on dir/list to test out changes. * user_router.js Add check for existence of Globus collection * user_router.js Remove existence check, move to filter method; Add notes on covered cases; add notes on code * datafed-ws.js Temporary handling of need consent response. * user_router.js Add queryParam validation for new params via joilib. ClientWorker.cpp Add refresh token try/catch to send through flow if refresh fails * datafed-ws.js Use correct converted casing for protobuf messages; redirect to error when needs_consent * datafed-ws.js No redirect, show error directly in list. * datafed-ws.js Early return to prevent further requests * support.js Update AccessTokenType enum. user_token.js New class introduced for handling logic for validating params when getting tokens and building GetAccessToken responses. user_router.js Replace logic with calls to new lib * user_token.js Add jsdoc definition for return type and format function. user_router.js Remove unnecessary comment. * user_token.js More jsdoc definition; Add validation for collection token document required fields. user_token.test.js Add preliminary unit tests for static methods. * CMakeLists.txt Add new unit tests to build. user_token.test.js Modify naming to match convention. * CMakeLists.txt Fix variable get. user_token.js Fix export. * user_fixture.js Add new user for get/token tests. user_router.test.js Add simple tests for get/token endpoint. user_token.js Fix naming for token document scopes field. user_token.test.js Changes for scope field bugfix. * user_router.js Formatting; Throw error if more than one token matches, add note about token type. * ClientWorker.cpp Address some TODOs and extraneous comments. DatabaseAPI.cpp Remove TODOs. SDMS_Auth.proto Remove unused field. * user_token.js Update formatUserToken to always return object values. DatabaseAPI.cpp Add note on possibly missing field. user_token.test.js Update tests to check for new expected values. * user_router.js Clean up some comments for clarity. datafed-ws.js Formatting. * ClientWorker.cpp Rewrite comment regarding mapped token refresh. * user_token.js Formatting. * CHANGELOG.md update for feature * user_router.js Re-introduce accidentally removed `globus_collection` object. * CMakeLists.txt Remove comment about necessity * Add collection_type and collection_id to dat/put task state * support.js Update getAccessToken method to work based on collection information, JSDoc. tasks.js Include context required in call to getAccessToken, add comment on refresh. TaskWorker.cpp Identify where changes will be reflected. * Update token lookup logic to use `byExample`, add test case for missing Globus collection. Add globus_coll fixture. * SDMS_Auth.proto Change UserAccessTokenReply needs_consent to optional field. * user_router.js Update token/get to default to start consent flow when collection tokens are present. user_router.test.js Update test from error to needs_consent. * Formatting changes. * SDMS_Auth.proto Add new fields to DataGetRequest * Begin refactor of token decision logic. * Address bugs in fetching and mapping data, make unit tests pass. * globus_token.js Match other models for get * tasks.js Pull in new logic for determining tokens, replace tokens; Needs to be verified. user_token.js Add new format function for compatibility with transfer doc replacement. * DatabaseAPI.cpp Sane serialization of json body for taskInitDataPut. TaskWorker.cpp Comments around work items. * DatabaseAPI.cpp Sane serialization of json body for taskInitDataGet. * ClientWorker.cpp Stub out required changes for consent flow when creating tasks * ClientWorker.cpp More thorough consent flow for procDataPutRequest, stub on needs_consent * tasks.js Throw error when initializing data put * tasks.js Format error throwing. ClientWorker.cpp comment out potentially unnecessary code. * user_token.js Add exists method to abstract check. tasks.js User exists method when checking token. * user_token.js JSDoc typing * tasks.js Add more required fields to taskRunDataPut for refresh in TaskWorker. * tasks.js Add collection_id to params in taskRunDataPut for refresh in TaskWorker. TaskWorker.cpp Refresh conditionals * DatabaseAPI.cpp Update merged devel changes to address some leftover code. * DatabaseAPI.cpp Add collection specification to get request. * DatabaseAPI.cpp whitespace. ClientWorker.cpp Clean up unused code. user_router.js Switch to checking token existence through standard API. * user_token.js Add comment about type return. data_router.js Do validations at router level for user collection token relationship. tasks.js Remove validation from logic level; make dataGet mirror dataPut; Updates to naming and params structure. * ClientWorker.cpp Revert to minimize diff. TaskWorker.cpp Remove unnecessary TODO; move set client for refresh in transfer tasks. * models/ JSDocs, formatting, make token fetches follow defined model. user_token.js JSDocs and typing. * user_token.js Cover case where collection does not exist. * Small changes for UI detection of collection type * fix epview call for UI changes, add list view change to react appropriately to needing consent according to current code * extract ep view logic to function for either path to utilize * small changes and notes for UI updates * UI search based on path, not ID. camelCasing * TaskWorker.cpp small fix to only enforce requirement of additional refresh fields when token is transfer token. * add params for dat/get on UI * task.js Fix bug where client is incorrectly referenced. * Add user model tests. Extract DataFedOAuthToken.js from user_token.js for reusability. * Add globus collection model tests. * Add globus token tests. Add fixtures to support globus token testing. Make all model properties static for iteration purposes. Utilize constructors for typing. * Add user token tests. Make non-existent user throw an error when building user tokens. * Change error to be not found * Formatting changes * Remove unused changes in support.js getAccessToken * Changelog update, update comments, remove log. --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov>

2025-02-18

Feature DAPS 1215 foxx UserGetAccessTokenRequest mapped collection support (#1284) * Begin prototyping required code for mapped collection token retrieval. * SDMS_Auth.proto Add optional fields to UserGetAccessTokenRequest and UserAccessTokenReply to support finding correct tokens * user_router.js Add baseline query params and checks. DatabaseAPI.cpp & DatabaseAPI.hpp Use updated names and types. ClientWorker.cpp Update comments. * user_router.js Add logic for finding token based on collection; add token match and missing match cases * Return necessary data from Database to refresh tokens. Mock data on dir/list to test out changes. * user_router.js Add check for existence of Globus collection * user_router.js Remove existence check, move to filter method; Add notes on covered cases; add notes on code * datafed-ws.js Temporary handling of need consent response. * user_router.js Add queryParam validation for new params via joilib. ClientWorker.cpp Add refresh token try/catch to send through flow if refresh fails * datafed-ws.js Use correct converted casing for protobuf messages; redirect to error when needs_consent * datafed-ws.js No redirect, show error directly in list. * datafed-ws.js Early return to prevent further requests * support.js Update AccessTokenType enum. user_token.js New class introduced for handling logic for validating params when getting tokens and building GetAccessToken responses. user_router.js Replace logic with calls to new lib * user_token.js Add jsdoc definition for return type and format function. user_router.js Remove unnecessary comment. * user_token.js More jsdoc definition; Add validation for collection token document required fields. user_token.test.js Add preliminary unit tests for static methods. * CMakeLists.txt Add new unit tests to build. user_token.test.js Modify naming to match convention. * CMakeLists.txt Fix variable get. user_token.js Fix export. * user_fixture.js Add new user for get/token tests. user_router.test.js Add simple tests for get/token endpoint. user_token.js Fix naming for token document scopes field. user_token.test.js Changes for scope field bugfix. * user_router.js Formatting; Throw error if more than one token matches, add note about token type. * ClientWorker.cpp Address some TODOs and extraneous comments. DatabaseAPI.cpp Remove TODOs. SDMS_Auth.proto Remove unused field. * user_token.js Update formatUserToken to always return object values. DatabaseAPI.cpp Add note on possibly missing field. user_token.test.js Update tests to check for new expected values. * user_router.js Clean up some comments for clarity. datafed-ws.js Formatting. * ClientWorker.cpp Rewrite comment regarding mapped token refresh. * user_token.js Formatting. * CHANGELOG.md update for feature * user_router.js Re-introduce accidentally removed `globus_collection` object. * CMakeLists.txt Remove comment about necessity * Update token lookup logic to use `byExample`, add test case for missing Globus collection. Add globus_coll fixture. * SDMS_Auth.proto Change UserAccessTokenReply needs_consent to optional field. * user_router.js Update token/get to default to start consent flow when collection tokens are present. user_router.test.js Update test from error to needs_consent. * Formatting changes. --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov>

2025-02-07

SDMS_Auth.proto Change UserAccessTokenReply needs_consent to optional field.

2025-02-03

ClientWorker.cpp Address some TODOs and extraneous comments. DatabaseAPI.cpp Remove TODOs. SDMS_Auth.proto Remove unused field.

2025-01-27

user_router.js Add baseline query params and checks. DatabaseAPI.cpp & DatabaseAPI.hpp Use updated names and types. ClientWorker.cpp Update comments.

SDMS_Auth.proto Add optional fields to UserGetAccessTokenRequest and UserAccessTokenReply to support finding correct tokens

2025-01-14

Feature DLT 1127 relay new token data (#1217) * Add additional params to DatabaseAPI::userSetAccessToken to pass message with addtional info on to database API. * Add convenience method with old type signature of DatabaseAPI::userSetAccessToken for compatibility * Conditionally add other token options param when calling dbGetRaw. * Pass token type along to Database API. * Pass token type from message. * Extract logic for building URL from dbPost and dbGetRaw into separate function for easier testing and debugging * Extract url building logic from dbGet * Add some comments for moved code, address a TODO, add new TODO for AccessToken Message type field * Address default on token type, formatting * Add new AccessTokenType entries for a sentinel value and Globus default in SDMS.proto, use as desired in DatabaseAPI.cpp * Change query param to match backend changes * Restore formatting of SDMS.proto * Make built URL const where appropriate in DatabaseAPI.cpp and DatabaseAPI.hpp * Change default AccessTokenType for UserSetAccessTokenRequest in SDMS_Auth.proto. * Remove logging of potentially senitive data * Adjust AccessTokenType ACCESS_SENTINEL to have value of uint8 max. --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov>

2025-01-13

Adjust AccessTokenType ACCESS_SENTINEL to have value of uint8 max.

2025-01-09

Change default AccessTokenType for UserSetAccessTokenRequest in SDMS_Auth.proto.

2025-01-07

Restore formatting of SDMS.proto

2025-01-06

Add new AccessTokenType entries for a sentinel value and Globus default in SDMS.proto, use as desired in DatabaseAPI.cpp

2024-12-26

Address default on token type, formatting

2024-12-12

1106 updated proto messages for setting token (#1160) * updated proto messages for setting token * Add uth and transfer AccessTokenTypes to enum * Move AccessTokenType to SDMS.proto to avoid possible read error in datafed-ws.js --------- Co-authored-by: Anthony Ramirez <ramirezat@ornl.gov> Co-authored-by: JoshuaSBrown <brownjs@ornl.gov>

2023-11-03

Added retries attribute to protcol buffers, and to CommandLib dataPut and dataGet options

2023-04-19

Remove unneeded version.proto file it is generated by cmake

Working on getting versioning fixed

2023-04-18

Attempt to standardize versioning

2021-12-27

- Adding daily message support to core, cli

2021-11-11

- Fixed issue #809 and #810

2021-11-05

- fixed issues #802 and #800 - Fixed some typos in CLI - Bumper version number

2021-11-03

- Fixed security issue with user create - Fixed python build issue

2021-10-28

- Fixes for injection attacks

2021-08-20

- production version fix - new production public key

2021-07-14

Merge branch 'master' into devel

2021-07-12

- Removed hardcoded count limit - Bumbed version number

2021-07-08

- More API documentation - Renamed several messages

2021-07-07

- more api documentation

- Removed several deprecated messages and fields - Updated version system - More API documentation

- more docs

2021-07-02

- More documentation

2021-07-01

- Protobuf documentation

2021-06-30

- Documenting protobuf files

2021-05-27

- Fixed issue #759

2021-04-27

- saved queries working

2021-04-26

- Working on new saved query create / update

2021-04-20

- Update protofile format hint - Reduce logging

2021-04-13

- Fixed issue #674

2021-04-12

- Suport external data via CLI/CommandLib - Remove unsupported parameters from record update message

2021-04-09

- more work on external data

2021-04-07

- working on external data support

2021-03-23

- Working on new search

2021-03-19

- Fixed proto files (removed commented out messages) - Fixed CLI for change in auth/anon messages

2021-03-18

- more work on search refactor (mostly working)

2021-03-17

- Working on search refactor

- refactor of auth/anon messages - refactoring search functions / api - fixed spelling in data_gen

2021-03-15

- Removed sch ver from catalog search

2021-03-12

- added schema reset button to data dialog - Fixing bug with md error clearing - removed more schema version references

2021-03-11

- Work on merging schema ID and version

- Added option for warn/error for metadata validation - Support metadata validation error in record create/update - Fixe options bug in settings

2021-03-10

- Working on issue #660

2021-03-09

- work on issue #657, #33

2021-03-05

Merge branch 'devel' into devel-schema

2021-02-20

- Fixed versions - Improved CLI new version notice

2021-02-18

- bumbed version - fixed bug #635

2021-02-15

- Fixed issue #627 and #628 - Improved error messages when allocation missing

2021-02-12

- Fixed many issues - Reworked welcome and registration pages - Added session management - Made password optional on registration

2021-02-10

- Fixed issue #617, #618

2021-02-05

- bumped server version

2021-01-25

- Bumped py client version and released (hotfix) on PyPi

2020-12-08

- Working on query builder - Found several new UI bugs

2020-12-02

- Schema now searchable in catalog

2020-11-25

- Schema search is working

2020-11-20

- Lots of schema work (edit/create/revise/delete)

2020-11-19

- Schema edit/view dialog - Refactor schema ID to add version

2020-11-18

- Working schema create/edit in UI

2020-11-17

- Schema work - Debug gridftp

2020-11-13

- Added basic schema management API and initial UI

2020-11-12

- schem work in prog

2020-11-10

- More schema work (ui, create/update)

2020-11-06

- More schema work - Data edit UI - schema validate feedback