Proto commits in obiba/opal

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

2026-09-09

feat: Project-internal H2 database (#4196) * feat(database): add owner_project to the databases table A project will be able to own its storage: an H2 file Opal creates with the project and deletes with it, rather than a database an operator registered beforehand. Ownership is a column on `databases` - NULL for registered databases, the project name for the ones Opal manages - so an internal database stays an ordinary row that every existing listing, upgrade step and the checkpointer keep reaching. The column is deliberately not a foreign key to `projects`: the row outlives the project when the project is archived, and re-attaches when a project of that name is created again. `uk_databases_owner_project` makes "one internal database per project" a schema invariant; NULLs are distinct on H2 and PostgreSQL, so registered databases are unconstrained. Opal 6.0 is unreleased and every installation creates its configuration schema from scratch, so the column joins the initial `1-config-model` changeset rather than adding a migration. That changes the changeset's checksum: a server already running a 6.0 snapshot must delete ${OPAL_HOME}/data/config and let the OrientDB migration run again. Nothing sets the column yet. * feat(jdbc): give a project-owned database a URL form of its own A database a project owns lives in a folder named after the project, holding one store: jdbc:h2:file:<project name>/data, expanded against ${OPAL_HOME}/data/h2 when the connection is opened, like the registered form it sits beside. A folder rather than another file directly under data/h2, because a folder `CLSA/` and a registered database's file `CLSA.mv.db` are not the same name: a project can be called whatever an operator once called a database. It also makes deleting a project's data a folder removal rather than a guess at which files beside `CLSA.mv.db` belong to it. The folder name is a project name, and the pattern ProjectsResource enforces is repeated here because this is where the name becomes a path: no separator, no parent reference, no ';' with which H2 settings would be smuggled into the URL. The canonical containment check on top of it is unreachable in practice - it is there so that a future loosening of the project name rule fails a test before it reaches the file system. The two forms cannot be mistaken for one another: getDatabaseName rejects a '/', so a registered database cannot be given a project-shaped URL, and DataSourceFactory picks the expansion by ownership rather than by parsing. Nothing builds this URL yet. * feat(database): make a project-owned database Opal's to manage, not the operator's A database created for a project is listed like every other one - an operator is accountable for the disk, and a store holding 40 GB must not be invisible in the one page that shows what Opal keeps. What ownership changes is authority, not visibility: it cannot be edited or deleted while its project exists. The registry gains the two guarded paths and the two the application uses: - create() refuses a name starting with '_', now reserved for Opal's own, and refuses an ownership handed to it in a payload; - createProjectOwned() / deleteProjectOwned() are how Opal registers and removes one, named apart from create()/delete() rather than defeating their guards by an implicit ordering; - update() and delete() refuse a database whose owner project still exists, and update() carries the stored owner over so that editing a row an archived deletion left behind does not silently un-own it; - delete() of such a leftover takes its files with it, which is the one place a project-owned database is an operator's to remove. Whether the owner project still exists is what decides, not whether the database has a datasource: a project that failed to load still exists. hasDatabases() is the one listing that filters. It answers "has an operator provided storage" and drives the setup prompts, so a database Opal made for a project is not an answer to it. Every other listing includes them, which is what the upgrade steps, the Hibernate 5 sequence fix and the H2 checkpointer need. The cross-database H2 file-name check now skips project-owned rows: a project database is a folder, a registered one a file beside it, so they do not collide - and reading a plain file name out of a project URL would have thrown. Deleting the files renames the folder out of the way first, so a removal that stops half way leaves something unambiguous rather than what still looks like a live project database - and the rename is what fails, before anything is gone, where the store is still open. * feat(project): add the service that owns a project's internal database Everything about internal storage lives in ProjectDatabaseService, so that the project service can create and delete a project's storage without knowing what H2 is: the row it builds, the folder it lives in, and the sweeps that clean up after a deletion that did not finish. ensureInternalDatabase is idempotent - a project recreated after an archiving deletion gets its data back rather than a second database - and the JDBC datasource settings it applies are the ones the administration UI sends when an operator registers a SQL database, moved server side so both paths use one definition. Two projects whose names differ only in case would be one folder on a case insensitive file system, and the second would silently open the first one's database. That is refused on every platform, not only where the disk would actually collide: a pair of projects that worked on Linux and broke when OPAL_HOME was copied to a Mac is the worse failure. The check reads the rows, because the folder does not exist until the first connection, and the folders too, for a directory somebody put there by hand. listOrphanFolders reports what no row points at and never deletes it: an operator who moved a file in by hand, or a bug in this feature, must not cost anyone their data. deletePendingFolders is the other half - it finishes the removals a previous run began. Deliberately not transactional: unregister and deleteProjectOwned refuse to run inside a transaction, and so do the project service methods that call this one. * feat(project): create and delete a project's storage with the project A project is now saved with the storage it should have - none, a database an operator registered, or one of its own - rather than with a database name copied into it. ProjectStorage says which, and save(project) reads it back from what the project already holds, so saving a project for any other reason - a table was added, an identifiers mapping was removed - does not touch its storage. Creating a project with internal storage creates its database; deleting the project without archiving deletes it, files and all. That deletion replaces datasource.drop() rather than following it: walking every table with a DROP TABLE for a file that is about to be removed is minutes of work for no effect. An archiving deletion keeps both, unreferenced, for an operator to see and reclaim. Two refusals, both of which run before anything is created so that a refused save leaves nothing behind: - changing storage on a project that holds data. The administration UI has always refused it; nothing on the server did, and a PUT could silently drop a datasource. With internal storage that silence would delete a file, so the server now refuses it - for registered storage alike, closing an existing hole rather than adding a special case. - pointing a project at a database another project owns. Now that these are listed, a client can name one, and two projects in one of them is precisely what internal storage exists to prevent. A failure after the database was created removes it again, and only when this call is what created it: a save that fails on a project which already owned one must not take its data with it. At startup the removals an interrupted deletion began are finished, and folders no database refers to are reported with their size. Never deleted: an operator who moved a file in by hand, or a bug in this feature, must not cost anyone their data. * feat(rest): let a project ask for a database of its own A project says whether it has a database of its own rather than naming it: the name of that row is an implementation detail of the projects API, and the databases API is where it is named. A database says which project owns it, and whether that project still exists - which is the question the administration page actually has to ask, since a project that failed to load still exists and its database is still Opal's, while an archiving deletion leaves an owned row whose project is gone and which an operator may now remove. Both DTOs are output only: ownership is never set from a payload, and the registry carries the stored owner over on update, which covers every caller rather than only this one. The sharpest edge is the mapping back. fromDto used to copy the DTO's database into the project, so a PUT from a client that predates internal storage - which sends back the project it read, in which an internal database is not named - would detach it, and detaching an internal database deletes a file. The mapping now carries the project metadata only and the resources resolve storage: - internalDatabase says so; - a database name asks for that database; - an *empty* database name asks for no storage at all; - a payload that says nothing leaves the storage as it is. Detaching is asked for rather than left out, because leaving it out is what an older client does with a database it cannot name. A database that belongs to a living project answers 409 to an edit or a delete: a conflict rather than a forbidden, since the request succeeds once that project is gone. Refusing storage a project cannot have is a 400, which is what IllegalArgumentException already maps to. Listing databases reads the project names once for the whole page, and not at all when no listed row belongs to a project - which is every server that uses registered storage only. * feat(ui): let a project be created with a database of its own The database select becomes a storage select: an internal database, one of the databases an operator registered, or none. A database another project owns is not offered - the server refuses it too, so the filter is a courtesy rather than the guard. Which one is selected when the dialog opens: the registered database marked default storage if there is one, internal otherwise. An operator marking a database as the default says where project data is meant to go on this server, and this does not overrule that; what it replaces is the old fallback to the first database in the list, which with internal storage available is a worse default than the project's own. A hint under the select says what deletion will mean - Opal deletes an internal database with the project, and leaves a registered one alone - because that difference is the whole point of the choice. No storage at all is now asked for with an empty database name rather than by leaving the field out: leaving it out is what a client that cannot name an internal database does, and the server reads that as "leave the storage alone". The project administration page reports internal storage as such rather than naming the row, and no longer warns that a project has no database when it owns one. * feat(ui): show project-owned databases as Opal's to manage The databases page lists them like every other one - an operator is accountable for the disk, and a store holding 40 GB must not be invisible here - with a sortable Project column saying whose it is. What changes is authority, not visibility: - the edit button becomes a view button, opening the same dialog read only. The URL and the JDBC settings are exactly what an operator comes to this page to read, so the dialog is shown rather than withheld, with a line saying why it cannot be changed; - the delete button is gone while the owner project exists. For a row an archived deletion left behind it comes back, and its confirmation says the opposite of the registered one: the data and its files WILL be deleted, where unregistering a declared database deletes nothing; - testing the connection stays available on every row. The page decides from whether the owner project exists, not from the database having a datasource: a project that failed to load has no datasource but is still there, and offering a delete button that comes back with a conflict is worse than not offering one. The 409 remains the server's last word - the page can be stale, and a project can be created between the listing and the click - so the delete now reports what the server says instead of dropping it. * fix(database): stop the identifiers database from being cornered by its own name and file Three problems, all found registering an H2 database for the identifiers. The reserved namespace was a superset of what it protects. '_identifiers' is a name Opal picks itself, so refusing every name starting with an underscore refused the one database the reservation exists to keep company with. What is refused now is '_project_*', always - including for an identifiers database, which also closes the way the usedForIdentifiers flag could have been used to squat the name of a project's database. The wider underscore reservation applies to everything else. The identifiers datasource was built once and kept for the life of the server, dropped only at shutdown. Deleting the identifiers database therefore left it in place, answering from the database that was there before through a connection pool the registry had already closed, and registering another one did not replace it. It is now forgotten whenever the identifiers database is created, edited or deleted. This predates project-internal databases and would bite even where the credentials match. An H2 database is a file Opal creates in its own folder, and that file outlives the registration: it keeps the credentials it was created with, so a database registered again at the same URL with a different password could never open it. Removing the files is now offered - DELETE /system/database/{name}?deleteFiles=true, a checkbox in the unregister dialog - and never implied: what an operator declared may hold data nobody meant to lose, so delete(database) still leaves it alone. The files are matched on the name followed by a dot, so those of 'opal' are not those of 'opal-data'. A database a project owns takes its files in any case, so the choice is not offered there. Also carries incidental prettier quote normalisation in the i18n files, from running the formatter over them. * docs: describe project-internal databases in the upgrade notes Three sections under 6.0.x. What internal storage is, where its files are, and the sentence that matters before anyone chooses: deleting a project deletes its internal database and everything in it, while deleting a project stored in a registered database does not. Also what an archiving deletion leaves behind and how to reclaim it, that these databases are listed but managed by Opal, that creating a project now provisions storage, and that data/h2 is project data to back up. That unregistering an H2 database now offers to delete its files, and why it matters: H2 keeps the credentials inside the file, so a database registered again at the same URL with a different password could not be opened while the file stayed behind. And, scoped to installations already running a 6.0 snapshot and marked as such, that data/config has to be rebuilt: the ownership column joined the initial changeset rather than adding a migration, which changes its checksum, and Liquibase refuses to start rather than ignore it. The user guide lives in the opal-doc repository and is not updated here. * fix(project): read a project's storage from the owner column, not its name _project_ became Opal's prefix in 6.0, so a server upgraded from 5.x can hold a database an operator registered under that name, and a project stored in it. Three places read the prefix as "Opal owns this", and all three were wrong for that database: the project would report an internal database whose name the administration page then could not show, saving the project would try to create Opal's own database over the operator's and fail, and deleting the project would skip the drop that empties it. isInternal now asks the owner column once the prefix has ruled out the many projects that cannot be internal, and ProjectService.getStorage exposes that answer to the DTO layer. The other two sites do not need the answer at all. A payload that says nothing about storage now says exactly that - ProjectStorage.UNCHANGED - instead of being resolved from the stored name, so the resource never has to recognise storage it cannot name, and saving a project for an unrelated reason no longer looks its database up. Also fixes the "Project nome invalid" typo in the project name refusal. Reported by Copilot on #4196.

The documentation is generated from this commit.

2026-09-02

feat(datashield): usage quotas (#4193) * feat(datashield): execution time quotas (#3745) Limit the R execution time a user may spend on DataSHIELD over a rolling window of 24 hours or 7 days. A quota resolves as user > group > system, the most permissive among several groups, and no quota means unlimited. Over quota, a new session is refused with 403; sessions already open keep working. Users see their consumption on their profile page, administrators on a subject's profile page and on the DataSHIELD administration page. Also fixes the R activity panel being empty for non-administrators, by adding self-service endpoints that take no user parameter. * fix(r): record the session activity in the context the session was created for The execution context was set after the session was created, but a Rock app session opens (and runs its context initiator) inside its constructor: the first R operation, and so the RServerSessionStartedEvent, happened while the context was still "default". Provide the context at creation time instead. * fix: some quotas ui adjustments * feat(datashield): session time quotas A quota now names what it limits: execution time bills the cpu the R server spent on a user's commands and prices an idle session at zero, session time bills the R server they are holding whether it computes or not. `RQuota.metric` joins the natural key, so a subject can be given one of each; limits of different metrics are not comparable, so they are resolved independently and a new session is refused when either is spent. - `r_session_activities` gains a stored `session_time_millis`, maintained on the three activity events. Stored rather than derived because the quota query has to be an indexed aggregate over a numeric column. - The session manager posts the started event at creation, and the activity service ignores a second post, so a session that never runs a command still leaves a record - which is the case the metric is for. - Usage of an open session adds the time since its record was last written, otherwise a parked session would make the counter stand still. - `execution_time_limit_millis` becomes `limit_millis`, and the usage endpoints return one entry per metric. Unreleased, nothing to migrate, but the `2-r-quotas` changeset is amended in place: a local database that already ran it must be re-initialised. * fix(r): align the quota unique constraint with the natural key The JPA @UniqueConstraint listed three columns where the changelog and the repository lookup use four: metric is part of the key, one quota per subject per metric. Clamp getIdleTimeMillis() at zero. Rows written before session time was recorded have a session time of zero and an execution time that is not, and the difference there is meaningless rather than negative.

2026-09-01

feat(datashield): execution time quotas (#3745) Limit the R execution time a user may spend on DataSHIELD over a rolling window of 24 hours or 7 days. A quota resolves as user > group > system, the most permissive among several groups, and no quota means unlimited. Over quota, a new session is refused with 403; sessions already open keep working. Users see their consumption on their profile page, administrators on a subject's profile page and on the DataSHIELD administration page. Also fixes the R activity panel being empty for non-administrators, by adding self-service endpoints that take no user parameter.

2026-08-31

feat(storage): durability and disk space safeguards (#4192) * fix(database): reject the H2 settings that would disable the shutdown fsync * feat(storage): watch the free disk space of the volumes Opal writes to * feat(storage): refuse the large writes before the small ones start failing * feat(database): force the open H2 databases to disk periodically * fix: log back to normal message when current disk status is OK * fix: check for disk space to upload file after the permissions were verified * fix(storage): measure the free disk space against absolute thresholds

2026-08-29

feat(database): register H2 file databases for storage (#4185) * feat(database): register H2 file databases for storage Magma 5.5 supports H2 as a JDBC datasource, so Opal can offer an embedded SQL database that needs no server to install or administer. Registering one is the same flow as any other SQL database, with the driver picked from the list. H2 is embedded rather than reached over the network, which makes two things different from MySQL, MariaDB, PostgreSQL and SQL Server: Where the file lives. The database is registered by name alone, as jdbc:h2:file:<name>, and that short form is what is persisted; DataSourceFactory expands it to ${OPAL_HOME}/data/h2/<name> when the connection is opened, creating the folder. Storing the name rather than the resolved path keeps the configuration portable if OPAL_HOME moves, and that is also the only site in the codebase that turns SQL settings into a connection, so the connection test resolves the same way. Restricting the URL to a plain name is what confines the databases to that folder: a path separator or a parent reference cannot be expressed, so there is nothing to escape with, and no canonical path comparison is needed to prove it. What it can be used for. There is no pre-existing database to import from or export to, so H2 is storage only. DefaultDatabaseRegistry.persist enforces this and the URL rule, being the single point both POST /system/databases and PUT /system/database/{name} pass through. Identifiers databases are registered with storage usage, so H2 serves there too. The dialog derives the offered usages from a new supportedUsages on JdbcDriverDto rather than naming the driver itself, so picking H2 leaves Storage alone and hides the JDBC datasource settings that only apply to import and export. Liquibase moves to 5.0.3, the version Magma 5.5 is built against, so that Magma creates its metadata schema on the Liquibase that Opal ships rather than one it was never tested with. Opal has no Liquibase code of its own; liquibase-slf4j 5.0.0 stays, its OSGi range naming 4.x notwithstanding, as the two SPI types it implements are unchanged between the versions. H2 is pinned to 2.4.240 to match Magma, which means 1.x database files are unsupported: they are rejected on registration with a message to migrate rather than left to fail obscurely in the driver. H2JdbcDatasourceTest covers the combination Opal ships rather than the one Magma tests, creating a table and round-tripping a binary value through a real H2 file. Also fixes the save handler in the dialog reading error.response.data .message, which ClientErrorDto does not carry, against a db.save_error message that had no placeholder for it: a rejected registration reported nothing about why. It now goes through notifyError, which resolves the status and its arguments. * chore: magma 5.5.0 * fix(database): reject H2 connection settings A registered H2 URL was validated only up to the first ';', and expand() appended everything after it to the absolute URL verbatim. In H2's grammar ';' introduces connection settings, so the unchecked tail was a second language rather than part of the name: INIT alone runs SQL every time the connection is opened, RUNSCRIPT FROM included, from a remote URL. The name is now the whole URL. Settings are dropped rather than filtered: a denylist over H2's settings would have to stay correct across H2 versions, and nothing in Opal's usage needs one, as the database is opened by Opal alone, for storage, with defaults. H2 also reads its settings from the connection properties, and SqlSettings carries a properties field of its own that DataSourceFactory hands to the driver, so INIT was reachable without a ';' in the URL. validateProperties rejects it there: in DefaultDatabaseRegistry.persist, so registering the database fails with a message, and again in DataSourceFactory, so a row persisted before this change cannot open a connection either. Reported in https://github.com/obiba/opal/pull/4185#issuecomment-5457354184 * fix(database): reject a second registration of the same H2 file Databases are unique by their Opal name, so nothing stopped two of them naming the same H2 file: two registrations, one set of tables. Reject a duplicate at registration, in the same place the rest of the H2 rules live. The comparison ignores case, because on a case insensitive file system 'opal' and 'Opal' are the same file, and a pair of names that only holds on Linux would not survive a move of the H2 folder. It looks at every SQL database rather than the listed ones, as the identifiers database is an H2 candidate too and is filtered out of the listings.

2026-08-28

feat(database): register H2 file databases for storage Magma 5.5 supports H2 as a JDBC datasource, so Opal can offer an embedded SQL database that needs no server to install or administer. Registering one is the same flow as any other SQL database, with the driver picked from the list. H2 is embedded rather than reached over the network, which makes two things different from MySQL, MariaDB, PostgreSQL and SQL Server: Where the file lives. The database is registered by name alone, as jdbc:h2:file:<name>, and that short form is what is persisted; DataSourceFactory expands it to ${OPAL_HOME}/data/h2/<name> when the connection is opened, creating the folder. Storing the name rather than the resolved path keeps the configuration portable if OPAL_HOME moves, and that is also the only site in the codebase that turns SQL settings into a connection, so the connection test resolves the same way. Restricting the URL to a plain name is what confines the databases to that folder: a path separator or a parent reference cannot be expressed, so there is nothing to escape with, and no canonical path comparison is needed to prove it. What it can be used for. There is no pre-existing database to import from or export to, so H2 is storage only. DefaultDatabaseRegistry.persist enforces this and the URL rule, being the single point both POST /system/databases and PUT /system/database/{name} pass through. Identifiers databases are registered with storage usage, so H2 serves there too. The dialog derives the offered usages from a new supportedUsages on JdbcDriverDto rather than naming the driver itself, so picking H2 leaves Storage alone and hides the JDBC datasource settings that only apply to import and export. Liquibase moves to 5.0.3, the version Magma 5.5 is built against, so that Magma creates its metadata schema on the Liquibase that Opal ships rather than one it was never tested with. Opal has no Liquibase code of its own; liquibase-slf4j 5.0.0 stays, its OSGi range naming 4.x notwithstanding, as the two SPI types it implements are unchanged between the versions. H2 is pinned to 2.4.240 to match Magma, which means 1.x database files are unsupported: they are rejected on registration with a message to migrate rather than left to fail obscurely in the driver. H2JdbcDatasourceTest covers the combination Opal ships rather than the one Magma tests, creating a table and round-tripping a binary value through a real H2 file. Also fixes the save handler in the dialog reading error.response.data .message, which ClientErrorDto does not carry, against a db.save_error message that had no placeholder for it: a rejected registration reported nothing about why. It now goes through notifyError, which resolves the status and its arguments.

2026-05-19

feat: added job queue per R cluster (#4147) * feat: added job queue per R cluster * fix: code review

2026-04-13

feat: added file bundle task for preparing download of large files (#4137) * feat: added file bundle command and options * feat: file bundle command implementation, add command result interface, applied to file download * feat: added support for multiple files bundling * feat: added file bundle progress message * feat: file bundle command can be cancelled * feat: file downloads can be reinstated * fix: always include project files link * fix: make sure file bundle command is available to everyone * fix: code review

2026-04-12

feat: added audit all permission allowing read only access to system (#4136) * feat: added audit all permission allowing read only access to system * feat: audit all permission added using wildcards * feat: updated ui to support audit all perm * feat: safeguard names from wildcard * chore: tr updated

2026-01-16

feat: added project entities counts per type and files count (#4092) * feat: added entity counts per type * feat: added project files count * chore: code cleaning

2026-01-14

feat: added setting to restrict R/DataSHIELD usage to personal access token authentication (#4090)

feat: added userinfo to subject profile when available from oidc realm (#4088) * feat: added userinfo to subject profile when available from oidc realm * fix: sanitize user info values

2025-11-30

feat: add re-auth on critical paths (#4079) * feat: added support for prompt and max_age in OIDC config, to help with reauthentication if provider supports it * feat: added ReAuthenticate to apply to REST resources that need to be protected with an authentication timeout, UI updated accordingly * feat: reauth in a dialog to not loose form data in a redirect * fix: cleaned add user dialog layout * fix: reauthenticate with credentials when user is from realm managed by agate * feat: reauth timeout for critical paths is configurable and can be disabled with a non positive value * feat: one resignin dialog for the app, api does the interception * feat: re-auth endpoints are configurable * feat: use simple wildcard in reauth config paths * chore: code cleaning

2025-10-04

feat: R session has a state (#4062) * feat: R session task manager added * feat: R session state and events added * feat: rock pod R session init moved to RockPodSession * fix: added logs and check r session is not pending before closing it * feat: display R session state/status in R admin page * fix: break loop when thread is interrupted * fix: break loop when thread is interrupted * fix: use a custom thread pool for rock pod sessions init * chore: code cleaning

2025-08-31

feat: improved management of pods (#4051) * feat: added liveness and readiness to rock pod #4048 * feat: added rock pod delay and max attempts to config #4049 * feat: attempts to reload project #4047 * feat: list or delete pod refs per spec * chore: log debug

2025-06-27

feat: added labels, node name, node selector and tolerances to pod spec (#4032) * feat: added labels, node name and selector to pod spec * feat: tolerances added to pod spec

2025-06-20

feat: kubernetes integration, rock spawner capability (#4031) * feat: legacy Rserve support removed * feat: added rock spawner app type (wip) * feat: added rock spawner app, with discovery and registration as an R cluster entry point * feat: rock spawner integration (wip) * feat: added k8s pod spec, ref and service * feat: added rock pod cluster of servers * fix: case rserver name does not have expected syntax * feat: ensure no potential conflict with pod name prefix * feat: improved rock spawner init phase * feat: R sessions ui * feat: added application lock file to avoid conflict when restarting opal pod * feat: download R server logs of the running pods * feat: added a beta badge to kubernetes admin page * fix: ensure work dir exists when locking * feat: rock spawner service has a running flag * feat: validate image references, can be restricted to a list of allowed ones * feat: added ui validations for pod specs * feat: make sure resource providers are initialized in k8s context * feat(ui): added pod spec removal

2024-10-29

Added query count API to get correct total counts (#3970) Co-authored-by: Ramin Haeri Azad <rhaeri@maelstrom-reseach.org>

2024-10-26

Clear only when starting search, added lastDoc for pagination (#3967) Co-authored-by: Ramin Haeri Azad <rhaeri@maelstrom-reseach.org>

2024-09-20

feat: R packages management system setting added, disabled by default #3858

2024-09-08

feat: reporting feature removed (#3924)

2024-05-05

feat(ui): added import data (wip), generate ts from proto

2024-04-02

feat: upgrade to Java 21, spring 6 and more (#3828) * Replaced @PostConstruct and @PreDestroy annotations by InitializingBean and DisposableBean interfaces * Upgrade of GWT, not working! * feat: java 17 wip * feat: upgrades orientdb graalvm resteasy (wip) * first java 17 compile and test * travis uses java 17 * deps versions fixes * jaxb * java 21 * opal-gwt-client removed * feat: upgraded vfs * feat: deb control updated with recommended packages * fix: upgraded slf4j, graal js engine usage to retrieve resourcer settings and utils * resteasy upgraded to latest * configure polyglot log * unzip plugin files in start scripts * orientdb and atomikos configurations * fix: ensure mime type on file download * code cleanup * do not load datasources concurrently (for now, to avoid liquibase errors) * removed windows start script because of missing plugins in classpath * orientdb create index is thread safe * opal upgrade is done in a separate, preliminary, process, because it was confusing some third party libs * jgit updated * makefile updated: removed gwt and update orientdb version

2024-03-15

feat(backend): added configuration option for enforcing 2fa to users from ini and opal realms #3724

2023-06-25

R session activity web services added

2023-06-17

#3802 R package commands added

2023-04-20

DataSHIELD activity logging (#3794) * #3744 #3745 WIP * Datashield context added for logging * DS error logging * DS workspace actions logging * Download all or latest logs. Record time spent in the R server. * Log assignments when async * Log rm * r_size added to track result size

2022-09-15

#3598 Use the OIDC end_session_endpoint when defined (and when enabled, default is true), to logout both from opal and the Identity provider

2022-09-06

#3756 Added a configurable logout URL

2022-07-18

#3663 Added a list of emails to notify when report generation fails

2022-06-11

#3743 Opal file system bean extracted from the opal runtime one

#3743 Opal file system bean extracted from the opal runtime one

2022-05-24

#3735 Apply domain specific callback url to ID providers

#3735 Apply domain specific callback url to ID providers

2022-04-26

Use of table status

2022-04-24

#3626 Added token expiration (default -1, no expiration) and activity (60d) timeouts

2022-04-22

Two-factor authentication mechanism added (#3711) * #3710 2FA feature added based on the TOTP mechanism * OTP exception allows HTTP header spec * #3710 2FA settings can be administrated

2022-03-28

#3569 All columns property added

#3569 No mutation script support (security), script edition not available for resource variables

2022-03-08

#3569 Resource view can be associated to a R server profile

2022-03-07

value table status added

2022-02-18

#3569 Continuous summary added, without normal distribution and support of missing categories

2022-02-09

#3569 ResourceView first implementation, reusing RValueTable helper classes

2021-06-29

DS profile R parser version can be modified

2021-06-28

#3655 DS R parser version handling, based on ds4j 2.0

2021-06-24

#3529 Both ID provider username (with claim name) and groups (with claim name and/or JS script) can be configured

2021-06-14

R profiles added

2021-06-11

DataSHIELD/R server profiles (#3648) * Get the R server by profile name. Datashield config storage move to orientdb started. * Datashield config moved to orientdb. DS methods handling refactored. * R administration page displays R server clusters * R cluster display layout * DataSHIELD administration page to handle profiles * Datashield Packages and Profiles sections, WIP * Reset DS profile * Reset DS profile and many UI improvements * Package DS settings can be (un)published in bulk * Confirm DS settings reset * Case there are multiple packages with same name (different servers and location) * Datashield profiles can be enabled/disabled and have their own REST resources * RServerProfile interface added * DataSHIELD profile access can require permission * Fix missing default R server profile definitio * Fix missing default R server profile definitio * DS methods and options can be removed in bulk * Handle case a R cluster is missing and associated profiles are still there * Fix resource providers discovery in the context of multiple clusters, ds admin permissions and default R server usage * DS profile display, addition, deletion * New DS profile is initialized with base profile settings

2021-04-20

#3500 Read only token added

2021-04-18

#3616 SQL service added to Personal access token

2021-04-16

#3616 SQL history refactored as a log file

2021-04-14

#3621 Project admin section added to personl access token

2021-04-07

#3616 SQL query audit added: GET /service/sql/history

2021-03-09

Fix App self-registration

Fix App self-registration

2021-03-08

Cluster name added to App, refactored Rock app info

2021-03-05

#3579 Entity ID column name can be specified at export time

2021-03-03

R errors handling reviewed

2021-02-22

#2044 Extract R server host nb of cores and amount of free memory

2021-02-18

R packages with cluster and R server name

2021-02-16

#2044 R server clusters (#3610) * Refactor R server interactions with adapter pattern, to not expose Rserve implementation and allow Rock implementation * R command queue sync fix, more doc * Fixed DS result setter * R server cluster management added * RDatasource updated to read JSON objects * Write file to Rock * Sync resource providers seek after R servers have been init, R variable categories can be provided by labels attribute * Rock credentials config, code cleaning * Rserve integrated to default cluster, code cleaning * REST entry point added to manage R clusters and associated R servers (get/start/stop/log). * R sessions are associated to the R cluster and server where they were created * Rock R server discovery * App rejection * App have tags and not necessarily a server public address (case of app discovery) * DataSHIELD packages management * DataSHIELD packages management, Rserve fix * Merge R logs from a cluster of R servers * More R server web services, either at cluster or server level * R servers of default R cluster listed in the UI * App authentication, Opal connects as an admin if manager/user is not defined * Notify when R server is stopped, clean managed R sessions accordingly * Renamed default apps properties * Check for legacy R server service availability * Refresh R sessions after R server stop event * Stop/start R server individually * Apps administration UI added * R server logs can be downloaded individually or concatenated * Rock app config UI added * Rock config user credentials UI * Data assignment refactored so that base64 content is just forwarded to Rock

2021-01-10

#2044 App registry added, to prepare R server app registration and usage.

2020-06-05

#3563 Option to backup views as tables

2020-06-04

#3563 Restore zip archive, possibly encrypted. Override option is false by default

2020-06-03

#3563 Project backup command added, restore command in progress.

2020-05-28

#2171 User and group suggestions, based of opal realm, profiles and permissions

2020-05-20

Resource providers status added

2020-05-12

#3566 Project's dashboard added. Async load of projects' datasources on startup.

2020-05-11

#3567 Description field added to resource reference

2020-05-06

Resource files are merged in one

2020-05-05

Resource services refactored to use R packages configuration instead of a plugin

2020-02-11

Resources (#3556) * Resource plugins SPI added * #3538 Resource plugin web services and dtos * #3538 Resource reference model, dto, service, resource * #3538 Resource reference web service for CRUD operations * #3540 Resources tab added to project view, listing of resources * #3540 Project resources UI first commit. Credentials data are encrypted in the database. * #3539 R/DataSHIELD resource assignment, first commit * Case resource name has a dot * JSON schema form enum items can have title * Resource factories can be grouped * Resources list with more columns * Html integer input box added * Resources permissions added * Clean resources on resource deletion * Refactored resources UI to have a view per resource (to apply permission at this level) * Fix resource permissions * Added edit and remove buttons to resource page * Resource duplication, removal in batch, with confirmation * Fix project page authorizations * More project authorizers added * Required package can be identified from the form and loaded at assignment time * Fix case stream value is null

2019-08-15

News panel and web service added, based on obiba.org data

2019-08-13

#3529 Groups claim setting added to ID provider config

2019-08-06

#3508 Tokens have a scope (#3521) * #3508 Access token scope can be restricted to enumerated projects * #3508 Project commands added to the scope of the token * #3508 Services added to the scope of the token * Handle token access to transient datasources * Token UI adjustments * Token UI adjustments (2) * #3508 #3477 opal.token option added to report template UI. Report parameters are stored encrypted and opal.password/opal.token options are hidden from the user. * Report UI fix * #3508 Sys admin added to token's scope * #3508 More system operations are submitted to token to be granted system admin * #3508 Opal REST client can authenticate with token. Applied to data import from a Opal server

2019-08-02

#3508 API access token added and handled to authenticate and grant permissions (#3520) * #3508 Opal token realm added, based on subject token object (managed by service and resources) * #3508 Token can give readonly access * #3508 Groups are saved in subject profile for reuse by token realm * #3508 No readonly mode, authz cache is managed by event subscription * #3508 UI to manage own's token added * Fix token realm shiro credentials (after token has been hashed) * Delete current session even when session id is specified (legacy)

2019-06-25

Subject profile page can include the account link when defined in the identity provider

2019-06-05

Project id mapping (#3481) * Added ProjectIdentifiersMapping in server/client code. * Remove project id mapping upon identifiers table/variable removal. * Added APIs to get a project ID mapping list or by entityType; refresh a project ID mapping list upon opening project admin TAB. * Added ID mapping when importing/exporting. * Added Edit/Remove project ID mapping. * Select best ID mapping when exporting several tables. * Bugfixes and cleanups.

2019-05-21

OpenID Connect configuration UI (#3469) * Added OIDC configurations web services. Configurations can be disabled * OIDC configuration UI added * Display ID provider user account login page link when available * More fields and validations on ID provider form * Fix Spring component init

2019-05-13

refresh command and conflict management

Added a project default export folder. (#3466)

2019-05-10

#3320 OpenID Connect support added. Pac4j dependency dropped.

2019-05-08

Login page shows OIDC clients

opal-pac4j added

2018-12-15

Changed analysisId to analysisName, name validation, fixed var chooser bug. (#3403)

2018-12-13

Added last result success/failed count and added sorting to analyses table. (#3398)

2018-12-11

set last analysis result status in analysis (#3390)

2018-12-08

analysis name must be unique for project table (#3388) * analysis name must be unique for project table * remove id form analysis

2018-12-05

Added icon for status in results panel, added duplicate action. (#3383)

2018-12-03

Added anlysis editing, running from list, view+run. (#3373)

2018-11-28

add pluginName in Analysis (#3368)

2018-11-27

Implemented export-analysis. (#3363) * Implemented export-analysis. * Removed limit and order and used lastResult instead.

2018-11-22

analysis plugin resources (#3358)

2018-11-20

analysis project and table GET resources (#3352) * analysis project and table GET resource * use dtos for list

2018-11-09

Implemented the Analyze command (#3332) * Implemented the Analyze command * Used RUtils for R symbol names.

2018-10-30

Added an ErrorDto for R runtime. (#3322) * Added an ErrorDto for R runtime. * Specialized the ErrorDto for OpalR WEB. * Added RRuntimeException and fixed PR comments. * Used the correct ErrorDto for RRuntimeException.

2018-09-24

#3307 Report datasource plugin group and use it in the UI

2018-09-05

Remove limesurvey and related code (#3294) * Remove limesurvey and related code * Add upgrade step to remove all limesurvey databases * save databases as properties files for backup * make sure that file is created * call mkdirs to make sure directory tree exists * catch noSuchElementException * intall the plugin, forgo the check * set default value for textArea * add "usage.IMPORT." prefix to help streamline copy pasting th properties

2018-07-31

Use pluginDatasourceFactoryDtoParser

2018-07-27

WIP - Plugin Datasource resource