Proto commits in diagonalworks/diagonal-b6

These 73 commits are when the Protocol Buffers files have changed:

2024-12-19

Resolve duplicate clicking in UI (#355) * Resolve duplicate clicking in UI We add a few features here: - A "target" function to the header; allowing an item to be located. - Fixing the "duplicate popup" issue when clicking on something open; it now just refocuses te element It's not perfect; but some comments have been added to indicate the areas for improvement. * Allow header-things to be closeable or not * Added copy and toggle-visibility back * Allow a little bit more toggle-visibility

The documentation is generated from this commit.

2024-12-02

Some simple useful features (#351) * Add is_valid and get_centroid * Test for is_valid filter * Test for get_centroid

2024-05-14

Added a /compare endpoint. (#228) Added a /compare endpoint, which reruns an analysis in a number of worlds, and returns a comparison of the results. In passing, correct and add tests for: - nested lambda expressions - changing searchable tags on overlaid features - filling collections from protos

2024-05-09

Support conditional map layers, and clamp bucket values. (#218) Also adjust render ordering, so geojson layers appear on top of building histogram colours.

2024-05-08

Added a skeleton API definition for scenario comparison. (#211) Also added add-world-with-change to b6.

2024-05-07

Add functions to list and delete worlds from Python. (#208)

2024-03-27

Cleanup naming of root vs context. (#194)

2024-03-26

Added an 'action' line to the design system. (#191) Added an 'action' line system, which allows an expression to be evaluated, optionally in the context of the stack it's being rendered within. This is still an exploratory concept.

2024-03-23

Render histograms by creating a collection in the world. (#181) Render histograms by creating a collection in the world, and passing the ID of that collection through to the tile renderer, instead of including the IDs of bucketed features in the JSON response. This makes the size of the response tractable. These features are currently leaked, but that'll be addressed in the future (by deleting features when the relevant stack is closed in the UI). Introduces a new method on CollectionFeature to find the first value for a given key, which can make use of binary search if the collection is sorted (otherwise tile rendering becomes very slow with large collections). In passing, make the tests for histogram rendering non-flakey by explictly ordering keys as they're encountered.

Render histograms by creating a collection in the world. Render histograms by creating a collection in the world, and passing the ID of that collection through to the tile renderer, instead of including the IDs of bucketed features in the JSON response. This makes the size of the response tractable. These features are currently leaked, but that'll be addressed in the future (by deleting features when the relevant stack is closed in the UI). Introduces a new method on CollectionFeature to find the first value for a given key, which can make use of binary search if the collection is sorted (otherwise tile rendering becomes very slow with large collections). In passing, make the tests for histogram rendering non-flakey by explictly ordering keys as they're encountered.

2024-03-20

Apply changes to the world made in the UI. (#179) Apply changes to the world made in the UI, telling the frontend to reload tiles if necessary. Factors out locking the world against concurrent modification into MutableWorlds (although our semantics around modifying the world when there are live iterators are still sketchy).

2024-03-18

Support separate worlds, keyed by the root feature ID. (#177) Support separate worlds, using the ID that's currently used for the feature that configures the UI to select which world. Support passing this ID through Python, and add it to the tile requests from javascript. TODO: this ID is currently called "context", "root", or "UI context", we should standardise on something.

2023-12-07

Add accessible-routes. (#163) Also rename accessible to accessible-all, and connect-all-to-network to connect-to-network-all, to form a convention that the -all suffix of a function is a more efficient implementation of the mapping the unprefixed version across a collection.

2023-12-06

Support logging events in the UI (#162)

2023-11-26

Add support for share icons in headers. (#149)

2023-11-25

Support naming expressions. (#146) Support namining expressions, so that parts of an expression can be named in an analysis with a value that can appear in the UX. Simplify queries from Python before running them, and support simplification of search query literals. In passing, correct a misplaced bucket colour.

2023-11-22

Add a header line to stacks, with a close button. (#143)

2023-11-20

Support condition GeoJSON and atoms. (#139)

Support bucketed values on the map based on dropdown chips. (#138) Also added a Swatch line type.

2023-11-15

Make materialise() add collections and expressions in the world. (#133) Change the interface of materialse(), so that it now takes a function that returns a collection, calls that function, and then adds both that collection, and the function that created it to the world. Includes the Python API for CollectionFeature and ExpressionFeature, which necessitated merging feature.proto and api.proto, as they now depend on each other.

2023-10-13

Support rendering query layers in different map positions. Also support shift-clicking on features from query layers, and dragging stacks from result lines.

2023-09-22

Render the coastline, and copy text on single click.

2023-09-18

Introduce the concept of locked stacks explictly. Introduce the concept of locked stacks explicitly. Actions on a locked stack result in further locked stacks, but some actions (like shift-click) allow you to break out.

2023-09-17

Support colouring by bucket, and outliners and click positions.

2023-09-16

Support multiple atoms on the left hand side of left/right line. Also use colour names, rather than numbers, for outliner classes.

Support adding relations, and passing a context relation to the UI.

Support rendering stacks at startup, which we're calling the 'dock'.

2023-09-11

Set the map center from the evaluated result.

Use an atom for the range UI element of histogram bars.

Use protos based on the b6 design system to communicate with JS. Use protos based on the design system that has emerged from the current UI for communication with JS code, replacing the existing ad-hoc JSON struct definitions. Also replace to badly-aliased PNG icons with SVG versions.

2023-06-04

Introduce a new UX for the web interface. Introduce a new UX for the web interface, rendering the steps of a pipeline as distinct blocks, and allowing multiple blocks to be shown at the same time.

2023-05-14

Add generated files to version control, with a pre-commit to update them. Add generated files to version control, so it's possible to include the diagonal.works/b6 go package as a dependency without generating the files. Add a pre-commit hook to ensure the generated files are rebuilt when their dependencies are changed.

2023-05-03

Use semver versions for the compact index. Use semver versions for the compact index, to match the approach for the API, and to track changes that don't break backward compatability. Use a protobuf to encode the header data, to allow flexibility where memory isn't at a premium. Invalidates all previous indices by changing the header magic.

Show the version in the UI, and verify it in GRPC requests. Stamp the current version number into the b6 backend binary, and show it in the web UI. Also include it in the generated Python client code, and send it along with GRPC requests. Fail requests that come from a Python library with a different major version number. Also use the version number in the PIP project description, and the binary tarball in the docker build.

2023-04-17

Add a skeleton b6 binary, which currently runs a GRPC server.

2023-03-21

Make proto definitons match Go code for queries.

2023-03-20

Introduce b6.Query, and replace the use of pb.QueryProto and search.Query. Introduce a new query representation, b6.Query, that's at the semantic level of features, and use that across the codebase, rather than a mixture of pb.QueryProto and search.Query.

2023-03-13

Extract diagonal.works/b6, which we'll open source.

2023-02-22

Use explicitly named arguments to lambdas. Use explicitly named arguments to lambdas. This enables the Python interface to use the full expressive power of b6, as the b6 shell could previously use partial evaluation to support nested parameters, something that isn't supported by the Python API.

2023-02-19

Shell usability improvements. Implement history, and copying tokens when clicking on them, in the shell.

2023-02-10

Treat tags as first class entities in the shell.

2022-10-12

Render feature titles and tags in the shell.

2022-08-31

Parse string based namespaces as tokens

2022-08-30

Make namespaces strings instead of integers. Make namespaces strings, instead of a set of hardcoded integers. While less efficient, it allows users to integrate arbitrary datasets under a namespace of their choice, without predefined restrictions. Namespaces are based on URLs related to the relevant data source. This change allows us to remove the "user space" implementation of namespaces in a5. Note that this change breaks index formats, and so we bump the version number.

2022-06-25

Add snap_area_edges, flattern and points. Add snap_area_edges, which returns a new area by snapped the edges of an area to paths matching a given query, intended to be useful for conflating boundary geometries to road networks. Also add the useful remnants of a failed approach snapping at the point level, specifically flattern, which flatterns collections, and points, which extracts points from geometries. Collections are now typed on the wire, allowing the type of empty collections to be determined. Consequentely, the definition of Type is begining to move into protocol buffers.

2022-06-09

Reimplement A5, adding pattern matching and first class functions. Make functions a first class entity in A5, and add lambda functions and Haskell style pattern matching. This allows many real-world analyses to be performed entirely server side, when previously they needed an expensive per-feature round trip. This change breaks the current A5 interface, and therefore historic analyses until they're updated. Setting tags and exporting as GeoJSON now happens via A5 functions and evaluate(), rather than evaluate_as_tags and evaluate_as_geojson. Concurrent reading and writing is also currently broken (with tests skipped) until locking is reintroduced.

2022-03-30

Expose point connection via A5. Expose point connection via A5. Introduces a new return type, WorldModification, which describes changes to a MutableWorld instance, and handles ID allocation. When used as the return value from a expression, A5 uses it to modify the world.

2021-09-28

Add JoinPaths and Interpolate to A5.

2021-08-25

Complete exposure of sightline in A5.

2021-07-22

Add SamplePointsAlongPaths to A5

2021-06-08

Make distance a paramter for reachable() In passing, make the use of tag keys in the Python test match the new index format (eg #highway, not highway), and make it possible to return nil values from A5, to match the behaviour outside A5 of looking up non-existant IDs.

2021-04-08

Add basic support for searching within polygons defined by WKT. In passing, correct the year used for importing ONS authority boundaries, and add a missing Makefile for ONS data.

2021-04-06

Add a progress bar widget in iPython. Add a progress bar widget in iPython, which uses a JSON endpoint to poll for updates. The involves changing the streaming GRPC interface - rather than incrementally streaming progress, it sends a single tracker ID before the result. Also add a new binary - scaffold - that makes it easy to play with HTML fragments, like the widget.

2021-03-26

Use oneof in the respone protos for ...WithProgress.

Adjust the implementation of timeouts in DFE. Long running streaming GRPC requests were cancelled due to the WriteTimeout in DFE. Handle timeouts ourselves to avoid this. Also add scripts to start atlas and dfe locally with common options.

2021-03-25

Allow a5.within() to search Features. Also add world.uk_ons_boundary_id().

Add SetTagFromGraphWithProgress.

2021-03-24

Add a streaming version of RunGraph which reports progess. In passing, also add a script to start a personal copy of Atlas on a 64 processor VM.

2021-03-16

Add countCollection to A5. Currently this only works for FeatureCollection, unfortunately, until we've sorted out the type system.

2021-03-15

Return GeoJSON results to Python via A5. Also includes adding GeoJSON properties for tags.

2021-02-15

Compute the areas reachable from a given point. Also make it possible to override the colours of boundaries, making it easier to render the results.

2021-02-14

Make it possible to find and colour paths from relations.

2021-02-09

Implement searching for features within other features in A5. Also add ingestons, a pipeline for ingesting boundary data from the ONS. It exists outside of the standard ingest binary, since it depends on the native GDAL library, complicating cross-compilation.

2021-02-02

Rework the Python query API, and add a simple area restriction.

Introduce a simple type system into A5, validating graphs before running.

2021-01-24

Extend the A5 graph interface, rewrite the colour example with it

2021-01-21

Explorations of an experimental computation API. Also added a missing test for deriving colours from a gradient.

2021-01-12

Implement the remaining World methods on MutableTagsOverlayWorld. Implement the remaining World methods on MutableTagsOverlayWorld. Also use this implementation of World by default in the gRPC API, and use it to implement a ModifyTags method, exposing it to Python.

2021-01-06

Add FindFeatures to the GRPC API.

Add an experimental Python client for the WorldService.

2021-01-05

Rename FeatureProto2 to FeatureProto With the beam ingest pipleine removed, the naming conflict is no longer present.

Add the beginings of a GRPC based API. Currently, the API simply supports looking up a feature by ID via a service wrapper on top of world.World. Implementing the API involved upgrading the version of protocol buffers we depend on, and switching from a forked protocol buffer implementation we were accidentally depending on. This, in turn, triggers a warning about duplicate metrics.proto files being registered (from Beam and Prometheus, presumably), and actually changes the number of elements read from our test PBF files. The warning will be addressed in a future change.

2019-12-02

First end-to-end pass at vector tile rendering.