Proto commits in pganalyze/libpg_query

These 22 commits are when the Protocol Buffers files have changed:

2024-10-31

Run `./scripts/generate_protobuf_and_funcs.rb`

The documentation is generated from this commit.

2024-01-09

Correctly handle AclMode as an uint64 (not "long") This mainly helps with resolving compiler warnings, and should not have any effective difference, since the relevant struct (RTEPermissionInfo) is not actually used in raw parsing.

2023-12-22

Generate protobuf and funcs

2023-10-03

pg_query.proto: Add Token ASCII_36; Add scan test (#211) Postgres's scanner will scan $ into a single token. For example: SELECT $identifier; The parser will later reject this. However, since the scanner returns this value, we should define a Token for it in the protobuf. test/scan.c: * Add a check so the test does not crash if token_kind is NULL, which it was before I defined the token. The test now prints an error: INVALID result for "...": scan_result token 1 token_kind == NULL * Make tests constant to prevent accidental modification. Unnecessary but could be helpful. * Define testsCount automatically so it no longer needs to be changed. * Add an assert to help ensure tests are defined correctly.

2022-12-12

Upgrade to PSQL 15.1 This upgrades `libpg_query` to Postgres 15.1. All tests have been updated, and tests were added for new constructs. - `Boolean` nodes have been added to the parser, replacing usage of integers with the values `1`/`0`. - In the postgres parser, `Value` was changed to be a union stored in the `A_Const` type. For this upgrade, we block `A_Const` from code generation, and instead manually implement the relevant required functions. - The `COPY` statement deparser was also updated to attempt deparsing into the old Postgres 8.4-style syntax (e.g. `COPY foo FROM STDIN FREEZE CSV`). This ensures that the regression tests keep working, as new changes in the parse structures mean that these two syntaxes result in equivalent but slightly different parse trees. - Support was added for `MERGE` statements. - Support was added for more advanced publication objects, e.g. `CREATE PUBLICATION foo FOR TABLES IN SCHEMA CURRENT_SCHEMA` - Support was added for `ALTER TABLE ALL IN TABLESPACE ...` statements

2022-11-23

Remove bare label status from scan output This reverts commit 33f42515d957c5625d94a287db8af6b2502a1236.

2022-11-21

Output bare label status in scan tests and example

2022-11-15

Merge branch '13-latest' into HEAD

2022-11-02

Update Postgres to 13.8 and backport bc7a40b4 to fix new Xcode build See https://postgr.es/m/1803927.1665938411@sss.pgh.pa.us for discussion of the Xcode build issue. Fixed in bc7a40b4, which is on REL_13_STABLE but not yet in an official Postgres release. In passing this also fixes a extract source issue with newer clang versions.

2022-09-06

PG14: run scripts/generate_protobuf_and_funcs Signed-off-by: Wolfgang Walther <walther@technowledgy.de>

run other scripts on latest Arch Linux Signed-off-by: Wolfgang Walther <walther@technowledgy.de>

2021-05-28

Fix crash in pg_scan function when encountering backslash escapes (#109) This also updates the protobuf-c library to 1.4.0.

2021-03-18

Improve output of Bitmapset fields, add to Protobufs Note that Bitmapset fields are currently only used in parse analysis trees, not raw parse trees, but this future-proofs our output code.

Protobuf: Always start enum values at 1 instead of 0 This avoids the automatic skipping of zero-values in the JSON-mapped version of the Protobuf output. To match, the simple JSON output (without the Protobuf library), now also always outputs the enum value.

Add automatically generated readfuncs to turn Protobuf into Postgres AST This will be used in a subsequent commit to enable deparsing of the Protobuf-encoded tree back into a SQL query.

Extract headers: Correctly handle 3+ enum values in the same line

Refresh parser/lexer patches for Postgres 13.1 This also fixes a few edge cases where parsing wasn't working as expected, and resolves an issue that broke JSON operator parsing. The new set of patches pass "make check" cleanly for Postgres, and also apply cleanly to Postgres 13.1 to make pg_query-specific information work.

Merge protobuf files into one, introduce top-level parse result

Add experimental Protobuf C++ outfuncs, converge JSON output to match This introduces the following changes to the JSON output format, in order to match the Protobuf JSON mapping scheme: - Default values are no longer output for Enum values - Whitespace between control characters in JSON is no longer added - "<" and ">" characters are escaped to avoid browser HTML injections Separately, this commit also unifies all outfuncs related code into the "pg_query_outfuncs" prefixed files, to make it easier to understand. The Protobuf C++ build can be utilized by building with "make USE_PROTOBUF_CPP=1". Note that the Protobuf C++ requirement is for building only, and the library itself remains a C library (with a bundled version of Protobuf-C).

Scanner: Generate correct scan tokens based on yytokentype enum Previously we used a hand-generated list that wasn't automatically updated with new Postgres releases.

Introduce new Protobuf parse tree output format Up until now, this library relied on JSON to pass the parse result back to the caller, which has a number of downsides, most importantly that we don't have a readily available parser for JSON thats not tied to a running Postgres server. That in turn makes it hard to provide cross-language features such as deparsing directly in this library (which would require reading back a parse tree that gets passed in). Protobuf isn't perfect, but its straightforward enough to generate the schema definitions for the parse tree nodes, and output the tree using a bundled C protobuf library, which has a small enough SLOC count (~3k) to not be noticeable in the big picture.

Add support for returning Postgres scanner result This allows utilizing pg_query for use cases that need the raw token information, instead of a parse tree. Due to additional modifications to the Postgres source, this also contains information about comments in the query string, and their location.