Proto commits in IBAX-io/go-ibax

These 36 commits are when the Protocol Buffers files have changed:

Commit:a8ecb71
Author:powerpook

P7: block version 4 commits the validators set of the next height A block of version 4 carries next_validators_hash, the hash of the validators set of the next height as its execution leaves it, and state_hash, the state hash of a snapshot it confirms; both enter the block hash and its signature. On a chain of the BFT consensus the trial of a generated block sets the validators hash from block 2 on, and a block of another node, proposed or committed by a certificate, is valid only if its header has the hash of the set its execution leaves and no state hash, until the nodes take snapshots. The block size bound counts both hashes at their length, and the shell of a streaming proposal leaves them out with the other fields the trial sets.

Commit:ee72801
Author:powerpook

P9: execute a block once, verify before the lock, send compact proposals - A trial holds its database transaction and the memory state it leaves until the height decides; Commit commits the held trial of the certified block and executes only another one. - A validator verifies the signatures of a proposal in parallel before it takes DBLock; DBLock guards only the execution and the writing of a block, not the machine or the messages. - A proposal carries its block in the compact form, the hashes of its transactions in the place of their bodies; a validator fills it in from its queue and fetches the rest from the proposer (RequestTypeBftTransactions), apart from the consensus goroutine. - A block carries after_txs no more: every node recomputes it; the block size counts the transaction bodies only.

Commit:3ee37fe
Author:powerpook

Settle the fees of a block once per account after its last transaction A transaction debits the payer of its fees and leaves the credits to the block: the block sums them by ecosystem and account and, after the last transaction, makes the missing accounts, credits every account once and writes one history row of type Settlement with its balance, in the order of the ecosystem and the account. No transaction writes the rows of the producer and of the taxes account any more, which every transaction paid. The rows the settlement writes have the transaction hash SettlementHash; the rollback of a block takes them back first. The block size keeps room for them as the accounts come. The history row of a fee keeps the balance of the payer; the balance of the recipient is in the settlement row.

Commit:9cae186
Author:powerpook

Remove the SQL DML block sync remnants DbTransaction.BinLogSql collected every executed statement for a block sync by SQL DML that is no longer implemented, and nothing read it. Remove the field, the commented-out code around it, the --sync flag and its config field, which selected that sync method without effect, and the BlockSyncMethod enum.

Commit:e349d7d
Author:powerpook
Committer:GitHub

Height-gated execution rules: one binary replays testnet and mainnet from block 1; Go 1.27 and bug fixes (#2175) * fix(crypto): derive the public key before signing, fixing the P256 signing panic on Go >= 1.20 P256.Sign only set the private key D. Since Go 1.20, ecdsa.Sign needs the public key for NIST curves, so signing panicked with a nil pointer. Nodes sign blocks (types/block_data.go) and system transactions (transaction/smart_contract.go) through this path, so a chain configured with ECC_P256 could not produce blocks. - p256 and secp256k1 derive the public key from D before signing - ecies.GetPrivateKeys fills in the public key as well - New sign/verify test: 3 signature algorithms x 4 hashes, 1000 random private keys each * fix(crypto): sign the genesis block with the node key; refuse out-of-range private keys - generateFirstBlock reads the node keys (syspar.ReadNodeKeys) before building the genesis block. It used to sign with an empty private key: P256 failed outright ("private key scalar is zero or negative"), and the other algorithms produced an invalid signature. - Sign for P256 / secp256k1 / SM2 checks that the private key satisfies 0 < d < n and returns ErrInvalidPrivateKey otherwise; SM2 also checks for an empty hash (ErrSigningEmpty). - New TestSignRefusesInvalidPrivateKey: nil, 0 and n are refused on all three curves. Found and verified by Weaver's local chain e2e: a 2-node network per suite for all 12 suites, with the peer checking and syncing every block. * fix(api): keep the JWT secret across restarts and make it unguessable The node made its API token secret at init from 15 letters of math/rand seeded with the start time, so whoever knew roughly when a node started could recover the secret and forge a token of any account, and every restart logged all users out. Start now loads a 32-byte crypto/rand secret from KeysDir/JWTSecret (hex), creating it on the first start with mode 0600 through a temporary file, and gives it to both the REST and the JSON-RPC API. A secret file that is not 32 hex-encoded bytes, or that other users can access, stops the node instead of being replaced. RandSeq, still used by the API tests, draws from crypto/rand. * Conditions run apart from the transaction's contract see its $stack ContractConditions, the conditions of rows and tables and the read filter ran their code with an extend built by getExtend, which had no stack: $stack there was an unknown extend identifier. getExtend now carries a copy of the contracts called so far, and ContractConditions pushes its contract before building it, so its conditions see themselves on top as the transaction's contract does. * fix: pass node, compiler and contract text as data, not as a format string Every one of these calls passed text from elsewhere (a node's result, a compiler message, contract output) as the format string, so a '%' in that text came out mangled. gas.go also named the recipient in the error about a missing sender; it now names the sender. These are the calls go vet refuses once the go directive is 1.24 or later, which is why they went unnoticed on 1.20. Ported from develop ac585d5e. * fix: name the field of the StopNetworkResponse literal go vet refuses the unkeyed literal from Go 1.26. Ported from develop 073781c1. * test: follow the current signatures the tests had not kept up with - metric: CollectorFunc takes the block time, ToMap returns a types.Map and Collector.Values takes the block time - gen: contract.AppID is a string and escape no longer doubles '%' Neither compiled under go vet before; both are test losses from earlier signature changes that main never picked up. * build: require Go 1.27 The go directive moves from 1.20 to 1.27.0 and the README from 1.17 to 1.27. Nothing in main needs ML-DSA, circl or any 1.25+ crypto: this only raises the toolchain, so that vet and the compiler of a supported release build main. * fix(script): call an extended function without parameters The VM indexed the parameters of such a function at -1 and crashed. No function the node gives contracts is such, so contracts run as before. Ported from the runtime.go part of develop 399c2cef. * fix: TransferSelf history and ecosystem 1 for plain transactions apply from testnet block 1729 b962ceab (#2064) changed how blocks execute without regard to height, so a node syncing the testnet from block 1 computed other rollbacks hashes than the network for the blocks made before it. Block execution rules now take effect per network from a fixed block, kept in packages/conf/rules; on other networks they apply from the first block. * fix(language): LangRes no longer depends on how long the node has run UpdateLang made a cache of the one changed source when the ecosystem's sources were not loaded yet, and LangText then took that cache as complete: after a restart, a NewLang or EditLanguage hid every other source of the ecosystem from LangRes. A node syncing the testnet from block 3052 wrote untranslated names into 1_notifications at block 3067 where the network had the translations. UpdateLang now leaves an unloaded ecosystem to LangText, which loads all of its sources from the database. * fix: bytea rollback data is hex before testnet block 3055 Blocks before f1a3650c kept the rollback data of a bytea column in hex; testnet blocks 1746-2755 carry such rows. ByteRollbackRaw takes effect from testnet block 3055. * fix: count fees and VM cost as testnet blocks before 2561 do Blocks made before d967f160 count fees through price_create_rate and the fuel rate rather than in the digits of the paying ecosystem, charge an element fee, charge a contract's base cost for each of its functions, and end a run that goes over its cost limit instead of failing at once. The rule FeeByDigits keeps these semantics for testnet blocks before 2561. * fix: take the jsonb column order testnet blocks before 2561 carry * fix: charge UTXO ecosystem fees as testnet blocks before 2425 do * fix: bytea rollback data is double hex only in testnet blocks 2657-3054 Since 630191f5 a row is read with its bytea columns in hex, so the rollback data of such a column is single hex in testnet blocks up to 1764 (made before 630191f5, which encoded the raw bytes once) and from 3055 (f1a3650c), and double hex in between (2657-2755). ByteRollbackRaw is replaced by ByteRollbackHexTwice and ByteRollbackHexOnce. A rollback restored a bytea column as text: the column name was quoted before IsByteColumn looked it up. It now decodes the rollback data, twice where it is double hex, so the column gets its bytes back. Claude-Session: https://claude.ai/code/session_01Rj4pFusXALK5Ybec9uBJyz * fix: create the ecosystems table access testnet's first block created The first block of a network made before #2072 gives neither @1TeEmission access to the ecosystems table nor, before #2133, @1NewToken access to its digits; a later export of the table (testnet block 2630) carries that access. TeEmissionAccess and DigitsAccess select it per network; testnet's first block predates both. Claude-Session: https://claude.ai/code/session_01Rj4pFusXALK5Ybec9uBJyz * fix: activate the height rules on mainnet where its blocks change Mainnet's first block predates #2072, and its blocks follow d967f160 from block 16993 (2023-06-09); it carries no bytea rollback data and no UTXO transfers outside ecosystem 1. Claude-Session: https://claude.ai/code/session_01Rj4pFusXALK5Ybec9uBJyz * every node records the status of the transactions in a block it plays transactions_status had a row only on the node that took the transaction through /sendTx: AfterPlayTxs updated rows it never inserted, so any other node answered E_HASHNOTFOUND for a transaction in its own chain, and a client that moved to another node could not follow what it had sent. The block's statuses are now upserted (hash, time, wallet, block, result, penalty), as gorm parameters in place of the SQL that was built from the results. A copy of a transaction in a block, sent again, is refused in the queue as a duplicate; MarkTransactionBad then wrote that refusal over the status of the one in the block. SetError now leaves a status with a block alone. (cherry picked from commit 20a8f04f) Claude-Session: https://claude.ai/code/session_01Rj4pFusXALK5Ybec9uBJyz * script: the cache of compiled conditions is safe for concurrent use EvalIf kept the conditions it compiled in a package map without a lock, and read it again after compiling. Blocks are not its only callers: the data API holds each table read to its read condition and templates filter by permissions, each request in its own goroutine. Two requests meeting a new condition at once wrote the map together, and the runtime stopped the node with "concurrent map writes" (seen with a client that loads its pages after restoring a session). The map is now guarded by a RWMutex, and EvalIf runs the block it compiled or found. (cherry picked from commit 3728ab17) Claude-Session: https://claude.ai/code/session_01Rj4pFusXALK5Ybec9uBJyz * asymalgo: a generated private key is written in 32 bytes P256, Secp256k1 and SM2 returned big.Int.Bytes() of the scalar, which drops leading zero bytes: one key in 256 came out in 31 bytes, and the PrivateKey and NodePrivateKey files held 62 hex digits. Clients take a key of these curves in 32 bytes only, so such a founder or node key could not sign in there. The scalar is now padded on the left, as the public key coordinates already were. (cherry picked from commit da484535) Claude-Session: https://claude.ai/code/session_01Rj4pFusXALK5Ybec9uBJyz * The compiler refuses an assignment without a value An assignment without an expression after '=' (`$result = }`) compiled to code that underflowed the stack when it ran. The assignment now requires an expression, as if and while do. (cherry picked from commit bb97156c) Claude-Session: https://claude.ai/code/session_01Rj4pFusXALK5Ybec9uBJyz * daemons: a node with its own 1block starts from it, also with boot nodes loadFirstBlock fetched block 1 from a boot node whenever boot nodes were set, ignoring the node's 1block file. Nodes of a new network that boot from one another could then never start cold: with no peer up, each failed with "All nodes unvailabale". The node now reads its own 1block and fetches block 1 only when it has no such file; a file it cannot read is an error. A 1block file now wins over the boot nodes, so on testnet and mainnet a file made for another network would start a chain of its own. The rules package keeps the hash of their first blocks, and the node refuses a block 1 with another hash. (ported from commit 5538ecf5) Claude-Session: https://claude.ai/code/session_01Rj4pFusXALK5Ybec9uBJyz * fix(utxo): number a transaction's inputs in a fixed order UpdateTxInputs numbered the inputs of a transaction while ranging over the map of its keys, so with inputs of more than one key (ecosystem and key ID) the input_index of each depended on the map's random order, and two nodes could write different spent_info rows. The rollbacks hash covers the spent_info rows a block creates, input_index included, when one of them is spent in the same block. The keys are now taken in the order of ecosystem and key ID. Claude-Session: https://claude.ai/code/session_01Rj4pFusXALK5Ybec9uBJyz * test(migration): check the first ecosystem script instead of writing eco.sql TestGetEcosystemScript wrote the generated script to packages/migration/eco.sql in the source tree. It now checks that the script mentions the founder wallet. (ported from commit ed83f5d5) Claude-Session: https://claude.ai/code/session_01Rj4pFusXALK5Ybec9uBJyz * Fix the utils and metric tests TestNtp queried pool.ntp.org a thousand times, two seconds apart, and asserted an error: it needed the network, took over half an hour and failed on a good clock. sntpDrift takes the server's address, and TestSntpDrift measures the drift against a local NTP server whose clock is shifted by a known offset. Each query closes its connection when it ends, not when sntpDrift returns. The metric tests had not followed CollectorFunc taking the block time and ToMap returning a types.Map. (cherry picked from commit 5bbe52fb; the metric test changes were already in) Claude-Session: https://claude.ai/code/session_01Rj4pFusXALK5Ybec9uBJyz * Make go test ./... pass - script: the expectations of the VM, EvalIf and lexer tests are the messages and results of the current VM; TestVMCompile reports every wrong case instead of stopping at the first - api, protocols, scheduler, sqldb: the tests follow the current signatures and fixtures - syspar: TestHonorNode expects the current error messages, and drops the key_id case: honor_nodes entries have no key_id (ported from commit 399c2cef, whose runtime.go change is in 12e88031) Claude-Session: https://claude.ai/code/session_01Rj4pFusXALK5Ybec9uBJyz

The documentation is generated from this commit.

Commit:4e21897
Author:powerpook

Accept PIV account keys: ECC_P384, RSA2048 and RSA3072 (RSASSA-PSS) PIV cards hold P-256, P-384 and RSA signature keys; accounts now take all of them. The new algorithms are account keys only: no node signs with them. RSA signs with RSASSA-PSS, the PSS and MGF1 hash being the network hash with a salt as long as it; account_algorithms refuses RSA on KECCAK256 and SM3 networks. The public exponent is fixed at 65537 and the public key is the modulus.

Commit:741516b
Author:powerpook

account_algorithms: the algorithms account keys may have, set at genesis The platform parameter lists each algorithm with its last registration and signing days (UTC, block time). A change may only bring days earlier and may remove an algorithm only when no key has it. The first block carries the initial set, which must hold the node algorithm, and a founder account key; CheckGenesis checks both. A node that cannot verify every algorithm of the set stops. generateKeys --accountAlgo writes an account key, and generateFirstBlock --accountAlgorithms sets the initial value (F13 step 2).

Commit:d60f915
Author:powerpook

feat(crypto): add ML-DSA-87, SHA-384 and SHA-512 - AsymAlgo MLDSA87 (FIPS 204 ML-DSA-87, context IBAX-MLDSA-87-v1) on the Go 1.27 crypto/mldsa; ML-DSA-65 and -87 share one implementation (asymalgo/mldsa.go). HashAlgo SHA384 and SHA512 from crypto/sha512. - AsymProvider.SignatureSize and HashProvider.Size; crypto.HashSize() replaces consts.HashSize, so the network protocol, the JSON-RPC block lookup and the hash columns follow the chain's hasher instead of assuming 32 bytes. 1_binaries.hash and the child chain's hash column hold a SHA-512 digest in hex (128). - cryptovectors: 30 suites (5 cryptoers x 6 hashers), 90 signature vectors, 36 ML-DSA context-free signatures; every suite has transfers the node accepts. Co-authored-by: Lin Wan <founder@sygnux.com>

Commit:2a38265
Author:powerpook

feat(crypto): ML-DSA-65 (FIPS 204) as cryptoer MLDSA65 A chain can now run with --cryptoer=MLDSA65 under any of the four hashers. Ported from the ML-DSA work in ibax-federal (bb772cf) and reworked so that it fits the node's key handling and verifies like the other cryptoers: - The private key is the 32-byte FIPS 204 seed, not the 4032-byte expanded key, so key files, encrypted wallet keys and imports keep their size, and any client key is a valid seed. The public key (1952 bytes) and the signature (3309 bytes) are the FIPS 204 encodings. - Signatures are pure ML-DSA over Hash(data), the digest every cryptoer signs, with the context string IBAX-MLDSA-65-v1, and hedged (fresh randomness in each signature) rather than deterministic. - Verify returns ErrIncorrectSign for a signature that does not match, and a separate error for malformed input, as the curves do; it used to return (false, nil). - Account ids come from crypto.Address like every other key. AsymProvider gains PublicKeySize, so code that checks node keys can ask the network's cryptoer instead of assuming 64 bytes. Not ported: the batch verifier and the mnemonic/HKDF key derivation (neither has callers; clients derive the seed as for the other cryptoers), and the federal directory, FIPS build script and vector tool (go-ibax already has tools/cryptovectors). The provider's public key for a fixed seed and its signatures were checked against @noble/post-quantum, which Weaver signs with, in both directions. Co-authored-by: Lin Wan <founder@sygnux.com>

Commit:0b89a68
Author:scottafk
Committer:GitHub

rm element fee, fix tx rollback (#2104)

Commit:b962cea
Author:powerpook
Committer:GitHub

fix: ecosystem default 1 and TransferSelf history (#2064)

Commit:5235252
Author:scottafk
Committer:GitHub

Improved block (#2013) * upgrade keys update perm * shelved tx bin log * shelved tx data into log * add network_id to block header

Commit:325d86c
Author:scottafk

fix rollback hash change compare method to array of rollback diff hash

Commit:45eb48b
Author:scottafk

improved rollbacks hash

Commit:d6cdeae
Author:powerpook

merge code

Commit:6b5d9a0
Author:scottafk

fix rollback when tx penalty, add rollback data_hash col

Commit:4a5471c
Author:powerpook

merge code

Commit:3a61a60
Author:scottafk

improved tx process

Commit:a125ce9
Author:scottafk

improved code * condensed error log * add more info for ecosystem * add col to table

Commit:f697593
Author:scottafk

fix conflict candidate

Commit:1174303
Author:scottafk

fix sql dml

Commit:926b291
Author:scottafk

impl sync block for sql

Commit:f3c9e2a
Author:scottafk

change block struct to proto * improve block marshal efficiency

Commit:1062041
Author:scottafk

add algo Secp256k1 and Keccak256, improved crypto

Commit:642c1fe
Author:scottafk

improved gas, fix gorm version

Commit:2dcfc84
Author:scottafk

improved gas

Commit:e83cfb5
Author:scottafk

add tx proto, add ContractName to model

Commit:18bffc4
Author:scottafk

fix combustion, add proto

Commit:1924a78
Author:scottafk

add per tx maybe to combustion

Commit:9c27024
Author:scottafk

fix runtime * fix object type func * rename Extend to ExtVar

Commit:79f54cc
Author:scottafk

refinement gas fee

Commit:74c3136
Author:scottafk

add more pay

Commit:6cc27b8
Author:scottafk

fix proto and recover err info

Commit:58044fd
Author:scottafk

add protobuf

Commit:24f77fb
Author:scottafk

adjust struct * mv code to common * rm unused code * rename param

Commit:9bc559e
Author:powerpook

revert SmartHeader -> Header