…package-binary-port

4845: Implement Entity and Package information requests (redux) r=EdHastingsCasperAssociation a=EdHastingsCasperAssociation

This is a fork of casper-network#4835 to handle merge conflicts

Co-authored-by: Jacek Malec <[email protected]>
Co-authored-by: edhastings <[email protected]>
Co-authored-by: Ed Hastings <[email protected]>

4844: Port the changes related to handling of unbonds in `global-state-update-gen` to `feat-2.0` r=fizyk20 a=fizyk20

This ports the changes made in casper-network#4828 to `feat-2.0`.

Co-authored-by: Bartłomiej Kamiński <[email protected]>

4823: Introducing calltable serialization to `TransactionV1` structures subtree. r=darthsiroftardis a=zajko

This is an extension to the `bytesrepr` serialization used in the rest of the node and it also uses the `
FromBytes` and `ToBytes` traits.
    From now on selected types will use a specific serialization approach which will be called "calltable" serialization.
    A "calltable-serialized" type is serialized as follow:

- For structs:
    -  Given type X with N attributes we assign a unique (in the scope of X) `u16` index to each of the structs attribute. The assumption is that the indexes are 0-based
    - We serialize each of the structs attribute using it's `ToBytes` implementation, producing binary payloads: _binary_vec<sub>0</sub>, binary_vec<sub>1</sub>, ..., binary_ve
c<sub>N-1</sub>_. We don't allow any of the _binary_vec<sub>i</sub>_ to be empty.
    - We construct a _bytes_ vector which is a concatenation of _binary_vec<sub>0</sub>, binary_vec<sub>1</sub>, ..., binary_vec<sub>N-1</sub>_
    - We construct a _fields_ vector. The _i-th_ element of _fields_ is a description of the _i-th_ attribute:
        - For field with index 0 this description will be:
              _field<sub>0</sub>_ = Field {
                 index: 0
                 offset: 0
              }
        - For each next field:
              *field<sub>i</sub>* = Field {
                  index: *i*
                  offset: *field<sub>i - 1</sub>.offset* + length(*binary_vec<sub>i - 1</sub>*)
              }
    - We construct an instance of `CalltableSerializationEnvelope {fields, bytes}`  and serialize it using it's `ToBytes` implementation.
- For enums:
    - Given an enum E with M variants we assign a `u8` unique index for each of the variant. The variant-index should start from 0 and be contiguous.
    - Given an enum instance *e* of variant *E<sub>j</sub>* (having variant index *j*) which has N attributes:
        - we assign a unique (in the scope of *E<sub>j</sub>) `u16` index to each of the enum variants attribute. The assumption is that the indexes are 1-based.
        - We serialize value of *j* as *binary_vec<sub>0</sub>* and each of *e* attribute as subsequent *binary_vec* (binary_vec<sub>1</sub>, binary_vec<sub>2</sub>,..., bi
nary_vec<sub>N</sub>). We don't allow any of the *binary_vec<sub>i</sub>* to be empty.
        -  We construct a *bytes* vector which is a concatenation of *binary_vec<sub>0</sub>, binary_vec<sub>1</sub>, ..., binary_vec<sub>N</sub>*.
        - We construct a *fields* vector. The *i-th* element of *fields* is a description of the *i-th* attribute:
            - For field with index 0 this description will be:
                 _field<sub>0</sub>_ = Field {
                   index: 0
                   offset: 0
                 }
            - For each next field:
                 _field<sub>i</sub>_ = Field {
                     index: *i*
                     offset: *field<sub>i - 1</sub>.offset* + length(*binary_vec<sub>i - 1</sub>*)
                 }
        - We construct an instance of `CalltableSerializationEnvelope {fields, bytes}`  and serialize it using it's `ToBytes` implementation.
        - To recap, each enum variant we serialize analogously to a `struct`, but we prepend a synthetic `u8` field which identifies the variant.
    
This PR applies calltable serialization to the following types:
- `InitiatorAddr` enum
- `PricingMode` enum
- `TransactionEntryPoint` enum
- `TransactionInvocationTarget` enum
- `TransactionScheduling` enum
- `TransactionTarget` enum
- `TransactionV1` struct
- `TransactionV1Body` struct
- `TransactionV1Header` struct
    
All other types (especially legacy ones, including those nested in "calltable-serialized" types) should use "regular" serialization logic.

Co-authored-by: Jakub Zajkowski <[email protected]>

… amelioration

… amelioration

… rustSDK-feat-2.0

# Conflicts:
#	Makefile
#	types/src/transaction/transaction_v1/transaction_v1_builder.rs

# Conflicts:
#	utils/global-state-update-gen/src/generic/state_tracker.rs

4891: Fix `node_should_rejoin_after_ejection` being flaky r=fizyk20 a=fizyk20

This PR changes the bid amount in the `node_should_rejoin_after_ejection` test to be equal to the minimum account balance, set [here](https://github.com/casper-network/casper-node/blob/e6456b709ec1da1c6b703db2a04beeba960651c5/node/src/reactor/main_reactor/tests.rs#L301).

Since the fixture is initialized with random balances for nodes, and the minimum possible balance was less than the bid amount, sometimes the call to `add_bid` would fail with the `InsufficientFunds` error, causing the test to be flaky. This change should fix the issue.

Closes casper-network#4859 


Co-authored-by: Bartłomiej Kamiński <[email protected]>

Add a link to Zug whitepaper to the Readme

4895: Add a WAL to Highway r=EdHastingsCasperAssociation a=fizyk20

This will enable validators in a network running Highway to resume validating after a crash without the necessity to download the protocol state from other nodes. The WAL (Write-Ahead Log) will contain all the units added to the protocol state during the node's operation, which combined with the information from the stored blocks will make it possible to restore the protocol state to the point from before a crash.

Since Zug already uses a WAL, this will close the only remaining hole in the protocol state persistence story.

Closes casper-network#3904 


Co-authored-by: Bartłomiej Kamiński <[email protected]>

4893: Do not notify consensus about historical blocks r=EdHastingsCasperAssociation a=fizyk20

What it says on the tin. This should also stop consensus from emitting the log message about missing eras.

Closes casper-network#4378 


Co-authored-by: Bartłomiej Kamiński <[email protected]>

Uses most recent I could find that does not emit invalid table
reference.

4773: Fix: remove unused connection symmetries and other code cleanups r=EdHastingsCasperAssociation a=moubctez

Changes:
* remove unused connection symmetries (fixes memory draining)
* remove unnecessary cloning
* don't derive DataSize when all fields are skipped
* hide some code that is needed only for tests
* code cleanup

Co-authored-by: Adam Ciarciński <[email protected]>

…s of hash, payload and approvals. payload is a merge of header and body concepts from before. body is not represented in a type-constrained way, instead we use an amorphic fields of type BTreeMap<u16, Bytes> which is more resistant to future changes.

…of GetRequest which allowed to promote state_identifier to a field directly in GlobalStateRequest

4894: Bump casper-wasmi, add sign-ext support r=igor-casper a=igor-casper

This PR addresses casper-network#4855 by bumping casper-wasmi to 0.14.0

As a key takeaway, [sign-ext](https://github.com/WebAssembly/sign-extension-ops/blob/master/proposals/sign-extension-ops/Overview.md) ops were previously unsupported, which is no longer the case with the newest interpreter. The engine was updated to accommodate this change:
- chainspec was modified to include ''wasm.opcode_costs.sign'' field which dictates the cost of these ops
- certain execution engine tests no longer make sense, so they've been removed:
  - should_not_accept_sign_ext_i32_e8s_proposal_wasm
  - should_not_accept_sign_ext_i32_e16s_proposal_wasm
  - should_not_accept_sign_ext_i64_e8s_proposal_wasm
  - should_not_accept_sign_ext_i64_e16s_proposal_wasm
  - should_not_accept_sign_ext_i64_e32s_proposal_wasm
- the appropriate serialization error no longer mentions unsupported sign extension operators

This PR also bumps the supported nightly toolchain to 2024-07-31.

Co-authored-by: igor-casper <[email protected]>
Co-authored-by: Michał Papierski <[email protected]>
Co-authored-by: igor-casper <[email protected]>

 into generic-hash-fn

… rustSDK-feat-2.0

# Conflicts:
#	types/Cargo.toml

…e' into rustSDK-feat-2.0-transaction

# Conflicts:
#	types/src/chainspec/transaction_config/transaction_v1_config.rs
#	types/src/transaction.rs
#	types/src/transaction/transaction_v1/transaction_v1_body.rs
#	types/src/transaction/transaction_v1/transaction_v1_builder.rs