Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[pull] master from paritytech:master #12

Closed
wants to merge 78 commits into from
Closed

[pull] master from paritytech:master #12

wants to merge 78 commits into from

Conversation

pull[bot]
Copy link

@pull pull bot commented May 28, 2024 •
edited
Loading

See Commits and Changes for more details.

Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

lexnv and others added 5 commits May 27, 2024 13:55
@lexnv
network: Update litep2p to v0.5.0 (#4570)
ce3e9b7 
## [0.5.0] - 2023-05-24

This is a small patch release that makes the `FindNode` command a bit
more robst:

- The `FindNode` command now retains the K (replication factor) best
results.
- The `FindNode` command has been updated to handle errors and
unexpected states without panicking.

### Changed

- kad: Refactor FindNode query, keep K best results and add tests
([#114](paritytech/litep2p#114))

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@michalkucharczyk
chain-spec-builder: help updated (#4597)
16887b6 
Added some clarification on output file.
@skunert
check-weight: Disable total pov size check for mandatory extrinsics (#…
70dd67a 
…4571)

So in some pallets we like
[here](https://github.com/paritytech/polkadot-sdk/blob/5dc522d02fe0b53be1517f8b8979176e489a388b/substrate/frame/session/src/lib.rs#L556)
we use `max_block` as return value for `on_initialize` (ideally we would
not).

This means the block is already full when we try to apply the inherents,
which lead to the error seen in #4559 because we are unable to include
the required inherents. This was not erroring before #4326 because we
were running into this branch:

https://github.com/paritytech/polkadot-sdk/blob/e4b89cc50c8d17868d6c8b122f2e156d678c7525/substrate/frame/system/src/extensions/check_weight.rs#L222-L224

The inherents are of `DispatchClass::Mandatory` and therefore have a
`reserved` value of `None` in all runtimes I have inspected. So they
will always pass the normal check.

So in this PR I adjust the `check_combined_proof_size` to return an
early `Ok(())` for mandatory extrinsics.

If we agree on this PR I will backport it to the 1.12.0 branch.

closes #4559

---------

Co-authored-by: command-bot <>
@AndreiEres
[subsystem-benchmarks] Add statement-distribution benchmarks (#3863)
a709768 
Fixes #3748

Adds a subsystem benchmark for statements-distribution subsystem.

Results in CI (reference hw):
```
$ cargo bench -p polkadot-statement-distribution --bench statement-distribution-regression-bench --features subsystem-benchmarks

[Sent to peers] standart_deviation 0.07%
[Received from peers] standart_deviation 0.00%
[statement-distribution] standart_deviation 0.97%
[test-environment] standart_deviation 1.03%

Network usage, KiB                     total   per block
Received from peers                1088.0000    108.8000
Sent to peers                      1238.1800    123.8180

CPU usage, seconds                     total   per block
statement-distribution                0.3897      0.0390
test-environment                      0.4715      0.0472
```
@michalkucharczyk
sc-chain-spec: deprecated code removed (#4410)
2d3a693 
This PR removes deprecated code:
- The `RuntimeGenesisConfig` generic type parameter in
`GenericChainSpec` struct.
- `ChainSpec::from_genesis` method allowing to create chain-spec using
closure providing runtime genesis struct
- `GenesisSource::Factory` variant together with no longer needed
`GenesisSource`'s generic parameter `G` (which was intended to be a
runtime genesis struct).


https://github.com/paritytech/polkadot-sdk/blob/17b56fae2d976a3df87f34076875de8c26da0355/substrate/client/chain-spec/src/chain_spec.rs#L559-L563
@pull pull bot added the ⤵️ pull label May 28, 2024
rzadp and others added 24 commits May 28, 2024 07:37
@rzadp
Remove workspace lints from templates (#4598)
09f07d5 
This detaches the templates from monorepo's workspace lints, so the
lints for the templates can evolve separately as needed.

Currently the templates [re-use the monorepo's
lints](https://github.com/paritytech/polkadot-sdk-minimal-template/blob/bd8afe66ec566d61f36b0e3d731145741a9e9e19/Cargo.toml#L16-L43)
which looks weird.

cc @kianenigma @gupnik
@alindima @pepoviola
Add availability-recovery from systematic chunks (#1644)
523e625 
**Don't look at the commit history, it's confusing, as this branch is
based on another branch that was merged**

Fixes #598 
Also implements [RFC
#47](polkadot-fellows/RFCs#47)

## Description

- Availability-recovery now first attempts to request the systematic
chunks for large POVs (which are the first ~n/3 chunks, which can
recover the full data without doing the costly reed-solomon decoding
process). This has a fallback of recovering from all chunks, if for some
reason the process fails. Additionally, backers are also used as a
backup for requesting the systematic chunks if the assigned validator is
not offering the chunk (each backer is only used for one systematic
chunk, to not overload them).
- Quite obviously, recovering from systematic chunks is much faster than
recovering from regular chunks (4000% faster as measured on my apple M2
Pro).
- Introduces a `ValidatorIndex` -> `ChunkIndex` mapping which is
different for every core, in order to avoid only querying the first n/3
validators over and over again in the same session. The mapping is the
one described in RFC 47.
- The mapping is feature-gated by the [NodeFeatures runtime
API](#2177) so that it
can only be enabled via a governance call once a sufficient majority of
validators have upgraded their client. If the feature is not enabled,
the mapping will be the identity mapping and backwards-compatibility
will be preserved.
- Adds a new chunk request protocol version (v2), which adds the
ChunkIndex to the response. This may or may not be checked against the
expected chunk index. For av-distribution and systematic recovery, this
will be checked, but for regular recovery, no. This is backwards
compatible. First, a v2 request is attempted. If that fails during
protocol negotiation, v1 is used.
- Systematic recovery is only attempted during approval-voting, where we
have easy access to the core_index. For disputes and collator
pov_recovery, regular chunk requests are used, just as before.

## Performance results

Some results from subsystem-bench:

with regular chunk recovery: CPU usage per block 39.82s
with recovery from backers: CPU usage per block 16.03s
with systematic recovery: CPU usage per block 19.07s

End-to-end results here:
#598 (comment)

#### TODO:

- [x] [RFC #47](polkadot-fellows/RFCs#47)
- [x] merge #2177 and
rebase on top of those changes
- [x] merge #2771 and
rebase
- [x] add tests
- [x] preliminary performance measure on Versi: see
#598 (comment)
- [x] Rewrite the implementer's guide documentation
- [x] #3065 
- [x] paritytech/zombienet#1705 and fix
zombienet tests
- [x] security audit
- [x] final versi test and performance measure

---------

Signed-off-by: alindima <alin@parity.io>
Co-authored-by: Javier Viola <javier@parity.io>
@AndreiEres
[subsytem-bench] Remove redundant banchmark_name param (#4540)
3bf283f 
Fixes #3601

Since we print benchmark results manually, we don't need to save
benchmark_name anywhere, better just put the name inside `println!`.
@ggwpez
[CI] Deny adding git deps (#4572)
6ed0200 
Adds a small CI check to match the existing Git deps agains a known-bad
list.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
@AndWeHaveAPlan
Conditional required checks (#4544)
ea46ad5 
Workaround for skipped but `required` github checks. The idea is to
trigger the workflow but filter out unaffected jobs or steps. See [ci_cd
998](paritytech/ci_cd#988) for details

In `.github/workflows/check-changed-files.yml` there is a reusable
workflow thad does all the checks and publishes results as outputs.
Example usage:
```
jobs:
  changes:
    permissions:
      pull-requests: read
    uses: ./.github/workflows/check-changed-files.yml
  some-job:
    needs: changes
    if: ${{ needs.changes.outputs.rust }}
 ....... 
 ```
@bolajahmad @Overkillus
Improve On_demand_assigner events (#4339)
650b124 
title: Improving `on_demand_assigner` emitted events

doc:
  - audience: Rutime User
description: OnDemandOrderPlaced event that is useful for indexers to
save data related to on demand orders. Check [discussion
here](https://substrate.stackexchange.com/questions/11366/ondemandassignmentprovider-ondemandorderplaced-event-was-removed/11389#11389).

Closes #4254 

crates: [ 'runtime-parachain]

---------

Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io>
@alexggh @ordian
Add metric to measure the time it takes to gather enough assignments (#…
ad22fa6 
…4587)

To understand with high granularity how many assignment tranches are
triggered before we concur that we have enough assignments.

This metric is important because the triggering of an assignment creates
a lot of work in the system for approving the candidate and gossiping
the necessary messages.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: ordian <write@reusable.software>
@bkchr
parachain-inherent: Make para_id more prominent (#4555)
2b1c606 
This should make it more obvious that at instantiation of the
`MockValidationDataInherentDataProvider` the `para_id` needs to be
passed.
@rzadp
Filter workspace dependencies in the templates (#4599)
d6cf147 
This detaches the templates from monorepo's workspace dependencies.

Currently the templates [re-use the monorepo's
dependencies](https://github.com/paritytech/polkadot-sdk-minimal-template/blob/bd8afe66ec566d61f36b0e3d731145741a9e9e19/Cargo.toml#L45-L58),
most of which are not needed.

The simplest approach is to specify versions directly and not use
workspace dependencies in the templates.

Another approach would be to programmatically filter dependencies that
are actually needed - but not sure if it's worth it, given that it would
complicate the synchronization job.

cc @kianenigma @gupnik
@codekitz @bkchr @kianenigma
Moves runtime macro out of experimental flag (#4249)
5f68c93 
Step in #3688

Now that the `runtime` macro (Construct Runtime V2) has been
successfully deployed on Westend, this PR moves it out of the
experimental feature flag and makes it generally available for runtime
devs.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
@EgorPopelyaev @bkontur
Add omni bencher & chain-spec-builder bins to release (#4557)
89604da 
Closes: #4354

This PR adds the steps to build and attach `frame-omni-bencher` and
`chain-spec-builder` binaries to the release draft

## TODO
- [x] add also chain-spec-builder binary
- [ ] ~~check/investigate Kian's comment: `chain spec builder. Ideally I
want it to match the version of the sp-genesis-builder crate`~~ see
[comment](#4518 (comment))
- [ ] Backport to `polkadot-sdk@1.11` release, so we can use it for next
fellows release: polkadot-fellows/runtimes#324
- [ ] Backport to `polkadot-sdk@1.12` release

---------

Co-authored-by: Branislav Kontur <bkontur@gmail.com>
@kianenigma @bkontur
Publish chain-spec-builder (#4518)
dfcfa4a 
marking it as release-able, attaching the same version number that is
attached to other binaries such as `polkadot` and `polkadot-parachain`.

I have more thoughts about the version number, though. The chain-spec
builder is mainly a user of the `sp-genesis-builder` api. So the
versioning should be such that it helps users know give a version of
`sp-genesis-builder` in their runtime, which version of
`chain-spec-builder` should they use?

With this, we can possibly alter the version number to always match
`sp-genesis-builder`.

Fixes #4352

- [x] Add to release artifacts ~~similar to
#4405 done here:
#4557

---------

Co-authored-by: Branislav Kontur <bkontur@gmail.com>
@joshuacheong @kianenigma
Update README.md (#4623)
aa32faa 
Minor edit to a broken link for Rust Docs on the README.md

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
@franciscoaguirre @bkchr
Change XcmDryRunApi::dry_run_extrinsic to take a call instead (#4621)
d5053ac 
Follow-up to the new `XcmDryRunApi` runtime API introduced in
#3872.

Taking an extrinsic means the frontend has to sign first to dry-run and
once again to submit.
This is bad UX which is solved by taking an `origin` and a `call`.
This also has the benefit of being able to dry-run as any account, since
it needs no signature.

This is a breaking change since I changed `dry_run_extrinsic` to
`dry_run_call`, however, this API is still only on testnets.
The crates are bumped accordingly.

As a part of this PR, I changed the name of the API from `XcmDryRunApi`
to just `DryRunApi`, since it can be used for general dry-running :)

Step towards #690.

Example of calling the API with PAPI, not the best code, just testing :)

```ts
// We just build a call, the arguments make it look very big though.
const call = localApi.tx.XcmPallet.transfer_assets({
  dest: XcmVersionedLocation.V4({ parents: 0, interior: XcmV4Junctions.X1(XcmV4Junction.Parachain(1000)) }),
  beneficiary: XcmVersionedLocation.V4({ parents: 0, interior: XcmV4Junctions.X1(XcmV4Junction.AccountId32({ network: undefined, id: Binary.fromBytes(encodeAccount(account.address)) })) }),
  weight_limit: XcmV3WeightLimit.Unlimited(),
  assets: XcmVersionedAssets.V4([{
    id: { parents: 0, interior: XcmV4Junctions.Here() },
    fun: XcmV3MultiassetFungibility.Fungible(1_000_000_000_000n) }
  ]),
  fee_asset_item: 0,
});
// We call the API passing in a signed origin 
const result = await localApi.apis.XcmDryRunApi.dry_run_call(
  WestendRuntimeOriginCaller.system(DispatchRawOrigin.Signed(account.address)),
  call.decodedCall
);
if (result.success && result.value.execution_result.success) {
  // We find the forwarded XCM we want. The first one going to AssetHub in this case.
  const xcmsToAssetHub = result.value.forwarded_xcms.find(([location, _]) => (
    location.type === "V4" &&
      location.value.parents === 0 &&
      location.value.interior.type === "X1"
      && location.value.interior.value.type === "Parachain"
      && location.value.interior.value.value === 1000
  ))!;

  // We can even find the delivery fees for that forwarded XCM.
  const deliveryFeesQuery = await localApi.apis.XcmPaymentApi.query_delivery_fees(xcmsToAssetHub[0], xcmsToAssetHub[1][0]);

  if (deliveryFeesQuery.success) {
    const amount = deliveryFeesQuery.value.type === "V4" && deliveryFeesQuery.value.value[0].fun.type === "Fungible" && deliveryFeesQuery.value.value[0].fun.value.valueOf() || 0n;
    // We store them in state somewhere.
    setDeliveryFees(formatAmount(BigInt(amount)));
  }
}
```

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
@eskimor @seadanda
Broker new price adapter (#4521)
f4dc8d2 
Fixes #4360 

Also rename: AllowedRenewals -> PotentialRenewals to avoid confusion of
future readers. (An entry in `AllowedRenewals` is not enough to allow a
renewal, the assignment also has to be complete, which is only checked
afterwards.)

- [x] Does not work with renewals as is - fix.
- [x] More tests
- [x] PR docs

Edit 1:
(Relevant blog post:
https://grillapp.net/12935/agile-coretime-pricing-explained-166522?ref=29715)

---------

Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: command-bot <>
@bkchr
pallet-staking: Put tests behind cfg(debug_assertions) (#4620)
4ab078d 
Otherwise these tests are failing if you don't run with
`debug_assertions` enabled, which happens if you run tests locally in
release mode.
@drskalman @davxy
@Lederstrumpf @acatangiu
Beefy client generic on aduthority Id (#1816)
bcab07a 
Revived version of paritytech/substrate#13311 .
Except Signature is not generic and is dictated by AuthorityId.

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
@conr2d
Fix broken windows build (#4636)
d539778 
Fixes #4625.

Specifically, the `cfg` attribute `windows` refers to the compile target
and not the build environment, and in the case of cross-compilation, the
build environment and target can differ. However, the line modified is
related to documentation generation, so there should be no critical
issue with this change.
@codekitz @kianenigma
Adds ability to specify chain type in chain-spec-builder (#4542)
78c24ec 
Currently, `chain-spec-builder` only creates a spec with `Live` chain
type. This PR adds the ability to specify it while keeping the same
default.

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
@kianenigma @bkchr
Update runtime_type ref doc with the new "Associated Type Bounds" (#…
71f4f5a 
…4624)

Co-authored-by: Bastian Köcher <git@kchr.de>
@sandreim @bkchr
collator-protocol: remove elastic-scaling-experimental feature (#4595)
0ae7219 
Validators already have been upgraded so they could already receive the
new `CollationWithParentHeadData` response when fetching collation.
However this is only sent by collators when the parachain has more than
1 core is assigned.

TODO:
- [x] PRDoc

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
@rzadp
Use Unlicense for templates (#4628)
8d8c0e1 
Addresses
[this](#3155 (comment)).
@franciscoaguirre
Implement XcmPaymentApi and DryRunApi on all system parachains (#…
fc6c318 
…4634)

Depends on #4621.

Implemented the
[`XcmPaymentApi`](#3607)
and [`DryRunApi`](#3872)
on all system parachains.

More scenarios can be tested on both rococo and westend if all system
parachains implement this APIs.
The objective is for all XCM-enabled runtimes to implement them.
After demonstrating fee estimation in a UI on the testnets, come the
fellowship runtimes.

Step towards #690.
@codekitz @bkchr
Better error for missing index in CRV2 (#4643)
f81751e 
Fixes #4552

---------

Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <info@kchr.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
⤵️ pull
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.