chore: update genbutemple branch (#2308)

* feat: add AC for vAMM during opening auction

* chore: typo

* fix: reword

* feat: Updates to fix up equations (#2303)

* feat: tx trade ordering (#2231)

* feat: Order trade transactions within a block

* feat: AMM Estimates (#2282)

* feat: Estimate within range

* refactor: first stab at max block auction

* refactor: add defaults

* chore: update genbutemple

---------

Co-authored-by: Jiajia-Cui <jiajia@vega.xyz>
Co-authored-by: David Siska <62546419+davidsiska-vega@users.noreply.github.com>
Co-authored-by: Tom <tom@vegaprotocol.io>
Co-authored-by: Witold <gawlikowicz@gmail.com>

.github/workflows/quality_check.yml

@@ -11,6 +11,7 @@ name: "Quality checks"
       - palazzo
       - colosseo
       - colosseo_II
+      - genbutemple
 
 env:
   GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

non-protocol-specs/0014-NP-VAMM-bounds-estimations.md

@@ -14,6 +14,7 @@ The API should take a pool's specification parameters and output various metrics
  1. Leverage At Upper Price
  1. Leverage At Lower Price
  1. Commitment Amount
+ 1. Market ID
  1. Optional: Party Key
 
 And then return the metrics:
@@ -55,7 +56,7 @@ $$
 r_f = \min(l_b, \frac{1}{ (f_s + f_l) \cdotp f_i}) ,
 $$
 
-where $l_b$ is the sided value `leverage_at_bounds` (`upper ratio` if the upper band is being considered and `lower ratio` if the lower band is), $f_s$ is the market's sided risk factor (different for long and short positions), $f_l$ is the market's linear slippage component and $f_i$ is the market's initial margin factor.
+where $l_b$ is the sided value `leverage_at_bounds` (`upper ratio` if the upper band is being considered and `lower ratio` if the lower band is), $f_s$ is the market's sided risk factor (different for long and short positions), $f_l$ is the market's linear slippage component and $f_i$ is the market's initial margin factor. This will result in two separate values to use for $r_f$, one for the lower range and one for the upper range.
 
 
 ### Position at Bounds
@@ -69,15 +70,15 @@ $$
 P_{v_u} = \frac{r_f b}{p_u (1 + r_f) - r_f p_a} ,
 $$
 
-where $r_f$ is the `short` factor for the upper range and the `long` factor for the lower range, `b` is the current total balance of the vAMM across all accounts, $P_{v_l}$ is the theoretical volume and the bottom of the lower bound and $P_{v_u}$ is the (absolute value of the) theoretical volume at the top of the upper bound.
+where $r_f$ is the `short` factor for the upper range and the `long` factor for the lower range, `b` is the current total balance of the vAMM across all accounts, $p_u$ and $p_l$ are as defined above, $P_{v_l}$ is the theoretical volume and the bottom of the lower bound and $P_{v_u}$ is the (absolute value of the) theoretical volume at the top of the upper bound.
 
 
 ### Loss on Commitment at Bound
 
 For the loss on commitment at bound, one needs to use the average entry price, bound price and the position at the bounds
 
 $$
-l_c = |p_a - p_b| \cdot P_b ,
+l_c = |(p_a - p_b) \cdot P_b |,
 $$
 
 where $P_b$ is the position at bounds (Either $P_{v_l}$ or $P_{v_u}$), $p_a$ is the average entry price and $p_b$ is the price at the corresponding outer bound. Note that this is an absolute value of loss, so outstanding balance at bounds would be `initial balance - $L_c$`.
@@ -88,10 +89,10 @@ where $P_b$ is the position at bounds (Either $P_{v_l}$ or $P_{v_u}$), $p_a$ is
 Using a similar methodology to the standard estimations for liquidation price and the above calculated values, an estimate for liquidation prices above and below the range can be obtained with
 
 $$
-p_{liq} = \frac{b - l_c - P_b \cdot p_b}{\abs{P_b} \cdot m_r - P_b} ,
+p_{liq} = \frac{b - l_c - P_b \cdot p_b}{|P_b| \cdot (f_l + m_r) - P_b} ,
 $$
 
-where $p_{liq}$ is the liquidation price (above or below the specified ranges), $b$ is the original commitment balance, $l_c$ is the loss on commitment at the relevant bound, $P_b$ is the position at the relevant bound, $p_b$ is the price at the bound and $m_r$ is the market's long or short risk factor (short for the upper price bound as the position will be negative and long for the lower).
+where $p_{liq}$ is the liquidation price (above or below the specified ranges), $b$ is the original commitment balance, $l_c$ is the loss on commitment at the relevant bound, $P_b$ is the position at the relevant bound, $p_b$ is the price at the bound, $f_l$ is the linear slippage factor for the market and $m_r$ is the market's long or short risk factor (short for the upper price bound as the position will be negative and long for the lower).
 
 
 ## Specified Key
@@ -101,7 +102,7 @@ When a key is specified, the existence of any current vAMM should be checked and
 
 ## Acceptance criteria
 
-- For a request specifying (base, upper, lower, leverage_upper, leverage_lower, commitment) as (1000, 1100, 900, 2, 2, 100) the response is (<a name="0014-NP-VAMM-001" href="#0014-NP-VAMM-001">0014-NP-VAMM-001</a>):
+- For a request specifying (base, upper, lower, leverage_upper, leverage_lower, commitment, short risk factor, long risk factor, market slippage factor) as (1000, 1100, 900, 2, 2, 100, 0.01, 0.01, 0) the response is (<a name="0014-NP-VAMM-001" href="#0014-NP-VAMM-001">0014-NP-VAMM-001</a>):
 
  1. Loss on Commitment at Upper Bound: 8.515
  1. Loss on Commitment at Lower Bound: 9.762
@@ -111,7 +112,7 @@ When a key is specified, the existence of any current vAMM should be checked and
  1. Liquidation Price at Lower Bound: 454.545
 
 
-- For a request specifying (base, upper, lower, leverage_upper, leverage_lower, commitment) as (1000, 1300, 900, 1, 5, 100) the response is (<a name="0014-NP-VAMM-004" href="#0014-NP-VAMM-004">0014-NP-VAMM-004</a>):
+- For a request specifying (base, upper, lower, leverage_upper, leverage_lower, commitment, short risk factor, long risk factor, market slippage factor) as (1000, 1300, 900, 1, 5, 100, 0.01, 0.01, 0) the response is (<a name="0014-NP-VAMM-004" href="#0014-NP-VAMM-004">0014-NP-VAMM-004</a>):
 
  1. Loss on Commitment at Upper Bound: 10.948
  1. Loss on Commitment at Lower Bound: 21.289

protocol/0001-MKTF-market_framework.md

@@ -33,7 +33,8 @@ Data:
   - If this is negative e.g. -3 this means that the smallest order and position is of size 1000.
   - Accepted values are `-6,...,-1,0,1,2,...,6`.
 - **Tick size**: the minimum change in quote price for the market. Order prices and offsets for pegged orders must be given as an exact multiple of the tick size. For example if the tick size is 0.02 USD. then a price of 100.02 USD is acceptable and a price of 100.03 USD is not. The tick size of a market can be updated through governance. Note, the tick size should be specified in terms of the market decimals, e.g. for a scaled tick size of `0.02` (USDT) in a market using `5` decimal places, the tick size would be set to `2000`.
-- **Liquidation strategy**: A field specifying the liquidation strategy for the market. Please refer to [0012-POSR-position_resolution](./0012-POSR-position_resolution.md#managing-networks-position) for supported strategies.  
+- **Liquidation strategy**: A field specifying the liquidation strategy for the market. Please refer to [0012-POSR-position_resolution](./0012-POSR-position_resolution.md#managing-networks-position) for supported strategies.
+- **Transaction Prioritisation**: A boolean, whether to enable [transaction prioritisation](./0092-TRTO-trading_transaction_ordering.md).
 
 Note: it is agreed that initially the integer representation of the full precision of both order and positions can be required to fit into an int64, so this means that the largest position/order size possible reduces by a factor of ten for every extra decimal place used. This also means that, for instance, it would not be possible to create a `BTCUSD` market that allows order/position sizes equivalent to 1 sat.
 

protocol/0026-AUCT-auctions.md

@@ -51,7 +51,7 @@ We can also imagine that an auction period could come to an end once a give numb
 
 Once the auction period finishes, vega needs to figure out the best price for the order range in the book which can be uncrossed. The first stage in this is to calculate the Volume Maximising Price Range - the range of prices (which will be a contiguous range in an unconstrained order book) at which the highest total quantity of trades can occur.
 
-For including [AMM](./0090-VAMM-automated_market_maker.md) configurations in the calculation of expected uncrossing price of the auction process and volume maximising price range, create two lists of independent copies of all currently active vAMMs. Ensure that the price range under consideration (lowest ask - highest bid) is expanded to cover the current best bid/ask prices of all vAMMs. When iterating through the price ranges the vAMMs should be updated after each price level checked as if they have traded that volume before checking the next level. vAMMs may quote on both sides of the book as long as they are not in reduce-only mode. When actually uncrossing the market, vAMMs should act as normal when trades from one side are being sent in to matching.
+For including [AMM](./0090-VAMM-automated_market_maker.md) configurations in the calculation of expected uncrossing price of the auction process and volume maximising price range, create two lists of independent copies of all currently active vAMMs. Ensure that the price range under consideration (lowest ask - highest bid) is expanded to cover the current best bid/ask prices of all vAMMs. Separately, calculate a price step size defined by either the smallest tick size or, if this would result in more steps between lowest ask and highest bid than allowed by `market.liquidity.maxAmmCalculationLevels` instead divide the price range into `market.liquidity.maxAmmCalculationLevels` and calculate the price step required for this. Additionally, add all price bounds for vAMMs which are within the auction bounds to the checked levels. Then iterate through these price levels, moving the vAMMs to each by querying their volume between the two bounds and counting this as a single trade at the price farther from mid. Add these volumes to those of individual limit orders when calculating the uncrossing price. vAMMs may quote on both sides of the book as long as they are not in reduce-only mode. When actually uncrossing the market, vAMMs should act as normal when trades from one side are being sent in to matching.
 
 Initially we will use the mid price within this range. For example, if the volume maximising range is 98-102, we would price all trades in the uncrossing at 100 ((minimum price of range+maximum price of range)/2). In future there will be other options, which will be selectable via a network parameter specified at market creation, and changeable through governance. These other options are not yet specified.
 

protocol/0028-GOVE-governance.md

@@ -579,6 +579,8 @@ APIs should also exist for clients to:
 - A market proposal with position decimal places not in `{-6,...,-1,0,1,2,...,6}` gets rejected. (<a name="0028-GOVE-062" href="#0028-GOVE-062">0028-GOVE-062</a>) For product spot: (<a name="0028-GOVE-075" href="#0028-GOVE-075">0028-GOVE-075</a>)
 - A market proposal with a tick size less than or equal to `0` gets rejected (<a name="0028-GOVE-180" href="#0028-GOVE-180">0028-GOVE-180</a>).
 - At enactment, a market change proposal updating the tick size leaves in place all orders where the quoted price is not an exact multiple of `10^-mdp` (where `mdp` is the market decimal places) (<a name="0028-GOVE-182" href="#0028-GOVE-182">0028-GOVE-182</a>).
+- At enactment, a market with `Transaction Prioritisation` enabled will have transactions re-prioritised as defined in [transaction prioritisation](./0092-TRTO-trading_transaction_ordering.md) (<a name="0028-GOVE-192" href="#0028-GOVE-192">0028-GOVE-192</a>).
+- At enactment, a market with `Transaction Prioritisation` disabled will not have transactions re-prioritised (<a name="0028-GOVE-193" href="#0028-GOVE-193">0028-GOVE-193</a>).
 
 #### Market change proposals
 
@@ -623,6 +625,7 @@ APIs should also exist for clients to:
 - A market change proposal specifying a new tick size less than or equal to `0` gets rejected (<a name="0028-GOVE-184" href="#0028-GOVE-184">0028-GOVE-184</a>).
 - At enactment, a market change proposal updating the tick size cancels all pegged orders where their offset is no longer an exact integer multiple of the tick size (<a name="0028-GOVE-183" href="#0028-GOVE-183">0028-GOVE-183</a>).
 - A market LP with ELS > 0 can vote on a market change proposal even if the key doesn't meet the `governance.proposal.updateMarket.minVoterBalance` for governance token.  (<a name="0028-GOVE-185" href="#0028-GOVE-185">0028-GOVE-185</a>).
+- An amendment to a market to enable or disable `Transaction Prioritisation` will have that effect immediately upon enactment. (<a name="0028-GOVE-194" href="#0028-GOVE-194">0028-GOVE-194</a>)
 
 
 #### Network parameter change proposals

protocol/0042-LIQF-setting_fees_and_rewarding_lps.md

@@ -474,6 +474,8 @@ Example 1, generated with [supplementary worksheet](https://docs.google.com/spre
 - A vAMM which was active on the market with an average of `10000` liquidity units (`price * volume`) provided for half the epoch, and then `0` for the second half of the epoch (as the price was out of the vAMM's configured range), and where the `market.liquidity.stakeToCcyVolume` value is `100`, will have an implied commitment of `50`. (<a name="0042-LIQF-109" href="#0042-LIQF-109">0042-LIQF-109</a>)
 - A vAMM which was active on the market with an average of `10000` liquidity units (`price * volume`) provided for half the epoch, and then is cancelled for the second half of the epoch, and where the `market.liquidity.stakeToCcyVolume` value is `100`, will have an implied commitment of `50`. (<a name="0042-LIQF-110" href="#0042-LIQF-110">0042-LIQF-110</a>)
 - A vAMM which was active on the market with an average of `10000` liquidity units (`price * volume`) provided for half the epoch, and then `5000` for the second half of the epoch (as the price was out of the vAMM's configured range), and where the `market.liquidity.stakeToCcyVolume` value is `100`, will have an implied commitment of `75`. (<a name="0042-LIQF-111" href="#0042-LIQF-111">0042-LIQF-111</a>)
+- If a vAMM was active during the market's opening auction if the opening auction ended and if trades were placed before the end of an epoch the vAMM should receive liquidity fee at epoch boundary (just like a normal LP that submitted bond during opening auction and then met the SLA) (<a name="0042-LIQF-112" href="#0042-LIQF-112">0042-LIQF-112</a>)
+
 
 ### Explicit instantaneous liquidity scoring function
 

protocol/0090-VAMM-automated_market_maker.ipynb

@@ -7,7 +7,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import numpy\n",
+    "import numpy as np\n",
     "from typing import Optional, Tuple, List\n",
     "from collections import namedtuple\n",
     "import matplotlib.pyplot as plt"
@@ -761,6 +761,131 @@
     "print(f\"Position at lower would be {volume_at_lower} and position at upper would be {-1 * volume_at_upper}  (Lower AEP {aep_lower}, Upper AEP {aep_upper})\")\n",
     "print(f\"Implying leverage {volume_at_lower * mm.lower_price / (balance + loss_at_lower)} at lower and {volume_at_upper * mm.upper_price / (balance - loss_at_upper)} at upper\")"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2553a194",
+   "metadata": {},
+   "source": [
+    "### Curve Estimator"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "aec95eb6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "upper_factor = 1300/1000 - 1\n",
+    "lower_factor = 0.1\n",
+    "base = 1000\n",
+    "\n",
+    "mm = CFMMarketMaker(\n",
+    "    initial_price=base,\n",
+    "    price_width_above=upper_factor,\n",
+    "    price_width_below=lower_factor,\n",
+    "    margin_multiple_at_upper=10,\n",
+    "    margin_multiple_at_lower=1,\n",
+    "    num_levels=8000,\n",
+    "    tick_spacing=1,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e2fc7416",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "trade_size = 100\n",
+    "\n",
+    "def execution_price(volume_to_trade: float, price_levels: list[(float, float)]):\n",
+    "    '''First value in price_levels is price, second is volume\n",
+    "    '''\n",
+    "    weighted_execution_price = 0\n",
+    "    traded_vol = 0\n",
+    "    for level in price_levels:\n",
+    "        if level[1] > volume_to_trade:\n",
+    "            weighted_execution_price += volume_to_trade * level[0]\n",
+    "            traded_vol += volume_to_trade\n",
+    "            volume_to_trade = 0\n",
+    "        else:\n",
+    "            weighted_execution_price += level[1] * level[0]\n",
+    "            volume_to_trade -= level[1]\n",
+    "            traded_vol += level[1]\n",
+    "        if volume_to_trade <= 0:\n",
+    "            break\n",
+    "    return weighted_execution_price / traded_vol\n",
+    "\n",
+    "def gen_levels(start_px: float, end_px: float, num_divisions: int, amm: CFMMarketMaker, amm_balance: float) -> list[(float, float)]:\n",
+    "    last_level = start_px\n",
+    "\n",
+    "    unit_upper_L = (\n",
+    "        amm.upper_price_sqrt\n",
+    "        * amm.base_price_sqrt\n",
+    "        / (amm.upper_price_sqrt - amm.base_price_sqrt)\n",
+    "    )\n",
+    "    aep = -1 * unit_upper_L * amm.upper_price_sqrt * ((unit_upper_L / (unit_upper_L + amm.upper_price_sqrt)) - 1)\n",
+    "    volume_at_upper_incoming = amm.margin_multiple_at_upper * amm_balance / (amm.upper_price * (1 + amm.margin_multiple_at_upper) - amm.margin_multiple_at_upper * aep)\n",
+    "\n",
+    "    upper_L = (\n",
+    "        volume_at_upper_incoming\n",
+    "        * amm.upper_price_sqrt\n",
+    "        * amm.base_price_sqrt\n",
+    "        / (amm.upper_price_sqrt - amm.base_price_sqrt)\n",
+    "    )\n",
+    "\n",
+    "    orders = []\n",
+    "    for level in np.linspace(start_px, end_px, num=num_divisions):\n",
+    "        if level == start_px:\n",
+    "            continue\n",
+    "        orders.append((level, amm._quantity_for_move(last_level**0.5, level**0.5, amm.upper_price_sqrt, upper_L)))\n",
+    "        last_level = level\n",
+    "    return orders"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "24a0ee68",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "STEPS_IN_RANGE = 100\n",
+    "RANGE_LOW = base\n",
+    "RANGE_HIGH = 2 * RANGE_LOW\n",
+    "TICK_SIZE = 0.01\n",
+    "TRADE_SIZE = 100\n",
+    "MM_BALANCE = 10_000\n",
+    "\n",
+    "num_divisions_to_test = [x for x in range(100, 10000, 5)]\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b9b009e9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prices = []\n",
+    "\n",
+    "for div in num_divisions_to_test:\n",
+    "    levels = gen_levels(RANGE_LOW, RANGE_HIGH, div, mm, MM_BALANCE)\n",
+    "    prices.append(execution_price(TRADE_SIZE, levels))\n",
+    "\n",
+    "plt.plot(num_divisions_to_test, prices)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "37264648",
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
@@ -779,7 +904,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.8"
+   "version": "3.10.14"
   }
  },
  "nbformat": 4,