Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate AgentMarket.outcome_token_pool into betting strategies #389

Merged
merged 17 commits into from
Sep 12, 2024

Conversation

evangriffiths
Copy link
Contributor

@evangriffiths evangriffiths commented Sep 9, 2024

  • Adding a 'full' kelly bet based using the olas code
  • Using this in the KellyBettingStrategy if supported
  • Refactoring the market-moving-bet function to not be OmenMarket-specific

Copy link

coderabbitai bot commented Sep 9, 2024

Walkthrough

The changes in this pull request modify the betting strategy logic within the prediction_market_agent_tooling package. Key updates include the introduction of two new functions, get_kelly_bet_full and get_kelly_bet_simplified, which enhance the flexibility of the betting strategy. The calculate_trades method has been updated to utilize these new functions based on market conditions. Additionally, the KellyBet class has been removed in favor of a more streamlined SimpleBet class, and the get_market_moving_bet function has been refactored for improved clarity and usability.

Changes

Files Change Summary
prediction_market_agent_tooling/deploy/betting_strategy.py Introduced get_kelly_bet_full and get_kelly_bet_simplified functions; updated calculate_trades to utilize these based on token pool availability; modified import statements.
prediction_market_agent_tooling/tools/betting_strategies/kelly_criterion.py Removed KellyBet class; renamed get_kelly_bet to get_kelly_bet_simplified with updated return type; added get_kelly_bet_full with enhanced calculation logic and validation checks.
prediction_market_agent_tooling/tools/betting_strategies/market_moving.py Modified get_market_moving_bet function to accept new parameters for outcome pool sizes and current probabilities; changed return type to SimpleBet for clarity; refactored internal logic for direct calculations without OmenMarket dependencies.
tests/markets/test_betting_strategies.py Updated import statements and modified test functions to align with new betting strategy functions and parameter names; enhanced clarity and precision in testing logic for the Kelly criterion and market moving bets.
prediction_market_agent_tooling/markets/omen/omen.py Added get_buy_token_amount and get_new_p_yes methods to calculate token amounts and updated probabilities based on bet amounts and directions; includes validation for token pool availability.
tests/markets/omen/test_omen.py Introduced test_get_new_p_yes to validate the behavior of get_new_p_yes, ensuring correct probability updates based on bet direction; included sanity checks against expected outcomes.

Possibly related PRs


Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between d2818ba and a54abbe.

Files selected for processing (1)
  • prediction_market_agent_tooling/markets/omen/omen.py (1 hunks)
Additional comments not posted (2)
prediction_market_agent_tooling/markets/omen/omen.py (2)

580-592: LGTM!

The get_buy_token_amount function is correctly implemented and follows the expected logic:

  1. It calls the calcBuyAmount method of the market contract to calculate the buy amount in Wei.
  2. It converts the buy amount from Wei to xDai using the wei_to_xdai function.
  3. It returns the received token amount as a TokenAmount object with the calculated amount and the market's currency.

The code changes are approved.


594-621: LGTM!

The get_new_p_yes function is correctly implemented and follows the expected logic:

  1. It checks for the existence of the outcome token pool and raises a ValueError if it's missing.
  2. It retrieves the sizes of the "yes" and "no" outcome pools from the outcome_token_pool attribute.
  3. It calculates the new sizes of the outcome pools by adding the bet amount adjusted by the fee.
  4. It calculates the received token amount using the get_buy_token_amount method.
  5. Depending on the bet direction, it subtracts the received token amount from the respective outcome pool size.
  6. It computes the new probability of the "yes" outcome as the ratio of the updated "no" outcome pool size to the total of both updated outcome pool sizes.
  7. It returns the new probability as a Probability object.

The code changes are approved.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Outside diff range, codebase verification and nitpick comments (3)
prediction_market_agent_tooling/tools/betting_strategies/kelly_criterion.py (1)

Line range hint 9-55: Refactor suggestion for get_kelly_bet_simplified.

The function get_kelly_bet_simplified has been correctly updated to return a SimpleBet object. The logic within the function appears sound, adhering to the Kelly Criterion formula. However, consider refactoring the handling of market_prob == 0 to avoid potential division by zero in a more explicit manner.

Consider adding a small epsilon to market_prob directly when it's calculated to ensure it never hits zero:

-    if market_prob == 0:
-        market_prob = 1e-10
+    market_prob = max(market_prob, 1e-10)
tests/markets/test_betting_strategies.py (2)

162-190: Review of test_get_market_moving_bet.

The test has been updated to reflect the new parameter names and expected outcomes. The assertions are appropriate for verifying the functionality of get_market_moving_bet. However, consider adding a comment explaining the tolerance used in the np.isclose function to clarify why a specific tolerance level was chosen.

Add a comment to explain the choice of tolerance in the np.isclose function:

# Tolerance set to 2.0 to allow for minor deviations in floating-point calculations

219-241: Limited Test Coverage for get_kelly_bet_full.

The current test file tests/markets/test_betting_strategies.py contains only one test case for get_kelly_bet_full, with no visible variations in parameters. To ensure comprehensive validation of the function, consider adding more test cases that cover a diverse range of scenarios and parameter variations.

  • Add additional test cases for get_kelly_bet_full with varied parameters to ensure robust testing.
Analysis chain

Review of test_kelly_bet.

The test has been updated to include both get_kelly_bet_simplified and get_kelly_bet_full. The logic checks the direction of the bet based on the estimated probability, which is a good practice for verifying the correctness of the betting strategy. Ensure that the tests cover a range of scenarios to fully validate the new and more complex get_kelly_bet_full function.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the coverage of test cases for `get_kelly_bet_full`.

# Test: Search for test cases related to `get_kelly_bet_full`. Expect: A diverse range of test scenarios.
rg --type python -A 5 $'get_kelly_bet_full'

Length of output: 73


Script:

#!/bin/bash
# Description: Verify the coverage of test cases for `get_kelly_bet_full`.

# Corrected Test: Search for test cases related to `get_kelly_bet_full`. Expect: A diverse range of test scenarios.
rg --type py -A 5 'get_kelly_bet_full'

Length of output: 2742


Script:

#!/bin/bash
# Description: Extract detailed information about test cases for `get_kelly_bet_full` in `tests/markets/test_betting_strategies.py`.

# Extract the relevant test cases and their context to assess coverage and diversity.
rg --type py -A 20 'get_kelly_bet_full' tests/markets/test_betting_strategies.py

Length of output: 1235


Script:

#!/bin/bash
# Description: Search for all occurrences of `get_kelly_bet_full` in the test file to assess the diversity of test scenarios.

# Search for all instances of `get_kelly_bet_full` in the test file to identify multiple test cases or parameter variations.
rg --type py 'get_kelly_bet_full' tests/markets/test_betting_strategies.py

Length of output: 126

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 429e1a8 and 9bbf51f.

Files selected for processing (5)
  • prediction_market_agent_tooling/deploy/betting_strategy.py (2 hunks)
  • prediction_market_agent_tooling/tools/betting_strategies/kelly_criterion.py (2 hunks)
  • prediction_market_agent_tooling/tools/betting_strategies/market_moving.py (2 hunks)
  • prediction_market_agent_tooling/tools/betting_strategies/utils.py (1 hunks)
  • tests/markets/test_betting_strategies.py (5 hunks)
Additional context used
Learnings (1)
Common learnings
Learnt from: kongzii
PR: gnosis/prediction-market-agent-tooling#224
File: prediction_market_agent_tooling/markets/markets.py:67-67
Timestamp: 2024-04-29T14:14:38.751Z
Learning: The function `have_bet_on_market_since` is suggested to be refactored into a method on the `AgentMarket` class, potentially reusing code from `DeployedAgent.get_resolved_bets`. This task is planned for future work, not within the scope of the current PR.
Ruff
prediction_market_agent_tooling/tools/betting_strategies/market_moving.py

37-37: Remove unnecessary True if ... else False

Remove unnecessary True if ... else False

(SIM210)

Additional comments not posted (7)
prediction_market_agent_tooling/tools/betting_strategies/utils.py (1)

4-6: LGTM!

The SimpleBet class is well-defined with appropriate field types and uses Pydantic for data validation.

prediction_market_agent_tooling/tools/betting_strategies/market_moving.py (2)

5-6: Approved import changes.

The imports are correctly updated to reflect the new dependencies and usage within the file.


82-82: Approved return statement.

The function correctly returns a SimpleBet object, enhancing the clarity and usability of the function's output.

prediction_market_agent_tooling/deploy/betting_strategy.py (1)

8-11: Approved import changes.

The imports are correctly updated to include the new functions and constants necessary for the updated betting strategy logic.

prediction_market_agent_tooling/tools/betting_strategies/kelly_criterion.py (2)

1-1: Approved import statement.

The import of SimpleBet from the utils module is correct and necessary due to the removal of KellyBet and the introduction of SimpleBet.


3-7: Ensure probability validation is robust.

The function check_is_valid_probability correctly checks if a probability value is within the acceptable range [0, 1]. This is crucial for the integrity of the betting calculations that follow.

tests/markets/test_betting_strategies.py (1)

32-33: Approved import updates.

The updated imports for get_kelly_bet_full and get_kelly_bet_simplified are correct and reflect the changes in the betting strategy logic.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 9bbf51f and 29ab1cf.

Files selected for processing (1)
  • prediction_market_agent_tooling/tools/betting_strategies/market_moving.py (2 hunks)
Additional context used
Learnings (1)
Common learnings
Learnt from: kongzii
PR: gnosis/prediction-market-agent-tooling#224
File: prediction_market_agent_tooling/markets/markets.py:67-67
Timestamp: 2024-04-29T14:14:38.751Z
Learning: The function `have_bet_on_market_since` is suggested to be refactored into a method on the `AgentMarket` class, potentially reusing code from `DeployedAgent.get_resolved_bets`. This task is planned for future work, not within the scope of the current PR.
Ruff
prediction_market_agent_tooling/tools/betting_strategies/market_moving.py

37-37: Remove unnecessary True if ... else False

Remove unnecessary True if ... else False

(SIM210)

Additional comments not posted (2)
prediction_market_agent_tooling/tools/betting_strategies/market_moving.py (2)

10-16: LGTM!

The function signature changes are approved. The new parameters and return type improve the flexibility and usability of the function.


36-42: The previous review comment on this code segment is still valid. Refer to the comment below:

"""
coderabbitai[bot]: Refactor suggestion: Set a more realistic upper bound.

The TODO comment acknowledges the need for a better upper bound for max_bet_amount. Consider using market data or historical bet sizes to set this dynamically.

Tools
Ruff

37-37: Remove unnecessary True if ... else False

Remove unnecessary True if ... else False

(SIM210)

"""

Tools
Ruff

37-37: Remove unnecessary True if ... else False

Remove unnecessary True if ... else False

(SIM210)

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 29ab1cf and d7de8b6.

Files selected for processing (3)
  • prediction_market_agent_tooling/deploy/betting_strategy.py (2 hunks)
  • prediction_market_agent_tooling/tools/betting_strategies/kelly_criterion.py (2 hunks)
  • prediction_market_agent_tooling/tools/betting_strategies/market_moving.py (2 hunks)
Additional context used
Learnings (1)
Common learnings
Learnt from: kongzii
PR: gnosis/prediction-market-agent-tooling#224
File: prediction_market_agent_tooling/markets/markets.py:67-67
Timestamp: 2024-04-29T14:14:38.751Z
Learning: The function `have_bet_on_market_since` is suggested to be refactored into a method on the `AgentMarket` class, potentially reusing code from `DeployedAgent.get_resolved_bets`. This task is planned for future work, not within the scope of the current PR.
Additional comments not posted (5)
prediction_market_agent_tooling/tools/betting_strategies/market_moving.py (1)

Line range hint 10-82: LGTM!

The changes to the get_market_moving_bet function are approved. The refactoring has improved the function's flexibility and usability by:

  • Modifying the function signature to accept pool sizes, market probabilities, and other parameters directly, instead of relying on an OmenMarket object.
  • Returning a SimpleBet object, which encapsulates the bet's direction and size, enhancing clarity.
  • Retaining the binary search mechanism for determining the optimal bet amount while adjusting the calculations to reflect the new parameters.
  • Maintaining the assertions and checks for market invariants based on the new structure of the amounts.

The function is well-documented with a clear explanation of the underlying mathematical formula, and it correctly handles potential issues like length mismatches between vectors.

prediction_market_agent_tooling/tools/betting_strategies/kelly_criterion.py (3)

Line range hint 9-55: LGTM!

The changes to the get_kelly_bet_simplified function (previously get_kelly_bet) are approved. The refactoring has improved the function's clarity and consistency by:

  • Renaming the function to get_kelly_bet_simplified, which accurately reflects its purpose and distinguishes it from the new get_kelly_bet_full function.
  • Changing the return type from KellyBet to SimpleBet, aligning with the streamlined representation of bets introduced in other parts of the codebase.

The function remains well-documented with a clear explanation of the underlying mathematical formula, and it includes appropriate validation checks for the input probabilities and confidence levels.


58-137: Approve the addition of get_kelly_bet_full with a suggestion to verify the mathematical formula.

The new get_kelly_bet_full function is a valuable addition to the betting strategy logic, as it introduces a more comprehensive calculation for the Kelly Criterion, accounting for additional market dynamics such as outcome pool sizes and fees.

The function is well-documented with a clear explanation of the underlying mathematical formula and its source. It includes appropriate validation checks for the input probabilities, confidence level, and fee, and correctly handles edge cases where max_bet or the denominator is 0.

The use of the SimpleBet return type is consistent with the streamlined representation of bets introduced in other parts of the codebase.

However, given the mathematical complexity of the formula, it is crucial to ensure its correctness. Consider the following suggestions:

  1. Implement comprehensive unit tests to verify the function's behavior under various input scenarios, including edge cases.
  2. Have the mathematical formula reviewed by a domain expert to confirm its accuracy and applicability to the specific use case.

Would you like assistance in setting up additional unit tests or facilitating a review of the mathematical formula by a domain expert?


1-1: Approve the removal of the KellyBet class.

The removal of the KellyBet class is a positive change that aligns with the shift towards a more streamlined representation of bets using the SimpleBet class throughout the codebase.

The SimpleBet class likely provides similar functionality to the removed KellyBet class, making the latter redundant. The use of the SimpleBet class in the return types of get_kelly_bet_simplified and get_kelly_bet_full ensures consistency with other parts of the codebase.

prediction_market_agent_tooling/deploy/betting_strategy.py (1)

173-192: LGTM!

The changes in the calculate_trades method of the KellyBettingStrategy class are approved. The refactoring enhances the flexibility of the betting strategy by:

  • Allowing the method to choose between the full and simplified Kelly Criterion calculations based on the availability of a token pool in the market.
  • Using get_kelly_bet_full when a token pool is available, enabling a more comprehensive calculation that accounts for additional market dynamics such as outcome pool sizes and fees.
  • Using get_kelly_bet_simplified when a token pool is not available, providing a fallback option that uses fewer parameters and relies on the current market probability.

The check_not_none function is used appropriately to ensure that the outcome_token_pool is not None before accessing its values, preventing potential errors.

The method correctly constructs the amounts dictionary and target_position based on the direction and size of the kelly_bet, ensuring consistency with the rest of the betting strategy logic.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between d7de8b6 and 5d5a681.

Files selected for processing (2)
  • prediction_market_agent_tooling/tools/betting_strategies/market_moving.py (2 hunks)
  • tests/markets/test_betting_strategies.py (5 hunks)
Additional context used
Learnings (1)
Common learnings
Learnt from: kongzii
PR: gnosis/prediction-market-agent-tooling#224
File: prediction_market_agent_tooling/markets/markets.py:67-67
Timestamp: 2024-04-29T14:14:38.751Z
Learning: The function `have_bet_on_market_since` is suggested to be refactored into a method on the `AgentMarket` class, potentially reusing code from `DeployedAgent.get_resolved_bets`. This task is planned for future work, not within the scope of the current PR.
Additional comments not posted (5)
prediction_market_agent_tooling/tools/betting_strategies/market_moving.py (2)

12-18: LGTM!

The changes to the get_market_moving_bet function are approved. The refactoring improves the function's interface and logic by:

  • Modifying the function signature to accept pool sizes and other parameters directly, instead of relying on an OmenMarket object.
  • Returning a SimpleBet object to encapsulate the bet's direction and size, enhancing clarity and usability.
  • Refactoring the internal logic to use the provided pool sizes for calculations, eliminating the dependency on the OmenMarket class.

These changes make the function more flexible and easier to use with different market configurations.

Also applies to: 38-44, 48-63, 84-84


87-134: LGTM!

The addition of the _sanity_check_omen_market_moving_bet function is approved. This utility function is a useful addition for sanity checking the market moving bets by:

  • Calling the market's calcBuyAmount method from the smart contract.
  • Using the adjusted outcome pool sizes to calculate the new p_yes.
  • Checking that the market's new p_yes is equal to the target_p_yes.

This function helps ensure the correctness of the market moving bets.

tests/markets/test_betting_strategies.py (3)

164-193: LGTM!

The changes to the test_get_market_moving_bet function are approved. The updates align with the modifications made to the get_market_moving_bet function by:

  • Altering the parameterization of the function to use the new parameter names and types.
  • Updating the logic to accommodate the new parameter names and types.
  • Adjusting the call to get_market_moving_bet to pass the new parameters.
  • Modifying the assertions to check against the new expected values.

These changes ensure that the test function accurately evaluates the updated get_market_moving_bet function.


195-216: LGTM!

The addition of the test_sanity_check_market_moving_bet function is approved. This function is a valid test for the _sanity_check_omen_market_moving_bet function added in the prediction_market_agent_tooling/tools/betting_strategies/market_moving.py file. It:

  • Retrieves a binary market using the OmenAgentMarket.get_binary_markets method.
  • Calls the get_market_moving_bet function with the market's pool sizes, current p_yes, target p_yes, and fee.
  • Calls the _sanity_check_omen_market_moving_bet function with the market moving bet, market, and target p_yes.

This test function helps ensure the correctness of the _sanity_check_omen_market_moving_bet function.


246-268: LGTM!

The changes to the test_kelly_bet function are approved. The modifications enhance the testing of the Kelly criterion betting strategy by:

  • Replacing the call to get_kelly_bet with get_kelly_bet_simplified.
  • Introducing a new assertion that utilizes get_kelly_bet_full.

These changes allow for a more comprehensive evaluation of the bet direction based on different calculation methods, improving the robustness of the test function.

Co-authored-by: Peter Jung <peter@jung.ninja>
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 5d5a681 and d6f9005.

Files selected for processing (1)
  • prediction_market_agent_tooling/deploy/betting_strategy.py (2 hunks)
Additional context used
Learnings (1)
Common learnings
Learnt from: kongzii
PR: gnosis/prediction-market-agent-tooling#224
File: prediction_market_agent_tooling/markets/markets.py:67-67
Timestamp: 2024-04-29T14:14:38.751Z
Learning: The function `have_bet_on_market_since` is suggested to be refactored into a method on the `AgentMarket` class, potentially reusing code from `DeployedAgent.get_resolved_bets`. This task is planned for future work, not within the scope of the current PR.
Additional comments not posted (3)
prediction_market_agent_tooling/deploy/betting_strategy.py (3)

14-15: LGTM!

The import statements are approved.


17-17: LGTM!

The import statement is approved.


173-191: Skip the previous review comment.

The previous review comment suggesting the use of OMEN_FALSE_OUTCOME for no_outcome_pool_size is not applicable to the current code changes. The code correctly uses market.get_outcome_str_from_bool(True) and market.get_outcome_str_from_bool(False) for accessing the yes and no outcome pools respectively.

prediction_market_agent_tooling/deploy/betting_strategy.py Outdated Show resolved Hide resolved
@evangriffiths
Copy link
Contributor Author

evangriffiths commented Sep 10, 2024

Trying to verify the calculation in ``, and it looks like there's a bug somewhere. You would expect the kelly bet to asymptote to the market-moving bet, but see from the plots (for two different markets) that this isn't the case:

Screenshot 2024-09-10 at 01 09 22
Screenshot 2024-09-10 at 01 09 53

Since I've added a test that checks the market-moving-bet vs the buy amount calculated by the smart contract, I'm convinced the bug is in the kelly bet code.

To reproduce:

import numpy as np
import pytest
from matplotlib import pyplot as plt
from prediction_market_agent_tooling.markets.agent_market import FilterBy, SortBy
from prediction_market_agent_tooling.markets.omen.omen import OmenAgentMarket
from prediction_market_agent_tooling.tools.betting_strategies.kelly_criterion import (
    get_kelly_bet_full,
)
from prediction_market_agent_tooling.tools.betting_strategies.market_moving import (
    _sanity_check_omen_market_moving_bet,
    get_market_moving_bet,
)

market = OmenAgentMarket.get_binary_markets(
    limit=1,
    sort_by=SortBy.CLOSING_SOONEST,
    filter_by=FilterBy.OPEN,
)[0]

yes_outcome_pool_size = market.outcome_token_pool[
    market.get_outcome_str_from_bool(True)
]
no_outcome_pool_size = market.outcome_token_pool[
    market.get_outcome_str_from_bool(False)
]
max_bets = np.linspace(0.0, 200, 100)

# Define a list of colors for different estimated_p_yes values
colors = plt.cm.viridis(np.linspace(0, 1, len([0.05, 0.42, 0.71])))

for i, estimated_p_yes in enumerate([0.57, 0.74, 0.92]):
    kelly_bets = []
    market_moving_bets = []
    market_moving_bet = get_market_moving_bet(
        yes_outcome_pool_size=yes_outcome_pool_size,
        no_outcome_pool_size=no_outcome_pool_size,
        market_p_yes=market.current_p_yes,
        target_p_yes=estimated_p_yes,
        fee=market.fee,
    )
    _sanity_check_omen_market_moving_bet(market_moving_bet, market, estimated_p_yes)

    market_moving_bet_signed = (
        market_moving_bet.size
        if market_moving_bet.direction == True
        else -market_moving_bet.size
    )

    for max_bet in max_bets:
        kelly_bet = get_kelly_bet_full(
            yes_outcome_pool_size=yes_outcome_pool_size,
            no_outcome_pool_size=no_outcome_pool_size,
            estimated_p_yes=estimated_p_yes,
            confidence=1.0,
            max_bet=max_bet,
            fee=market.fee,
        )
        kelly_bet_signed = (
            kelly_bet.size if kelly_bet.direction == True else -kelly_bet.size
        )
        kelly_bets.append(kelly_bet_signed)
        market_moving_bets.append(market_moving_bet_signed)

        # TODO kelly bet is not moving the market as expected!
        if max_bet == max_bets[-1]:
            with pytest.raises(ValueError) as e:
                _sanity_check_omen_market_moving_bet(kelly_bet, market, estimated_p_yes)
            assert e.match("Bet does not move market to target_p_yes")

    plt.plot(
        max_bets,
        kelly_bets,
        label=f"Kelly bet, {estimated_p_yes=:.2f}",
        color=colors[i],
        linestyle="-",
    )
    plt.plot(
        max_bets,
        market_moving_bets,
        label=f"Market-moving bet, target_p_yes={estimated_p_yes:.2f}",
        color=colors[i],
        linestyle="--",
    )

plt.xlabel(f"Max bet (xDai)")
plt.ylabel("Kelly bet size (xDai)")
plt.title(
    f"Market-moving vs. kelly bet, market_p_yes={market.current_p_yes:.2f}, pool_size={(yes_outcome_pool_size+no_outcome_pool_size):.2f}"
)
plt.legend(bbox_to_anchor=(1.05, 1), loc="upper left")
plt.tight_layout()
plt.show()

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (1)
prediction_market_agent_tooling/deploy/betting_strategy.py (1)

173-192: LGTM, but consider simplifying the conditional logic.

The changes to introduce the conditional logic for choosing between get_kelly_bet_full and get_kelly_bet_simplified based on the availability of a token pool in the market are approved. This enhances the flexibility of the betting strategy.

The use of check_not_none is a good practice to ensure that the outcome pool sizes are validated before use.

Consider simplifying the conditional logic by directly returning the result inside the conditional branches, reducing the need for the temporary variable kelly_bet:

return (
    get_kelly_bet_full(
        yes_outcome_pool_size=check_not_none(market.outcome_token_pool)[
            market.get_outcome_str_from_bool(True)
        ],
        no_outcome_pool_size=check_not_none(market.outcome_token_pool)[
            market.get_outcome_str_from_bool(False)
        ],
        estimated_p_yes=answer.p_yes,
        max_bet=self.max_bet_amount,
        confidence=answer.confidence,
    )
    if market.has_token_pool()
    else get_kelly_bet_simplified(
        self.max_bet_amount,
        market.current_p_yes,
        answer.p_yes,
        answer.confidence,
    )
)
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between d50c553 and 2038282.

Files selected for processing (1)
  • prediction_market_agent_tooling/deploy/betting_strategy.py (2 hunks)
Additional context used
Learnings (1)
Common learnings
Learnt from: kongzii
PR: gnosis/prediction-market-agent-tooling#224
File: prediction_market_agent_tooling/markets/markets.py:67-67
Timestamp: 2024-04-29T14:14:38.751Z
Learning: The function `have_bet_on_market_since` is suggested to be refactored into a method on the `AgentMarket` class, potentially reusing code from `DeployedAgent.get_resolved_bets`. This task is planned for future work, not within the scope of the current PR.
Additional comments not posted (3)
prediction_market_agent_tooling/deploy/betting_strategy.py (3)

14-15: LGTM!

The imports are necessary for the changes introduced later in the file.


17-17: LGTM!

The import is necessary for the changes introduced later in the file.


176-181: Refactor Required: Use Correct Indices for Outcome Pools

The code incorrectly uses OMEN_TRUE_OUTCOME for both yes_outcome_pool_size and no_outcome_pool_size. It should use OMEN_TRUE_OUTCOME for yes_outcome_pool_size and OMEN_FALSE_OUTCOME for no_outcome_pool_size to ensure correct functionality.

Run the following script to verify the correct indices for outcome pools:

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (1)
prediction_market_agent_tooling/deploy/betting_strategy.py (1)

173-192: LGTM, but consider simplifying the conditional logic.

The changes to introduce the conditional logic for choosing between get_kelly_bet_full and get_kelly_bet_simplified based on the availability of a token pool in the market are approved. This enhances the flexibility of the betting strategy.

The use of check_not_none is a good practice to ensure that the outcome pool sizes are validated before use.

Consider simplifying the conditional logic by directly returning the result inside the conditional branches, reducing the need for the temporary variable kelly_bet:

return (
    get_kelly_bet_full(
        yes_outcome_pool_size=check_not_none(market.outcome_token_pool)[
            market.get_outcome_str_from_bool(True)
        ],
        no_outcome_pool_size=check_not_none(market.outcome_token_pool)[
            market.get_outcome_str_from_bool(False)
        ],
        estimated_p_yes=answer.p_yes,
        max_bet=self.max_bet_amount,
        confidence=answer.confidence,
    )
    if market.has_token_pool()
    else get_kelly_bet_simplified(
        self.max_bet_amount,
        market.current_p_yes,
        answer.p_yes,
        answer.confidence,
    )
)
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 43b4b97 and e5fcd68.

Files selected for processing (1)
  • prediction_market_agent_tooling/deploy/betting_strategy.py (2 hunks)
Additional context used
Learnings (1)
Common learnings
Learnt from: kongzii
PR: gnosis/prediction-market-agent-tooling#224
File: prediction_market_agent_tooling/markets/markets.py:67-67
Timestamp: 2024-04-29T14:14:38.751Z
Learning: The function `have_bet_on_market_since` is suggested to be refactored into a method on the `AgentMarket` class, potentially reusing code from `DeployedAgent.get_resolved_bets`. This task is planned for future work, not within the scope of the current PR.
Additional comments not posted (4)
prediction_market_agent_tooling/deploy/betting_strategy.py (4)

14-15: LGTM!

The import statements are correctly updated to include the new functions get_kelly_bet_full and get_kelly_bet_simplified from kelly_criterion.py.


17-17: LGTM!

The import statement for check_not_none is correctly added to validate outcome pool sizes before use.


173-192: The suggestion to refactor the conditional logic for choosing between get_kelly_bet_full and get_kelly_bet_simplified to simplify it is still valid and applicable to the current code segment.


173-192: The potential issue with using the same index for both yes and no outcome pool sizes is not applicable to the current code segment, as it now uses market.get_outcome_str_from_bool(True) and market.get_outcome_str_from_bool(False) to get the correct indices.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range and nitpick comments (1)
prediction_market_agent_tooling/deploy/betting_strategy.py (1)

173-192: LGTM, but consider simplifying the conditional logic.

The changes to introduce the conditional logic for choosing between get_kelly_bet_full and get_kelly_bet_simplified based on the availability of a token pool in the market are approved. This enhances the flexibility of the betting strategy.

The use of check_not_none is a good practice to ensure that the outcome pool sizes are validated before use.

Consider simplifying the conditional logic by directly returning the result inside the conditional branches, reducing the need for the temporary variable kelly_bet:

return (
    get_kelly_bet_full(
        yes_outcome_pool_size=check_not_none(market.outcome_token_pool)[
            market.get_outcome_str_from_bool(True)
        ],
        no_outcome_pool_size=check_not_none(market.outcome_token_pool)[
            market.get_outcome_str_from_bool(False)
        ],
        estimated_p_yes=answer.p_yes,
        max_bet=self.max_bet_amount,
        confidence=answer.confidence,
    )
    if market.has_token_pool()
    else get_kelly_bet_simplified(
        self.max_bet_amount,
        market.current_p_yes,
        answer.p_yes,
        answer.confidence,
    )
)
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between e5fcd68 and 55159fb.

Files selected for processing (1)
  • prediction_market_agent_tooling/deploy/betting_strategy.py (2 hunks)
Additional comments not posted (3)
prediction_market_agent_tooling/deploy/betting_strategy.py (3)

14-15: LGTM!

The import changes are approved. They are consistent with the alterations to the declarations of exported or public entities.

Also applies to: 17-17


173-192: The refactoring suggestion for the conditional logic is still valid and applicable to the current code segment. Please refer to the previous comment for the suggested refactoring.


173-192: The issue with using the same index for both outcome pool sizes is not applicable to the current code segment as it uses market.get_outcome_str_from_bool(True) and market.get_outcome_str_from_bool(False) to get the correct indices.

Comment on lines +132 to +134
denominator = 2 * (x**2 * f - y**2 * f)
if denominator == 0:
return SimpleBet(direction=True, size=0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick question about the formula: if (x=yes_outcome_pool_size = 10, y=no_outcome_pool_size=10), this always goes to 0 thus bet_amount is hard-set to 0.
Is that correct? This would e.g. prevent us from placing bets in balanced markets using Kelly - but maybe I'm mistaken.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but that doesn't seem like desirable behaviour! Have made an issue to fix: #401

@evangriffiths evangriffiths merged commit 3bcf3ce into main Sep 12, 2024
8 checks passed
@evangriffiths evangriffiths deleted the evan/kelly-bet-full branch September 12, 2024 00:51
@evangriffiths
Copy link
Contributor Author

Script to reproduce

Screenshot 2024-09-12 at 01 36 00

on this PR #389

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants