Skip to content

Commit

Permalink
Add azure storage state handler support
Browse files Browse the repository at this point in the history
  • Loading branch information
MDUYN committed Jan 10, 2025
1 parent d6baab6 commit f7c5a12
Show file tree
Hide file tree
Showing 88 changed files with 3,279 additions and 2,522 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,10 @@
### Python template
# Byte-compiled / optimized / DLL files
__pycache__/
*__pycache__/
*.py[cod]
*$py.class
.DS_Store

# C extensions
*.so
Expand Down
53 changes: 13 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
[![GitHub stars](https://img.shields.io/github/stars/coding-kitties/investing-algorithm-framework.svg?style=social&label=Star&maxAge=1)](https://github.com/SeaQL/sea-orm/stargazers/) If you like what we do, consider starring, sharing and contributing!

###### Sponsors

<p align="left">
<a href="https://finterion.com">
<img alt="Finterion" src="static/sponsors/finterion.png" width="200px" />
Expand Down Expand Up @@ -36,30 +37,13 @@ Features:
The following algorithm connects to binance and buys BTC every 5 seconds. It also exposes an REST API that allows you to interact with the algorithm.

```python
import pathlib
import logging
from investing_algorithm_framework import create_app, PortfolioConfiguration, \
RESOURCE_DIRECTORY, TimeUnit, CCXTOHLCVMarketDataSource, Algorithm, \
CCXTTickerMarketDataSource, MarketCredential, SYMBOLS

# Define the symbols you want to trade for optimization, otherwise the
# algorithm will check if you have orders and balances on all available
# symbols on the market
symbols = ["BTC/EUR"]

# Define resource directory and the symbols you want to trade
config = {
RESOURCE_DIRECTORY: pathlib.Path(__file__).parent.resolve(),
SYMBOLS: symbols
}

state_manager = AzureBlobStorageStateManager(
account_name="<your account name>",
account_key="<your account key>",
container_name="<your container name>",
blob_name="<your blob name>",
)
TimeUnit, CCXTOHLCVMarketDataSource, Algorithm, \
CCXTTickerMarketDataSource, MarketCredential, DEFAULT_LOGGING_CONFIG

logging.config.dictConfig(DEFAULT_LOGGING_CONFIG)

# Define market data sources
# OHLCV data for candles
bitvavo_btc_eur_ohlcv_2h = CCXTOHLCVMarketDataSource(
identifier="BTC-ohlcv",
Expand All @@ -74,17 +58,10 @@ bitvavo_btc_eur_ticker = CCXTTickerMarketDataSource(
market="BITVAVO",
symbol="BTC/EUR",
)
app = create_app(
config=config,
sync_portfolio=True,
state_manager=state_manager
)
app = create_app()
algorithm = Algorithm()
app.add_market_credential(MarketCredential(
market="bitvavo",
api_key="<your api key>",
secret_key="<your secret key>",
))
# Bitvavo market credentials are read from .env file
app.add_market_credential(MarketCredential(market="bitvavo"))
app.add_portfolio_configuration(
PortfolioConfiguration(
market="bitvavo",
Expand All @@ -94,21 +71,18 @@ app.add_portfolio_configuration(
)
app.add_algorithm(algorithm)

# Run every two hours and register the data sources
@algorithm.strategy(
# Run every two hours
time_unit=TimeUnit.HOUR,
interval=2,
# Specify market data sources that need to be passed to the strategy
market_data_sources=[bitvavo_btc_eur_ticker, bitvavo_btc_eur_ohlcv_2h]
)
def perform_strategy(algorithm: Algorithm, market_data: dict):
# By default, ohlcv data is passed as polars df in the form of
# {"<identifier>": <dataframe>} https://pola.rs/,
# call to_pandas() to convert to pandas
# Access the data sources with the indentifier
polars_df = market_data["BTC-ohlcv"]
print(f"I have access to {len(polars_df)} candles of ohlcv data")

# Ticker data is passed as {"<identifier>": <ticker dict>}
# Convert the polars dataframe to a pandas dataframe
pandas_df = polars_df.to_pandas()
ticker_data = market_data["BTC-ticker"]
unallocated_balance = algorithm.get_unallocated()
positions = algorithm.get_positions()
Expand Down Expand Up @@ -257,7 +231,6 @@ app.add_portfolio_configuration(
PortfolioConfiguration(
market="<your market>",
initial_balance=400,
track_from="01/01/2022",
trading_symbol="EUR"
)
)
Expand Down
49 changes: 49 additions & 0 deletions examples/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
from dotenv import load_dotenv

from investing_algorithm_framework import create_app, PortfolioConfiguration, \
TimeUnit, CCXTOHLCVMarketDataSource, Algorithm, \
CCXTTickerMarketDataSource, MarketCredential, AzureBlobStorageStateHandler

load_dotenv()

# Define market data sources
# OHLCV data for candles
bitvavo_btc_eur_ohlcv_2h = CCXTOHLCVMarketDataSource(
identifier="BTC-ohlcv",
market="BITVAVO",
symbol="BTC/EUR",
time_frame="2h",
window_size=200
)
# Ticker data for orders, trades and positions
bitvavo_btc_eur_ticker = CCXTTickerMarketDataSource(
identifier="BTC-ticker",
market="BITVAVO",
symbol="BTC/EUR",
)
app = create_app(state_handler=AzureBlobStorageStateHandler())
app.add_market_data_source(bitvavo_btc_eur_ohlcv_2h)
algorithm = Algorithm()
app.add_market_credential(MarketCredential(market="bitvavo"))
app.add_portfolio_configuration(
PortfolioConfiguration(
market="bitvavo",
trading_symbol="EUR",
initial_balance=20
)
)
app.add_algorithm(algorithm)

@algorithm.strategy(
# Run every two hours
time_unit=TimeUnit.HOUR,
interval=2,
# Specify market data sources that need to be passed to the strategy
market_data_sources=[bitvavo_btc_eur_ticker, "BTC-ohlcv"]
)
def perform_strategy(algorithm: Algorithm, market_data: dict):
# By default, ohlcv data is passed as polars df in the form of
# {"<identifier>": <dataframe>} https://pola.rs/,
# call to_pandas() to convert to pandas
polars_df = market_data["BTC-ohlcv"]
print(f"I have access to {len(polars_df)} candles of ohlcv data")
29 changes: 10 additions & 19 deletions examples/bitvavo_trading_bot/bitvavo.py
Original file line number Diff line number Diff line change
@@ -1,31 +1,28 @@
import os

from investing_algorithm_framework import MarketCredential, TimeUnit, \
CCXTOHLCVMarketDataSource, CCXTTickerMarketDataSource, TradingStrategy, \
create_app, PortfolioConfiguration, Algorithm, SYMBOLS, RESOURCE_DIRECTORY
create_app, PortfolioConfiguration, Algorithm

"""
Bitvavo trading bot example with market data sources of bitvavo.
Bitvavo does not requires you to have an API key and secret key to access
their market data. If you just want to backtest your strategy,
Bitvavo does not requires you to have an API key and secret key to access
their market data. If you just want to backtest your strategy,
you don't need to add a market credential. If your running your strategy live,
you need to add a market credential to the app, that accesses your
you need to add a market credential to the app, that accesses your
account on bitvavo.
"""


# Define your market credential for bitvavo
bitvavo_market_credential = MarketCredential(
api_key="<your_api_key>",
secret_key="<your_secret_key>",
market="bitvavo",
api_key="your_api_key",
secret_key="your_secret_key"
)
# Define your market data sources for coinbase
bitvavo_btc_eur_ohlcv_2h = CCXTOHLCVMarketDataSource(
identifier="BTC/EUR-ohlcv",
market="bitvavo",
symbol="BTC/EUR",
timeframe="2h",
time_frame="2h",
window_size=200
)
bitvavo_btc_eur_ticker = CCXTTickerMarketDataSource(
Expand All @@ -36,26 +33,20 @@


class BitvavoTradingStrategy(TradingStrategy):
time_unit = TimeUnit.HOUR
interval = 2
time_unit = TimeUnit.SECOND
interval = 10
market_data_sources = [bitvavo_btc_eur_ohlcv_2h, bitvavo_btc_eur_ticker]

def apply_strategy(self, algorithm, market_data):
print(market_data["BTC/EUR-ohlcv"])
print(market_data["BTC/EUR-ticker"])


config = {
SYMBOLS: ["BTC/EUR"],
RESOURCE_DIRECTORY: os.path.join(os.path.dirname(__file__), "resources")
}

# Create an algorithm and link your trading strategy to it
algorithm = Algorithm()
algorithm.add_strategy(BitvavoTradingStrategy)

# Create an app and add the market data sources and market credentials to it
app = create_app(config=config)
app = create_app()
app.add_market_credential(bitvavo_market_credential)
app.add_market_data_source(bitvavo_btc_eur_ohlcv_2h)
app.add_market_data_source(bitvavo_btc_eur_ticker)
Expand Down
1 change: 1 addition & 0 deletions examples/deployments/azure_function/.funcignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

48 changes: 48 additions & 0 deletions examples/deployments/azure_function/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
bin
obj
csx
.vs
edge
Publish

*.user
*.suo
*.cscfg
*.Cache
project.lock.json

/packages
/TestResults

/tools/NuGet.exe
/App_Data
/secrets
/data
.secrets
appsettings.json
local.settings.json

node_modules
dist

# Local python packages
.python_packages/

# Python Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# Azurite artifacts
__blobstorage__
__queuestorage__
__azurite_db*__.json
76 changes: 76 additions & 0 deletions examples/deployments/azure_function/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Investing Algorithm Framework App Deployment to Azure Functions

This article demonstrates how to deploy your trading bot to Azure Functions.
We will deploy an example bot that uses the investing algorithm framework to
azure functions. In order to do that we will do the following:

1. Create a new app using the framework with the azure blob storage state handler.
2. Use the framework provided ci tools to create a azure functions ready application.
3. Use the framework provided ci tools to deploy the application to azure functions.

## Prerequisites

For this example, you need to have the following:

- An Azure account with an active subscription. [Create an account](https://azure.microsoft.com/en-us/free/)
- The Azure CLI installed. [Install the Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli)
- The Azure Functions Core Tools installed. [Install the Azure Functions Core Tools](https://docs.microsoft.com/en-us/azure/azure-functions/functions-run-local)
- The investing algorithm framework installed. [Install the framework]() or simply run `pip install investing_algorithm_framework`

### Creating a new app

First run the following command to create a new app azure functions ready app:

```bash
create_azure_function_trading_bot_app
```

This command will create a new app with the following structure:

```yaml
.
├── function_app.py
├── host.json
├── local.settings.json
└── requirements.txt
```

The function_app.py while import the app from the app.py file in the root of the project and run it as an azure function. It is therefore important that you have a file named `app.py` in the root of your project that defines the application in a variable named `app`.

Additionaly, because Azure Functions are stateless, you need to use a state storage solution. In this example, we will use Azure Blob Storage state handler provided by the framework. This state handler is specifically designed to work with Azure Functions and Azure Blob Storage.

The reason why we need a state storage solution is that Azure Functions are stateless. This means that each function execution is independent of the previous one. This is a problem for trading bots because they need to keep track of their state between executions (portfolios, order, positions and trades). In order to solve this problem, we need to use a state storage solution that can store the bot's databases between executions.

Combining all of this, the `app.py` file should look like this:

> When you are using the cli command 'deploy_trading_bot_to_azure_function' (which we will use later) you don't need to provide any connection strings. The command will take care of provisioning all
> resourses and configuration of all required parameters for the state handler.
```python
from investing_algorithm_framework import AzureBlobStorageStateHandler, create_app

app = create_app(state_handler=AzureBlobStorageStateHandler)

# Write here your code where your register your portfolio configurations, strategies and data providers
....
```

## Deployment to Azure

To deploy your trading bot to Azure Functions, you need to run the following command:

```bash
deploy_trading_bot_to_azure_function
```

This command will do the following:

- Create a new resource group in your Azure account.
- Create a new storage account in the resource group.
- Create a new blob container in the storage account.
- Create a new function app in the resource group.
- Deploy the trading bot to the function app.
- Configure the function app to use the blob container as the state handler.
- Print the URL of the function app.

After running the command, you will see the URL of the function app. You can use this URL to access your trading bot. Now your trading bot is running on Azure Functions!
File renamed without changes.
Loading

0 comments on commit f7c5a12

Please sign in to comment.