Everscale Wallet API

Overview

This is a light node + api for sending and tracking payments. The app listens for addresses from the database and indexes all transactions, putting information about them in the postsgres DB. All transactions with native EVERs are tracked, and there is a whitelist of root token addresses to be tracked in the settings. There is a callbacks table in the database, where you can specify the url of your backend to which callbacks will come for all transactions.

It takes about 20 minutes to synchronize the node. Both the ton-wallet-api and callback requests use HMAC signatures in the headers.

Runtime requirements

• CPU: 4 cores, 2 GHz
• RAM: 8 GB
• Storage: 100 GB fast SSD
• Network: 100 MBit/s
• Postgres: 11 or higher

How to run natively

To simplify the build and create some semblance of standardization in this repository there is a set of scripts for configuring the ton-wallet-api.

NOTE: scripts are prepared and tested on Ubuntu 20.04. You may need to modify them a little for other distros.

1. Setup ton-wallet-api service

   ./scripts/setup.sh -t native --database-url ${DATABASE_URL}

   DATABASE_URL - Postgres connection url (example: postgresql://postgres:postgres@127.0.0.1/ton_wallet_api)

   At this stage, a systemd service ton-wallet-api is created. Configs and keys will be in /etc/ton-wallet-api and TON node DB will be in /var/db/ton-wallet-api.

   Do not start this service yet!

2. Prepare config

   Either add the environment variables to the [Service] section of unit file. It is located at /etc/systemd/system/ton-wallet-api.service.

   [Service]
   ...
   Environment=DB_HOST=db_host
   Environment=DB_USER=db_user
   Environment=DB_PASSWORD=db_password
   Environment=DB_NAME=ton_wallet_api
   Environment=SECRET=secret
   Environment=SALT=salt
   ...
   

   SECRET - any string to encrypt/decrypt all addresses private keys.

   SALT - 16 bytes recommended in B64 for secret hashing.

   How to gen SALT

   cargo build --release
   ./target/release/ton-wallet-api salt

3. Create api service

     ./scripts/api_service.sh -t native --database-url ${DATABASE_URL} --id ${SERVICE_ID} --name ${SERVICE_NAME} --key ${SERVICE_KEY} --secret ${SERVICE_SECRET}

   DATABASE_URL - Postgres connection url (example: postgresql://postgres:postgres@127.0.0.1/ton_wallet_api)
   SERVICE_ID - Service id (UUID4) (example: 1fa337bd-2947-4809-9a7a-f04b4f9b738a)
   SERVICE_NAME - Service name (example: test)
   SERVICE_KEY - Public key (example: apiKey)
   SERVICE_SECRET - Secret key (example: apiSecret)

4. Enable and start ton-wallet-api service

   systemctl enable ton-wallet-api
   systemctl start ton-wallet-api
   
   # Optionally check if it is running normally. It will take some time to start.
   # ton-wallet-api is fully operational when it prints `listening on ${your_listen_address}`
   journalctl -fu ton-wallet-api

   Wallet API has a two built-in Prometheus metrics exporters: API and Node. You can enable API metrics by giving the value of api_metrics_addr in config. Node metrics exporter is configured in the node_metrics_settings section of the config. By default, node metrics are available at http://127.0.0.1:10000/

   Node metrics response example:

   ton_subscriber_ready 1
   ton_subscriber_current_utime 1639490380
   ton_subscriber_time_diff 4
   ton_subscriber_shard_client_time_diff 7
   ton_subscriber_mc_block_seqno 13179326
   ton_subscriber_shard_client_mc_block_seqno 13179326
   ton_subscriber_pending_message_count 0
   

5. Update service

     ./scripts/update.sh -t native --database-url ${DATABASE_URL}

   DATABASE_URL - Postgres connection url (example: postgresql://postgres:postgres@127.0.0.1/ton_wallet_api)

Let's start using Wallet API

1. Create address

   Create yourself a "system address" by calling /address/create with empty parameters. The response will return a EVER address. It is necessary to send EVERs on it, which will be consumed as gas for further work.

   For simplicity, you use the script

   API_KEY=${API_KEY} SECRET=${API_SECRET} HOST=${HOST} \
   ./scripts/wallet.sh -m create_account

2. Callbacks

   In the table api_service_callback we enter the address of our backend, which will deal with payment processing. After receiving or sending new transactions or token transactions Wallet API will call web hook with POST method on callback url. Body will contain AccountTransactionEvent from swagger.

3. Token Whitelist

   You can see the root-contract addresses at manifest. By default, the whitelist already includes all the tokens in this list.

   To add more tokens to the whitelist, use the script:

     ./scripts/root_token.sh -t native --database-url ${DATABASE_URL} --name ${TOKEN_NAME} --address ${TOKEN_ADDRESS} --version ${TOKEN_CONTRACT_VERSION}

   DATABASE_URL - Postgres connection url (example: postgresql://postgres:postgres@127.0.0.1/ton_wallet_api)
   TOKEN_NAME - Token name (example: WTON)
   TOKEN_ADDRESS - Token address (example: 0:0ee39330eddb680ce731cd6a443c71d9069db06d149a9bec9569d1eb8d04eb37) TOKEN_CONTRACT_VERSION - "Tip3" or "OldTip3v4"

4. Transfer EVER

   Example request:

   /transactions/create
   {
      // a random uuid that you generate yourself and store on your backend, to further track the status of the transaction
      "id":"00000000-0000-0000-0000-000000000000",
      // The address of the sender. For example, your system address.
      "fromAddress":"0:0000000000000000000000000000000000000000000000000000000000000000",
      "bounce":false,
      "outputs":[
         {
            // how much EVER to send. To send 1 EVER this value = 1000000000
            "value":"1000000000",
            // Set Normal to take the number of sent EVERs from the value
            "outputType":"Normal",
            // Recipient address of EVERs
            "recipientAddress":"0:0000000000000000000000000000000000000000000000000000000000000000"
         }
      ],
      // base64 encoded payload
      payload: "te6ccgEBAQEABgAACAVriBQ="
   }
   

   Or use the script:

   # Create transaction
   API_KEY=${API_KEY} SECRET=${API_SECRET} HOST=${HOST} \
   ./scripts/wallet.sh -m create_transaction \
   --src-addr {sender} --dst-addr {recipient} --amount {amount}

   You can track the status of a transaction with:

   1. (Recommended way) via callback AccountTransactionEvent, which has transactionStatus field:

      • expired - end state for failed transactions,
      • done - final state for successful transactions.

      If your backend was disabled during the callback or responded with an error, the event will have an Error state. In this case you should query all events /events in Error state at backend startup, process them and give each event a Done state by calling /events/mark.

   2. by polling the GET method /transactions/id/<uuid>

5. How to process a payment from a user on the backend

   We generate a deposit address for the user by calling /address/create with empty parameters. After receiving the payment, the backend receives a callback of the form AccountTransactionEvent (see swagger). You can also get such events in a list, using the /events method.

   If your backend was not working at the time of the callback or responded with an error, the event will have an Error status. In this case you should query all events /events in Error state at the start of the backend, process them and set each of them to Done state by calling /events/mark. Each event has an id (generated by ton-api). You can do extra checks on it to make sure that your backend doesn't re-process events.

6. Transfer tokens

   First, check the status and balance of the address you want to send tokens from by making a GET request to /address/{string}. The address you are sending tokens from must have at least 0.6 EVER (balance >= 600000000).

   To transfer tokens, use the method:

   /tokens/transactions/create
   {
      // a random uuid that you generate yourself and store on your backend, to further track the status of the transaction
      "id":"00000000-0000-0000-0000-000000000000",
      // The address of the sender. For example, your system address.
      "fromAddress":"0:0000000000000000000000000000000000000000000000000000000000000000",
      // Recipient address of EVERs
      "recipientAddress":"0:0000000000000000000000000000000000000000000000000000000000000000",
      // The number of tokens with decimals. For example, for transferring 1 USDT this value = "1000000"
      "value":"1000000000",
      // How much to apply EVER, the default recommended value is 0.5 EVER. The funds will be debited fromAddress.
      "fee": "5000000000",
      // The address to which to return the residuals EVER. For example, your system address.
      "sendGasTo":"0:0000000000000000000000000000000000000000000000000000000000000000",
      // Token Address from whitelist
      "rootAddress":"0:0000000000000000000000000000000000000000000000000000000000000000",
      // base64 encoded payload
      "payload": "te6ccgEBAQEABgAACAVriBQ="
   }
   

   Or use the script:

    # Create token transaction
   API_KEY=${API_KEY} SECRET=${API_SECRET} HOST=${HOST} \
   ./scripts/wallet.sh -m create_token_transaction \
   --src-addr {sender} --dst-addr {recipient} \
   --root-addr {root_token_address} --amount {amount}

   You can track the status of a transaction with:

   1. (Recommended way) via callback AccountTransactionEvent, which has transactionStatus field:

      • expired - end state for failed transactions,
      • done - final state for successful transactions.

      If your backend was down at the time of the callback or responded with an error, the event will have an Error state. In this case, you should query all events /token/events in Error state at backend startup, process them, and set each event to Done state by calling /token/events/mark.

   2. by polling with the GET method /tokens/transactions/id/<uuid>

Postman

pre-request-script.js is javascript for using with Postman's pre-request script feature. It generates HTTP request headers for HMAC authentication. Copy the contents of pre-request-script.js into the "Pre-request Script" tab in Postman to send signed request.

Authorization signatures

A request must undergo an authorization step, which involves verifying the request payload, URI, and timestamp against a known secret specific to each service. For GET requests, the body is empty, while for POST requests, the body is a string representing the serialized JSON payload. The payload must be pretty-printed (stored) with an indentation of 4 spaces. An example of the string to be signed is provided below. This setup generally applies to POST requests.

stringToSign="$timestamp$uri$body"
echo -en "$stringToSign" | openssl sha256 -hmac "$secret" -binary | base64

Example config

NOTE: The syntax ${VAR} can also be used everywhere in config. It will be replaced by the value of the environment variable VAR.

---
# Server address
server_addr: "0.0.0.0:8080"
# Database URL
database_url: "postgresql://${DB_USER}:${DB_PASSWORD}@${DB_HOST}/${DB_NAME}"
# Database Connection Pools
db_pool_size: 5
ton_core:
  # UDP port, used for ADNL node. Default: 30303
  adnl_port: 30303
  # Root directory for ton-wallet-api DB. Default: "./db"
  db_path: "/var/ton-wallet-api/db"
  # Path to ADNL keys.
  # NOTE: Will be generated if it was not there.
  # Default: "./adnl-keys.json"
  keys_path: "/var/ton-wallet-api/adnl-keys.json"
metrics_settings:
  # Listen address of metrics. Used by the client to gather prometheus metrics.
  # Default: "127.0.0.1:10000"
  listen_address: "127.0.0.1:10000"
  # URL path to the metrics. Default: "/"
  # Example: `curl http://127.0.0.1:10000/`
  metrics_path: "/"
  # Metrics update interval in seconds. Default: 10
  collection_interval_sec: 10
# log4rs settings.
# See https://docs.rs/log4rs/1.0.0/log4rs/ for more details
logger_settings:
  appenders:
    stdout:
      kind: console
      encoder:
        pattern: "{d(%Y-%m-%d %H:%M:%S %Z)(utc)} - {h({l})} {M} = {m} {n}"
  root:
    level: error
    appenders:
      - stdout
  loggers:
    ton_wallet_api:
      level: info
      appenders:
        - stdout
      additive: false
    ton_indexer:
      level: error
      appenders:
        - stdout
      additive: false

How to Run via Docker/Podman

This guide explains how to build and run the project using Podman (or Docker as an alternative). There are two Dockerfiles: one for building an intermediate image for setting and compiling the Rust project and another for deploying it. With that user may replace configuration files and start parameters and promptly rebuild deployment image.

Prerequisites

User must provide a link to running Postgres instance with user and password. It will be used for migrations and for deployment as well.

Building Images

To build the images, two Dockerfiles are used: builder.dockerfile for compiling the project and deploy.dockerfile for setting up the deployment environment.

1. Build the builder image: The builder.dockerfile is responsible for compiling the project using Rust. It builds the project based on the specified network (either everscale or venom) and prepares the database for the application using SQLx.

   Use the following command to build the builder image:

   podman build --layers --network=host -f builder.dockerfile -t builder --build-arg DATABASE_URL="postgresql://everscale:everscale@localhost:5432/everscale"

2. Build the deployment image: The deploy.dockerfile is used to create the runtime environment and copy the necessary binaries and configuration files from the builder stage.

   Build the deployment image using the following command:

   podman build --layers -f deploy.dockerfile -t ever-wallet

Running the Container

Once the images are built, you can run the container using Podman or Docker.

1. Running the application:

   To run the application, use the following command:

   podman run --network=host ever-wallet

   This will run the ton-wallet-api server using the default configuration files already existing in the container. Errors shall be expected at this step.

   WARN: Environment variable DB_USER was not set
   WARN: Environment variable DB_PASSWORD was not set
   WARN: Environment variable DB_HOST was not set
   WARN: Environment variable DB_NAME was not set

   To avoid these errors, the database connection and other settings (such as secrets) shall be provided. Alternatively, they may be added into deploy.dockerfile. For example:

   podman run --network=host \
     -v /tmp/everscale-data:/var/db/ton-wallet-api
     -e DB_USER=everscale \
     -e DB_PASSWORD=everscale \
     -e DB_HOST=localhost \
     -e DB_NAME=everscale \
     -e SECRET=0xAAAAA \
     -e SALT=OreOYYe5nHWTHnOPSvsmMQ \
     ever-wallet

   It generally allows dynamically setting environment variables for database credentials, secrets, and other configurations.

Troubleshooting

When the node is out of sync, which especially applies for Venom, removing database and re-syncing node may help to restore service operations.

rm -rf /var/db/ton-wallet-api