- Join us on our public discord channel for news, discussions, and status updates.
- Check out our medium for the latest posts and announcements.
- Functions
- System Design
- Modules
- Prerequisites
- Database
- Installation
- After Installation
- Configuration
- Work Process
- Pay for Filecoin by Polygon
- License
- Make payment from multi chain for filecoin storage
- Backup user's file to filecoin network
- Supports payment with tokens such as USDC on polygon
- Currently, USDC is supported for payment.
- WID is generated after you upload a file by calling upload file API, it includes 2 parts
- UUID: generated each time when upload, such as
ccceb1c1-ba11-4c95-8e62-d375ea80b0c3 - source file payload CID: generated by IPFS after uploading to IPFS server in upload web API, such as
QmbeBMxC8yBk67xZYPhAgyUMuxCW6DZdfFPhGbt9WFva3q - after upload a file, if the web API generates the above UUID and source file payload CID, then the WID will be
ccceb1c1-ba11-4c95-8e62-d375ea80b0c3QmbeBMxC8yBk67xZYPhAgyUMuxCW6DZdfFPhGbt9WFva3q, that is UUID + source file payload CID
- UUID: generated each time when upload, such as
- Users pay USDC or other tokens, which are called user tokens, when pay for a uploaded file.
- MCS uses FIL, which is called wrapped token, to pay when store data to filecoin network.
- User tokens should be changed to wrapped tokens by this module and this step is called token exchange(swap).
- Token exchange(swap) is done through Sushi Swap which is a DEX.
- After a file is uploaded, the money to be paid is estimated based on:
- the average price of all the miners on the entire network
- file size
- storage copy number
- duration
- Then the estimated amount of money will be locked to the payment contract address defined in Configuration
- In unlock step, the amount pay to filcoin network by swan platform fil wallet, will be transfered to mcs payment recipient address defined in Configuration
- In refund step, the overpayment part that is locked will be returned to user wallet
- If DAO detects that the file uploaded has been chained, it will trigger a signature operation
- OS: Ubuntu 20.04 LTS
- Mysql5.5+
- Lotus Node
- IPFS Client
- Lotus node is used for making car files and sending offline deals
- Install lotus node or lotus lite node in the same machine as MCS
- Lotus lite node is preferred since lotus full node is too heavy compared with lotus lite node
- Lotus lite node depends on a lotus node, so ensure that a lotus node exists somewhere when using lotus lite node
Option:one: install a lotus full node
Option:two: install a lotus lite node
- Please see schema create script in
./script/create_table.sql - Before installation, please create database and related tables using above script file
Option:one: Prebuilt package: See release assets
wget --no-check-certificate https://github.com/filswan/multi-chain-storage/releases/tag/v2.0.0/install.sh
chmod +x ./install.sh
./install.sh🔔go 1.16+ is required
git clone https://github.com/filswan/multi-chain-storage.git
cd multi-chain-storage
git checkout <release_branch>
./build_from_source.sh- Before executing, you should check your configuration in
~/.swan/mcs/config.tomlto ensure it is right.
vi ~/.swan/mcs/config.toml- Before executing, you should check your enviornment variable in
~/.swan/mcs/.envto ensure it is right.
vi ~/.swan/mcs/.env- After set your config and env variable in the related files, you can run MCS using one of the following methods
./multi-chain-storage-2.0.0-linux-amd64 polygon.mumbai|polygon.mainnet|bsc.testnet #After installation from Option 2
./build/multi-chain-storage polygon.mumbai|polygon.mainnet|bsc.testnet #After installation from Option 2- Logs are in directory
./logs - You can use the following methods to avoid it be stopped when you exit your OS session:
nohup ./multi-chain-storage-2.0.0-linux-amd64 polygon.mumbai|polygon.mainnet|bsc.testnet >> mcs.log & #After installation from Option 1
nohup ./build/multi-chain-storage polygon.mumbai|polygon.mainnet|bsc.testnet >> ./build/mcs.log & #After installation from Option 2- port: Web api port
- release: When work in release mode: set this to true, otherwise to false and enviornment variable GIN_MODE not to release
- filecoin_network: filecoin_calibration or filecoin_mainnet
- filecoin_wallet: The wallet address used to pay on the filecoin network
- flink_url: Deals data can be searched from here
- web3_api_url_polygon_mumbai: Web3 api url for polygon mumbai
- web3_api_url_bsc_testnet: Web3 api url for BSC testnet
- db_host: Host MCS database resides in
- db_port: Port of MCS database
- db_schema_name: MCS database name, see Database
- db_username: Username of MCS database
- db_password: Password of MCS database
- db_args: Use default value
charset=utf8mb4&parseTime=True&loc=Local
- api_url: Swan API address:
https://go-swan-server.filswan.com. ‼️ api_key: Your Swan API key. Acquire from Swan Platform -> "My Profile"->"Developer Settings".‼️ access_token: Your Swan API access token. Acquire from Swan Platform -> "My Profile"->"Developer Settings".
- client_api_url: Url of lotus client web api, such as:
http://[ip]:[port]/rpc/v0, generally the[port]is1234. - client_access_token: Access token of lotus client web api with admin access right. Get it from lotus node by command
lotus auth create-token --perm admin.
- download_url_prefix: Ipfs server url prefix, such as:
http://[ip]:[port]. Store car files for downloading by storage provider. Car file url will be[download_url_prefix]/ipfs/[file_hash] - upload_url_prefix: Ipfs server url for uploading files, such as
http://[ip]:[port]
- dir_deal: Directory to store source files, car files, and JSON files created by Swan Client API
- description: Task description
- curated_dataset: Task dataset
- max_price: Max price willing to pay per GiB/epoch for offline deal
- expired_days: Expected completion days for storage provider sealing data
- verified_deal: [true/false] Whether deals in this task are going to be sent as verified
- fast_retrieval: [true/false] Indicates that data should be available for fast retrieval
- start_epoch_hours: Start epoch for deals in hours from current time
- min_file_size: Source files size lower limit when merge them to a car file
- create_task_interval_second: Job running interval, unit: second, default: 120
- send_deal_interval_second: Job running interval, unit: second, default: 180
- scan_deal_status_interval_second: Job running interval, unit: second, default: 300
- Users upload a file they want to backup to filecoin network
- User pay currencies we support to send tokens to our payment contract address defined in Configuration
- MCS writes the transaction info to our system
- MCS scan those source files uploaded and paid but not yet created to car files, and then do the following steps:
- compute the max price for each source file, based on the source file size, token paid, and exchange rate betwee USDC and wFil
- if the scanned source file size sum is equal or greater than
[swan_task].min_file_sizedefined in Configuration, or the earliest source file to be merged to car file is more 1 day ago, then MCS will do the following steps by calling Swan Client API- create car files, use the minimum max price among the source files to be merged as the max price for the whole car file
- upload car files
- create task on swan platform
- Market Matcher allocate miners for the car file created in last step
- MCS send deals by calling Swan Client API
- MCS Scan Scheduler module scan the deal info from lotus
- When DAO organization find the deal active on lotus, they will sign to agree to unlock the user's payment for this deal.
- After success DAO signatures number equal or greater than DAO threshold defined in smart contract, and after 1 minute later of the last DAO signature, MCS will unlock the user's payment, release the money spent on send deal by Swan Client API to
[polygon].payment_recipient_addressdefined in Configuration - After all deals of a car file are unlocked, MCS refund the remaining money to user wallet address used when pay in step 2.