Boost Pre-release Testing

[LexLuthr]

Overview

The purpose of Boost is to replace the current version of Lotus Markets. It is currently being developed as a standalone repository to allow the team to iterate quickly on a redesign of the system, prove the merit of the new design early and often through internal and community facing demos, and to be releasable as a standalone binary that can run alongside the existing Lotus process.

Alpha Testing

The alpha testing for boost has already been completed with positive results. We have integrated the feedback from the initial testing back into the boost. You can check the results of alpha testing here.

Pre-release Testing

After the successful initial testing, Boost team is looking to test the product with a wider range of storage providers. We are inviting all the interested storage providers from the community to participate in this testing before the main launch. The objective of this program is to test boost's robustness and performance with different sized storage providers, deal sources and configuration.

Please note that there is no grant or reward to participate with this program.

Expectations

• Improve the storage deal flow
  • Make replay-ability of deal stages much easier when the process experiences a recoverable error, such as "out of funds".
  • Reduce state complexity and debuggability through reduction based state transitions
  • Provide more flexibility of each deal stage to enable future extensibility/customization
• Improve visibility for Storage Providers into their systems
  • Add a simple webui to leverage the new Markets APIs
  • Better visibility of fund allocation, deal states including data transfer, and sealing pipelines
• Extensible data transfer
  • Online deals support fetching a CAR file over HTTP
• Improve team velocity
  • Reduce complexity in deal stage management

Schedule

Event	Date
Start Testing	16th May 2022
End Testing	22nd May 2022

Instructions

• No Complete Rollback

  • Please note that Boost uses a sqllite DB for the deal metadata and logs. Once boost has been enabled, the new deals cannot be rolled back to the lotus market. If you decide to roll back after the pre-release testing, you will lose all the metadata for the deal made with boost. However, this will have no impact on the sealed data itself.

• Please upgrade your test lotus-miner to the latest v1.15.2 release.

• All instructions on using boost can be found on the boost website under migrating from lotus to boost section.

• For any issues with the documentation, please also include that directly in your feedback.

• Please fill out this report template with your results, as well as any qualitative feedback that you may have.

Title	Metric
Miner ID
Changing deal ask config from UI is working	[x]yes []no
Adding fund to market actor is working through UI	[x]yes []no
Deal transfer is working in UI for all deals	[x]yes []no
Publishing deals from UI is working	[x]yes []no
All the deal parameters and logs are available in deal detail page	[x]yes []no
All the legacy deals are available in the UI	[x]yes []no
Sealing Pipeline displays the correct stage for deal sealing	[x]yes []no
Success rate and number of boost deals made (Estuary, Bidbot, boost-client, etc)
Success rate and number of legacy deals made (Estuary, Bidbot, lotus-light, etc)
Hardware spec of Boost node
Average speed of transfers experienced previously with legacy markets (GraphSync)
Average speed of transfers on Boost
Network bandwidth
Number of deals initially migrated

Reminders

• Risk Mitigation
  • Please backup everything in the repositories before making any changes.
  • Roll back to legacy markets
• Reporting Issues
  • Please fill out this template
• Questions
  • Start here
  • @LexLuthr will be monitoring the relevant slack channels and Github
• User tutorials
  • We would appreciate help with user tutorials.
  • Example tutorial on how to store files with boost

[tmyuu]

Version

$ boostd -h
0.0.1+git.bc85712 (v1.0.0-rc2)

$ lotus-miner version
Daemon:  1.15.3-rc2+mainnet+git.ddbcf1fc6+api1.5.0
Local: lotus-miner version 1.15.3-rc2+mainnet+git.ddbcf1fc6

Title	Metric
Miner	ID
Changing deal ask config from UI is working	[x]yes []no
Adding fund to market actor is working through UI	[x]yes []no
Deal transfer is working in UI for all deals	[x]yes []no
Publishing deals from UI is working	[x]yes []no
All the deal parameters and logs are available in deal detail page	[x]yes []no
All the legacy deals are available in the UI	[x]yes []no
Sealing Pipeline displays the correct stage for deal sealing	[x]yes []no
Success rate and number of boost deals made (Estuary, Bidbot, boost-client, etc)
Success rate and number of legacy deals made (Estuary, Bidbot, lotus-light, etc)
Hardware spec of Boost node	LXD container using 8Core/100GB (TS-h3088XU-RP-W1270-128G)
Average speed of transfers experienced previously with legacy markets (GraphSync)	30Mbps
Average speed of transfers on Boost	400Mbps
Network bandwidth	10Gbps
Number of deals initially migrated	>30000

[scaseye]

Title	Metric
Miner ID	f01611097
Changing deal ask config from UI is working	[x]yes []no
Adding fund to market actor is working through UI	[x]yes []no
Deal transfer is working in UI for all deals	[x]yes []no
Publishing deals from UI is working	[x]yes []no
All the deal parameters and logs are available in deal detail page	[x]yes []no
All the legacy deals are available in the UI	[x]yes []no
Sealing Pipeline displays the correct stage for deal sealing	[x]yes []no
Success rate and number of boost deals made (Estuary, Bidbot, boost-client, etc)	100%
Success rate and number of legacy deals made (Estuary, Bidbot, lotus-light, etc)	100%
Hardware spec of Boost node	Processor: 2x AMD EPYC 7H12, Memory: 2TB ,NVME drives
Average speed of transfers experienced previously with legacy markets (GraphSync)	<15Mbps
Average speed of transfers on Boost	>300Mbps
Network bandwidth	1Gbps
Number of deals initially migrated	7500

[scaseye]

Backup

• docs need to address what the recommended process/steps to create backups on a regular basis.

[scaseye]

running boostd version 0.0.1+git.762fecc now. will report any issues.

[scaseye]

I have only done SNAP deals so far. ~67 new boost, 300 legacy boost imports.

[scaseye]

first error boost deal

Error: failed to verify CommP: commP mismatch, expected=baga6ea4seaqcgrfziyhq7a4kafge76xbvwrncyfnm7acstcqfoyutxf3glrhedq, actual=baga6ea4seaqmfoyql7ukt7tbhosht2suxsr53uv3xqbmvvhfpxzy3rf7n6tswdy

[10d9e]

Hi @scaseye - we are seeing the Error: failed to verify CommP: commP mismatch, ... in Estuary lately. Just wondering if you are able to reliably reproduce the error with this version of boost?