diff --git a/README.md b/README.md index 50fa7e12..0fa0379f 100644 --- a/README.md +++ b/README.md @@ -6,20 +6,30 @@ # Repository with E2E tests for Summer.fi DeFi web application -## Introduction +1. [ Introduction ](#1-introduction) +2. [ Installation ](#2-installation) +3. [ Environment variables](#3-environment-variables) +4. [ Browsers ](#4-browsers) +5. [ Running the tests ](#5-running-the-tests) +6. [ Test reports ](#6-test-reports) +7. [ Tags ](#7-tags) +8. [ Projects ](#8-projects) +9. [ License ](#9-license) + +## 1. Introduction This repository contains a repository of tests for the e2e testing of the Summer.fi DeFi web application. There are two main categories of e2e tests: - **No wallet**: Tests without a wallet connected. - **With wallet**: -Tests with a wallet (and a Tenderly fork) connected. +Tests with a wallet connected (Metamask) and, in most of the cases, with a Tenderly fork. The tests and framework have been written using Playwright and Typescript. -- These 'No wallet' tests use Playwright only. +- The 'No wallet' tests use Playwright only. - The 'With wallet' tests use Playwright, Synpress (for Metamask wallet) and Tenderly APIs (for forks). -## Installation +## 2. Installation First, install the programs required to run the application: - [Node.js](https://nodejs.org/en/download/) @@ -36,17 +46,13 @@ yarn install ``` -## Environment variables +## 3. Environment variables To run the tests locally you will need to create a `.env` file. You can create a copy from `.env.example` file by running: ``` cp .env.example .env ``` -The following environment variable is **mandatory** for both all tests: - -| Variable | Description | -| --- | --- | -| BASE_URL | This is the base url used to run the tests. As per `.env.example` file, you'll probably use `https://staging.summer.fi` as base url| +There aren't any mandatory environment variables for **no wallet** tests. The following environment variables are **mandatory** for **with wallet** tests: @@ -61,25 +67,25 @@ You can also use these other environment variables: | Variable | Default | Description | | --- | --- | -- | +| BASE_URL | This is the base url used to run the tests. As per `.env.example` file. If you don't set this variable the tests will run with `https://staging.summer.fi` as base url| | FULLY_PARALLEL | false | Playwright Test runs tests in parallel. In order to achieve that, it runs several worker processes that run at the same time. By default, **test files** are run in parallel. However, tests in a single file are run in order, in the same worker process. You can configure entire test run to concurrently execute all tests in all files using this option. Note that **with wallet** tests can run **only** serial mode, so this variable will be applied only to **no wallet tests**. | | WORKERS | 1 | Number of workers used to run the tests. To run the tests in pralallel you need to use more than 1 workers. Note that **with wallet** tests can run **only** with 1 worker, so this variable will be applied only to **no wallet tests**. | | TIMEOUT | 30000 | Timeout for each test in milliseconds. Note that some timeouts have been defined in `./utils/config.ts` file, so this TIMEOUT variable might not have any effect in some cases. | -| RETRIES | 0 | The maximum number of retry attempts given to failed tests. By default failing tests are not retried. | +| RETRIES | 2 | The maximum number of retry attempts given to failed tests. By default failing tests are not retried. | | HEADLESS | true | Whether to run browser in headless mode. Note that **with wallet** tests can run **only** in headed mode, so this variable will be applied only to **no wallet tests**. | | FLAGS | '' | You can pass one or more flags to enable hidden features in staging and dev environments. To pass more than one flag separate them with an empty space: `'flag1:true flag2:false'` | -## Browsers +## 4. Browsers For now the tests have been setup so that they run only on desktop Chrome. -## Running the tests +## 5. Running the tests **Note** that in oder to considerably reduce the test execution time, several scripts have been created so that **with wallet** tests can be run in parallel jobs (docker containers) in GitHub actions. 1. Create `.env` file. -2. Set `BASE_URL` in `.env` file. For example, `BASE_URL='https://staging.summer.fi'`. -For running **with wallet** tests also set `WALLET_ADDRESS`, `TENDERLY_PROJECT`, `TENDERLY_USER` and `TENDERLY_ACCESS_KEY` variables. +2. For running **with wallet** tests, set `WALLET_ADDRESS`, `TENDERLY_PROJECT`, `TENDERLY_USER` and `TENDERLY_ACCESS_KEY` variables. -3. Run tests by running: +3. Run tests by running, for example: - Run all tests: `yarn test:e2e` - Run all **regression** tests: @@ -93,19 +99,26 @@ For running **with wallet** tests also set `WALLET_ADDRESS`, `TENDERLY_PROJECT`, - Run **with wallet regression** tests: `yarn with-wallet:all:regression` -## Running the tests report -After running one or more tests you can run the following commands to open one or more test reports in the browser: +Check all the scripts in [ package.json ](https://github.com/OasisDEX/e2e-tests/blob/main/package.json) file + +## 6. Test reports +> The test reports for the test runs executed in gitHub can be found in https://github.com/OasisDEX/e2e-tests/actions. +> 1. Click on the test run you wish to see the report for. +> 2. Scroll down to the **Artifacts** section and click on the report name you wish to download. +> 3. Unzip the report file you just downloaded and double click on the **index.html** file to open the report in the browser. + +After running one or more tests locally you can run the following commands to open test reports in the browser: - `yarn test:report:no-wallet` - `yarn test:report:with-wallet:aave:ethereum` - `yarn test:report:with-wallet:aave:other` - `yarn test:report:with-wallet:maker-and-spark` -## Tags -Tests can be tagged so that we can only run the tests that have the certain tag. +## 7. Tags +Tests can be tagged so that we can run only the tests that have the certain tag. At the moment we are using the following tags: - `@regression`: Add this tag to all the tests that should be part of the regression test suite. -## Projects +## 8. Projects A project is a logical group of tests running with the same configuration. For now we are using projects so that we can achieve the two following goals: - Applying different configuration to **no wallet** and **with wallet** tests. - Splitting **with wallet** tests so that they can run in parallel containers, and so reducing significantly the test execution time. @@ -116,7 +129,7 @@ At the moment we are using the following projects: - `with-wallet-aave-ethereum` - `with-wallet-maker-and-spark` -## License +## 9. License Copyright (C) 2021 Oazo Apps Limited, Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy diff --git a/tests/noWallet/aaveV3/base/aaveV3EarnBase.spec.ts b/tests/noWallet/aaveV3/base/aaveV3EarnBase.spec.ts index b6f29cff..808e8e99 100644 --- a/tests/noWallet/aaveV3/base/aaveV3EarnBase.spec.ts +++ b/tests/noWallet/aaveV3/base/aaveV3EarnBase.spec.ts @@ -79,7 +79,7 @@ test.describe('Aave v3 Earn Base', async () => { }); }); - test('It should allow to simulate an Aave V3 Base Earn position before opening it - Adjust risk - Up and Down - No wallet connected @regression', async ({ + test('It should allow to simulate an Aave V3 Base Earn position before opening it - Adjust risk - Up and Down - No wallet connected @regression', async ({ app, }) => { test.info().annotations.push({ @@ -111,7 +111,7 @@ test.describe('Aave v3 Earn Base', async () => { exactAmount: true, }); await app.position.setup.shouldHaveLiquidationPrice({ - amount: '[0-3].[0-9]{3,4} CBETH', + amount: '[0-5].[0-9]{3,4} CBETH', }); await app.position.overview.shouldHaveLiquidationPriceAfterPill('[0-3].[0-9]{2}'); @@ -129,10 +129,10 @@ test.describe('Aave v3 Earn Base', async () => { exactAmount: true, }); await app.position.setup.shouldHaveLiquidationPrice({ - amount: '[2-6].[0-9]{3,4} CBETH', + amount: '[0-9]{1,2}.[0-9]{1,2}([0-9]{1,2})? CBETH', }); - await app.position.overview.shouldHaveLiquidationPriceAfterPill('[2-6].[0-9]{2}'); + await app.position.overview.shouldHaveLiquidationPriceAfterPill('[0-9]{1,2}.[0-9]{2}'); const updatedLiqPrice2 = await app.position.manage.getLiquidationPrice(); const updatedLoanToValue2 = await app.position.manage.getLoanToValue(); expect(updatedLiqPrice2).toBeGreaterThan(updatedLiqPrice); diff --git a/tests/withWallet/aaveV3/ethereum/aaveV3EarnEthereum.spec.ts b/tests/withWallet/aaveV3/ethereum/aaveV3EarnEthereum.spec.ts index 399609c7..112aeed4 100644 --- a/tests/withWallet/aaveV3/ethereum/aaveV3EarnEthereum.spec.ts +++ b/tests/withWallet/aaveV3/ethereum/aaveV3EarnEthereum.spec.ts @@ -83,7 +83,7 @@ test.describe('Aave V3 Earn - Ethereum - Wallet connected', async () => { await app.position.manage.ok(); await app.position.overview.shouldHaveNetValue({ - value: '[0-9]{2}.[0-9]{2}ETH', + value: '[0-9]{2}(.[0-9]{1,2})?ETH', }); await app.position.overview.shouldHaveTotalCollateral({ amount: '20.00000', token: 'WSTETH' }); }); diff --git a/tests/withWallet/spark/sparkBorrow.spec.ts b/tests/withWallet/spark/sparkBorrow.spec.ts index b484c223..f638d3ad 100644 --- a/tests/withWallet/spark/sparkBorrow.spec.ts +++ b/tests/withWallet/spark/sparkBorrow.spec.ts @@ -30,10 +30,10 @@ test.describe('Spark Borrow - Wallet connected', async () => { await resetState(); }); - test('It should adjust risk of an existent Spark Borrow position - Up @regression', async () => { + test('It should deposit extra collateral on an existent Spark Borrow position @regression', async () => { test.info().annotations.push({ type: 'Test case', - description: '13050', + description: '11406', }); test.setTimeout(extremelyLongTestTimeout); @@ -46,165 +46,6 @@ test.describe('Spark Borrow - Wallet connected', async () => { ({ forkId, walletAddress } = await setup({ app, network: 'mainnet' })); }); - await tenderly.changeAccountOwner({ - account: '0x3dc0f12ff0452cab029c3c185c9dc9061d1515c8', - newOwner: walletAddress, - forkId, - }); - - await app.page.goto('/ethereum/spark/v3/1474#overview'); - await app.position.manage.shouldBeVisible('Manage collateral'); - await app.position.manage.openManageOptions({ currentLabel: 'Manage ETH' }); - await app.position.manage.select('Adjust'); - - const initialLiqPrice = await app.position.manage.getLiquidationPrice(); - const initialLoanToValue = await app.position.manage.getLoanToValue(); - - await app.position.manage.waitForSliderToBeEditable(); - await app.position.manage.moveSlider({ value: 0.6 }); - - await app.position.manage.adjustRisk(); - - // Confirm action randomly fails - Retry until it's applied. - await expect(async () => { - await app.position.setup.confirmOrRetry(); - await test.step('Metamask: ConfirmPermissionToSpend', async () => { - await metamask.confirmPermissionToSpend(); - }); - await app.position.manage.shouldShowSuccessScreen(); - }).toPass({ timeout: longTestTimeout }); - - await app.position.manage.ok(); - - await app.position.manage.shouldBeVisible('Manage collateral'); - await app.position.manage.openManageOptions({ currentLabel: 'Manage ETH' }); - await app.position.manage.select('Adjust'); - const updatedLiqPrice = await app.position.manage.getLiquidationPrice(); - const updatedLoanToValue = await app.position.manage.getLoanToValue(); - expect(updatedLiqPrice).toBeGreaterThan(initialLiqPrice); - expect(updatedLoanToValue).toBeGreaterThan(initialLoanToValue); - }); - - test('It should adjust risk of an existent Spark Borrow position - Down @regression', async () => { - test.info().annotations.push({ - type: 'Test case', - description: '13051', - }); - - test.setTimeout(veryLongTestTimeout); - - await app.position.manage.shouldBeVisible('Manage Borrow position'); - const initialLiqPrice = await app.position.manage.getLiquidationPrice(); - const initialLoanToValue = await app.position.manage.getLoanToValue(); - - await app.position.manage.waitForSliderToBeEditable(); - await app.position.manage.moveSlider({ value: 0.3 }); - - await app.position.manage.adjustRisk(); - - // Confirm action randomly fails - Retry until it's applied. - await expect(async () => { - await app.position.setup.confirmOrRetry(); - await test.step('Metamask: ConfirmPermissionToSpend', async () => { - await metamask.confirmPermissionToSpend(); - }); - await app.position.manage.shouldShowSuccessScreen(); - }).toPass({ timeout: longTestTimeout }); - - await app.position.manage.ok(); - - await app.position.manage.shouldBeVisible('Manage collateral'); - await app.position.manage.openManageOptions({ currentLabel: 'Manage ETH' }); - await app.position.manage.select('Adjust'); - const updatedLiqPrice = await app.position.manage.getLiquidationPrice(); - const updatedLoanToValue = await app.position.manage.getLoanToValue(); - - expect(updatedLiqPrice).toBeLessThan(initialLiqPrice); - expect(updatedLoanToValue).toBeLessThan(initialLoanToValue); - }); - - test('It should close an existent Spark Borrow position - Close to debt token (DAI) @regression', async () => { - test.info().annotations.push({ - type: 'Test case', - description: '13052', - }); - - test.setTimeout(veryLongTestTimeout); - - await app.position.manage.shouldBeVisible('Manage Borrow position'); - await app.position.manage.openManageOptions({ currentLabel: 'Adjust' }); - await app.position.manage.select('Close position'); - await app.position.manage.closeTo('DAI'); - await app.position.manage.shouldHaveTokenAmountAfterClosing({ - token: 'DAI', - amount: '[0-3].[0-9]{3,4}', - }); - - // Confirm action randomly fails - Retry until it's applied. - await expect(async () => { - await app.position.setup.confirmOrRetry(); - await test.step('Metamask: ConfirmPermissionToSpend', async () => { - await metamask.confirmPermissionToSpend(); - }); - await app.position.manage.shouldShowSuccessScreen(); - }).toPass({ timeout: longTestTimeout }); - - await app.position.manage.ok(); - - await app.page.goto('/ethereum/spark/v3/1474#overview'); - await app.position.overview.shouldHaveLiquidationPrice({ price: '0.00', token: 'DAI' }); - await app.position.overview.shouldHaveLoanToValue('0.00'); - await app.position.overview.shouldHaveBorrowCost('0.00'); - await app.position.overview.shouldHaveNetValue({ value: '0.00', token: 'DAI' }); - await app.position.overview.shouldHaveExposure({ amount: '0.00000', token: 'ETH' }); - await app.position.overview.shouldHaveDebt({ amount: '0.0000', token: 'DAI' }); - }); - - test('It should open a Spark Borrow position @regression', async () => { - test.info().annotations.push({ - type: 'Test case', - description: '11811', - }); - - test.skip(baseUrl.includes('staging') || baseUrl.includes('//summer.fi')); - - test.setTimeout(extremelyLongTestTimeout); - - await app.page.goto('/ethereum/spark/v3/borrow/ethdai#simulate'); - // Depositing collateral too quickly after loading page returns wrong simulation results - await app.position.overview.waitForComponentToBeStable(); - await app.position.setup.deposit({ token: 'ETH', amount: '3' }); - await app.position.setup.borrow({ token: 'DAI', amount: '1' }); - await app.position.setup.createSmartDeFiAccount(); - // Confirmation button with same label - await app.position.setup.createSmartDeFiAccount(); - await test.step('Metamask: ConfirmAddToken', async () => { - await metamask.confirmAddToken(); - }); - await app.position.setup.continue(); - await app.position.setup.openBorrowPosition1Of2(); - - // Position creation randomly fails - Retry until it's created. - await expect(async () => { - await app.position.setup.confirmOrRetry(); - await test.step('Metamask: ConfirmPermissionToSpend', async () => { - await metamask.confirmPermissionToSpend(); - }); - await app.position.setup.goToPositionShouldBeVisible(); - }).toPass({ timeout: longTestTimeout }); - - await app.position.setup.goToPosition(); - await app.position.manage.shouldBeVisible('Manage '); - }); - - test('It should deposit extra collateral on an existent Spark Borrow position @regression', async () => { - test.info().annotations.push({ - type: 'Test case', - description: '11406', - }); - - test.setTimeout(extremelyLongTestTimeout); - // New fork needed await test.step('Test setup - New fork', async () => { ({ forkId } = await setupNewFork({ app, network: 'mainnet' })); @@ -465,6 +306,87 @@ test.describe('Spark Borrow - Wallet connected', async () => { await app.position.overview.shouldHaveDebt({ amount: '0.0000', token: 'DAI' }); }); + test('It should close an existent Spark Borrow position - Close to debt token (DAI) @regression', async () => { + test.info().annotations.push({ + type: 'Test case', + description: '13052', + }); + + test.setTimeout(veryLongTestTimeout); + + await tenderly.changeAccountOwner({ + account: '0x3dc0f12ff0452cab029c3c185c9dc9061d1515c8', + newOwner: walletAddress, + forkId, + }); + + await app.page.goto('/ethereum/spark/v3/1474#overview'); + await app.position.manage.shouldBeVisible('Manage Collateral'); + await app.position.manage.openManageOptions({ currentLabel: 'Manage ETH' }); + await app.position.manage.select('Close position'); + await app.position.manage.closeTo('DAI'); + await app.position.manage.shouldHaveTokenAmountAfterClosing({ + token: 'DAI', + amount: '[0-3].[0-9]{3,4}', + }); + + // Confirm action randomly fails - Retry until it's applied. + await expect(async () => { + await app.position.setup.confirmOrRetry(); + await test.step('Metamask: ConfirmPermissionToSpend', async () => { + await metamask.confirmPermissionToSpend(); + }); + await app.position.manage.shouldShowSuccessScreen(); + }).toPass({ timeout: longTestTimeout }); + + await app.position.manage.ok(); + + await app.page.goto('/ethereum/spark/v3/1474#overview'); + await app.position.overview.shouldHaveLiquidationPrice({ price: '0.00', token: 'DAI' }); + await app.position.overview.shouldHaveLoanToValue('0.00'); + await app.position.overview.shouldHaveBorrowCost('0.00'); + await app.position.overview.shouldHaveNetValue({ value: '0.00', token: 'DAI' }); + await app.position.overview.shouldHaveExposure({ amount: '0.00000', token: 'ETH' }); + await app.position.overview.shouldHaveDebt({ amount: '0.0000', token: 'DAI' }); + }); + + test('It should open a Spark Borrow position @regression', async () => { + test.info().annotations.push({ + type: 'Test case', + description: '11811', + }); + + test.skip(baseUrl.includes('staging') || baseUrl.includes('//summer.fi')); + + test.setTimeout(extremelyLongTestTimeout); + + await app.page.goto('/ethereum/spark/v3/borrow/ethdai#simulate'); + // Depositing collateral too quickly after loading page returns wrong simulation results + await app.position.overview.waitForComponentToBeStable(); + await app.position.setup.deposit({ token: 'ETH', amount: '3' }); + await app.position.setup.borrow({ token: 'DAI', amount: '1' }); + await app.position.setup.createSmartDeFiAccount(); + // Confirmation button with same label + await app.position.setup.createSmartDeFiAccount(); + await test.step('Metamask: ConfirmAddToken', async () => { + await metamask.confirmAddToken(); + }); + await app.position.setup.continue(); + await app.position.setup.openBorrowPosition1Of2(); + + // Position creation randomly fails - Retry until it's created. + await expect(async () => { + await app.position.setup.confirmOrRetry(); + await test.step('Metamask: ConfirmPermissionToSpend', async () => { + await metamask.confirmPermissionToSpend(); + }); + await app.position.setup.goToPositionShouldBeVisible(); + }).toPass({ timeout: longTestTimeout }); + + await app.position.setup.goToPosition(); + await app.position.manage.shouldBeVisible('Manage '); + }); + test.skip('It should list an opened Spark Borrow position in portfolio', async () => { test.info().annotations.push( {