Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop #1398

Merged
merged 78 commits into from
Oct 5, 2023
Merged

Develop #1398

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
e1ac0db
3911 // Make cypress ci tests easy to debug
Dinika Jun 26, 2023
155e8f9
3911 // Fix cypress tests by creating sesssions correctly
Dinika Jun 26, 2023
5fde41f
3911 // Modify cypress run in ci
Dinika Jun 27, 2023
59e6f26
Cypress tests should run without sudo permissions in ci
Dinika Jul 6, 2023
982ff7f
3911 // Tests run locally with localuser
Dinika Jul 6, 2023
6e199c3
Add cypress environment file
Dinika Jul 6, 2023
23c1222
3911 // Can be removed later - Simplify test assertions
Dinika Jul 6, 2023
cda3bfb
3911 // GitHub user should own the file
Dinika Jul 6, 2023
434d859
add debug logs
Dinika Jul 6, 2023
de4fda0
Run cypress container as user
Dinika Jul 6, 2023
f044c93
Use trigger click instead of click event
Dinika Jul 6, 2023
c8fa69d
Force click buttons and wait longer for the studio navigation
Dinika Jul 6, 2023
534cea5
Chown everything
Dinika Jul 6, 2023
f0b179a
1. Remove debug logs, dont copy into container, force click
Dinika Jul 6, 2023
d0a049d
f remove temo
Dinika Jul 6, 2023
b234d40
3911 // Add back analysis file
Dinika Jul 6, 2023
37b18db
wait for navaigation to studio
Dinika Jul 7, 2023
737c81d
3911 // Wait for redirection
Dinika Jul 7, 2023
f4bc900
3911 // hard wait
Dinika Jul 7, 2023
ef1f0b5
3911 // Run analysis plugin test also
Dinika Jul 7, 2023
6154160
3911 // Use latest chrome and cypress
Dinika Jul 7, 2023
79fa221
3911 // Run without recording on cypress cloud
Dinika Jul 7, 2023
3ca2605
Remove all cypress things owned by root
Dinika Jul 7, 2023
9ae807c
run as user 1000
Dinika Jul 7, 2023
3318ff1
3911 // Run tests by spec
Dinika Jul 7, 2023
16af98c
3911 // Run in headless mode in ci
Dinika Jul 7, 2023
eeff23e
3911 // no need to add user to docker compose instead pass user in exec
Dinika Jul 7, 2023
4d8d8ea
3911 // Remove chowing
Dinika Jul 7, 2023
740c350
3911 // Pass user id as env variabl to cypress container
Dinika Jul 7, 2023
dd47127
login to fix limit
Dinika Jul 7, 2023
d7dfd2b
3911 // Run without record key
Dinika Jul 7, 2023
53ba942
3911 // Run without user
Dinika Jul 7, 2023
129045c
3911 // Enable debug logs
Dinika Jul 7, 2023
4ef42ad
3911 // Remove cypress cache
Dinika Jul 7, 2023
24bd91e
3911 // Uninstall and reinstall cypress
Dinika Jul 7, 2023
a70066f
3911 // Inline script for running test
Dinika Jul 7, 2023
58b7dd4
3911 // Dont pass user to docker compose
Dinika Jul 7, 2023
25c0cc7
3911 // Remove recording
Dinika Jul 7, 2023
7cff20d
3911 // remove debug
Dinika Jul 7, 2023
fdbf79f
3911 // use script
Dinika Jul 7, 2023
a31b9e3
3911 // pass headless new as launcher option
Dinika Jul 8, 2023
bf36b74
3911 // increasr timeout remove wait
Dinika Jul 8, 2023
3d2e5e8
3911 // Increase timeout
Dinika Jul 8, 2023
54c129b
3911 // Run with github runner user
Dinika Jul 8, 2023
c4d5ce7
3911 // try removing wait
Dinika Jul 8, 2023
1ca9307
3911 // Check for dashboard item in list
Dinika Jul 8, 2023
b69ae5d
3911 // Simplify assertion before other one timesout
Dinika Jul 8, 2023
2d1e48d
3911 // Remove force clicks
Dinika Jul 8, 2023
a8bfd73
3911 // Inconsequential log
Dinika Jul 8, 2023
164fbec
3911 // Dont take screenshot on failure because it hangs cypress
Dinika Jul 8, 2023
acb8c17
3911 // Enable debugging
Dinika Jul 9, 2023
2d5b604
3911 // Disable sandbox to e2e tests stop running
Dinika Jul 9, 2023
e31a085
3911 // Disable uid because cypress hangs
Dinika Jul 9, 2023
bc989b8
3911 // copy contents to see if permissions issue is resolved
Dinika Jul 9, 2023
cd792db
3911 // Pass uid to exec to see logs
Dinika Jul 9, 2023
dc46d13
3911 // Pass user id in docker compose
Dinika Jul 9, 2023
4344b2d
3911 // Run without mounting volumes
Dinika Jul 9, 2023
d2845eb
3911 // Use yaml line folding
Dinika Jul 9, 2023
7fb4351
3911 // Decrease timeout
Dinika Jul 9, 2023
7df2f72
3911 // Add timeout to job
Dinika Jul 9, 2023
625dcec
3911 // Increase timeout to 50 sec
Dinika Jul 9, 2023
33af710
3911 // Bring back force clicks
Dinika Jul 9, 2023
e3ae826
3911 // Try using linux timeout command
Dinika Jul 9, 2023
2d660a6
3911 // dd verbose logs
Dinika Jul 9, 2023
fcec823
3911 // Remove old files
Dinika Jul 9, 2023
8a041e9
3911 // Remove changes that are not needed in source code
Dinika Jul 9, 2023
71aa1dc
3911 // Kill cypress if tests dont complete in 20 minutes
Dinika Jul 9, 2023
db7e5c1
3911 // Add documentation to run e2e tests
Dinika Jul 10, 2023
293b4c4
3911 // Revert change setting config
Dinika Jul 10, 2023
fb38164
3911 // Only push headless new if not headed
Dinika Jul 10, 2023
cefdde5
3911 // Headless local should run on same browser as headless ci(cy:ci)
Dinika Jul 11, 2023
fe24816
3911 // Let user choose browser when running cypress headless locally
Dinika Jul 11, 2023
797b53e
3911 // Remove test isolation
Dinika Sep 4, 2023
6f7fe77
4093 // Get exhaustive list of paths from graph analytics
Dinika Sep 1, 2023
8b819ef
4083 // Run predicates in backend
Dinika Sep 8, 2023
88f5a67
4289 // Add utility function for multi-get
Dinika Sep 22, 2023
d1cd709
Hotfix // Fix error when multi fetch is called with 0 resources
Dinika Oct 4, 2023
9e640fb
Merge branch 'main' into develop
bilalesi Oct 4, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 20 additions & 11 deletions .github/workflows/review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,10 @@ on: [pull_request]
jobs:
test-and-build:
runs-on: it

steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v3
with:
fetch-depth: 0
- uses: actions/setup-node@v3
with:
cache: 'yarn'
Expand All @@ -33,15 +34,23 @@ jobs:
- name: build fusion docker image
run: |
docker build . --tag=nexus-web:fresh
# TODO: The following have been commented out because cypress tests are failing in the CI. Uncomment these when 3911 is fixed.
# - name: Start services
# run: docker-compose -f ci/docker-compose.yml up -d && sleep 60
# - name: Copy nexus-web into Cypress container
# # avoids permission issue where cypress writes screenshots to host with root as user
# # which we can't then delete easily
# run: docker cp ./. cypress:/e2e
# - name: e2e tests
# run: echo | docker exec -e DEBUG=cypress:launcher:browsers -t cypress cypress run --config-file cypress.config.ts --browser chrome --record --key ${{ secrets.CYPRESS_RECORD_KEY }}
- name: Start services
run: docker-compose -f ci/docker-compose.yml up -d && sleep 60
- name: Copy nexus-web into Cypress container
# avoids permission issue where cypress writes screenshots to host with root as user
# which we can't then delete easily
run: docker cp ./. cypress:/e2e
- name: e2e tests
run: >-
echo | timeout --verbose 20m docker exec
-e 'DEBUG=cypress:launcher:browsers'
-t
cypress
cypress run
--config-file cypress.config.ts
--browser chrome
--record
--key ${{ secrets.CYPRESS_RECORD_KEY }}
- name: Cleanup Docker Containers
if: ${{ always() }}
run: docker-compose -f ci/docker-compose.yml down --rmi "local" --volumes
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,4 @@ cypress.env.json
*.sops*
.sops-*.*
*.key
*.asdcas
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,23 @@ Run unit tests:
yarn test
```

Run end to end tests:

1. Ensure you have the `cypress.env.json` file locally.
2.

a. To run the tests in headed mode:

```
yarn cy:open
```

b. To run the tests in headless mode:

```
yarn cy:run
```

## Build for production

Compile app in `dist/` folder.
Expand Down
2 changes: 1 addition & 1 deletion ci/config/delta-postgres.conf
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
app {
http {
base-uri = "http://delta:8080/v1"
base-uri = "http://delta:8098/v1"
interface = 0.0.0.0
}

Expand Down
15 changes: 10 additions & 5 deletions ci/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
version: '3.3'
services:
cypress:
image: 'cypress/included:10.3.1-typescript'
image: 'cypress/included:12.17.0'
depends_on:
- fusion
- delta
environment:
- CYPRESS_NEXUS_API_URL=http://delta.test:8080/v1
- CYPRESS_NEXUS_API_URL=http://delta.test:8098/v1
- CYPRESS_BASE_URL=http://fusion.test:8000

working_dir: /e2e
container_name: cypress
entrypoint: ['bash']
Expand All @@ -19,11 +18,13 @@ services:
- delta
environment:
ANALYSIS_PLUGIN_SHOW_ON_TYPES: 'Analysis,DetailedCircuit,AnalysisReport,Report'
API_ENDPOINT: 'http://delta.test:8080/v1'
API_ENDPOINT: 'http://delta.test:8098/v1'
CLIENT_ID: 'fusion'
image: nexus-web:fresh
expose:
- 8000
ports:
- 8000:8000
networks:
default:
aliases:
Expand Down Expand Up @@ -56,9 +57,12 @@ services:
'./bin/delta-app',
'-Xmx4G',
'-Dapp.cluster.remote-interface=delta',
'-Dapp.http.port=8098',
]
expose:
- 8080
- 8098
ports:
- 8098:8098
volumes:
- ./config:/config
- /tmp:/default-volume
Expand All @@ -79,6 +83,7 @@ services:
- --hostname-port=8080
- --hostname-strict=false
- --hostname-strict-backchannel=false
- --log-level=DEBUG
ports:
- 8080:8080
networks:
Expand Down
21 changes: 16 additions & 5 deletions cypress.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import {
createNexusOrgAndProject,
createResource,
deprecateNexusOrgAndProject,
} from './cypress/plugins/nexus';
} from './cypress/support/Utils/nexus';
import { uuidv4 } from './src/shared/utils';
import setup, { TestUsers } from './cypress/support/setupRealmsAndUsers';

Expand All @@ -14,23 +14,34 @@ export default defineConfig({
projectId: '1iihco',
viewportWidth: 1200,
video: true,
screenshotOnRunFailure: false,
e2e: {
baseUrl: 'http://localhost:8000',
fileServerFolder: '/cypress',
defaultCommandTimeout: 50000,
specPattern: 'cypress/e2e/**/*.cy.{js,jsx,ts,tsx}',
// @ts-ignore
experimentalSessionAndOrigin: true,
// testIsolation: false,
chromeWebSecurity: false,

env: {
DEBUG: 'cypress:launcher:browsers',
ELECTRON_DISABLE_GPU: 'true',
ELECTRON_EXTRA_LAUNCH_ARGS: '--disable-gpu',
},
setupNodeEvents(on, config) {
on('before:browser:launch', (browser, launchOptions) => {
launchOptions.args.push('--window-size=1920,1080');
if (browser.name == 'chrome') {
launchOptions.args.push('--disable-gpu');
launchOptions.args.push('--no-sandbox');

launchOptions.args.push('--disable-web-security');
launchOptions.args.push(
'--unsafely-treat-insecure-origin-as-secure=http://keycloak.test:8080'
);
if (!browser.isHeaded) {
console.log('Pushing Headless New');
launchOptions.args.push('--headless=new');
}
}
return launchOptions;
}),
Expand Down Expand Up @@ -65,7 +76,7 @@ export default defineConfig({
const projectLabel = `${projectLabelBase}-${uuidv4()}`;
const projectDescription =
'An project used for Cypress automated tests';

console.log('Auth TOKEN', authToken);
try {
const nexus = createNexusClient({
uri: nexusApiUrl,
Expand Down
22 changes: 11 additions & 11 deletions cypress/e2e/Studios.cy-ignore.ts → cypress/e2e/Studios.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,14 @@ describe('Studios', () => {
studioDetailsPage = new StudioDetailsPage();
});

// after(function() {
// cy.task('project:teardown', {
// nexusApiUrl: Cypress.env('NEXUS_API_URL'),
// authToken: this.nexusToken,
// orgLabel: Cypress.env('ORG_LABEL'),
// projectLabel: this.projectLabel,
// });
// });
after(function() {
cy.task('project:teardown', {
nexusApiUrl: Cypress.env('NEXUS_API_URL'),
authToken: this.nexusToken,
orgLabel: Cypress.env('ORG_LABEL'),
projectLabel: this.projectLabel,
});
});

it('user can create a studio with a workspace and dashboard', function() {
cy.visit(
Expand All @@ -70,9 +70,9 @@ describe('Studios', () => {

cy.findByText('Enable Filter').click();

return cy.findByRole('button', { name: /Save/ }).click();
})
.then(() => {
cy.findByRole('button', { name: /Save/ }).click({ force: true });
// cy.wait('@saveDashboardRequest');

studioDetailsPage.openEditDashboard();

cy.findByLabelText(/Enable Filter/i).should('be.checked');
Expand Down
20 changes: 11 additions & 9 deletions cypress/support/Studios/StudioDetails.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ export class StudioDetailsPage extends StudioListPage {
}

openEditDashboard() {
cy.findByRole('button', { name: /Dashboard/ }).click();
cy.findByRole('button', { name: /Dashboard/ }).click({ force: true });
cy.get('button[data-testid="edit-dashboard"]')
.contains(/Edit/)
.click();
.click({ force: true });
cy.findByRole('dialog').findByText('Edit');
}

Expand All @@ -42,14 +42,16 @@ export class StudioDetailsPage extends StudioListPage {
cy.findByRole('button', { name: /Add/ }).click();
cy.findByPlaceholderText(/Name/i).type(dashboardName);
cy.findByRole('combobox', { name: /View/ }).click();
cy.findByTitle(
'https://bluebrain.github.io/nexus/vocabulary/defaultSparqlIndex'
).click();
cy.get(
'[data-testid="https://bluebrain.github.io/nexus/vocabulary/defaultSparqlIndex"]'
);
cy.get(
'[data-testid="https://bluebrain.github.io/nexus/vocabulary/defaultSparqlIndex"]'
).trigger('click');

cy.findByRole('checkbox', { name: /Enable Sort/i }).click();
cy.findByRole('button', { name: /Save/ }).click();
cy.findByRole('menuitem', { name: new RegExp(workspaceName, 'i') }).click();
cy.get('ul')
.contains(new RegExp(dashboardName, 'i'))
.click();
console.log('Saved');
cy.findByText(new RegExp(dashboardName, 'i')).should('exist');
}
}
1 change: 0 additions & 1 deletion cypress/support/Studios/StudioList.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ export class StudioListPage {
cy.findByRole('button', { name: /Create Studio/i }).click();
cy.findByRole('textbox', { name: /Label/ }).type(name);
cy.findByRole('button', { name: /Save/ }).click();

const studioElement = cy.findByRole('heading', { name });
studioElement.contains(name);
return studioElement;
Expand Down
67 changes: 67 additions & 0 deletions cypress/support/Utils/nexus.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import { NexusClient, ResourcePayload } from '@bbp/nexus-sdk';

export const createNexusOrgAndProject = async ({
nexus,
orgLabel,
orgDescription,
projectLabel,
projectDescription,
}: {
nexus: NexusClient;
orgLabel: string;
orgDescription: string;
projectLabel: string;
projectDescription: string;
}) => {
// Create Org if does not exist
try {
const org = await nexus.Organization.get(orgLabel);

if (org._deprecated) {
throw new Error('Org deprecated');
}
} catch (e) {
await nexus.Organization.create(orgLabel, {
description: orgDescription,
});
}

// Create project if does not exist
try {
await nexus.Project.get(orgLabel, projectLabel);
} catch (e) {
await nexus.Project.create(orgLabel, projectLabel, {
description: projectDescription,
});
}
};

export const deprecateNexusOrgAndProject = async ({
nexus,
orgLabel,
projectLabel,
}: {
nexus: NexusClient;
orgLabel: string;
projectLabel: string;
}) => {
// Deprecate project
try {
const project = await nexus.Project.get(orgLabel, projectLabel);
await nexus.Project.deprecate(orgLabel, projectLabel, project._rev);
} catch (e) {
console.error('Encountered an error whilst trying to deprecate project', e);
}
};

export const createResource = async ({
nexus,
orgLabel,
projectLabel,
resource,
}: {
nexus: NexusClient;
orgLabel: string;
projectLabel: string;
resource: ResourcePayload;
}) => await nexus.Resource.create(orgLabel, projectLabel, resource);
2 changes: 1 addition & 1 deletion cypress/support/setupRealmsAndUsers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ const fetch = require('node-fetch');
const fs = require('fs');

const keycloakUrl = 'http://keycloak.test:8080';
const deltaBaseUrl = 'http://delta.test:8080/v1';
const deltaBaseUrl = 'http://delta.test:8098/v1';

// /* these are used for administrating keycloak */
const keycloakAdmin = {
Expand Down
Loading