Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Working Draft] Introduce Playwright test framework #368

Closed
wants to merge 50 commits into from
Closed

[Working Draft] Introduce Playwright test framework #368

wants to merge 50 commits into from

Conversation

terryzfeng
Copy link
Collaborator

@terryzfeng terryzfeng commented Jun 11, 2024 •
edited
Loading

Preparing Playwright Test Framework for AudioWorklet Testing

Tests written here src/tests/playwright (.spec.ts)
Test pages are in src/tests/playwrights/pages

More documentation is written here
https://docs.google.com/document/d/1e7G9jc4kuVZWvjylTtFE76DzQBsOG6lZSFmV19Kc5cY/edit#heading=h.wb3akxg1nco

@terryzfeng
Copy link
Collaborator Author

terryzfeng commented Jun 11, 2024 •
edited
Loading

I haven't yet figured out how to manage multiple versions of Chromium through Playwright, just have the latest Chromium build I believe.

@terryzfeng terryzfeng mentioned this pull request Jun 11, 2024
Closed
@terryzfeng terryzfeng linked an issue Jun 11, 2024 that may be closed by this pull request
Research various JS testing frameworks #365
Closed
Kizjkre and others added 5 commits June 25, 2024 17:26
@Kizjkre @terryzfeng
Create bufferinator and audio graph cache
0c41e3c 
Co-authored-by: terryzfeng <terryzfeng@users.noreply.github.com>
@Kizjkre
Change default exports to named exports
4ddc7e9
@terryzfeng
bump playwright version
0c9e322
@terryzfeng @Kizjkre
implement library functions for dsp graph building and realtime and o…
055cd61 
…ffline audio buffer comparison

Co-authored-by: kizjkre <kizjkre@users.noreply.github.com>
@terryzfeng
feat: add playwright multiple chromium version support
6ee4009 
feat: add beCloseTo impl

feat: impl buffer comparison and graph evaluation

refactor: realtimeSine test, recorder-main, etc.

test: add dsp graph eval test
@Kizjkre @terryzfeng
Fix bug where DSP graph evaluation could only be run once
21cbcac 
Co-authored-by: terryzfeng <terryzfeng@users.noreply.github.com>
@mjwilson-google mjwilson-google added the gsoc24 Google Summer of Code 2024 label Jun 28, 2024
Kizjkre and others added 13 commits July 2, 2024 15:45
@Kizjkre
Integrate evaluation function into live suite
5811046
@terryzfeng
refactor realtime-sine test as test example
126c6f1
@terryzfeng @Kizjkre
Small live suite fixes
c54cd52 
Co-authored-by: Kizjkre <kizjkre@users.noreply.github.com>
@Kizjkre
Capture console output in live suite
36f7b81
@Kizjkre
Add % match and tolerance to realtime sine
55bfdcf
@Kizjkre
Add assert functionality
681466a
@terryzfeng @Kizjkre
playwright test suite refactor and live suite implementation
fa96163 
Co-authored-by: Kizjkre <kizjkre@users.noreply.github.com>
@terryzfeng
Port gain perf test and optimize CI
d0255be 
Co-authored-by: Kizjkre <kixjkre@users.noreply.github.com>
@terryzfeng @Kizjkre
Port gain perf test and optimize CI
fe8120d 
Co-authored-by: Kizjkre <kizjkre@users.noreply.github.com>
@Kizjkre @terryzfeng
Increase tolerance
5e8e33d 
Co-authored-by: terryzfeng <terryzfeng@users.noreply.github.com>
@Kizjkre
Merge remote-tracking branch 'origin/playwright-test' into playwright…
7380986 
…-test
@terryzfeng
lower threshold
16b13fc
@terryzfeng
update record impl to record exact number of samples
c4df71c
hoch
hoch reviewed Jul 8, 2024
View reviewed changes
Copy link
Member

@hoch hoch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great in general! I glanced 2/3 of files in this PR.

One recommendation - many comments and many files usually mean that we need to narrow the scope of the PR. Can we keep this PR for the iteration, and tease out a small part of this and focus on it? That will definitely make the review/landing faster.

.github/workflows/playwright.yml Show resolved Hide resolved
.gitignore Show resolved Hide resolved
playwright.config.ts Outdated Show resolved Hide resolved
playwright.config.ts Outdated Show resolved Hide resolved
src/tests/playwright/pages/dsp-graph-evaluation.html Outdated Show resolved Hide resolved
src/tests/playwright/pages/perf-gain.html Outdated Show resolved Hide resolved
src/tests/playwright/pages/processors/recorder/recorder-main.js Outdated Show resolved Hide resolved
src/tests/playwright/pages/perf-gain.html Outdated Show resolved Hide resolved
src/tests/playwright/pages/perf-gain.html Outdated Show resolved Hide resolved
src/tests/playwright/pages/processors/recorder/recorder-processor.js Outdated Show resolved Hide resolved
@terryzfeng
Copy link
Collaborator Author

This looks great in general! I glanced 2/3 of files in this PR.

One recommendation - many comments and many files usually mean that we need to narrow the scope of the PR. Can we keep this PR for the iteration, and tease out a small part of this and focus on it? That will definitely make the review/landing faster.

I totally agree that this PR is much too big and quite complex. Thank you for taking the time to look at so much of it! I'm happy to remove extraneous features/functionality from this MVP. What's the best way to tease out a small part of this? With that, I think we would be able to wrap it up nicely this week!

@hoch
Copy link
Member

hoch commented Jul 8, 2024

For the first PR, how about focusing on src/tests/playwright/pages/realtime-sine.html and dependencies for it?

terryzfeng and others added 9 commits July 8, 2024 16:46
@terryzfeng
update record logic and style and add documentation
bd0360f
@Kizjkre @terryzfeng
PR review fixes
eca95f1 
Co-authored-by: terryzfeng <terryzfeng@users.noreply.github.com>
@terryzfeng @Kizjkre
Port gain perf test and optimize CI
eaaef2d 
Co-authored-by: Kizjkre <kizjkre@users.noreply.github.com>
@terryzfeng @Kizjkre
lower threshold
986691c
@terryzfeng @Kizjkre
update record impl to record exact number of samples
11cbfdd
@terryzfeng @Kizjkre
update record logic and style and add documentation
676d2b0
@Kizjkre @terryzfeng
PR review fixes
51409a5 
Co-authored-by: terryzfeng <terryzfeng@users.noreply.github.com>
@Kizjkre
Merge branch 'playwright-test' of https://github.com/GoogleChromeLabs…
a3bcd9e 
…/web-audio-samples into playwright-test
@Kizjkre @terryzfeng
Small fix
8a51198 
Co-authored-by: terryzfeng <terryzfeng@users.noreply.github.com>
@Kizjkre Kizjkre marked this pull request as draft July 10, 2024 00:36
@hoch hoch changed the title Create Playwright Test Framework [Working Draft] Introduce Playwright test framework Jul 10, 2024
@Kizjkre Kizjkre closed this Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hoch hoch hoch left review comments

@Kizjkre Kizjkre Awaiting requested review from Kizjkre

@mjwilson-google mjwilson-google Awaiting requested review from mjwilson-google

Assignees

@Kizjkre Kizjkre

@terryzfeng terryzfeng

Labels
gsoc24 Google Summer of Code 2024
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Research various JS testing frameworks
4 participants
@terryzfeng @hoch @Kizjkre @mjwilson-google