build: add alternative build process to enable faster developer builds

[davidmurdoch]

Closes https://github.com/MetaMask/MetaMask-planning/issues/1477 and https://github.com/MetaMask/MetaMask-planning/issues/1903

Prereq PR: #25456

About this PR

This PR adds an alternative build process that is much faster than the gulp build we have now, which is quite slow and doesn't make use of modern build system improvements. The speed up is possible by making use of the SWC compiler, and more modern build system. The build system is also a bit simpler and hopefully more maintainable.

This build doesn't yet support:

• HMR/chromereload (requires we get rid of our circular dependencies)
• lavamoat (neither running the build system in lavamoat, or adding lavamoat protections)
• production builds (because of not supporting lavamoat)
• MV3 (requires writing a webpack plugin)

Proposed Review Tactics

There is a lot to review in this PR. Depending on your own review methodologies, you might like the idea of breaking up review into several review passes, each one in a different categories/tactics. Some useful categories may cover things like:

• Style (eslint customizations, exports, imports, use of require)
• Architecture & Organization (naming, files and folder location, code separation)
• Accuracy (are there bugs, unhandled exceptions, etc?)
• Comments, Documentation, etc. (typos, inaccuracies, missing information?)
• API (the new cli as well as yarn scripts)
• Test Coverage and Test Organization, CI decisions (are tests lacking, are tests themselves maintainable and extensible, are there any unnecessary or missing tests?)

What's bad and should be discussed:

• We now have to maintain 2 build systems

Linking the test ticket here https://github.com/MetaMask/MetaMask-planning/issues/2705

[github-actions]

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

[socket-security]

🚨 Potential security issues detected. Learn more about Socket for GitHub ↗︎

To accept the risk, merge this PR and you will not be notified again.

Alert	Package	Note	Source	CI
Network access	npm/http-proxy-middleware@2.0.6	Module: http-proxy Location: Package overview	yarn.lock	🚫
Network access	npm/http-proxy@1.18.1	Module: http Location: Package overview	yarn.lock	🚫
Network access	npm/http-proxy@1.18.1	Module: https Location: Package overview	yarn.lock	🚫
Network access	npm/serve-index@1.9.1	Module: http-errors Location: Package overview	yarn.lock	🚫
Network access	npm/spdy@4.0.2	Module: http Location: Package overview	yarn.lock	🚫
Network access	npm/spdy@4.0.2	Module: https Location: Package overview	yarn.lock	🚫
Network access	npm/spdy@4.0.2	Module: net Location: Package overview	yarn.lock	🚫
Network access	npm/spdy@4.0.2	Module: http-deceiver Location: Package overview	yarn.lock	🚫
Network access	npm/spdy@4.0.2	Module: tls Location: Package overview	yarn.lock	🚫
Network access	npm/multicast-dns@7.2.5	Module: dns-packet Location: Package overview	yarn.lock	🚫
Network access	npm/multicast-dns@7.2.5	Module: dgram Location: Package overview	yarn.lock	🚫
Network access	npm/sockjs@0.3.24	Module: http Location: Package overview	yarn.lock	🚫
Shell access	npm/launch-editor@2.6.1	Module: child_process Location: Package overview	yarn.lock	🚫
Network access	npm/webpack-dev-server@5.0.3	Module: globalThis["fetch"] Location: Package overview	package.json yarn.lock	🚫
Network access	npm/webpack-dev-server@5.0.3	Module: net Location: Package overview	package.json yarn.lock	🚫
Network access	npm/webpack-dev-server@5.0.3	Module: http-proxy-middleware Location: Package overview	package.json yarn.lock	🚫
Network access	npm/@lydell/node-pty@1.0.1	Module: net Location: Package overview	package.json yarn.lock	🚫
Shell access	npm/@lydell/node-pty@1.0.1	Module: child_process Location: Package overview	package.json yarn.lock	🚫
Network access	npm/@metamask/snaps-utils@8.0.0	Module: globalThis["fetch"] Location: Package overview	package.json	🚫

View full report↗︎

Next steps

What is network access?

This module accesses the network.

Packages should remove all network access that is functionally unnecessary. Consumers should audit network access to ensure legitimate use.

What is shell access?

This module accesses the system shell. Accessing the system shell increases the risk of executing arbitrary code.

Packages should avoid accessing the shell which can reduce portability, and make it easier for malicious shell access to be introduced.

Take a deeper look at the dependency

Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev.

Remove the package

If you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency.

Mark a package as acceptable risk

To ignore an alert, reply with a comment starting with @SocketSecurity ignore followed by a space separated list of ecosystem/package-name@version specifiers. e.g. @SocketSecurity ignore npm/foo@1.0.0 or ignore all packages with @SocketSecurity ignore-all

• @SocketSecurity ignore npm/http-proxy-middleware@2.0.6
• @SocketSecurity ignore npm/http-proxy@1.18.1
• @SocketSecurity ignore npm/serve-index@1.9.1
• @SocketSecurity ignore npm/spdy@4.0.2
• @SocketSecurity ignore npm/multicast-dns@7.2.5
• @SocketSecurity ignore npm/sockjs@0.3.24
• @SocketSecurity ignore npm/launch-editor@2.6.1
• @SocketSecurity ignore npm/webpack-dev-server@5.0.3
• @SocketSecurity ignore npm/@lydell/node-pty@1.0.1
• @SocketSecurity ignore npm/@metamask/snaps-utils@8.0.0

[metamaskbot]

Builds ready [dea1e34]

• builds: chrome, firefox
• builds (beta): chrome
• builds (flask): chrome, firefox
• builds (MMI): chrome, firefox
• builds (test): chrome, firefox
• builds (test-flask): chrome, firefox
• build viz: Build System
• mv3: Background Module Init Stats
• mv3: UI Init Stats
• mv3: Module Load Stats
• mv3: Bundle Size Stats
• mv2: E2e Actions Stats
• code coverage: Report
• storybook: Storybook
• typescript migration: Dashboard
• all artifacts
• bundle viz:

  • background: 0, 1, 2, 3, 4, 5, 6
  • common: 0, 1, 10, 2, 3, 4, 5, 6, 7, 8, 9
  • content-script: 0
  • offscreen: 0
  • ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (622 ± 371 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	66	327	142	81	39
		domContentLoaded	9	135	38	38	18
		load	41	2136	622	773	371
		domInteractive	9	135	38	38	18

Bundle size diffs [🚨 Warning! Bundle size has increased!]

• background: 81.68 KiB (2.36%)
• ui: -11 Bytes (-0.00%)
• common: 31.38 KiB (0.45%)

[metamaskbot]

Builds ready [b7dc749]

• builds: chrome, firefox
• builds (beta): chrome
• builds (flask): chrome, firefox
• builds (MMI): chrome, firefox
• builds (test): chrome, firefox
• builds (test-flask): chrome, firefox
• build viz: Build System
• mv3: Background Module Init Stats
• mv3: UI Init Stats
• mv3: Module Load Stats
• mv3: Bundle Size Stats
• mv2: E2e Actions Stats
• code coverage: Report
• storybook: Storybook
• typescript migration: Dashboard
• all artifacts
• bundle viz:

  • background: 0, 1, 2, 3, 4, 5, 6
  • common: 0, 1, 10, 2, 3, 4, 5, 6, 7, 8, 9
  • content-script: 0
  • offscreen: 0
  • ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (146 ± 161 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	75	330	113	54	26
		domContentLoaded	9	69	24	16	8
		load	44	1604	146	335	161
		domInteractive	9	69	24	16	8

Bundle size diffs [🚨 Warning! Bundle size has increased!]

• background: 81.68 KiB (2.37%)
• ui: -11 Bytes (-0.00%)
• common: 31.38 KiB (0.45%)

[metamaskbot]

Builds ready [9ec67bd]

• builds: chrome, firefox
• builds (beta): chrome
• builds (flask): chrome, firefox
• builds (MMI): chrome, firefox
• builds (test): chrome, firefox
• builds (test-flask): chrome, firefox
• build viz: Build System
• mv3: Background Module Init Stats
• mv3: UI Init Stats
• mv3: Module Load Stats
• mv3: Bundle Size Stats
• mv2: E2e Actions Stats
• code coverage: Report
• storybook: Storybook
• typescript migration: Dashboard
• all artifacts
• bundle viz:

  • background: 0, 1, 2, 3, 4, 5, 6
  • common: 0, 1, 10, 2, 3, 4, 5, 6, 7, 8, 9
  • content-script: 0
  • offscreen: 0
  • ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (151 ± 205 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	62	131	85	19	9
		domContentLoaded	9	38	17	9	4
		load	38	2012	151	427	205
		domInteractive	9	38	17	9	4

Bundle size diffs [🚨 Warning! Bundle size has increased!]

• background: 81.68 KiB (2.37%)
• ui: -11 Bytes (-0.00%)
• common: 31.38 KiB (0.45%)

[hjetpoluru]

@davidmurdoch, looks like this step is failing Socket Security: Pull Request Alerts any idea.

[hjetpoluru]

LGTM!! Awesome work @davidmurdoch

[davidmurdoch]

@davidmurdoch, looks like this step is failing Socket Security: Pull Request Alerts any idea.

This will go away once we merge.

[sonarcloud]

Quality Gate passed

Issues
25 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[metamaskbot]

Builds ready [27872ae]

• builds: chrome, firefox
• builds (beta): chrome
• builds (flask): chrome, firefox
• builds (MMI): chrome, firefox
• builds (test): chrome, firefox
• builds (test-flask): chrome, firefox
• build viz: Build System
• mv3: Background Module Init Stats
• mv3: UI Init Stats
• mv3: Module Load Stats
• mv3: Bundle Size Stats
• mv2: E2e Actions Stats
• code coverage: Report
• storybook: Storybook
• typescript migration: Dashboard
• all artifacts
• bundle viz:

  • background: 0, 1, 2, 3, 4, 5, 6
  • common: 0, 1, 10, 2, 3, 4, 5, 6, 7, 8, 9
  • content-script: 0
  • offscreen: 0
  • ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (237 ± 244 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	69	145	103	19	9
		domContentLoaded	9	62	21	13	6
		load	41	1876	237	508	244
		domInteractive	9	62	21	13	6

Bundle size diffs [🚨 Warning! Bundle size has increased!]

• background: 81.68 KiB (2.37%)
• ui: -11 Bytes (-0.00%)
• common: 31.38 KiB (0.45%)