[RFC] decrease build times and simplify build process

[fschade]

---

Date: 2021-06-14
Relevant Team(s): Webteam
RFC Issue: #5259

Summary

The build pipeline we use at the moment started to get slow, the configuration is hard and error prone.
After working on the typescript pr it got even more complex and up to 2 times slower.

Motivation

Decrease build times, make app development less error prone and more pleasant:

Pain points

• slow rebuilds after filechange
• hard to understand whats going on
• hard to adopt when starting new projects

Supported filetypes:

*.js
*.vue
*.ts
*.css
*.sass

Build times

File	Build-Time per change
packages/web-app-draw-io/src/App.vue	5.1s
packages/web-app-draw-io/src/index.js	5.4s
packages/web-app-files/src/views/Personal.vue	5.6s
packages/web-app-files/src/helpers/collaborators.js	5.5s
packages/web-app-markdown-editor/src/App.vue	5.9s
packages/web-app-markdown-editor/src/store.js	5s
packages/web-app-media-viewer/src/App.vue	5.6s
packages/web-app-media-viewer/src/index.js	4s
packages/web-runtime/src/components/ApplicationsMenu.vue	5.7s
packages/web-runtime/src/components/ApplicationsMenu.vue	5.1s

App development configuration

• https://github.com/owncloud/web/blob/master/rollup.config.js
• https://github.com/owncloud/ocis-hello/blob/master/rollup.config.js
• https://github.com/owncloud/ocis/blob/master/settings/rollup.config.js
• https://github.com/owncloud/ocis/blob/master/accounts/rollup.config.js

Detailed design

We should introduce a abstract layer for our build-pipeline which makes it possible to change the underlying architecture without changing the api surface.

Then as a developer i'm still able to use the pipeline as it is, even if the underlying architecture changes over time.
The api could be as follows:

import { BuildEvent, BuildPipeline, bundles, tasks } from './dev/build'
import path from 'path'

const { DefaultBundle } = bundles
const options = {
  buildDest: 'dist',
  watch: process.env.WATCH === 'true'
}

/* define bundle */
const bundle = new DefaultBundle({
  input: {
    'web-app-draw-io': 'packages/web-app-draw-io/src/index.js',
    'web-app-files': 'packages/web-app-files/src/index.js',
    'web-app-markdown-editor': 'packages/web-app-markdown-editor/src/index.js',
    'web-app-media-viewer': 'packages/web-app-media-viewer/src/index.js',
    'web-runtime': 'packages/web-runtime/src/index.js'
  },
  watch: options.watch
})

/* define tasks */
const copyHTMLTask = new tasks.CopyTask({
  watch: options.watch,
  targets: [
    {
      source: 'packages/web-container/**/*.{html,json,svg,png}',
      dest: options.buildDest
    },
    {
      source: 'packages/web-runtime/themes/**/*',
      dest: path.join(options.buildDest, 'themes')
    },
    {
      source: 'config/config.json',
      dest: options.buildDest
    },
    {
      source: 'node_modules/requirejs/require.js',
      dest: path.join(options.buildDest, 'js')
    }
  ]
})

/* define pipeline */
const pipeline = new BuildPipeline({ watch: options.watch })

/* attach bundle */
pipeline.addBundle(bundle)

/* attach task */
pipeline.addTask(copyHTMLTask)

/* define pipeline events */
pipeline.on('event', (event: BuildEvent) => {
  // console.log('BuildPipeline', '--', event.key)
  // print event to stdout
  // copy a file
  // trigger livereload event
  // ...
})

/* define copyHTMLTask events */
copyHTMLTask.on('event', (event: BuildEvent) => {
  // console.log('CopyTask', '--', event.key)
  // print event to stdout
  // copy a file
  // trigger livereload event
  // ...
})

/* start the pipeline */
pipeline.start()

there are 3 main parts that are required to start a succesful build:

• import @ocweb/dev/build
• create a bundle
• create build tasks (for example copying files)
• create a pipeline, attach steps and bundles and then start it

Drawbacks

we need to take care of the configuration and provide best practices for app developers

Alternatives

use a preconfigured pipeline like vite, but this requires that the system addresses all our needs.

Unresolved questions

TBD

Acceptance criteria

• the build system is easy to use and extendable
• the build system is battery included and makes it possible to be extended (new tasks, new bundles or even plain rollup config)
• build times are up to 3 times faster in development mode

[fschade]

first POC available

https://github.com/owncloud/web/tree/build-pipeline-rfc-poc