Added webpack to the HQ deploy process #6284
Conversation
| tasks: | ||
| - import_role: {name: deploy_hq} | ||
| vars: | ||
| if_not_done: run_webpack |
There was a problem hiding this comment.
What do you think of adding the tasks in this file to staticfiles_collect.yml file? It would reduce deploy overhead to group them together since they are applied to the same set of hosts. This is doing the same kind of thing as other staticfiles collection tasks, so I think it makes sense.
There was a problem hiding this comment.
Is the primary motivation to reduce overhead or to create natural groups? From running this deploy process on staging, I don't think overhead should be a concern -- the webworkers task executed very quickly. The bottleneck is the compilejs18n command from staticfiles_collect.yml. If the motivation is instead to create natural groups, I'd argue that this grouping has the potential to create a lot of confusion. Webpack doesn't collect any static files. Instead, it is generally known as the step to compress/minify, and our dedicated "Compress static files" task lives outside of "Collect static files". I also wouldn't recommend combining tasks just because they use the same hosts. I suppressed a warning by including webworkers here, but in my head, this should really only be deployed to proxy servers.
If we wanted to split off a playbook dedicated to javascript tasks, that would make sense, but I think dedicated tasks for webpack make sense
There was a problem hiding this comment.
The overhead I'm speaking of is SSH connections made by Ansible, not the time it takes to actually run webpack.
staticfiles_collect.yml is where we do other js-related build steps (example: build_requirejs), so I think it makes sense to put webpack in there too since that is the same type of task.
There was a problem hiding this comment.
I've updated the associated ticket with logs from 2 runs to compare what changes when webpack is run from the deploy_hq task vs from within staticfiles_collect. I don't believe there is any difference with respect to SSH connections. From what I can tell, ansible will create connections for every task, and it is thus advised to configure SSH to re-use connections.
I continue to believe that moving webpack to staticfiles_collect would be confusing. Collecting into static is django terminology. I think it's unintuitive that we're building requireJS in this playbook, and I don't think we should double down here. Webpack deserves its own task file because it is capable of much more than collecting, and those responsibilities may not align with the same hosts that collecting does. In addition, it is essentially in feature-flag status at the moment. Including it as I'm doing minimizes log output for runs that do not use webpack.
There was a problem hiding this comment.
Looks like there would be two extra SSH connections (Check if done: run_webpack and the corresponding Completed: run_webpack) when run_webpack.yml is included via a separate block in this file. Admittedly not much overhead, but it's something.
For me putting it together with other tasks doing similar things (currently in staticfiles_collect.yml) is the more important point since that's where people who are familiar with the existing structure would go to look for that type of task. I could see an argument for renaming staticfiles_collect.yml to something more generic that includes JS bundling. Adding another top-level block in this file seems overkill for a feature flagged experiment.
There was a problem hiding this comment.
Noting that build_requirejs is the last task in staticfiles_collect.yml, I think because it depends collectstatic. Is there any chance (now or in the future) that webpack could also depend on collectstatic being run first?
There was a problem hiding this comment.
I looked at this, and I'm not entirely sure why requireJS runs after collectstatic. That said, webpack is creating artifacts that django will later serve, so those artifacts should be created before static assets are collected. I previously modified the webpack configuration such that webpack is able to follow django app paths, thus it looks for the location of files before collecting them into a single static folder
Associated ticket: https://dimagi.atlassian.net/browse/SAAS-15485
Adds support for running webpack while deploying HQ, if a webpack configuration file is found. This is not currently required by any live code, but allows us to test react and vue (and other javascript features) in the future
Environments Affected
Announce New Release
to announce a new commcare-cloud release.