Port bootstrap improvements from lecture capture preferences webapp #36

rjw57 · 2019-03-10T13:36:01Z

This PR uses a combination of git format-patch and git am to port the changes from uisautomation/lecture-capture-preferences-webapp#4 back into the bootstap application. As such the commit dates, authorship and description are as in the original PR with fixups as necessary to allow use in the template.

A deviation from the mechanical conversion of the changes is to make ucamlookup an optional feature. For extra-CUDN deployments, additional work is needed to use ucamlookup and we may not actually need to in all cases.

To speed up tox environment builds, allow tox to make use of packages already installed in the container.

The auto-generated migrations are rarely flake8 friendly so ignore them.

Copy the logging configuration from other projects to ensure that exceptions are recorded in the logs.

For running in production, make sure that the production Raven certificate is specified.

Add in the standard security features for our web applications: * HTTP -> HTTPS automatic redirect. (Disabled in development and for tests.) * CORS headers * Clickjacking protection middleware

Add in the Django Debug Toolbar configuration for development.

Add in the Django REST Framework and associated packages which we use. configure the DRF to allow for session-based and token authentication. The token authentication is useful to allow for "service account" or "robot" users.

Add in auto-generated Swagger API documentation and, in development, a container which lets you browse it via Swagger UI.

Add in django-ucamlookup which is simpler to use than lookupproxy if we're deploying in-house. Since lookup required extra configuration to use from outside of the CUDN, we make it opt-in.

The README and documentation generated by the bootstrap included some incorrect references to the repository location and some out-of-date advice. Update them accordingly.

Having a default of "YES" breaks tests for the moment since tests are run outside of the CUDN.

rjw57 · 2019-03-10T13:46:52Z

I'm not quite sure why the tests are failing because they're in an unrelated area of the template. Will investigate.

abrahammartin · 2019-03-10T16:48:31Z

{{ cookiecutter.project_slug }}/{{ cookiecutter.project_module }}/settings/base.py

@@ -148,3 +151,27 @@

 #: Allow members who are not current members to log in?
 UCAMWEBAUTH_NOT_CURRENT = False
+
+# Allow all origins to access API.
+CORS_URLS_REGEX = r'^/api/.*$'


Unrelated but found this while looking at CORS info in DRF: https://www.django-rest-framework.org/topics/ajax-csrf-cors/

Do we have CSRF protection enabled in DRF?

I fact, we cannot trivially disable it just for the DRF since the "unsafe" method protection is provided by the CSRF protection middleware.

abrahammartin · 2019-03-10T16:52:07Z

{{ cookiecutter.project_slug }}/{{ cookiecutter.project_module }}/settings/base.py

+    SECURE_REDIRECT_EXEMPT = ['^healthz/?$']
+
+    SECURE_SSL_REDIRECT = True
+    SECURE_HSTS_SECONDS = 31536000  # == 1 year


HSTS doesn't accept self-signed certificates or non HTTP connections, making the website impossible possible to visit after the first installation (with a self-signed certificate while it tries to get the real one using let's encrypt) or if let's encrypt fails to renew. It's worth noting that and thinking about it before going to that route.

This is the same route we've gone down for the media platform and preferences webapp (And probably others.) If we're going for consistency, we should keep it. If we're unsure then we should remove it from where we've used it at the moment.

abrahammartin · 2019-03-10T16:56:32Z

{{ cookiecutter.project_slug }}/{{ cookiecutter.project_module }}/settings/base.py

+# TLS termination for us. In future this setting might need to be moved to settings.docker or to be
+# configured via an environment variable if we want to support a wider range of TLS terminating
+# load balancers.
+SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')


Is this also needed when deploying to GCP or only internally using the UIS load balancer?

In any case it's a dangerous setting and we shouldn't leave it by default to be on, see django docs warning on this topic: https://docs.djangoproject.com/en/2.1/ref/settings/#secure-proxy-ssl-header

We need it with GCP too both for nginx and Google-provided Ingresses.

Fair point about dangerous by default. I'll put it behind an option like ucamwebauth.

abrahammartin · 2019-03-10T17:00:26Z

{{ cookiecutter.project_slug }}/{{ cookiecutter.project_module }}/settings/docker.py

+        '': {
+            'handlers': ['console'],
+            'propagate': True,
+            'level': 'INFO'


This conf will be used for all environments (as it is in docker.py), hence I believe 'level' should be configurable via an env variable and put into the readme. We may not want to log INFO in production systems, for example.

OK, we need to port that to the preferences webapp then.

abrahammartin · 2019-03-10T17:01:29Z

{{ cookiecutter.project_slug }}/{{ cookiecutter.project_module }}/settings/docker.py

+    },
+    'loggers': {
+        '': {
+            'handlers': ['console'],


I'd a 'email' handler for 'ERROR' level logs in production systems as well.

OK, we need to port that to the preferences webapp then.

abrahammartin · 2019-03-10T17:07:00Z

{{ cookiecutter.project_slug }}/requirements/base.txt

+djangorestframework
+django-cors-headers
+drf-yasg
+django-crispy-forms


Are we using this at all?

As this is a cherry-pick of the preferences webapp PR, I suppose we should see if there was a subsequent commit in the preferences webapp which removed django-crispy-forms? AFAICT it's still in the current master.

abrahammartin · 2019-03-10T17:07:57Z

{{ cookiecutter.project_slug }}/{{ cookiecutter.project_module }}/urls.py

+schema_view = get_schema_view(
+   openapi.Info(
+      title='{{ cookiecutter.project_name }}',
+      default_version='v1',


v1Alpha by default until finished?

Possibly. If so, we'd need to update the preferences webapp.

rjw57 and others added 12 commits March 10, 2019 12:51

tox: allow use of sitepackages

0c7acf3

To speed up tox environment builds, allow tox to make use of packages already installed in the container.

flake8: ignore generated migrations

aba0203

The auto-generated migrations are rarely flake8 friendly so ignore them.

README: add note on building documentation

1184599

settings.docker: add more verbose logging

be21610

Copy the logging configuration from other projects to ensure that exceptions are recorded in the logs.

settings.docker: add production Raven configuration

8344e8b

For running in production, make sure that the production Raven certificate is specified.

add security features

7a681ad

Add in the standard security features for our web applications: * HTTP -> HTTPS automatic redirect. (Disabled in development and for tests.) * CORS headers * Clickjacking protection middleware

add Django Debug Toolbar

516ce96

Add in the Django Debug Toolbar configuration for development.

add Django REST Framework

85f18f5

Add in the Django REST Framework and associated packages which we use. configure the DRF to allow for session-based and token authentication. The token authentication is useful to allow for "service account" or "robot" users.

add Swagger API documentation

90efb59

Add in auto-generated Swagger API documentation and, in development, a container which lets you browse it via Swagger UI.

add django-ucamlookup integration

10dafa1

Add in django-ucamlookup which is simpler to use than lookupproxy if we're deploying in-house. Since lookup required extra configuration to use from outside of the CUDN, we make it opt-in.

fix bootstrap docs and readme

5eb09ca

The README and documentation generated by the bootstrap included some incorrect references to the repository location and some out-of-date advice. Update them accordingly.

ucamlookup: change default to "NO"

55a02c5

Having a default of "YES" breaks tests for the moment since tests are run outside of the CUDN.

rjw57 requested a review from a team March 10, 2019 13:46

abrahammartin suggested changes Mar 10, 2019

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Port bootstrap improvements from lecture capture preferences webapp #36

Port bootstrap improvements from lecture capture preferences webapp #36

rjw57 commented Mar 10, 2019

rjw57 commented Mar 10, 2019

abrahammartin Mar 10, 2019

rjw57 Mar 10, 2019

abrahammartin Mar 10, 2019

rjw57 Mar 10, 2019

abrahammartin Mar 10, 2019

rjw57 Mar 10, 2019

abrahammartin Mar 10, 2019

rjw57 Mar 10, 2019

abrahammartin Mar 10, 2019

rjw57 Mar 10, 2019

abrahammartin Mar 10, 2019

rjw57 Mar 10, 2019

abrahammartin Mar 10, 2019

rjw57 Mar 10, 2019

Port bootstrap improvements from lecture capture preferences webapp #36

Are you sure you want to change the base?

Port bootstrap improvements from lecture capture preferences webapp #36

Conversation

rjw57 commented Mar 10, 2019

rjw57 commented Mar 10, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment