nodejs-training

Backend part for our frontend (react/angular) training

• REST API docs (generated with http://apidocjs.com/)
• See nodejs-modules for common packages.

Deployment/Running:

Create a new OAuth App at Github ( https://github.com/settings/developers ) and Google ( https://console.developers.google.com/apis/dashboard ) and use the client id and the secret to configure the nodejs app.

For local development it is better to use nvm or nvm-windows for managing installed (and used) local node versions (instead of installing Node manually).

Heroku

heroku addons:create papertrail:choklad
heroku addons:create heroku-postgresql:hobby-dev
heroku config:set PGSSLMODE=require
heroku config:set NPM_CONFIG_PRODUCTION=false # to install dev dependencies
heroku config:set JWT_KEY=<some random string>
heroku config:set DOMAIN=<url of your app>
heroku config:set GITHUB_CLIENT_ID=<insert>
heroku config:set GITHUB_CLIENT_SECRET=<insert>
heroku config:set GOOGLE_CLIENT_ID=<insert>
heroku config:set GOOGLE_CLIENT_SECRET=<insert>

Outside Heroku

Copy .env.dev to .env and update settings:

• ALLOW_UNSECURE - do not try to redirect http to https
• PORT - the port where API will be running
• DOMAIN - URL where API will be running
• DATABASE_URL - connection string to Postgres DB

Create files

• secret_jwt_key.txt - random secure key for signing session payload
• secret_github_client_id.txt, secret_github_client_secret.txt, secret_google_client_id.txt, secret_google_client_secret.txt with OAuth credentials from apps you created

Optionally you can update .env file with SSL settings (but running LB or proxy is recommended):

• SSL_KEY_FILE - path to SSL key file
• SSL_KEY - SSL key file contents
• CERT_SSL_FILE - path to SSL cert file
• CERT_SSL - SSL cert file contents

And run either via

• npm run start:dev or npm start (you need to have node, Postgres and everything installed locally)
• (docker-compose build when dependencies change) and docker-compose up (you need only docker and there should be no "works on my machine" issue 🎉)
• for running in production use app docker image directly or use docker swarm
  • change image in docker-compose.production.yml
  • run docker-compose -f docker-compose.yml -f docker-compose.production.yml config > docker-compose.prod.yml to squash config files together (check docker-compose.prod.yml secrets section - there should be relative paths)
  • create files secret_database_url.txt and secret_db_password.txt with DB connection url/password
  • create swarm cluster and deploy (see Brownbag presentation)
  • enjoy :)

Training plan

There is no nodejs way (like there is an Angular way or React way etc.). Just pick framework (we like Koa) and it comes with a function signature for writing middleware function (all frameworks have a similar structure - request in, response out). All other depends on the type of project, used technologies (REST/SOAP/GraphQL, Postgres/Mongo/Redis/ElasticSearch/Kafka/...) and on you. Here I present some patterns you may find useful.

Try to stick with The Twelve-Factor App paradigm.

App's entry point is in /src/index.js where http(s) server is configured, routes are added and server starts listening on the specified port. Routes/Actions are defined in /src/actions, directory structure on disk represents directory structure of URLs.

Koa middleware TL;DR

Koa middleware is async function with ctx parameter and optional next parameter (must be called to process next middlewares, if present). Value assigned to ctx.body is returned to client with ctx.status status code (if ctx.body is set defaults to 200, if ctx.body is not set defaults to 404). You can read route params from ctx.params and POST params from ctx.request.body. All thrown exceptions will be caught and transformed to JSON for the client (see errorMiddleware).

Authentication

The authentication layer is provided via passport package, using JWT authentication header (no cookie, no session). OAuth examples provided through Google and Github, also logging in via local user/password is present. After successful authentication JS code is sent to the client which stores JWT token to local storage and reloads page (so client app can retrieve and use this token). (You should use cookies for storing JWT if all your client apps support it, as cookies are usually more secure than local storage) Secured endpoints should use authMiddleware (Checks JWT token in the header and tries to populate ctx.state.user with user object).

Authorization layer not provided. (Usually, Role Based Access Control is sufficient)

Testing

Use mocha/jest/ava/... whatever you want.

I can recommend using supertest (see testing users API /src/actions/v1/users/index.spec.js) for testing your API calls and puppeteer for testing front-end code (see puppeteer example at /tests/puppeteer.spec.js).

Request data

We are using cls-hooked to share context inside a request. It is used for keeping a request id accessible for logging even on places where you do not have direct access to the request. If you are requesting external (micro)services, you should keep this request id for better logging. e.g.

const axios = require("axios");
const { getRequestId } = require("@salsita/koa-server");
axios.get("http://example.com", {
  headers: { "x-request-id": getRequestId() },
});

If really necessary you can also use this storage for your data. e.g.

const { getNamespace } = require("cls-hooked");
const { requestNSName } = require("@salsita/koa-server");

const request = getNamespace(requestNSName);

const data = {};
request.set("mydata", data);

// somewhere else
const { getNamespace } = require("cls-hooked");
const { requestNSName } = require("@salsita/koa-server");

const request = getNamespace(requestNSName);
const data = request.get("mydata");

What we use:

Services:

• Circle CI - CI/test server
• Renovate - Automated dependency management
• Docker - For united development experience
• Heroku - For deployment and review apps
• Snyk - Find/Protect against vulnerabilities in 3rd party code
• ngrok - Public URL for your localhost (https://ngrok.com/)

Dependencies:

• koa - server
• @koa/router - routing
• koa-passport - authentication via passport
• koa-bodyparser - parse json/form/text request
• koa-compress - zipping response
• koa-cors - enabling Cross-origin resource sharing (CORS)
• koa-helmet - securing HTTP(s) communication
• koa-morgan - HTTP(s) request logger (also see koa-logger and koa-json)
• koa-send - serving static files
• koa-static - serving static directories
• joi - validation (e.g. io-ts is better suited for TypeScript projects)
• pg - DB
• node-pg-migrate - DB migrations
• squel - SQL building (also see knex and sequelize)
• winston - logging
• pretty-error - formatting node.js errors
• snyk - protect against vulnerabilities
• dotenv - reading .env file
• uuid - generating uuids

Dev:

• apidoc - API documentation
• nodemon - restarting dev server on change
• supertest - API tests
• cross-env - environment variables across platforms
• prettier - automatic code formatting
• eslint - code linting
• husky - git hooks
• mocha - test runner
• sinon - spies, stubs and mocks
• chai - assertions
• puppeteer - headless Chrome API

Others:

• luxon or date-fns - date and time
• node-schedule - task scheduling
• socket.io - WebSockets
• mongoose - MongoDB object modeling
• ioredis - Redis client
• apollo-server-koa - GraphQL server
• webpack - module bundler (also see rollbar and parcel-bundler)
• babel - JS transpiler
• kue - priority job queue

Links

• node compatibility table
• API docs
• Awesome node.js (aka node.cool)
• Node.JS best practices list