From 16291a50a9ebe6eaea1a54b67c8ddbaad7ebb372 Mon Sep 17 00:00:00 2001 From: Dexter Codo Date: Sun, 17 Mar 2024 20:48:19 +1100 Subject: [PATCH] Chore/update readme (#1) * update: README.md * update: README.md * update: README.md * update: README.md * update: added env vars docs * update: added routers docs * update: added handlers docs * update: make command for docker * update: README.md * update: README.md * update: added a post handler * update: cleanup * update: README.md --------- Co-authored-by: JocelynPinto <162846923+JocelynPinto@users.noreply.github.com> --- README.md | 461 +++++++++++++++++++++++++++++++++++------------------- 1 file changed, 300 insertions(+), 161 deletions(-) diff --git a/README.md b/README.md index dbce149..7d2f2aa 100644 --- a/README.md +++ b/README.md @@ -7,9 +7,9 @@ [![Codacy Security Scan](https://github.com/codoworks/go-boilerplate/actions/workflows/codacy.yml/badge.svg)](https://github.com/codoworks/go-boilerplate/actions/workflows/codacy.yml) -This is a backend service skeleton or a boilerplate to speed up development process. Over time this package has become opinionated and behaves more like a framework with a set of pre-defined features. +This is a backend service skeleton or boilerplate to speed up development process. Over time, this package has become opinionated and behaves more like a framework with a set of predefined features. This package was developed using `GoLang v1.21`. -Thank you for using codoworks service skeleton. +Thank you for using Codoworks Go Boilerplate. @@ -24,12 +24,18 @@ Thank you for using codoworks service skeleton.
Mindset + +This service is designed for developers to build backend API as quickly as possible - almost as simply as copy and pasting components. The goal is to be able to clone this repository, rename it and get started with your first RESTful CRUD API within minutes. -There are many http libraries on github that enables quick and easy api development but as your project scales it can quickly become messy without proper structure and workflows, let alone preparing for production and managing security etc. +There are many HTTP libraries on GitHub that enable quick and easy API development, but as your project scales it can quickly become messy without proper structure and workflows, and even more when preparing for production or managing security. -This service was designed to enable backend api development as quickly as possible, almost by simply copy pasting components. The goal is to be able to clone this repository, rename it and get started with your first RESTful CRUD API within minutes. +Codoworks Go Boilerplate has all your production needs taken care of in advance, so you can focus your efforts on creating business and application logics. -It's designed with 3 separate routers (public, protected and hidden), each router has their individual configuration that you can customize to your needs. This can enable the following structure +It's worth noting that this package builds upon [Echo](https://echo.labstack.com) but you can easily adapt it to a different framework. + +#### Multi-routers + +It's designed with 3 separate routers (public, protected and hidden). Each router has its individual configuration that you can customise to your needs. This enables the following structure: ``` https://your-domain.com/api/ # Publicly consumable API @@ -37,37 +43,49 @@ https://api.your-domain.com/ # Your application's AP https://api.your-domain.com/.admin/
-Feature list +Full Feature List -- Cli commands (via Cobra) +- CLI commands (via Cobra) - HTTP server (via Echo) - - public router - - protected router - - hidden router + - Public router + - Protected router + - Hidden router - Daemon processes or workers - Tasks for custom one-off operations - Middlewares - - HTTP headers checks and setters + - HTTP header checks and setters - Auto error handling and response - - Authentication via ory kratos - - Authorization via ory keto + - Authentication via Ory Kratos + - Authorisation via Ory Keto - CORS handling - Auto trim trailing slashes - Request timeout @@ -81,22 +99,22 @@ Database migrations, logging, routing, hot-reloading, CORS, timeout and even gra - JSON forms and model mapping - Data validation - Clients - - Forward Http client to forward authorization headers + - Forward HTTP client to forward authorization headers - Ory Kratos Client - authentication - Ory Keto Client - authorization - Custom logger - Graceful shutdown -- Feature Toggle: [ory_kratos, ory_keto, db, redis] +- Feature toggle: [ory_kratos, ory_keto, db, redis]
-# Getting started +# Getting Started 1. Clone this repository `git clone git@github.com:codoworks/go-boilerplate.git` 2. Run `cd go-boilerplate` 3. Run `go get` 4. Run `go run . db migrate` 5. Run `go run . db seed` -6. Run `go run . start` to start the server, you should see the following +6. Run `go run . start` to start the server, you should see the following: ``` ⇨ http server started on [::]:8081 ⇨ http server started on [::]:8080 @@ -104,44 +122,44 @@ Database migrations, logging, routing, hot-reloading, CORS, timeout and even gra ``` 7. List available routes using `go run . info protected-api-routes` and use your favourite API client to test. or use the following to get started and make sure you're up and running. ```bash -curl -H "Accept: application/json" http://127.0.0.1:8080/health/alive -curl -H "Accept: application/json" http://127.0.0.1:8080/health/ready +curl -H "Accept: application/json" http://127.0.0.1:8081/health/alive +curl -H "Accept: application/json" http://127.0.0.1:8081/health/ready ``` > Recommended: run `go run .` and explore all available options, it should be straightforward. For more details on running and using the service, scroll down to "[Operations](#operations)" section. -To learn about developing and extending this service, scroll down to "[Make it your own](#make-it-your-own)" section. +To learn about developing and extending this service, scroll down to "[make it your own](#make-it-your-own)" section. -#### Simplified architecture diagram +#### Simplified Architecture Diagram
Docker -The service is shipped with a few docker compose files to get you started, all of which are automated with a Makefile to make things consistent. +The service is shipped with a few Docker compose files to get you started, all of which are automated with a Makefile to make things consistent. -#### Quick start +#### Quick Start -From the boilerplate root folder, run the quick-start target from the Makefile +From the boilerplate root folder, run the quick-start target from the Makefile. ```bash make quick-start ``` -#### Quick start with mysql +#### Quick Start with MySQL -To run an example using mysql database, from the boilerplate root folder, run the quick-start-mysql target from the Makefile +To run an example using MySQL database, from the boilerplate root folder, run the quick-start-mysql target from the Makefile. ```bash make quick-start-mysql ``` -#### Quick start with postgres +#### Quick Start with Postgres -To run an example using postgres database, from the boilerplate root folder, run the quick-start-mysql target from the Makefile +To run an example using Postgres database, from the boilerplate root folder, run the quick-start-mysql target from the Makefile. ```bash make quick-start-postgres @@ -152,31 +170,16 @@ make quick-start-postgres
Usage -### Env vars +### Env Vars -Environment variables are evaluated based on the following order to allow flexibility when running in production: +Environment variables are evaluated in the following order to allow flexibility when running in production: 1. `.env` file 2. environment variables 3. cmd flags (if available) -Example: if you have `HOST=127.0.0.1` in the `.env` file and `$ EXPORT HOST=0.0.0.0` in a terminal, the service will first read the first value in the env file, which is then overriden by the value in the terminal environment. +During development, it is recommended to use a `.env` file. You can find a reference under /.env.sample` to get started. -During development, it's recommended to use a `.env` file. You can find a reference under `./.env.sample` to get you started. - -If you're using docker for development you can uncomment the following line from `./ci/compose` to signal docker to pick up the `.env` file. -```Dockerfile -... -services: - go-boilerplate: - ... - # env_file: ../../.env - ... -... -``` - -> This was left commented out by default to avoid errors because `.env` is ignored via `.gitignore` , plus all env vars are optional. - -To ease your development process, we've included a command to print the environment to better understand why your app is behaving the way it is, simply run `go run . info env`. Together with `go run . info features` you should be able to get to the bottom of whatever is happening. +To ease your development process, we've included a command to print the environment to better understand your app behaviour. Simply run `go run . info env`. Together with `go run . info features`, you should be able to get to the bottom of an issue.
@@ -185,36 +188,36 @@ To ease your development process, we've included a command to print the environm | Var Name | Required | Description | | -------- | -------- | ----------- | | `HOST` | optional | service host address. default: 0.0.0.0 | -| `PORT` | optional | service port. default: 8080 | -| `PUBLIC_PORT` | optional | service port. default: 8081 | -| `HIDDEN_PORT` | optional | service port. default: 8079 | -| `DB_HOST` | optional | database host | -| `DB_PORT` | optional | database port | -| `DB_USER` | optional | database username | -| `DB_PASSWORD` | optional | database password | -| `DB_NAME` | optional | database name | -| `DB_TIMEZONE` | optional | database timezone. required with postgres platform | -| `DB_PLATFORM` | optional | enum: ["postgres", "mysql", "sqlite"]. default: "sqlite" | -| `KRATOS_PUBLIC_SERVICE` | optional | ory kratos public api url | -| `KRATOS_ADMIN_SERVICE` | optional | ory kratos admin api url | -| `KETO_READ_SERVICE` | optional | ory keto read api url | -| `KETO_WRITE_SERVICE` | optional | ory keto write api url | -| `REDIS_HOST` | optional | redis host url. required if redis is enabled | -| `REDIS_PORT` | optional | redis port. required if redis is enabled | -| `REDIS_PASSWORD` | optional | redis password. required if redis is enabled | -| `LOG_LEVEL` | optional | enum: ["INFO", "WARN", "DEBUG", "ERROR"]. default: "INFO" | -| `CORS_ALLOW_ORIGINS` | optional | allowed origins. default: "*" | -| `REQUEST_TIMEOUT_DURATION` | optional | number in seconds. default: "60" | -| `DISABLE_FEATURES` | optional | list of features to disable in runtime, make sure its comma separated without spaces | +| `PROTECTED_API_PORT` | optional | Service port. Default: 8080 | +| `PUBLIC_API_PORT` | optional | Service port. Default: 8081 | +| `HIDDEN_API_PORT` | optional | Service port. Default: 8079 | +| `DB_HOST` | optional | Database host | +| `DB_PORT` | optional | Database port | +| `DB_USER` | optional | Database username | +| `DB_PASSWORD` | optional | Database password | +| `DB_NAME` | optional | Database name | +| `DB_TIMEZONE` | optional | Database timezone. Required with Postgres platform | +| `DB_PLATFORM` | optional | Enum: ["postgres", "mysql", "sqlite"]. default: "sqlite" | +| `KRATOS_PUBLIC_SERVICE` | optional | Ory Kratos public API URL | +| `KRATOS_ADMIN_SERVICE` | optional | Ory Kratos admin API URL | +| `KETO_READ_SERVICE` | optional | Ory Keto read API URL | +| `KETO_WRITE_SERVICE` | optional | Ory Keto write API URL | +| `REDIS_HOST` | optional | Redis host URL. Required if Redis is enabled | +| `REDIS_PORT` | optional | Redis port. Required if Redis is enabled | +| `REDIS_PASSWORD` | optional | Redis password. Required if Redis is enabled | +| `LOG_LEVEL` | optional | Enum: ["info", "warn", "debug", "error"]. default: "info" | +| `CORS_ALLOW_ORIGINS` | optional | Allowed origins. Default: "*" | +| `REQUEST_TIMEOUT_DURATION` | optional | Number in seconds. Default: "60" | +| `DISABLE_FEATURES` | optional | List of features to disable in runtime, make sure its comma separated without spaces |
-### Execution modes +### Execution Modes -The service can run in one of two modes, either production and development modes. +The service can run in one of two modes: production mode or development mode. -Development mode is activated using the `-d` or `--dev` flag. Running in this mode will lock the service host to `127.0.0.1` to avoid firewall issues when developing using macOS. You can override this setting using `-H 0.0.0.0` if needed. +Development mode is activated using the `-d` or `--dev` flag. Running in this mode will lock the service host to `127.0.0.1` to avoid firewall issues when developing using MacOS. You can override this setting using `-H 0.0.0.0` if needed. -Development mode will also activate useful middlewares that help print incoming request body and input data validation errors for debugging, as well as set the logger level to debug to ease development. Everything else is identical to running in production mode. +Development mode will also activate useful middlewares that help print incoming request body, input data validation errors for debugging, and set the logger level to debug for ease of development. Everything else is identical to running in production mode. You can change the behaviour of the service using flags, see the list of flags below for more. @@ -223,29 +226,29 @@ You can change the behaviour of the service using flags, see the list of flags b | Flag Name | Shorthand | type | Description | | --------- | --------- | ---- | ----------- | -| `--dev` | `-d` | bool | run in development mode | -| `--env` | `-e` | bool | print environment variables | -| `--host` | `-H` | string | (optional) service host. overrides env vars | -| `--port` | `-P` | string | (optional) service port. overrides env vars | -| `--watcher` | (N/A) | bool | (optional) start watcher in the backgoround | -| `--log` | `-l` | string | (optional) log level | +| `--dev` | `-d` | bool | Run in development mode | +| `--env` | `-e` | bool | Print environment variables | +| `--host` | `-H` | string | (optional) Service host. Overrides env vars | +| `--port` | `-P` | string | (optional) Service port. Overrides env vars | +| `--watcher` | (N/A) | bool | (optional) Start watcher in the backgoround | +| `--log` | `-l` | string | (optional) Log level |
-### Live reload / hot-swap +### Live Reload / Hot-swap -It's convenient to automatically restart the service every time you save your changes, for that you can use [air](https://github.com/cosmtrek/air), which is a separate go package you can install using the following command +It is convenient to automatically restart the service every time you save your changes. For that, you can use [air](https://github.com/cosmtrek/air), which is a separate Go package you can install using the following command: ```bash go install github.com/cosmtrek/air@latest ``` -once `air` is installed, you simply need to run `air` to start the service. configurations for that can be found under `./.air.toml` +Once `air` is installed, you simply need to run `air` to start the service. Configurations for this can be found under `./.air.toml`. -Live reloading will also work in docker, the `Dockerfile.dev` is configured to install and run the service via `air` +Live reloading will also work in Docker. The `Dockerfile.dev` is configured to install and run the service via `air`. ### Operations -This service is shipped with a cmd client, which means you can run `./go-boilerplate` to view all available commands and help menu. +This service is shipped with a cmd client, which means you can use `./go-boilerplate` to view all available commands and help menu. > - You need to build the service first before you can use `go-boilerplate` > - both `./go-boilerplate` and `go run .` can be followed by any flags, commands and sub-commands @@ -258,64 +261,64 @@ It also requires `Content-Type: application/json` with `POST`, `PUT` and `DELETE ### Native Development -If you're writing a small project with a few endpoints then running Go in your terminal shouldn't be much of a problem. You can use the [live-reload](#live-reload) while you're editing your code in your favourite editor. +If you're writing a small project with a few endpoints then running Go in your terminal shouldn't be much of a problem. You can use [live-reload](#live-reload) while you're editing your code in your favourite editor. -To run the service without building, run `go run .` this will achieve the same result as `./go-boilerplate` after building the binary. +To run the service without building, run `go run .` which will achieve the same result as running `./go-boilerplate` after building the binary. -> The name `go-boilerplate` will change if you have changed the package name [as mentioned here](#change-pkg-name) +> The name `go-boilerplate` will change if you change the package name [as mentioned here](#change-pkg-name). ### In-Docker Development -However when you're running a large project with multiple micro-services (multiple instances of this boilerplate), it can be handy to live edit your code while in docker, for that we've designed the `Dockerfile.dev` to get your started. +However, when you are running a large project with multiple micro-services (multiple instances of this boilerplate), it can be handy to live edit your code while in Docker. For this, we have designed the `Dockerfile.dev` to get you started. -Simply run `make docker-dev` to get up and running, and to stop it, use `make stop-docker-dev`. Make sure you set your env correctly. +Simply run `make quick-start` to get up and running. To stop it, use `Ctrl+C`. ### Build -To build, run `go build .` this will generate a binary with the default name of the package, in this case it will be `./go-boilerplate` unless you've changed it (which is recommend). +To build, run `go build .` which will generate a binary with the default name of the package. In this case, it will be `./go-boilerplate` unless you change it (which is recommended). -If you've executed the above, you may notice the version by running `./go-boilerplate version` is set to `2.x.x-default`, that's because this is the second interation of this boilerplate. It's recommended that you burn the version into the binary in build time to create versioned builds. To do that, use the following command to build instead +If you have executed the above, you may notice that the version `./go-boilerplate version` is set to `2.x.x-default` during run time. That's because it is the second iteration of this boilerplate. It is recommended that you burn the version into the binary in build time to create versioned builds. To do that, use the following command to build: ```bash -go build -ldflags="-w -s -X main.VERSION=" +go build -ldflags="-w -s -extldflags '-static' -X main.VERSION=" # Example -go build -ldflags="-w -s -X main.VERSION=1.0.0" +go build -ldflags="-w -s -extldflags '-static' -X main.VERSION=1.0.0" ./go-boilerplate version # v1.0.0 ``` -Once built, a single binary file is generated, it's an executable that you can rename and place in any folder as long as your profile PATH can find it. a good place to place it on your local machine would be `/usr/bin` which is where most binaries are. +Once built, a single binary file is generated. It is an executable file that you can rename and place in any folder as long as your profile PATH can find it. A good place to place it on your local machine would be in `/usr/bin` which is where most binaries are. ### Deployment -If you wish to deploy this service locally, all you need to do is build from the above secion and then ship the outputted binary into a location where your terminal's PATH can find it, you should be able to use it by just calling it's name in your terminal. +If you wish to deploy this service locally, all you need to do is build as per the section above then ship the outputted binary into a location where your terminal's PATH can find it. You should be able to use it just by calling its name in your terminal. -The "Usage" section should get you familiar with all configurations required to configure, operate and maintain this service, deploying it should be a piece of cake onto any dockerized environment. +The "Usage" section should get you familiarised with all the parameters that are configurable. Deploying it should not be a problem in any dockerised environment. -From a container point of view, i'd encourage you to place this binary in an empty container, i.e. `FROM scratch` in Dockerfile. this helps keep the container size minimal. When tested on an M1 Mac Machine, we got an 18MB container. +From a containerisation perspective, I'd encourage you to place this binary in an empty container i.e. `FROM scratch` in your Dockerfile. This helps keep the container size to a minimum. When tested on an M1 Mac Machine, we got an 18MB container.
-Extending the service (Make it your own) +Extending the service (make it your own) This section is all about extending the service to create your own application and APIs. -> The first thing you'd want to do is to change the package name, find `codoworks/go-boilerplate` in all files and replace it with your desired package name. You can choose to use the general `(org-name)/(project-name)` naming pattern for consistency. +> The first thing you should do is to change the package name, find `github.com/codoworks/go-boilerplate` in all the files and replace it with your own package name. You can choose to use the general `github.com/(org-name)/(project-name)` naming pattern for consistency.
Migrations -Migrations help create your database and track how it eveolves overtime, here we use [GoMigrate](https://github.com/go-gormigrate/gormigrate) to achieve this along with an added implementation to enable easy extendability and better logs throughout your development process. +Migrations help create your database and track how it evolves overtime. Here, we use [GoMigrate](https://github.com/go-gormigrate/gormigrate) to achieve this. Some added complexity is added to enable easy extendability and generate better logs throughout your development process. -Migrations go under `pkg/db/migrations/.go`. The way it's implemented uses `Go`'s `init()` function which means they're added to the list in the alphabetical order they appear in, they get migrated in that order (top to bottom), and rollbacked in the reverse order (bottom up). For this it's best to maintain the naming convention of `YYYYMMDD[00-99]_migration_description`. +Migrations go under `pkg/db/migrations/.go`. Its implemention uses `Go`'s `init()` function, which means they're added to the list in alphabetical order. They migrate in that order (top to bottom) and rollback in the reverse order (bottom up). For this, it is best to maintain the naming convention of `YYYYMMDD[00-99]_migration_description`. Here's a sample migration to get you started: -```go +```Go func init() { m := &gormigrate.Migration{} @@ -343,12 +346,12 @@ func init() { } ``` -The variable `m` holds the migration details and is added to the list of migrations at the end. `m.ID` is the identifier used by `gomigrate` to keep track of the migrations that already ran so make sure to change that for every migration. +The variable `m` holds the migration details and is added to the list of migrations at the end. `m.ID` is the identifier used by `gomigrate` to keep track of the migrations that already ran. So, make sure to change that for every migration. -Every migration has 2 methods to be implemented, a `Migrate()` and `Rollback()` methods as described above. Make sure you use the `logSuccess`, `logFail` and `AutoMigrateAndLog()` functions to print the migrations that ran. This would come in very handy with remote deployments. +Every migration has 2 methods to be implemented, the `Migrate()` and `Rollback()` method as described above. Make sure you use the `logSuccess`, `logFail` and `AutoMigrateAndLog()` functions to print the migrations that ran. This will come in very handy for remote deployments. -It's recommended to declare your models within each migration (separately from models) to keep track of how the database schema is changing throughout the project. You can add/delete columns, rename columns and execute raw SQL in migrations. +It's recommended to declare your models within each migration (separately from the models package) to keep track of the database schema change through time. You can add or delete columns, rename columns, and execute raw SQL in migrations. > A general good practice would be to flatten your migrations once your application achieves version 1, leaving only neat table creation in each migration.
@@ -356,13 +359,13 @@ It's recommended to declare your models within each migration (separately from m
Seeds -Seeds are very similar to migrations, the key difference between the two is that seeds don't implement the `Rollback` function. That's because seeds are intended to create content inside the database, they don't modify the database structure in any way so there's no need for rollbacks. +Seeds are very similar to migrations, but seeds do not implement the `Rollback` function. -Just like migrations, seeds are applied once, and tracked using their unique identifier `ID` by [GoMigrate](https://github.com/go-gormigrate/gormigrate). +Just like migrations, seeds are applied once and tracked using their unique identifier `ID` by [GoMigrate](https://github.com/go-gormigrate/gormigrate). -Seeds are part of the whole package which allows you to access models, clients and other components directly to configure the application or perhaps provide dummy data to enable streamlined development. +Seeds are part of the whole package which allows you to access models, clients and other components directly to configure the application, and perhaps provide dummy data to help with development. -Here's a seed skeleton to get you started, all you need to do is copy the following structure into a new file under seeds and change the `s.ID` property. +Here's a seed skeleton to get you started. Copy the following structure into a new file under seeds and change the `s.ID` property. ```Go func init() { @@ -380,7 +383,7 @@ func init() { } ``` -And here's a sample seed to give an idea of how you can utilize seeds. +And here's a sample seed to give an idea of how you can utilise seeds. ```Go func init() { @@ -423,13 +426,13 @@ func init() {
Models -Models can sometimes be a complex aspect of any application, in this section you'll find a breakdown on how you can compose your models or database entities. +Models can sometimes be a complex aspect of any application. In this section, you'll find a rundown on how you can compose your models or database entities. -#### Model structure +#### Model Structure -The first thing you'd want to do is to create a struct that matches your database schema, almost all models should embed the `ModelBase` struct that provides the `ID`, `CreatedAt` and `UpdatedAt` properties, excepts can be things like a many to many table where you only need to store 2 identifiers. To learn more about model declarations you can refer to [Gorm](https://gorm.io) official comprehensive documentation. +The first thing is to create a struct that matches your database schema. Almost all models should embed the `ModelBase` struct that provides the `ID`, `CreatedAt` and `UpdatedAt` properties. Exceptions may include a many-to-many table where you only need to store 2 identifiers. To learn more about model declarations, you can refer to [Gorm](https://gorm.io)'s official comprehensive documentation. -Here's a Cat model that should correspond to a Cats table in a database, and contains 5 properties, `ID`, `CreatedAt`, `UpdatedAt`, `Name` and `Type` +Here's a Cat model that should correspond to a Cats table that contains 5 properties i.e. `ID`, `CreatedAt`, `UpdatedAt`, `Name` and `Type` in a database. ```Go type Cat struct { @@ -438,15 +441,15 @@ type Cat struct { Type string `gorm:"size:255"` } ``` -Notice how every property contains a `gorm` decoration to specify things like field size, uniqueness or foreign keys etc. For more details please refer to [Gorm](https://gorm.io)'s documentation. +Notice how every property contains a `gorm` decoration to specify things like field size, uniqueness or foreign keys etc. For more details, please refer to [Gorm](https://gorm.io)'s documentation. -Your model may sometimes contain properties that do not correspond to a database column, to do that you simply need to use the `gorm:"-"` decoration. +Your model may sometimes contain properties that do not correspond to a database column. To do that, you simply need to use the `gorm:"-"` decoration. -> Note: given this package is designed to work with multiple database servers like mysql or postgres. Some data types may be available in some servers and not others, it's worth testing your application with differnet servers from time to time as you go to accomodate the ease of switching database server, unless your use-case relies on that specific data type in which case you're making an informed decision to lock your application to that server. +> Note: Given that this package is designed to work with multiple database servers like MySQL or Postgres, some data types may be available in some servers and not others. It's worth testing your application with differnet servers from time to time to accomodate easy switching of database server, unless your use case relies on a specific data type - in which case you're making a calculated decision to lock your application to that server. -#### Common basic functionality +#### Common Basic Functionality -Now that you have a structure that corresponds to a table in your database, some common functionality is in order. Generally one would expect at least the basic CRUD functionality, and ofcourse reading can mean one or more records so more functions can be added. Here's a basic CRUD implementation that is required for any model. +Now that you have a structure that corresponds to a table in your database, some common functionality is in order. Generally, one would at least expect the basic CRUD functionality. Here's a basic CRUD implementation that is required for any model: - `FindAll()`, for retrieving all records in the table ```Go @@ -493,12 +496,12 @@ func (model *Cat) Delete(id string) error { } ``` -All of the above functions will return an error if they weren't able to perform what they're supposed to do, that's useful to inform the user if the data they're looking for exists or is stored. For detailed utilization of these functions, have a look the handlers section. +All of the above functions will return an error if they cannot perform what they're supposed to. That's useful to inform users if the data they're looking for exists or is stored. For detailed utilisation of these functions, check out the handlers folder. -These functions are not abstracted to allow granular control over each model as each individual model can quickly morph into something very large with child elements, preload functions and pagination. +These functions are not abstracted to allow granular control over each model, as each individual model can quickly morph into something very large with child elements, preload functions and pagination. -#### Model accessibility +#### Model Accessibility Given the basic functionality defined in the previous section, we've created the ability to do something like the following: ```Go @@ -512,7 +515,7 @@ if err != nil { ... ``` -The problem with the above code is that you'd need to instantiate a new struct `catModel` from `&Cat{}` in order to have a pointer receiver that can call the `Find()` function. You can avoid that by using the following common structure for all models, right at the top of the model before it's declaration to maintain consistency. +The problem with the code above is that you will need to instantiate a new struct `catModel` from `&Cat{}` in order to have a pointer receiver that can call the `Find()` function. You can avoid that by using the following common getter structure for all models, right at the top of the model before its declaration to maintain consistency. ```Go var cat *Cat = &Cat{} @@ -520,17 +523,17 @@ func CatModel() *Cat { return cat } ``` -The above will now create a singleton pattern that you can access from any component within the package like `models.CatModel().Find()` +The above will now create a singleton pattern that you can access from any component within the package like `models.CatModel().Find()`. -> Note: the `CatModel()` method should only be used to fetch data from the database, saving, updating and deleting data should be applied to an actual instance that has been returned through a `Find()`, `FindAll()` or `FindMany()` functions. +> Note: the `CatModel()` method should only be used to fetch data from the database. Saving, updating and deleting data should be applied to an actual instance that has been returned through a `Find()`, `FindAll()` or `FindMany()` function. -#### Working with json forms +#### Working with JSON Forms -Once you have retrieved the records needed from the database, you may want to send those records as a response to the user. To do that, forms have been created, while every model is expected to have at least one method named `MapToForm()` that returns a json representation of that model. +Once you have retrieved the records needed from the database, you may want to send those records as a response. To do that, you can use forms. Every model is expected to have at least one method named `MapToForm()` that returns a JSON representation of that model. -Forms are basic structures that may or may not exactly match all the properties that a model has, the reason why it's done this way is to enable multiple forms where one can contain all model properties, intended for an admin user to view, while another may contain a sanitized version of that model, intended only for a read-only user. +Forms are basic structures that may or may not exactly match all the properties that a model has. The reason it has been done this way is to enable multiple forms where one can contain all model properties e.g. intended for an admin user to view, while another may contain a sanitised version of that model e.g. intended only for a read-only user. -For more details on creating a form, scroll down to the forms section below. Here you'll find a sample implementation of `MapToForm()` function +For more details on creating a form, scroll down to the forms section below. Here you'll find a sample implementation of `MapToForm()` function. ```Go func (model *Cat) MapToForm() *CatForm { form := &CatForm{ @@ -599,17 +602,17 @@ func (model *Cat) Delete(id string) error { return db.Model(model).Where("ID=?", id).Delete(&model).Error } ``` -Feel free to copy the above code and replace the name `Cat` to get started. +Copy the code above and replace the name `Cat` to get started.
Forms -Forms are data contracts that are used to send responses to clients and receive/bind user input. +Forms are data contracts that are used to send responses to clients and receive/ bind user input. -Each model can have many forms to enable sending specific values with different endpoints. An example scenario would be having an admin with full access to all data in a record whereas a customer has access to a subset of that data. +Each model can have many forms to enable sending specific values with different endpoints. An example scenario would be having an admin with full access to all data in a record whereas a customer has access only to a subset of that data. -Data validation is applied to fields in forms. Here's a sample form to get you started +Data validation is applied to fields in forms. Here's a sample form to get you started. ```Go type CatForm struct { FormBase @@ -627,29 +630,110 @@ func (form *CatForm) MapToModel() *Cat { The `FormBase` struct provides the `ID`, `CreatedAt` and `UpdatedAt` fields. -Each field should specify the name mapping in json format along with validation rules. For more on validations check out the [Playground Validator documentation](https://github.com/go-playground/validator). +Each field should specify the name mapping in JSON format along with validation rules. For more on validations check out the [Playground Validator documentation](https://github.com/go-playground/validator). To skip validations all together, use `validate:"-"`. -Finally, each form should have a `MapToModel()` function that returns a form so it can be stored after it's been mapped and validated. +Finally, each form should have a `MapToModel()` function that returns a model, so it can be stored after it has been validated. Note that forms do not set a model's `ID` property as that is the job of the model. Instead, it must be set manually prior to a database operation. Think of this like an actual form you fill up that has a section "for office use only".
Handlers +> Note: This go boilerplate uses [Echo](https://echo.labstack.com). If you're ever in doubt, you can refer to Echo's documentation for more details on what's possible with routers. + +A handler is any function with the `func (c echo.Context) error` signature. All handlers should be stored under `pkg/api/handlers` and categorized in directories following their entity name in plural form. For readability and maintainability, we encourage maintaining a single handler in a single file as we all know that Go files can quickly grow. + +Handlers should also be nested, which means a Cats handlers directory can contain a sub directory, such as `cats/tags`, that helps avoid long file names and improve readability. + +How handlers look will largely depend on your project's business logic and requirements. For reference, here's a quick sample to give an idea on how you should construct your handler. + +```Go +func Get(c echo.Context) error { + + id := c.Param("id") + + if id == "" { + return helpers.Error(c, constants.ERROR_ID_NOT_FOUND, nil) + } + + m, err := models.CatModel().Find(id) + + if err != nil { + return helpers.Error(c, err, nil) + } + + return c.JSON(http.StatusOK, handlers.Success(m.MapToForm())) + +} +``` +And perhaps another example to demonstrate how to receive user input and store a model. +```Go +func Post(c echo.Context) error { + + f := &models.CatForm{} + + if err := c.Bind(f); err != nil { + return helpers.Error(c, constants.ERROR_BINDING_BODY, err) + } + + if err := helpers.Validate(f); err != nil { + return c.JSON(http.StatusBadRequest, handlers.ValidationErrors(err)) + } + + m := f.MapToModel() + + if err := m.Save(); err != nil { + return helpers.Error(c, err, nil) + } + + return c.JSON(http.StatusOK, handlers.Success(m.MapToForm())) + +} +``` +
Routers +> Note: This go boilerplate uses [Echo](https://echo.labstack.com). If you're ever in doubt, you can refer to Echo's documentation for more details on what's possible with routers. + +This boilerplate is shipped with 3 routers, public, privdate and hidden routers - all of which ollow the same structure and procedure with slight differences in what is registered within each. + +Why have 3 routers? Well, some projects may have public and protected routes, and such use case is straightforward. The latter implements an authentication middleware while the first does not. Attempting to achieve such behaviour within a single router can be tricky, so isolated routers running on different ports are used instead. The third "hidden" router is provided to enable a pattern commonly used to allow one microservice to communicate with another without exposing those routes to the public internet. With that said, wiring those 3 routers can easily be achieved through a different service like Kubernetes or NGINX. + +All routers should go through the following process: +1. Initialisation +2. Checking for DevMode and enabling related middlewares +3. Register common middleware +4. Register health routes +5. Register security middleware +6. Register user-defined routes +7. Register error handler + +Have a look at `pkg/api/routers/protectedApi.go` to familiarise yourself with router initialisation process. If you've already created your handlers from the previous section, all you need is to add your new route to this file as such: +```Go +func registerProtectedAPIRoutes() { + cats := protectedApiRouter.Echo.Group("/cats") // Your new REST resource + cats.GET("", catsHandlers.Index) // GET "/cats/" route and handler + cats.GET("/:id", catsHandlers.Get) // GET "/cats/:id" route and handler + cats.POST("", catsHandlers.Post) // POST "/cats/" route and handler + cats.PUT("/:id", catsHandlers.Put) // PUT "/cats/:id" route and handler + cats.DELETE("/:id", catsHandlers.Delete) // DELETE "/cats/:id" route and handler + + // add more routes here ... +} +``` +
Tasks -Tasks is a way to extend the command line cli without having to go through the trouble of understanding the initialization process. +Tasks is a way to extend the command line CLI without having to go through the trouble of understanding the initialisation process. -To create a new task, simply add the following sample into a new file `./pkg/tasks/.go` +To create a new task, simply add the following sample into a new file `./pkg/tasks/.go`: -```go +```Go func init() { var t = &Task{ @@ -668,51 +752,105 @@ func execMyTask(env *TaskEnv, args map[string]string) error { } ``` -Tasks are automatically injected with an `env` object that contains the environment. they're also injected with an `args` map containing any values added to the exec command, as long as they're separated by '=' like `key1=value1 key2=value2 key3=value3 `. +Tasks are automatically injected with an `env` object that contains the environment. They are also injected with an `args` map containing any values added to the exec command, as long as they're separated by '=' e.g. `key1=value1 key2=value2 key3=value3 `. -You can also set the required arguments in `myTask.RequiredArgs = []string{"key1", "key2"}` to prevent execution unless all arguments are provided. +You can also set the required arguments in `myTask.RequiredArgs = []string{"key1", "key2"}` to prevent execution until all arguments are provided. -To execute the above task, simply run `go run . task exec myTask` and you should get the "My first task is executed!" message. +To execute the task above, simply run `go run . task exec myTask` and you should get the "My first task is executed!" message.
Error Handling +This boilerplate automates error handling and error responses. + +First, let's talk about logging errors. When using proper logging mechanics and log levels, you can then leave all your logs in the code and have them printed depending on their severity. The package is shipped with the function `helpers.Error()`, a wrapper that's intended to log an error and return it. These logs will only be visible if the `LOG_LEVEL` env var permits. Avoid using `fmt.Println()` at all times, instead, use `logger.Debug()` or if you're within a handler, you can use the `c.Logger().Debug()` helper. + +Given that each handler can return an error, the router is configured to handle the error using `pkg/api/handlers/errors/automatedHttpError.go` which will unwrap the error and match it with a list of registered errors under `pkg/api/handlers/errors/errors.go`. Finally, it will construct an error response and respond to that request. + +Validation errors are no different, except they're unwrapped further and sent to the user as individual form inputs so they can be displayed. + +You're encouraged to register and maintain as many errors as you can in the same way. It's useful to have a specific error code mapped to each error, that way we can determine exactly what went wrong in each user flow. +
Adding Env Vars & Features +All environment variables reside in `pkg/config/features`. They're categorised within their respective features such as `database.go` or `service.go`. Each env var must have a `mapstructure:` decoration that spells it in caps when parsing an ENV. You can add your own, it's as simple as adding a new line in any of these files, or create your own. + +Below is a sample of `pkg/config/features/service.go`: +```Go +type ServiceConfig struct { + Host string `mapstructure:"HOST"` + ProtectedApiPort string `mapstructure:"PROTECTED_API_PORT"` + PublicApiPort string `mapstructure:"PUBLIC_API_PORT"` + HiddenApiPort string `mapstructure:"HIDDEN_API_PORT"` + LogLevel string `mapstructure:"LOG_LEVEL"` + RequestTimeoutDuration string `mapstructure:"REQUEST_TIMEOUT_DURATION"` + WatcherSleepInterval string `mapstructure:"WATCHER_SLEEP_INTERVAL"` +} + +var service = &Feature{ + Name: constants.FEATURE_SERVICE, + Config: &ServiceConfig{}, + enabled: true, + configured: false, + ready: false, + requirements: []string{ + "Host", + "ProtectedApiPort", + "PublicApiPort", + "HiddenApiPort", + "LogLevel", + "RequestTimeoutDuration", + "WatcherSleepInterval", + }, +} + +func init() { + Features.Add(service) +} +``` + +From the example above, you can find a type `ServiceConfig` that states what env vars are to be expected. These are automatically read from the environment. Env vars must belong to a feature which can be toggled on or off. A feature can also define which env vars are required for it to start. + +If you wish to disable a feature, you can mention it in the list of `DISABLE_FEATURES` var in run-time. + +Reading the env vars is the job of `pkg/config/envVars.go`. Each config struct must be registered in `envVars.go`. The config struct is then automatically injected to its respective feature after initialisation. + +It is possible to set a default value for each variable, this can be done in `pkg/config/envVars.go` under `setDefaults()`. + +By the time the CMD calls the Proc, all env vars should have already been read and injected into their features, making them available for the rest of the package. +
Folder Structure -### root folder structure +### Root Folder Structure The package is split into 3 directories | Directory | Description | | --------- | ----------- | -| `/ci` | Contains all files related to building or deploying the service such as docker, docker compose, configuration and k8s files| +| `/ci` | Contains all files related to building or deploying the service such as Docker, Docker compose, configuration and K8S files| | `/cmd` | Contains all available commands | | `/pkg` | Contains all source code files. This is where you'll be spending most of your time | -### pkg folder structure - -Every folder must have a file with the same name. example: in `db` folder, there must be a `db.go` file +### pkg Folder Structure | Directory | Description | | --------- | ----------- | | `/pkg/api` | Everything related to `Echo`, routers and handlers go in here | -| `/pkg/clients` | These are clients used throughout the service. They can be third party services or simple config providers for workflows | +| `/pkg/clients` | These are clients used throughout the service. They can be third-party services or simple config providers for workflows | | `/pkg/config` | Service configuration and environment variable management | -| `/pkg/db` | Everything related to database entities and models, as well as migrations and seed data | +| `/pkg/db` | Everything related to database entities and models, migrations, and seed data | | `/pkg/proc` | Entry points for all processes | -| `/pkg/tasks` | User defined tasks available via the command line cli | -| `/pkg/utils` | General utilities used throughout the package that don't belong to any specific package | +| `/pkg/tasks` | User-defined tasks available via the command line CLI | +| `/pkg/utils` | General utilities used throughout the package that do not belong to any specific package |
@@ -794,9 +932,9 @@ Every folder must have a file with the same name. example: in `db` folder, there
Dependencies -This package is purely written in Go, which eases dependency management. All dependencies can be easily installed using the `go get` command. +This package is purely written in Go, which helps with dependency management. All dependencies can be easily installed using the `go get` command. -There are only 2 optional dependencies that must be installed separately, you can choose to install them or not, the first is [Air](https://github.com/cosmtrek/air) used for [live-reload](#live-reload), and the other is [Docker](https://www.docker.com/products/docker-desktop/). +There are only 2 optional dependencies that can be installed separately. The first is [Air](https://github.com/cosmtrek/air) used for [live-reload](#live-reload), and the other is [Docker](https://www.docker.com/products/docker-desktop/). List of run-time dependencies: @@ -825,7 +963,7 @@ List of development dependencies:
Known Issues -- Gorm v1.25.6 and v1.25.7 are known to cause issues with PostgreSQL database. If you experience an error `this driver does not support LastInsertID()`, try downgrading gorm to v1.25.5 +- Gorm v1.25.6 and v1.25.7 are known to cause issues with PostgreSQL database. If you experience an error `this driver does not support LastInsertID()`, try downgrading Gorm to v1.25.5
@@ -841,7 +979,7 @@ List of development dependencies: - [ ] Enhanced error handling - [ ] Quick start examples - [ ] Example with Kratos for authentication - - [ ] Example with Keto for authorization + - [ ] Example with Keto for authorisation - [ ] Code cleanup and in-line documentation - [ ] Swagger integration - [ ] Postman collection @@ -855,8 +993,9 @@ List of development dependencies: ### Contribution -We're looking for contributors, all ideas are welcome. Feel free start a new discussion, open a new PR etc, if you wish to join the team just let us know using the contact below. +Feel free to start a new discussion, submit a new PR, make a feature request or etc.. If you would like to join the team, reach out to us on Discord. We are always looking for contributors! ### Contacts +- [Join our Discord channel](https://discord.gg/Q27kgPVub7) - [Dexter Codo](mailto:dexter@dexterexplains.com)