Skip to content

Commit

Permalink
Move submodule update command to rivershared + check for go/toolcha…
Browse files Browse the repository at this point in the history
…in + workspace (#528)

This one's a bit of a grab bucket, but do a few refactors and additions:

* Bring the project over to use a Go workspace, thereby letting us run
  tests and programs across module directories and cut out our `replace`
  statements in `go.mod` all over the place.

* Move the River version update program for `go.mod` files over to
  `rivershared` so that it can be used by other River projects. It's
  modified so that instead of hardcoding a submodule list, it can now
  read them straight out of the project's workspace in `go.work`.

* Add a new program that's similar to this one whose job it is to check
  that the `go`/`toolchain` directives across all `go.mod`s in the
  workspace match, which will help prevent accidental regressions of
  #522. It can also be used locally to update `go`/`toolchain`
  directives to new values in case we want to do that in the future.

* Do a lot of repaving in `Makefile` so that we can use list of
  submodules from the workspace to run a series of commands instead of
  needing to maintain independent lists for each target, a practice that
  was often unreliable.
  • Loading branch information
brandur authored Aug 18, 2024
1 parent e97f2a3 commit 3eda791
Show file tree
Hide file tree
Showing 21 changed files with 499 additions and 174 deletions.
24 changes: 18 additions & 6 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ jobs:
strategy:
matrix:
go-version:
- "1.22"
- "1.21"
postgres-version: [16, 15, 14]
- "1.22"
postgres-version: [14, 15, 16]
fail-fast: false
timeout-minutes: 5

Expand Down Expand Up @@ -59,10 +59,7 @@ jobs:
run: go version

- name: Install dependencies
run: |
echo "::group::go get"
go get -t ./...
echo "::endgroup::"
run: go get -t ./...

- name: Set up test DBs
run: go run ./internal/cmd/testdbman create
Expand Down Expand Up @@ -265,3 +262,18 @@ jobs:
run: |
echo "Make sure that all sqlc changes are checked in"
make verify/sqlc
submodule_check:
runs-on: ubuntu-latest

steps:
- uses: actions/setup-go@v5
with:
go-version: "stable"
check-latest: true

- name: Checkout
uses: actions/checkout@v4

- name: Check all go/toolchain directives match
run: CHECK=true make update-mod-go
88 changes: 52 additions & 36 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,71 +1,87 @@
.PHONY: generate
generate:
generate: generate/migrations
generate: generate/sqlc
.DEFAULT_GOAL := help

.PHONY: db/reset
db/reset: ## drop, create, and migrate dev and test databases
db/reset: ## Drop, create, and migrate dev and test databases
db/reset: db/reset/dev
db/reset: db/reset/test

.PHONY: db/reset/dev
db/reset/dev: ## drop, create, and migrate dev database
db/reset/dev: ## Drop, create, and migrate dev database
dropdb river_dev --force --if-exists
createdb river_dev
cd cmd/river && go run . migrate-up --database-url "postgres://localhost/river_dev"

.PHONY: db/reset/test
db/reset/test: ## drop, create, and migrate test databases
db/reset/test: ## Drop, create, and migrate test databases
go run ./internal/cmd/testdbman reset

.PHONY: generate
generate: ## Generate generated artifacts
generate: generate/migrations
generate: generate/sqlc

.PHONY: generate/migrations
generate/migrations: ## sync changes of pgxv5 migrations to database/sql
generate/migrations: ## Sync changes of pgxv5 migrations to database/sql
rsync -au --delete "riverdriver/riverpgxv5/migration/" "riverdriver/riverdatabasesql/migration/"

.PHONY: generate/sqlc
generate/sqlc:
generate/sqlc: ## Generate sqlc
cd riverdriver/riverdatabasesql/internal/dbsqlc && sqlc generate
cd riverdriver/riverpgxv5/internal/dbsqlc && sqlc generate

# Looks at comments using ## on targets and uses them to produce a help output.
.PHONY: help
help: ALIGN=22
help: ## Print this message
@awk -F '::? .*## ' -- "/^[^':]+::? .*## /"' { printf "'$$(tput bold)'%-$(ALIGN)s'$$(tput sgr0)' %s\n", $$1, $$2 }' $(MAKEFILE_LIST)

# Each directory of a submodule in the Go workspace. Go commands provide no
# built-in way to run for all workspace submodules. Add a new submodule to the
# workspace with `go work use ./driver/new`.
submodules := $(shell go list -f '{{.Dir}}' -m)

# Definitions of following tasks look ugly, but they're done this way because to
# produce the best/most comprehensible output by far (e.g. compared to a shell
# loop).
.PHONY: lint
lint:
cd . && golangci-lint run --fix
cd cmd/river && golangci-lint run --fix
cd riverdriver && golangci-lint run --fix
cd riverdriver/riverdatabasesql && golangci-lint run --fix
cd riverdriver/riverpgxv5 && golangci-lint run --fix
cd rivershared && golangci-lint run --fix
cd rivertype && golangci-lint run --fix
lint:: ## Run linter (golangci-lint) for all submodules
define lint-target
lint:: ; cd $1 && golangci-lint run --fix
endef
$(foreach mod,$(submodules),$(eval $(call lint-target,$(mod))))

.PHONY: test
test:
cd . && go test ./... -p 1
cd cmd/river && go test ./...
cd riverdriver && go test ./...
cd riverdriver/riverdatabasesql && go test ./...
cd riverdriver/riverpgxv5 && go test ./...
cd rivershared && go test ./...
cd rivertype && go test ./...
test:: ## Run test suite for all submodules
define test-target
test:: ; cd $1 && go test ./... -p 1
endef
$(foreach mod,$(submodules),$(eval $(call test-target,$(mod))))

.PHONY: tidy
tidy:
cd . && go mod tidy
cd cmd/river && go mod tidy
cd riverdriver && go mod tidy
cd riverdriver/riverdatabasesql && go mod tidy
cd riverdriver/riverpgxv5 && go mod tidy
cd rivertype && go mod tidy
tidy:: ## Run `go mod tidy` for all submodules
define tidy-target
tidy:: ; cd $1 && go mod tidy
endef
$(foreach mod,$(submodules),$(eval $(call tidy-target,$(mod))))

.PHONY: update-mod-go
update-mod-go: ## Update `go`/`toolchain` directives in all submodules to match `go.work`
go run ./rivershared/cmd/update-mod-go ./go.work

.PHONY: update-mod-version
update-mod-version: ## Update River packages in all submodules to $VERSION
go run ./rivershared/cmd/update-mod-version ./go.work

.PHONY: verify
verify:
verify: ## Verify generated artifacts
verify: verify/migrations
verify: verify/sqlc

.PHONY: verify/migrations
verify/migrations:
verify/migrations: ## Verify synced migrations
diff -qr riverdriver/riverpgxv5/migration riverdriver/riverdatabasesql/migration

.PHONY: verify/sqlc
verify/sqlc:
verify/sqlc: ## Verify generated sqlc
cd riverdriver/riverdatabasesql/internal/dbsqlc && sqlc diff
cd riverdriver/riverpgxv5/internal/dbsqlc && sqlc diff
cd riverdriver/riverpgxv5/internal/dbsqlc && sqlc diff
12 changes: 0 additions & 12 deletions cmd/river/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,6 @@ go 1.21

toolchain go1.22.5

// replace github.com/riverqueue/river => ../..

// replace github.com/riverqueue/river/riverdriver => ../../riverdriver

// replace github.com/riverqueue/river/riverdriver/riverdatabasesql => ../../riverdriver/riverdatabasesql

// replace github.com/riverqueue/river/riverdriver/riverpgxv5 => ../../riverdriver/riverpgxv5

// replace github.com/riverqueue/river/rivershared => ../../rivershared

// replace github.com/riverqueue/river/rivertype => ../../rivertype

require (
github.com/jackc/pgx/v5 v5.6.0
github.com/lmittmann/tint v1.0.4
Expand Down
2 changes: 1 addition & 1 deletion docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ queries. After changing an sqlc `.sql` file, generate Go with:
```shell
git checkout master && git pull --rebase
export VERSION=v0.x.y
go run ./internal/cmd/update-submodule-versions/main.go
make update-mod-version
git checkout -b $USER-$VERSION
```

Expand Down
11 changes: 0 additions & 11 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,6 @@ go 1.21

toolchain go1.22.5

replace github.com/riverqueue/river/riverdriver => ./riverdriver

replace github.com/riverqueue/river/riverdriver/riverpgxv5 => ./riverdriver/riverpgxv5

replace github.com/riverqueue/river/riverdriver/riverdatabasesql => ./riverdriver/riverdatabasesql

replace github.com/riverqueue/river/rivershared => ./rivershared

replace github.com/riverqueue/river/rivertype => ./rivertype

require (
github.com/jackc/pgerrcode v0.0.0-20220416144525-469b46aa5efa
github.com/jackc/pgx/v5 v5.6.0
Expand All @@ -26,7 +16,6 @@ require (
github.com/robfig/cron/v3 v3.0.1
github.com/stretchr/testify v1.9.0
go.uber.org/goleak v1.3.0
golang.org/x/mod v0.20.0
golang.org/x/sync v0.8.0
golang.org/x/text v0.17.0
)
Expand Down
12 changes: 10 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,16 @@ github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/riverqueue/river/riverdriver v0.11.2 h1:2xC+R0Y+CFEOSDWKyeFef0wqQLuvhk3PsLkos7MLa1w=
github.com/riverqueue/river/riverdriver v0.11.2/go.mod h1:RhMuAjEtNGexwOFnz445G1iFNZVOnYQ90HDYxHMI+jM=
github.com/riverqueue/river/riverdriver/riverdatabasesql v0.11.2 h1:I4ye1YEa35kqB6Jd3xVPNxbGDL6S1gpSTkZu25qffhc=
github.com/riverqueue/river/riverdriver/riverdatabasesql v0.11.2/go.mod h1:+cOcD4U+8ugUeRZVTGqVhtScy0FS7LPyp+ZsoPIeoMI=
github.com/riverqueue/river/riverdriver/riverpgxv5 v0.11.2 h1:yxFi09ECN02iAr2uO0n7QhFKAyyGZ+Rn9fzKTt2TGhk=
github.com/riverqueue/river/riverdriver/riverpgxv5 v0.11.2/go.mod h1:ajPqIw7OgYBfR24MqH3VGI/SiYVgq0DkvdM7wrs+uDA=
github.com/riverqueue/river/rivershared v0.11.2 h1:VbuLE6zm68R24xBi1elfnerhLBBn6X7DUxR9j4mcTR4=
github.com/riverqueue/river/rivershared v0.11.2/go.mod h1:J4U3qm8MbjHY1o5OlRNiWaminYagec1o8sHYX4ZQ4S4=
github.com/riverqueue/river/rivertype v0.11.2 h1:YREWOGxDMDe1DTdvttwr2DVq/ql65u6e4jkw3VxuNyU=
github.com/riverqueue/river/rivertype v0.11.2/go.mod h1:bm5EMOGAEWhtXKqo27POWnViqSD5nHMZDP/jsrJc530=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
Expand All @@ -32,8 +42,6 @@ go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
golang.org/x/crypto v0.17.0 h1:r8bRNjWL3GshPW3gkd+RpvzWrZAwPS49OmTGZ/uhM4k=
golang.org/x/crypto v0.17.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
golang.org/x/mod v0.20.0 h1:utOm6MM3R3dnawAiJgn0y+xvuYRsm1RKM/4giyfDgV0=
golang.org/x/mod v0.20.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/sync v0.8.0 h1:3NFvSEYkUoMifnESzZl15y791HH1qU2xm6eCJU5ZPXQ=
golang.org/x/sync v0.8.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/text v0.17.0 h1:XtiM5bkSOt+ewxlOE/aE/AKEHibwj/6gvWMl9Rsh0Qc=
Expand Down
13 changes: 13 additions & 0 deletions go.work
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
go 1.21

toolchain go1.22.5

use (
.
./cmd/river
./riverdriver
./riverdriver/riverdatabasesql
./riverdriver/riverpgxv5
./rivershared
./rivertype
)
38 changes: 38 additions & 0 deletions go.work.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
github.com/cpuguy83/go-md2man/v2 v2.0.3 h1:qMCsGGgs+MAzDFyp9LpAe1Lqy/fY/qCovCm0qnXZOBM=
github.com/creack/pty v1.1.9 h1:uDmaGzcdjhF4i/plgjmEsriH11Y0o7RKapEf/LDaM3w=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/kr/pty v1.1.1 h1:VkoXIwSboBpnk99O/KFauAEILuNHv5DVFKZMBN/gUgw=
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/yuin/goldmark v1.4.13 h1:fVcFKWvrslecOb/tg+Cc05dkeYx540o0FuFt3nUVDoE=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.20.0 h1:utOm6MM3R3dnawAiJgn0y+xvuYRsm1RKM/4giyfDgV0=
golang.org/x/mod v0.20.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0 h1:d/OCCoBEUq33pjydKrGQhw7IlUPI2Oylr+8qLx49kac=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.23.0 h1:YfKFowiIMvtgl1UERQoTPPToxltDeZfbj4H7dVUCwmM=
golang.org/x/sys v0.23.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2 h1:IRJeR9r1pYWsHKTRe/IInb7lYvbBVIqOgsX/u0mbOWY=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
golang.org/x/term v0.23.0 h1:F6D4vR+EHoL9/sWAWgAR1H2DcHr4PareCbAaCo1RpuU=
golang.org/x/term v0.23.0/go.mod h1:DgV24QBUrK6jhZXl+20l6UWznPlwAHm1Q1mGHtydmSk=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
gopkg.in/errgo.v2 v2.1.0 h1:0vLT13EuvQ0hNvakwLuFZ/jYrLp5F3kcWHXdRggjCE8=
72 changes: 0 additions & 72 deletions internal/cmd/update-submodule-versions/main_test.go

This file was deleted.

2 changes: 0 additions & 2 deletions riverdriver/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,4 @@ go 1.21

toolchain go1.22.5

replace github.com/riverqueue/river/rivertype => ../rivertype

require github.com/riverqueue/river/rivertype v0.11.2
2 changes: 2 additions & 0 deletions riverdriver/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/riverqueue/river/rivertype v0.11.2 h1:YREWOGxDMDe1DTdvttwr2DVq/ql65u6e4jkw3VxuNyU=
github.com/riverqueue/river/rivertype v0.11.2/go.mod h1:bm5EMOGAEWhtXKqo27POWnViqSD5nHMZDP/jsrJc530=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
Expand Down
Loading

0 comments on commit 3eda791

Please sign in to comment.