Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Io test 4 #159

Closed
wants to merge 24 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
fe0f2f7
ci: Use "devel" as the pre-release name instead of the commit title
wolfgangwalther May 4, 2024
58d8133
ci: Enable docker push for releases
wolfgangwalther May 4, 2024
9fe90bf
ci: Fix pushing of devel-arm docker tag
wolfgangwalther May 4, 2024
eec35d4
ci: Avoid recreating devel release
wolfgangwalther May 4, 2024
2f98d83
ci: Push docker hub description automatically on main branch
wolfgangwalther May 4, 2024
c67f1c3
ci: Fix docker release job after 2f98d837
wolfgangwalther May 5, 2024
8433f98
ci: Replace assets of existing devel release properly
wolfgangwalther May 5, 2024
d9ba9a8
ci: Avoid devel release failure when no assets to clean up exist
wolfgangwalther May 5, 2024
a57d12b
ci: Use the new DOCKER_ vars for docker-arm job
wolfgangwalther May 5, 2024
463dfb5
chore(deps): update actions/checkout hash for release/prepare job
wolfgangwalther May 5, 2024
cabe744
chore(deps): update ubuntu docker tag to v24
renovate[bot] May 4, 2024
21bc48a
fix: fix wrong 503 Service Unavailable on pg error 53400
taimoorzaeem Apr 10, 2024
7e91e53
fix: not adding application_name on all URIs
steve-chavez May 3, 2024
1d4f315
changelog: move 3340 from fixed to added
steve-chavez May 6, 2024
b34c00c
changelog: add missing entry for 3184
steve-chavez May 6, 2024
df9b373
docs: better place for application_name
steve-chavez May 6, 2024
b006016
ci: Make "release / tag" job detect existing tags
wolfgangwalther May 7, 2024
cb6151e
ci: Make artifact-from-cirrus action succeed when cirrus job doesn't …
wolfgangwalther May 7, 2024
1374178
ci: Improve performance for nix jobs in CI
wolfgangwalther May 7, 2024
1b584f7
refactor: is ready Admin logic to AppState
steve-chavez May 7, 2024
f9e9740
nix: add postgrest-ctags command
steve-chavez May 8, 2024
0060abe
feat: /live and /ready respond with 500 on failure
steve-chavez May 7, 2024
50000a5
test: Reset statement timeout after each test
wolfgangwalther May 9, 2024
c869433
test: Prevent test_admin_ready_includes_schema_cache_state from timin…
wolfgangwalther May 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions .github/actions/artifact-from-cirrus/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,14 @@ runs:
echo "Cirrus CI task has not started, yet. Waiting..."
sleep 10
else
echo "check_suite_found=1" >> "$GITHUB_OUTPUT"
echo "check_runs_url=$check_runs_url" >> "$GITHUB_OUTPUT"
exit 0
fi
done
>&2 echo "Cirrus CI check suite not found. Is Cirrus CI enabled for this repo?"
echo "check_suite_found=0" >> "$GITHUB_OUTPUT"
- name: Find task by name
id: find-task
if: steps.check-suite.outputs.check_suite_found
if: steps.check-suite.outputs.check_runs_url
shell: bash
run: |
get_number_of_tasks() {
Expand Down
4 changes: 4 additions & 0 deletions .github/actions/setup-nix/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ runs:
using: composite
steps:
- uses: nixbuild/nix-quick-install-action@60e9c39264d4714139af3cdf15f691b19eec3530 # v28
with:
nix_conf: |-
always-allow-substitutes = true
max-jobs = auto
- uses: cachix/cachix-action@18cf96c7c98e048e10a83abd92116114cd8504be # v14
with:
name: postgrest
Expand Down
2 changes: 1 addition & 1 deletion .github/scripts/arm/docker-env/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# PostgREST docker hub image

FROM ubuntu:jammy@sha256:a6d2b38300ce017add71440577d5b0a90460d0e57fd7aec21dd0d1b0761bbfb2 AS postgrest
FROM ubuntu:noble@sha256:3f85b7caad41a95462cf5b787d8a04604c8262cdcdf9a472b8c52ef83375fe15 AS postgrest

RUN apt-get update -y \
&& apt install -y --no-install-recommends libpq-dev zlib1g-dev jq gcc libnuma-dev \
Expand Down
14 changes: 10 additions & 4 deletions .github/scripts/arm/docker-publish.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,13 @@ DOCKER_REPO="$2"
DOCKER_USER="$3"
DOCKER_PASS="$4"
SCRIPT_DIR="$5"
PGRST_VERSION="v$6"
PGRST_VERSION="$6"

if [ "$PGRST_VERSION" == "devel" ]; then
PGRST_TAG="$PGRST_VERSION"
else
PGRST_TAG="v$PGRST_VERSION"
fi

DOCKER_BUILD_DIR="$SCRIPT_DIR/docker-env"

Expand All @@ -37,13 +43,13 @@ cd ~/$DOCKER_BUILD_DIR
# be added to the manifest if they are not in the registry beforehand.
# This image must be manually deleted from Docker Hub at the end of the process.
sudo docker buildx build --build-arg PGRST_GITHUB_COMMIT=$PGRST_GITHUB_COMMIT \
-t $DOCKER_REPO/postgrest:$PGRST_VERSION-arm \
-t $DOCKER_REPO/postgrest:$PGRST_TAG-arm \
--push .

# Add the arm images to the manifest
# NOTE: This assumes that there already is a `postgrest:<version>` image
# for the amd64 architecture pushed to Docker Hub
sudo docker buildx imagetools create --append -t $DOCKER_REPO/postgrest:$PGRST_VERSION $DOCKER_REPO/postgrest:$PGRST_VERSION-arm
[ "$PGRST_VERSION" != "devel" ] && sudo docker buildx imagetools create --append -t $DOCKER_REPO/postgrest:latest $DOCKER_REPO/postgrest:$PGRST_VERSION-arm
sudo docker buildx imagetools create --append -t $DOCKER_REPO/postgrest:$PGRST_TAG $DOCKER_REPO/postgrest:$PGRST_TAG-arm
[ "$PGRST_VERSION" != "devel" ] && sudo docker buildx imagetools create --append -t $DOCKER_REPO/postgrest:latest $DOCKER_REPO/postgrest:$PGRST_TAG-arm

sudo docker logout
72 changes: 42 additions & 30 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ jobs:
steps:
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
with:
fetch-tags: true
ssh-key: ${{ secrets.POSTGREST_SSH_KEY }}
- name: Tag latest commit
run: |
Expand Down Expand Up @@ -151,7 +152,7 @@ jobs:
- build
- arm
steps:
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
- name: Check the version to be released
run: |
cabal_version="$(grep -oP '^version:\s*\K.*' postgrest.cabal)"
Expand Down Expand Up @@ -229,35 +230,41 @@ jobs:
echo "Releasing version ${GITHUB_REF_NAME} on GitHub..."

if [ "${GITHUB_REF_NAME}" == "devel" ]; then
isprerelease=1
# To replace the existing release, we must first delete the old assets,
# then modify the release, then add the new assets.
gh release view devel --json assets \
| jq -r '.assets[] | .name' \
| xargs -rn1 \
gh release delete-asset -y devel
gh release edit devel \
-t devel \
--verify-tag \
-F artifacts/release-changes/CHANGES.md \
--prerelease
gh release upload --clobber devel release-bundle/*
else
gh release create "${GITHUB_REF_NAME}" \
-t "${GITHUB_REF_NAME}" \
--verify-tag \
-F artifacts/release-changes/CHANGES.md \
release-bundle/*
fi

gh release delete "${GITHUB_REF_NAME}" || true
gh release create "${GITHUB_REF_NAME}" \
-F artifacts/release-changes/CHANGES.md \
${isprerelease:+"--prerelease"} \
release-bundle/*


docker:
name: Release / Docker Hub
runs-on: ubuntu-22.04
needs:
- prepare
if: |
vars.DOCKER_USER &&
vars.DOCKER_REPO && vars.DOCKER_USER &&
(success() || needs.prepare.result == 'success')
env:
DOCKER_REPO: postgrest
DOCKER_USER: stevechavez
DOCKER_REPO: ${{ vars.DOCKER_REPO }}
DOCKER_USER: ${{ vars.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
steps:
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
- name: Setup Nix Environment
uses: ./.github/actions/setup-nix
with:
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
tools: release.dockerHubDescription.bin
- name: Download Docker image
uses: actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e # v4.1.7
with:
Expand All @@ -278,18 +285,23 @@ jobs:
else
echo "Skipping push to 'latest' tag for pre-release..."
fi
# TODO: Enable dockerhub description update again, once a solution for the permission problem is found:
# https://github.com/docker/hub-feedback/issues/1927
# - name: Update descriptions on Docker Hub
# env:
# DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
# run: |
# if [[ -z "$ISPRERELEASE" ]]; then
# echo "Updating description on Docker Hub..."
# postgrest-release-dockerhub-description
# else
# echo "Skipping updating description for pre-release..."
# fi


docker-description:
name: Release / Docker Hub Description
runs-on: ubuntu-22.04
if: |
vars.DOCKER_REPO && vars.DOCKER_USER &&
github.ref == 'refs/tags/devel'
steps:
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
- uses: peter-evans/dockerhub-description@e98e4d1628a5f3be2be7c231e50981aee98723ae # v4.0.0
with:
username: ${{ vars.DOCKER_USER }}
password: ${{ secrets.DOCKER_PASS }}
repository: ${{ vars.DOCKER_REPO }}/postgrest
short-description: ${{ github.event.repository.description }}
readme-filepath: ./nix/tools/release/docker-hub-full-description.md


docker-arm:
Expand All @@ -300,8 +312,8 @@ jobs:
- docker
env:
GITHUB_COMMIT: ${{ github.sha }}
DOCKER_REPO: postgrest
DOCKER_USER: stevechavez
DOCKER_REPO: ${{ vars.DOCKER_REPO }}
DOCKER_USER: ${{ vars.DOCKER_USER }}
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
steps:
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4.1.4
Expand Down
7 changes: 6 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ This project adheres to [Semantic Versioning](http://semver.org/).
- #3404, Show extra information in the `PGRST121` (could not parse RAISE 'PGRST') error - @laurenceisla
+ Shows the failed MESSAGE or DETAIL in the `details` field
+ Shows the correct JSON format in the `hints` field
- #3340, Log when the LISTEN channel gets a notification - @steve-chavez
- #3184, Log full pg version to stderr on connection - @steve-chavez

### Fixed

Expand All @@ -32,13 +34,16 @@ This project adheres to [Semantic Versioning](http://semver.org/).
- #3237, Dump media handlers and timezones with --dump-schema - @wolfgangwalther
- #3323, #3324, Don't hide error on LISTEN channel failure - @steve-chavez
- #3330, Incorrect admin server `/ready` response on slow schema cache loads - @steve-chavez
- #3340, Log when the LISTEN channel gets a notification - @steve-chavez
- #3345, Fix in-database configuration values not loading for `pgrst.server_trace_header` and `pgrst.server_cors_allowed_origins` - @laurenceisla
- #3361, Clarify the `PGRST204` (column not found) error message - @steve-chavez
- #3373, Remove rejected mediatype `application/vnd.pgrst.object+json` from response - @taimoorzaeem
- #3418, Fix OpenAPI not tagging a FK column correctly on O2O relationships - @laurenceisla
- #3256, Fix wrong http status for pg error `42P17 infinite recursion` - @taimoorzaeem
- #3404, Clarify the `PGRST121` (could not parse RAISE 'PGRST') error message - @laurenceisla
- #3267, Fix wrong `503 Service Unavailable` on pg error `53400` - @taimoorzaeem
- #2985, Fix not adding `application_name` on all connection strings - @steve-chavez
- #3424, Admin `/live` and `/ready` now differentiates a failure as 500 status - @steve-chavez
+ 503 status is still given when postgREST is in a recovering state

### Deprecated

Expand Down
1 change: 0 additions & 1 deletion docs/postgrest.dict
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,6 @@ stateful
stdout
supervisees
SvelteKit
syslog
systemd
todo
todos
Expand Down
8 changes: 5 additions & 3 deletions docs/references/admin_server.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Admin Server
############

PostgREST provides an admin server that can be enabled by setting :ref:`admin-server-port` to the port number of your preference.
PostgREST provides an admin server that can be enabled by setting :ref:`admin-server-port`.

.. _health_check:

Expand All @@ -23,7 +23,7 @@ Two endpoints ``live`` and ``ready`` will then be available.
Live
----

The ``live`` endpoint verifies if PostgREST is running on its configured port. A request will return ``200 OK`` if PostgREST is alive or ``503`` otherwise.
The ``live`` endpoint verifies if PostgREST is running on its configured port. A request will return ``200 OK`` if PostgREST is alive or ``500`` otherwise.

For instance, to verify if PostgREST is running while the ``admin-server-port`` is set to ``3001``:

Expand All @@ -38,7 +38,7 @@ For instance, to verify if PostgREST is running while the ``admin-server-port``
Ready
-----

In addition, the ``ready`` endpoint checks the state of the :ref:`connection_pool` and the :ref:`schema_cache`. A request will return ``200 OK`` if both are good or ``503`` if not.
Additionally to the ``live`` check, the ``ready`` endpoint checks the state of the :ref:`connection_pool` and the :ref:`schema_cache`. A request will return ``200 OK`` if both are good or ``503`` if not.

.. code-block:: bash

Expand All @@ -48,6 +48,8 @@ In addition, the ``ready`` endpoint checks the state of the :ref:`connection_poo

HTTP/1.1 200 OK

PostgREST will try to recover from the ``503`` state with :ref:`automatic_recovery`.

Metrics
=======

Expand Down
2 changes: 1 addition & 1 deletion docs/references/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ admin-server-port
**In-Database** `n/a`
=============== =======================

Specifies the port for the :ref:`health_check` endpoints.
Specifies the port for the :ref:`admin_server`.

.. _app.settings.*:

Expand Down
25 changes: 20 additions & 5 deletions docs/references/connection_pool.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,6 @@ A connection pool is a cache of reusable database connections. It allows serving

Minimizing connections is paramount to performance. Each PostgreSQL connection creates a process, having too many can exhaust available resources.

Connection String
-----------------

For connecting to the database, the pool requires a connection string. You can configure it using :ref:`db-uri`.

.. _pool_growth_limit:
.. _dyn_conn_pool:

Expand All @@ -22,6 +17,26 @@ To conserve system resources, PostgREST uses a dynamic connection pool. This ena

- If all the connections are being used, a new connection is added. The pool can grow until it reaches the :ref:`db-pool` size. Note that it’s pointless to set this higher than the ``max_connections`` setting in your database.
- If a connection is unused for a period of time (:ref:`db-pool-max-idletime`), it will be released.
- For connecting to the database, the :ref:`authenticator <roles>` role is used. You can configure this using :ref:`db-uri`.

Connection Application Name
~~~~~~~~~~~~~~~~~~~~~~~~~~~

PostgREST sets the connection `application_name <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-FALLBACK-APPLICATION-NAME>`_ for all of its used connections.
This is useful for PostgreSQL statistics and logs.

For example, you can query `pg_stat_activity <https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-ACTIVITY-VIEW>`_ to get the PostgREST version:

.. code-block:: postgres

select distinct usename, application_name
from pg_stat_activity
where usename = 'authenticator';

usename | application_name
---------------+--------------------------
authenticator | PostgREST 12.1


Connection lifetime
-------------------
Expand Down
Loading
Loading