What are the steps for deploying to a single server?

[albertski]

I want to deploy my application to a single Digital Ocean Droplet (database & app on one droplet). I'm following the Deploy Rails app and Postgres with Kamal(previously MRSK) on single DigitalOcean server but I get the following error:

ActiveRecord::ConnectionNotEstablished: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?


Caused by:
PG::ConnectionBad: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

deploy.yml:

# Name of your application. Used to uniquely configure containers.
service: my_site

# Name of the container image.
image: me/my_site

# Deploy to these servers.
servers:
  web:
    - 1.1.1.1

registry:
  username: my_user_name

  # Always use an access token rather than real password when possible.
  password:
    - KAMAL_REGISTRY_PASSWORD
# Inject ENV variables into containers (secrets come from .env).
env:
  clear:
    DB_HOST: KAMAL_DB_HOST
  secret:
    - RAILS_MASTER_KEY
    - KAMAL_POSTGRES_PASSWORD

accessories:
  db:
    image: postgres:14
    host: 1.1.1.1
    port: 5432
    env:
      clear:
        POSTGRES_USER: KAMAL_POSTGRES_USER
        POSTGRES_DB: KAMAL_POSTGRES_DB
    directories:
      - data:/var/lib/postgresql/data
  redis:
    image: redis:7.0
    host: 1.1.1.1
    port: 6379
    directories:
      - data:/data

Dockerfile

# syntax = docker/dockerfile:1

# Make sure RUBY_VERSION matches the Ruby version in .ruby-version and Gemfile
ARG RUBY_VERSION=3.1.1
FROM ruby:$RUBY_VERSION-slim as base

# Rails app lives here
WORKDIR /rails

# Set production environment
ENV RAILS_ENV="production" \
    BUNDLE_DEPLOYMENT="1" \
    BUNDLE_PATH="/usr/local/bundle" \
    BUNDLE_WITHOUT="development"


# Throw-away build stage to reduce size of final image
FROM base as build

# Install packages needed to build gems
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y build-essential postgresql postgresql-contrib git curl libpq-dev libvips pkg-config

# Install application gems
COPY Gemfile Gemfile.lock ./
RUN bundle install && \
    rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git && \
    bundle exec bootsnap precompile --gemfile


# Copy application code
COPY . .

# Precompile bootsnap code for faster boot times
RUN bundle exec bootsnap precompile app/ lib/

# Precompiling assets for production without requiring secret RAILS_MASTER_KEY
RUN SECRET_KEY_BASE_DUMMY=1 ./bin/rails assets:precompile


# Final stage for app image
FROM base

# Install packages needed for deployment
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y libsqlite3-0 libvips postgresql-client curl && \
    rm -rf /var/lib/apt/lists /var/cache/apt/archives

# Copy built artifacts: gems, application
COPY --from=build /usr/local/bundle /usr/local/bundle
COPY --from=build /rails /rails

# Run and own only the runtime files as a non-root user for security
RUN useradd rails --home /rails --shell /bin/bash && \
    chown -R rails:rails db log storage tmp
USER rails:rails

# Entrypoint prepares the database.
ENTRYPOINT ["/rails/bin/docker-entrypoint"]

# Start the server by default, this can be overwritten at runtime
EXPOSE 3000
CMD ["./bin/rails", "server"]

Would anyone have any suggestions on getting this to work? I wonder if I need to install the database on the server manually.

[jonduarte]

I am successfully running DB, redis and app server in a single instance.
My setup is pretty similar to yours.

The steps were:

1. Add the required libraries to Dockerfile
2. Setup accessories in config/deploy.yml
3. Run initially kamal setup

If you have run setup before, you can kamal accessory boot [NAME], this will setup the accessory directly in the remote server for you.

[albertski]

Thanks for your help. I ran a kamal remove and then ran kamal accessory boot db but got back:

Status: Downloaded newer image for postgres:14
docker: Error response from daemon: driver failed programming external connectivity on endpoint my_db(c8c0cff84feb6fd7094028bfc9bd9b3ced8c4ee827ef2682f2b235439968dbee): Error starting userland proxy: listen tcp4 0.0.0.0:5432: bind: address already in use.

Do you do anything with Postgres in Docker?

[sonic182]

Seems that you need to specify the database host by ip (127.0.0.1) instead of (localhost) to avoid it trying to connect via unix sockets (files)

[albertski]

When I run docker ps I get back:

CONTAINER ID   IMAGE          COMMAND                  CREATED      STATUS                          PORTS                                       NAMES
14c8818bf06f   traefik:v2.9   "/entrypoint.sh --pr…"   3 days ago   Up 3 days                       0.0.0.0:80->80/tcp, :::80->80/tcp           traefik
4119bd8b96fc   redis:7.0      "docker-entrypoint.s…"   3 days ago   Up 3 days                       0.0.0.0:6379->6379/tcp, :::6379->6379/tcp   my-site-redis
dc527c15c10f   postgres:14    "docker-entrypoint.s…"   3 days ago   Restarting (1) 38 seconds ago                                               my_site

I ran it a couple of times and it always tries to do a restart under status.

[scart88]

You can see the logs why your postgresql is failing

kamal accessory logs db --follow

You can try this config and you can use this tool to tune your Postgresql database https://pgtune.leopard.in.ua/

  env:
    clear:
      DB_HOST: 10.0.0.1

  accessories:
  db:
    image: postgres:15.2
    host: 10.0.0.1
    port: "10.0.0.1:5432:5432"
    add-host: 10.0.0.1
    env:
      clear:
        POSTGRES_USER: <%= ENV["DB_USER"] %>
        POSTGRES_DB: <%= ENV["DB_NAME"] %>
        POSTGRES_PASSWORD: <%= ENV["POSTGRES_PASSWORD"] %>
    directories:
      - db:/var/lib/postgresql/data
    cmd: postgres -c max_connections=100 -c shared_buffers=2GB -c effective_cache_size=6GB -c maintenance_work_mem=512MB -c checkpoint_completion_target=0.9 -c wal_buffers=16MB -c default_statistics_target=100 -c random_page_cost=1.1 -c effective_io_concurrency=200 -c work_mem=10485kB -c min_wal_size=1GB -c max_wal_size=4GB -c max_worker_processes=4 -c max_parallel_workers_per_gather=2 -c max_parallel_workers=4 -c max_parallel_maintenance_workers=2

# config/database.yml
production:
  primary:
    <<: *default
    host: <%= ENV["DB_HOST"] %>
    database: <%= ENV["DB_NAME"] %>
    username: <%= ENV["DB_USER"] %>
    password: <%= ENV['POSTGRES_PASSWORD'] %>
  

kamal accessory boot db or kamal accessory reboot db

You can go even further and create postgresql backups and upload them to S3 using the instructions from here: #355 (comment)

[albertski]

Thanks for sharing all this. Running kamal accessory logs db --follow shows that I am running out of room. I am using the $6 droplet. Perhaps I need to add swap space.

2023-09-25T03:26:05.710541564Z 2023-09-25 03:26:05.708 UTC [1] FATAL:  could not map anonymous shared memory: Cannot allocate memory
2023-09-25T03:26:05.710597932Z 2023-09-25 03:26:05.708 UTC [1] HINT:  This error usually means that PostgreSQL's request for a shared memory segment exceeded available memory, swap space, or huge pages. To reduce the request size (currently 2220384256 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections.
2023-09-25T03:26:05.710604163Z 2023-09-25 03:26:05.708 UTC [1] LOG:  database system is shut down
2023-09-25T03:27:06.257928221Z 
2023-09-25T03:27:06.257973407Z PostgreSQL Database directory appears to contain a database; Skipping initialization
2023-09-25T03:27:06.257978335Z 
2023-09-25T03:27:06.300901814Z 2023-09-25 03:27:06.298 UTC [1] FATAL:  could not map anonymous shared memory: Cannot allocate memory
2023-09-25T03:27:06.300939890Z 2023-09-25 03:27:06.298 UTC [1] HINT:  This error usually means that PostgreSQL's request for a shared memory segment exceeded available memory, swap space, or huge pages. To reduce the request size (currently 2220384256 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections.
2023-09-25T03:27:06.300945988Z 2023-09-25 03:27:06.298 UTC [1] LOG:  database system is shut down

[scart88]

If you use the settings from my example, please note that I added those settings to a VM with 4vCPU and 8GB Ram.
You will have to decrease the number of max connections, shared_buffer and so on.

cmd: postgres -c max_connections=100 -c shared_buffers=2GB -c effective_cache_size=6GB -c maintenance_work_mem=512MB -c checkpoint_completion_target=0.9 -c wal_buffers=16MB -c default_statistics_target=100 -c random_page_cost=1.1 -c effective_io_concurrency=200 -c work_mem=10485kB -c min_wal_size=1GB -c max_wal_size=4GB -c max_worker_processes=4 -c max_parallel_workers_per_gather=2 -c max_parallel_workers=4 -c max_parallel_maintenance_workers=2

You can try something like this

max_connections = 20
shared_buffers = 128MB

[sonic182]

The error you placed at the beigining:

ActiveRecord::ConnectionNotEstablished: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

Indicates that the ruby/rails container is trying to connect to a unix socket for the postgres

You need to indicate the postgres container ip, in the database host variable for the rails container

[albertski]

Thanks everyone for your help. Turns out my issue was that I didn't have my database.yml set correctly:

production:
  <<: *default
  database: <%= ENV["KAMAL_POSTGRES_DB"] %>
  username: <%= ENV["KAMAL_POSTGRES_USER"] %>
  password: <%= ENV["KAMAL_POSTGRES_PASSWORD"] %>
  host: <%= ENV["KAMAL_POSTGRES_HOST"] %>

.env

DOCKER_REGISTRY_TOKEN=
KAMAL_REGISTRY_PASSWORD=
KAMAL_POSTGRES_DB=
KAMAL_POSTGRES_HOST=
KAMAL_POSTGRES_USER=
KAMAL_POSTGRES_PASSWORD=
KAMAL_POSTGRES_PORT=