Use ENVs when using docker

[kdev]

When I setup this using docker I found it very unhelpful I had to build it myself after having to put everything in a config file instead of just using docker pull to pull this from Docker Hub or GH Packages and then using ENVs to configure.
Would it be possible to rewrite this to use ENVs for easier setup and administration?

[PhlexPlexico]

Hoo boy, that would be quite a bit of a task and involve a lot more testing to ensure everything is working as intended. I think what I could do is maybe have some checks for .env files first upon creation? @ebuttonsdude what are your thoughts on this? Is there some Docker magic that you could pull that would involve less change to the application itself?

[PhlexPlexico]

I think the whole point behind that is still following the best practices for docker - one application per container. That way you can spin up several instances of the API and just one instance of the database (load balancing example) without having any extra impact on resources if you had multiple database running in the app container.

[kdev]

Might be the case but I still have to create an external DB when using Docker Compose and cannot just use it as I would creating the DB over the compose file and then just connecting to it.
(This basically relates to the ENV stuff this was initially about)

[kdev]

Any updates on this?

This mainly requires 2 things:

1. read variables from env and put them into a config file/object
2. migrate the DB once at first start (which can be done using an ORM or checking it via the mysql package)

[PhlexPlexico]

Hey! I've been pretty busy with work/other projects going on atm. But I'm just waiting to hear back from a buddy of mine in regrads to this... I would prefer it to be external to the app (i.e. Docker scripts doing this). I don't really see the difference of just copying/pasting the config file and just bringing that across installations. Is this not possible? If not, maybe there could be a way the Docker script can pick these up. I'd like to avoid making changes to configs as much as possible at this point as I'm not completely sold on .env files and making them interchangeable with the current json configs.

As for your second point - the Dockerfile already does this. Upon first launch it creates and migrates the database to the given port/host:

G5API/Dockerfile

Line 38 in aaba738

RUN yarn migrate-create-prod && yarn migrate-prod-upgrade && yarn

[kdev]

ey! I've been pretty busy with work/other projects going on atm. But I'm just waiting to hear back from a buddy of mine in regrads to this... I would prefer it to be external to the app (i.e. Docker scripts doing this). I don't really see the difference of just copying/pasting the config file and just bringing that across installations. Is this not possible? If not, maybe there could be a way the Docker script can pick these up. I'd like to avoid making changes to configs as much as possible at this point as I'm not completely sold on .env files and making them interchangeable with the current json configs.

For now it's not a huge problem to do this but actually putting the config in a Docker Image prevents (at least me) from putting it on some platform like Docker Hub.

This line is executed when the Image is built, not on the first launch of the application which is not nice when I do not have a DB setup when building the image and it also prevents me from using this in combination with Docker Compose.

G5API/Dockerfile

Line 38 in aaba738

RUN yarn migrate-create-prod && yarn migrate-prod-upgrade && yarn

[ebuttonsdude]

I created a pull request for this:
#105

I don't think that SQL should be integrated into this as some people may want to use a hosted database. You should be able to simply add mariadb into a docker-compose script and have it start up before the API

[kdev]

I don't think that SQL should be integrated into this as some people may want to use a hosted database.

I think I wasn't clear enough to be honest. Putting the DB into the image isn't a good thing but having it required on startup isn't a nice option either because then you can never build an image using CI which is somehow contraproductive.

[PhlexPlexico]

I mean, if you're using CI to build and deploy wouldn't it just be the step prior to if you need to create a database?

[kdev]

Not necessarily. If I'm building the image I need to setup a DB first so when using it with docker-compose I want to first build the image and then use a DB configured in my docker-compose.yml

[kdev]

Could this be a DB Connection issue?
I am running this inside Docker Compose so it shouldn't but still could be a misconfiguration on my side

g5api                                | $ MYSQL_FLAGS="-CONNECT_WITH_DB" db-migrate --env production --config config/production.json db:create get5
g5api                                | [ERROR] SyntaxError: Unexpected token r in JSON at position 695
g5api                                |     at JSON.parse (<anonymous>)
g5api                                |     at exports.parse (/Get5API/G5API/node_modules/rc/lib/utils.js:15:17)
g5api                                |     at addConfigFile (/Get5API/G5API/node_modules/rc/index.js:30:20)
g5api                                |     at module.exports (/Get5API/G5API/node_modules/rc/index.js:46:20)
g5api                                |     at module.exports (/Get5API/G5API/node_modules/db-migrate/lib/commands/set-default-argv.js:122:47)
g5api                                |     at new dbmigrate (/Get5API/G5API/node_modules/db-migrate/api.js:75:10)
g5api                                |     at Object.module.exports.getInstance (/Get5API/G5API/node_modules/db-migrate/index.js:85:10)
g5api                                |     at /Get5API/G5API/node_modules/db-migrate/bin/db-migrate:32:25
g5api                                |     at /Get5API/G5API/node_modules/resolve/lib/async.js:133:25
g5api                                |     at maybeRealpath (/Get5API/G5API/node_modules/resolve/lib/async.js:41:9)

[kdev]

Using no 's does not change a thing except from the position being 702 not 695 anymore

[kdev]

My docker-compose file:

version: "2"

services:
# ... some nginx config

  db:
    restart: always
    image: mariadb
    environment:
      MYSQL_ROOT_PASSWORD: <removed>
      MYSQL_DATABASE: get5
      MYSQL_USER: get5
      MYSQL_PASSWORD: <removed>
    
    volumes:
      - ./volumes/db:/docker-entrypoint-initdb.d
      - ./volumes/db/data:/va/lib/mysql
      
    networks:
      - db_network

  g5api-container:
    restart: always
    build: ./G5API
    networks:
      - db_network
    depends_on:
      - db
    container_name: g5api
    environment:
      VIRTUAL_HOST: <removed>
      VIRTUAL_NETWORK: nginx-proxy
      VIRTUAL_PORT: 80
      LETSENCRYPT_HOST: <removed>
      LETSENCRYPT_EMAIL: <removed>

      PORT: 80
      HOSTNAME: <removed>
      DBKEY: <removed>
      STEAMAPIKEY: <removed>
      SHAREDSECRET: <removed>
      CLIENTHOME: http://localhost:80
      APIURL: <removed>
      SQLUSER: get5
      SQLPASSWORD: <removed>
      DATABASE: get5
      SQLHOST: 127.0.0.1
      SQLPORT: 3306
      SUPERADMINS: 76561198288255973,76561197996838261
      REDISPASSWORD: <removed>
    volumes:
     - ./volumes/redis:/RedisFiles

networks:
  db_network:

[PhlexPlexico]

Okay, well this is going to sound bad (and an issue should be created, should this fix the problem), but try and change your passwords to not include any special characters (&,!,?, Etc). And see if that fixes it? As for verifying as well, if you grab the production.json file and run it through a JSON Parser it may tell you what's wrong, too.

[PhlexPlexico]

And I think CLIENTHOME should just be http://localhost since port 80 in is defined up at the top of that block?

[kdev]

Okay, well this is going to sound bad (and an issue should be created, should this fix the problem), but try and change your passwords to not include any special characters (&,!,?, Etc). And see if that fixes it? As for verifying as well, if you grab the production.json file and run it through a JSON Parser it may tell you what's wrong, too.

I will try to remove any special chars ^^

And I think CLIENTHOME should just be http://localhost since port 80 in is defined up at the top of that block?

Will do ^^