Deployment Options

[JoeToeniskoetter]

Wondering what deployment options everyone is using. Having the option to embed this library into an existing go rest API was extremely attractive to me but I'm having a hard time figuring out a deployment strategy.

I was hoping to deploy the entire application (REST API with embedded MQTT broker) on something like Digital Ocean App Platform. Where the rest api would be accessible via https://mydomain.com/api and the broker would be available to my IOT devices via mydomain.com:1883. Unfortunately it looks like DO app platform does not allow raw TCP connections.

Any suggestions for platforms that would allow for this type of setup?

Thanks

[dgduncan]

Hey @JoeToeniskoetter . What I do is that I have a separate repository that serves as the broker. I have written scripts so that is uses certbot and Let'sEncrypt to generate certificates for SSL. I build everything into a docker image that then gets deployed to a GCP Compute Instance using Cloud Build. GCP Compute does not have any such limitations for raw TCP connections as what you described is the limitation for Digital Ocean.

GCP Compute or AWS EC2s are fantastic for these types of things; however, I would not consider them the most "user friendly".

[JoeToeniskoetter]

@dgduncan Ah nice! Digital Ocean also offers a bare VPS solution similar to GCP Compute. I just like the convenience of pointing to my github repo and clicking 'deploy'.

Do you also have a REST/GraphQL API running on the same Compute instance? If so, how do you communicate between the broker and API servers?

As far as the SSL certs, I'm not sure if it's possible but I believe I would need two different certs pairs in that instance. One for the API server that can be regularly refreshed, and one for the MQTT broker that is valid for a long time so the IoT devices that connect to it don't have to be re-flashed every time the cert expires. Unless I am misunderstanding SSL certs which is likely.

[dgduncan]

I technically do; however, it is mostly used for testing so I have not configured any SSL certificates for that. Also the port the API is exposed is specifically locked down to an internal VPC connector compared to the MQTTS port that is public.

The one I used gets invalidated on a relatively short period; however, it can be validated using one of the longer lived intermediate certificates that lives on device. I use ESPs which I can update the certificate store via OTA in the case that the certificate will expire within months.

[mochi-co]

If either of you have any good out-of-the-box solutions for these deployments, I would be happy to add them to repo somehow 👍🏻

[dgduncan]

I can think of some ways to make an "out of the box" solution for deployments; however, I do feel that outside of a more paramaterized and/or config driven approach to deployments, it won't be very helpful.

Also I have been working on a drop in solution that will configure certbot auto renewal that may be interesting for people looking to use a more secure solution that all runs in process.

[SimonMacIntyre]

I have a bunch of services in a local development docker compose setup. I want to simply add the mochi as a service, this actually works really well (with some limitations)

  mqtt:
    build: https://github.com/mochi-co/mqtt.git
    ports:
      - 1883:1883
      - 1882:1882
      - 8080:8080

This enough to get the default setup going and working, and benchmark, etc. Of course, I actually have a few things I want to do next:

• TLS, so mount my certificates (no problem, just docker compose mount them and reference them)
• Modify the main.go or override it, to have WSS, MQTTS, and the 8080 default as listeners.

The most common workflow for my local docker services is to extend from a base (like this repository, baked into a default image), and then override some of the files and stuff. In more complex use cases, of course we make our own dockerfile to do the same thing. In this case that is a little difficult without cloning the entire repository into my project.

I guess that is not possible yet until the Dockerfile config file from the roadmap is tackled. I would try it, but I am still very new to Golang!

At this time I don't see any other path forward for my use case locally besides a git submodule, so I am going to try that!

If anyone has advice beyond what I reasoned above, I'd be happy to hear it! I'm not raising any problems or solutions here, just curious if anyone had any insight and also adding my use case to the discussion of deployments!

My plan is basically
- use a git submodule
- overwrite some files in it manually
- build the dockerfile (which now has my file overwrites)

It is a bit hacky, but might work for the interim for my local work. The reason I am exploring this approach, is I need this to just work for other team members. I don't want any manual steps for others. Basically I need it scriptable to reach the final state and running docker container.~~

I also explored other hacky solutions like cloning the repo into a Dockerfile 😆

Update: cloning it in the Dockerfile is actually very quick and easy, and will do for now!

[SimonMacIntyre]

Actually, I think git cloning inside the Dockerfile, for now, is more KISS-y. This is my dockerfile which seems to be working, basically a copy of this source one with minor changes to allow override. My TLS listener's are working with my custom certs 🥳

FROM golang:1.19.0-alpine3.15 AS builder

# Update apk and install git
RUN apk update
RUN apk add git

# Clone the repository
WORKDIR /app
RUN git clone https://github.com/mochi-co/mqtt.git .

# Copy your own main.go file, overwriting the original
COPY main.go ./cmd/main.go

# Install dependencies
RUN go mod download

# Build the application
RUN go build -o /mochi ./cmd

# Start a new build stage
FROM alpine

# Set the working directory
WORKDIR /

# Copy the built application
COPY --from=builder /mochi .

# Expose necessary ports
EXPOSE 8883
EXPOSE 8882
EXPOSE 8080

# Set the entrypoint
ENTRYPOINT [ "/mochi" ]

The service in the docker compose

  mqtt:
    build:
      context: ./.local/mqtt/mochi
      dockerfile: Dockerfile.local
    ports:
      - 8883:8883
      - 8882:8882
      - 8080:8080
    environment:
      VIRTUAL_HOST: mqtt.your-host.anything
    volumes:
      - mount-your-certs

Then all I have so far is a single main.go which you see me copying in the Dockerfile.

[mochi-co]

We're going to be looking into improving the docker and configuration support in the near future, so it would be great to have your input @SimonMacIntyre!

[dgduncan]

I opened up a discussion thread for file based configuration here for anyone interested