README.md

Rodbot

Minimalistic yet polyglot framework to build chat bots on top of a Roda backend for chatops and fun.

• Homepage
• API
• Author: Sven Schwyn - Bitcetera

Thank you for supporting free and open-source software by sponsoring on GitHub or on Donorbox. Any gesture is appreciated, from a single Euro for a ☕️ cup of coffee to 🍹 early retirement.

Table of Contents

Install
Anatomy
   App Service
   Relay Services
   Schedule Service
CLI
Request
Say
Routes and Commands
Database
Environments
Credentials
Plugins
Environment Variables
Development

Install

Security

This gem is cryptographically signed in order to assure it hasn't been tampered with. Unless already done, please add the author's public key as a trusted certificate now:

gem cert --add <(curl -Ls https://raw.github.com/svoop/rodbot/main/certs/svoop.pem)

Generate New Bot

Similar to other frameworks, generate the files for your new bot as follows:

gem install rodbot --trust-policy MediumSecurity
rodbot new my_bot
cd my_bot

For the bot to be useful at all, you should choose one of the supported relay service plugins. Say, you'd like to interact via Matrix:

bundle config set --local with matrix
bundle install

You can use more than one plugin of course. Please note that you have to list them separated with a space:

bundle config set --local with matrix slack
bundle install

Please refer to the Matrix plugin README for more on how to configure and authorise this relay service.

Time to add Git to the mix. Both gems.locked and .bundle are included in order to use the same gems and versions both for local development and deployment to production:

git init
git add .
git commit -m "Bootstrap Rodbot"

You're all set, let's have a look at what Rodbot can do for you:

bundle exec rodbot --help

Anatomy

The bot consists of three kinds of services interacting with one another:

RODBOT                                                            EXTERNAL
╭╴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ╶╮
╷ ╭──────────────────╮  <─────>  ╭──────────────────╮       ╷
╷ │ APP              │  <───╮    │ RELAY - Matrix   ├╮  <──────>  [1] Matrix
╷ ╰──────────────────╯  <─╮ │    ╰┬─────────────────╯├╮  <──┼──>  [1] simulator
╷                         │ │     ╰┬─────────────────╯│   <────>  [1] ...
╷                         │ │      ╰──────────────────╯     ╵
╷                         │ │                               ╵
╷                         │ │     ╭──────────────────╮      ╵
╷                         │ ╰──>  │ SCHEDULE         │  <───┼───  [2] clock
╷                         │       ╰──────────────────╯      ╷
╷                         │                                 ╷
╷                         ╰─────────────────────────────────────  [3] webhook caller
╰╴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ╶╯

App Service

The app service is a Roda app where the real action happens. It acts on and responds to HTTP requests from:

• commands forwarded by relay services
• timed events triggered by the schedule service
• third party webhook calls e.g. from GitLab, GitHub etc

See Rodbot::Config::DEFAULTS for available config settings and their defaults.

Roda

The Roda app is located in the app directory. It contains:

• app.rb – Roda app class where new routes are added using run statements
• routes\ – Directory which contains one route file for every run statement
• views\ – Directory which contains layouts and views called with view in route files

For an example, take a look at app/routes/help.rb generated as part of every new Rodbot app.

The app.rb loads the Rodbot plugin with plugin :rodbot. This Roda plugin is a necessary dependency for many Rodbot plugins and does two things.

It loads the following Roda plugins:

• multi_run
• environments
• heartbeat
• public
• run_append_slash
• halt
• unescape_path
• render

It also loads the following Roda extension provided by Rodbot:

• Shortcut r.arguments for r.params['arguments']

Host

The app service binds to localhost by default and therefore isolates it from the internet. In case you want to make it publicly reachable, you have to set the RODBOT_APP_HOST environment variable to a public IP. Or to bind to all IPs of all interfaces:

export RODBOT_APP_HOST=0.0.0.0

Ports

The app service binds to the base port 7200 by default. However, each relay service needs a predictable port to bind to as well, which is why the next few following ports must not be in use already. If you have to, you can change the base port in config/rodbot.rb:

port 12345

Commands

All top level GET requests such as GET /foobar are commands and therefore accessible by relays, for instance using !foobar on Matrix.

Responses have to be either of the following content types:

• text/plain; charset=utf-8
• text/markdown; charset=utf-8

Please note that the Markdown might get stripped on communication networks which feature only limited or no support for Markdown.

The response may contain special tags which have to be replaced appropriately by the corresponding relay service:

Tag	Replaced with
[[SENDER]]	Mention the sender of the command.
[[EVERYBODY]]	Mention everybody.

Other Routes

All higher level requests such as GET /foo/bar are not accessible by relays. Use them to implement other aspects of your bot such as webhooks or schedule tasks.

Relay Services

The relay service act as glue between the app service and external communication networks such as Matrix.

Each relay service does three things:

• Proactive: It creates and listens to a local TCP socket which accepts and forwards messages. See below for more on this.
• Reactive: It reads messages, detects commands usually beginning with a !, forwards them to the app service and writes the HTTP response back as a message to the communication network.
• Test: It detects the !ping command and replies with "pong" without hitting the app service.

You can simulate such a communication network locally:

rodbot simulator

Enter the command !pay EUR 123 and you see the request GET /pay?argument=EUR+123 hitting the app service.

TCP Socket

The TCP socket is primarily used by other Rodbot services to forward messages to the corresponding external communication network. However, you can use these sockets for non-Rodbot processes as well e.g. to issue notifications when events happen on the host running Rodbot.

Simply connect to a socket and submit the message as plain text or Markdown in UTF-8. Multiple lines are allowed, to finish and post the message, append the EOT character (\x04 alias Ctrl-D).

Such simple messages are always posted to the primary room (aka: channel, group etc) of the communication network. For more complex scenarios, please take a look at message objects which may contain meta information as well.

Schedule Service

The schedule service is a Clockwork process which triggers Ruby code asynchronously as configured in config/schedule.rb.

It's a good idea to have the app service do the heavy lifting while the schedule simply fires the corresponding HTTP request.

A word or two on time zones since they are particularly important for schedules:

Automatic discovery of the local time zone and DST status is rather unreliable. Therefore, Rodbot expects you to set the time zone in config/rodbot.rb using time_zone. See ls /usr/share/zoneinfo for valid values. To correctly handle DST, you should use geographical zones like Europe/Paris rather than technical zones like CET. If time_zone is not defined, the environment variable TZ is read instead. And if TZ isn't set neither, Rodbot falls back to Etc/UTC.

Also, make sure the time zone data is available where you deploy your bot to. The official Alpine-based Ruby images for instance doesn't come with it preinstalled, so you either have to RUN apk add --no-cache tzdata in the Dockerfile or add the tzinfo-data gem to the bundle for TZ to have any effect at all.

CLI

The rodbot CLI is the main tool to manage your bot. For a full list of functions:

rodbot --help

Starting and Stopping Services

While working on the app service, you certainly want to try routes:

rodbot start app

This starts the server in the current terminal. You can set breakpoints with binding.irb, however, if you prefer a real debugger:

rodbot start app --debugger

This requires the debug gem and adds the ability to set breakpoints with debugger.

Here's how to start single services in the background:

rodbot start app --daemonize

You can also start all services at once in which case the services must run in the background and therefore the --daemonize is implied and may be omitted:

rodbot start

Finally, to stop all running Rodbot services:

rodbot stop

Deployment

While controlling Rodbot as mentioned in the previous section is okay for local development, deploying the bot to production comes in a gazillion scenarios. Rodbot helps you with scaffolds for some of them. To get the list of all deploy scaffolds:

rodbot deploy --help

⚠️ It's near impossible to include such deployment scenarios in the test suite. If you find an error or have an improvement, please submit an issue!

Let's take a quick look at the two most common scenarios:

Docker

To run all of Rodbot in one single Docker service:

rodbot deploy docker

In case you prefer to split each service into its own container:

rodbot deploy docker --split

The Docker deployment is a compose.yml file, so you might want to write it to disk:

rodbot deploy docker >compose.yml

Procfile

The Procfile was introduced by Heroku and is nowadays supported many cloud providers as well as tools for local development.

While a monolith approach is certainly possible, it makes more sense to split each service into its own process:

rodbot deploy procfile --split

As per convention, the Procfile should be placed in the root of the project:

rodbot deploy procfile --split >Procfile

It's easy to test drive using a process manager such as Foreman:

gem install foreman
foreman start

For more control and debug features, you might want to try Overmind instead e.g. installed via Homebrew:

brew install overmind
overmind start

Request

To query the app service, you can either use the bundled HTTPX gem or the following convenience wrapper:

response = Rodbot.request('/time', params: { zone: 'UTC' })

This uses the default method: :get and the default timeout: 10 seconds, it returns an instance of HTTPX::Response:

response.code   # => 200
response.body   # => '2023-09-06 22:51:50.231703 UTC'

Say

You can send proactive messages to communication networks with Rodbot.say.

Since you're not limited to just one relay plugin, you have to configure which of them shall post messages submitted with Rodbot.say by adding say true in config/rodbot.rb. Here's an example for the Matrix relay plugin:

plugin :matrix do
  say true
  (...)
end

With this in place, you can now submit messages from just about anywhere, most notably app service routes and schedule service jobs.

say("Hello, World!")

You can further narrow where to post the message if you specify the relay plugin explicitly:

say("Hello, Slack!", on: :slack)

Routes and Commands

Adding new tricks to your bot boils down to adding routes to the app service which is powered by Roda, a simple yet very powerful framework for web applications: Easy to learn (like Sinatra) but really fast and efficient. Take a minute and get familiar with the basics of Roda.

Rodbot relies on MultiRun to spread routes over more than one routing file. This is necessary for Rodbot plugins but is entirely optional for your own routes.

⚠️ At this point, keep in mind that any routes at the root level like /pay or /calculate can be accessed via chat commands such as !pay and !calculate. Routes which are nested further down, say, /myapi/users are off limits and should be used to trigger schedule events and such. Make sure you don't accidentally add routes to the root level you don't want people to access via chat commands, not even by accident.

To add a simple "Hello, World!" command, all you have to do is add a route /hello. A good place to do so is app/routes/hello.rb:

module Routes
  class Hello < App

    route do |r|

      # GET /hello
      r.root do
        response['Content-Type'] = 'text/plain; charset=utf-8'
        'Hello, World!'
      end

    end

  end
end

To try, start the app service with rodbot start app and fire up the simulator with rodbot simulator:

rodbot> !hello
Hello, World!

Try to keep these route files thin and extract the heavy lifting into service classes. Put those into the lib directory where they will be autoloaded by Zeitwerk.

Database

Your bot might be happy dealing with every command as an isolated event. However, some implementations require data to be persisted between requests. A good example is the OTP plugin which needs a database to assure each one-time password is accepted once only.

Rodbot implements a very simple key/value database which is completely optional and supports a few different backends.

Redis

For the Redis backend to work, you have to install the corresponding Bundler group:

bundle config set --local with redis
bundle install

Then set the connection URL in config/rodbot.rb:

db 'redis://localhost:6379/10'

Hash

The Hash backend is not thread-safe and therefore shouldn't be used in production. To use it, simply add the following to config/rodbot.rb:

db 'hash'

Write and Read Data

With this in place, you can access the database with Rodbot.db:

Rodbot.db.flush                                        # => Rodbot::Db

Rodbot.db.set('foo') { 'bar' }                         # => 'bar'
Rodbot.db.get('foo')                                   # => 'bar'
Rodbot.db.scan('*')                                    # => ['foo']
Rodbot.db.delete('foo')                                # => 'bar'
Rodbot.db.get('foo')                                   # => nil

Rodbot.db.set('lifetime', expires_in: 1) { 'short' }   # => 'short'
Rodbot.db.get('lifetime')                              # => 'short'
sleep 1
Rodbot.db.get('lifetime')                              # => nil

For a few more tricks, see the Rodbot::Db docs.

Environments

Similar to other frameworks, Rodbot features different environments which affect the way certain processes work. Use the environment variable RODBOT_ENV to set control this:

Value	Meaning
development	This is the default environment used for local develoment.
production	Use this environment when you deploy Rodbot.
test	This environment is set for the automated tests of Rodbot.

The current environment can be programmatically queried:

ENV['RODBOT_ENV'] = "production"
Rodbot.env.current        # => "production"
Rodbot.env.production?    # => true
Rodbot.env.development?   # => false

You can use the more generic alternative APP_ENV as well, however, if RODBOT_ENV is defined, it takes precedence over APP_ENV.

Credentials

In order not to commit secrets to repositories or environment variables, Rodbot bundles the dry-credentials gem and exposes it via the rodbot credentials CLI command. The secrets are then available in your code like Rodbot.credentials.my_secret and the encrypted files are written to config/credentials.

Plugins

Rodbot aims to keep its core small and add features via plugins, either built-in or provided by gems.

Built-In Plugins

Name	Dependencies	Description
:matrix	yes	relay service for the Matrix communication network
:slack	yes	relay service for the Slack communication network
:otp	yes	guard commands with one-time passwords
:gitlab_webhook	no	event announcements from GitLab
:github_webhook	no	event announcements from GitHub
:hal	no	feel like Dave (demo)
:word_of_the_day	no	word of the day announcements (demo)

You have to install the corresponding Bundler group in case the plugin depends on extra gems. Here's an example for the :otp plugin listed above:

bundle config set --local with otp
bundle install

How Plugins Work

Given the following config/rodbot.rb:

plugin :my_plugin do
  color 'red'
end

Plugins provide one or more extensions each of which extends one of the services. In order only to spin things up when needed, the plugin may contain the following files:

• rodbot/plugins/my_plugin/app.rb – add routes and/or extend Roda
• rodbot/plugins/my_plugin/relay.rb – add a relay
• rodbot/plugins/my_plugin/schedule.rb – add schedules to Clockwork

Whenever a service boots, the corresponding file is required.

In order to keep these plugin files slim, you should extract functionality into service classes. Just put them into rodbot/plugins/my_plugin/lib/ and use require_relative where you need them.

Create Plugins

You can create plugins in any of the following places:

• inside your Rodbot instance:
  /lib/rodbot/plugins/my_plugin
• in a vendored gem "rodbot-my_plugin":
  /lib/rodbot/vendor/gems/rodbot-my_plugin/lib/rodbot/my_plugin
• in a published gem "rodbot-my_plugin":
  /lib/rodbot/plugins/my_plugin

Please adhere to common naming conventions and use the dashed prefix rodbot- (and Module Rodbot), however, underscores in case the remaining gem name consists of several words.

App Extension

An app extension rodbot/plugins/my_plugin/app.rb defines the module App and looks something like this:

module Rodbot
  class Plugins
    module MyPlugin
      module App

        module Routes < ::App
          route do |r|
            # GET /my_plugin
            r.root do
              # called by command !my_plugin
            end

            # GET /my_plugin/whatever
            r.get('whatever') do
              # not reachable by any command
            end
          end
        end

        module ResponseMethods
          # (...)
        end

      end
    end
  end
end

The Routes module contains all the routes you would like to inject.

The App module can be used to extend all aspects of Roda.

For an example, take a look at the :hal plugin.

Relay Extension

A relay extension rodbot/plugins/my_plugin/relay.rb defines the class Relay and looks something like this:

module Rodbot
  class Plugins
    module MyPlugin
      class Relay < Rodbot::Relay

        def loops
          SomeAwesomeCommunicationNetwork.connect
          [method(:read_loop), method(:write_loop)]
        end

        private

        def read_loop
          loop do
            # Listen in on the communication network
          end
        end

        def write_loop
          loop do
            # Post something to the communication network
          end
        end

      end
    end
  end
end

The loops method must returns an array of callables (e.g. a Proc or Method) which will be called when this relay service is started. The loops must trap the INT signal.

Proactive messages require other parts of Rodbot to forward a message directly. To do so, the relay has to implement a TCP socket. This socket must bind to the IP and port you get from the bind method which returns an array like ["localhost", 7201].

For an example, take a look at the :matrix plugin.

Schedule Extension

A schedule extension rodbot/plugins/my_plugin/schedule.rb defines the class Schedule and looks something like this:

module Rodbot
  class Plugins
    module MyPlugin
      class Schedule

        def initialize
          Clockwork.every(1.day, -> { tea }, at: '16:00')
        end

        private

        def tea
          Rodbot.say "Time for a cup of tea!"
        end

      end
    end
  end
end

The initializer must set at least one schedule using Clockwork.every – see the Clockwork docs.

For an example, take a look at the :word_of_the_day plugin.

Toolbox

Before you write a plugin, familiarise yourself with the following bundled helpers:

• Rodbot::Refinements – just a few handy extensions to Ruby core classes
• Rodbot::Memoize – environment-aware memoization for method return values

Environment Variables

Environment variables are used for the configuration bits which cannot or should not be part of config/rodbot.rb mainly because they have to be set on the infrastructure level.

Variable	Description	Default
RODBOT_ENV	Environment	development
RODBOT_CREDENTIALS_DIR	Override the directory containing encrypted credentials files	config/credentials/
RODBOT_APP_HOST	Override where to locally bind the app service	localhost
RODBOT_APP_URL	Override where to locally reach the app service	http://localhost
RODBOT_RELAY_HOST	Override where to bind the relay services	localhost
RODBOT_RELAY_URL_XXX	Override where to locally reach the given relay service XXX (e.g. MATRIX)	tcp://localhost

Development

To install the development dependencies and then run the test suite:

bundle install
bundle exec rake    # run tests once
bundle exec guard   # run tests whenever files are modified

Some tests require Redis and will be skipped by default. You can enable them by setting the following environment variable along the lines of:

export RODBOT_SPEC_REDIS_URL=redis://localhost:6379/10

You're welcome to join the discussion forum to ask questions or drop feature ideas, submit issues you may encounter or contribute code by forking this project and submitting pull requests.