Skip to content

Commit

Permalink
doc: Initialize documentation
Browse files Browse the repository at this point in the history 
Both structure and huge parts are copied from Icinga DB and were altered
afterwards. The already existing Channel Plugin documentation was
extended.
  • Loading branch information
@oxzi
oxzi committed Jul 4, 2024
1 parent bc3910f commit 3ef3a03
Show file tree
Hide file tree
Showing 11 changed files with 436 additions and 33 deletions.
2 changes: 1 addition & 1 deletion Makefile
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# These variables follow the naming convention from the GNU Make documentation
# but their defaults correspond to the rest of the code (note that changing
# libexecdir here wouldn't affect the default path for the channel plugin
# directory used by the dameon for example).
# directory used by the daemon for example).
#
# https://www.gnu.org/software/make/manual/html_node/Directory-Variables.html
prefix ?= /usr
Expand Down
40 changes: 10 additions & 30 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,40 +1,20 @@
# Icinga Notifications

> **Warning**
> This is an early preview version for you to try, but do not use this in production. There may still be severe bugs
> and incompatible changes may happen without any notice. At the moment, we don't yet provide any support for this.
> [!WARNING]
> This is an early preview version for you to try, but do not use this in production.
> There may still be severe bugs and incompatible changes may happen without any notice.
> At the moment, we don't yet provide any support for this.
Icinga Notifications is a set of components that processes received events from various sources, manages incidents and
forwards notifications to predefined contacts, consisting of:

* The Icinga Notifications daemon (this repository), which receives events and sends notifications
* An [Icinga Web module](https://github.com/Icinga/icinga-notifications-web) that provides graphical configuration and further processing of the data collected by the daemon
* And Icinga 2 and other custom sources that propagate state updates and acknowledgement events to the daemon
* The Icinga Notifications daemon (this repository), which receives events and sends notifications.
* The [Icinga Notifications Web](https://github.com/Icinga/icinga-notifications-web) module,
which provides graphical configuration.
* Icinga 2 or other sources that provide monitoring events that result in notifications.

## Installation

To install Icinga Notifications and get started, you first need to clone this repository.
```bash
git clone https://github.com/Icinga/icinga-notifications.git
```

Next, you need to provide a `config.yml` file, similar to the [example config](config.example.yml), for the daemon.
It is required that you have created a new database and imported the [schema](schema/pgsql/schema.sql) file beforehand.
> **Note**
> At the moment **PostgreSQL** is the only database backend we support.
Additionally, it also requires you to manually insert items into the **source** table before starting the daemon.
```sql
INSERT INTO source
(id, type, name, icinga2_base_url, icinga2_auth_user, icinga2_auth_pass, icinga2_insecure_tls)
VALUES
(1, 'icinga2', 'Local Icinga 2', 'https://localhost:5665', 'root', 'icinga', 'y');
```

Then, you can launch the daemon with the following command.
```go
go run ./cmd/icinga-notifications --config config.yml
```
For more information about how to install or use Icinga Notifications,
just follow the [documentation](https://icinga.com/docs/icinga-notifications/latest).

## License

Expand Down
55 changes: 55 additions & 0 deletions doc/01-About.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Icinga Notifications

!!! warning

This is an early preview version for you to try, but do not use this in production.
There may still be severe bugs and incompatible changes may happen without any notice.
At the moment, we don't yet provide any support for this.

Icinga Notifications is a set of components that processes received events from various sources, manages incidents and
forwards notifications to predefined contacts, consisting of:

* The Icinga Notifications daemon, which receives events and sends notifications.
* The [Icinga Notifications Web](https://icinga.com/docs/icinga-notifications-web/latest/doc/01-About/) module,
which provides graphical configuration.
* Icinga 2 or other sources that provide monitoring events that result in notifications.

## Big Picture

<!-- TODO: add a real picture -->

Because Icinga Notifications consists of several components,
this section tries to help understand how these components relate.

First, the Icinga Notifications configuration resides in a SQL database.
It can be conveniently tweaked via Icinga Notifications Web directly from a web browser.
The Icinga Notifications daemon uses this database to read the current configuration.

As in any Icinga setup, all host and service checks are defined in Icinga 2.
By querying the Icinga 2 API, the Icinga Notifications daemon retrieves state changes, acknowledgements, and other events.
These events are stored in the database and are available for further inspection in the Icinga Notifications Web.

Depending on its configuration, the daemon will take further action on these events.
This optionally includes escalations that are sent through a channel plugin.
Each such channel plugin implements a domain-specific transport, e.g., the `email` channel sends emails via SMTP.
When configured, Icinga Notifications will use channel plugins to notify end users or talk to other APIs.

## Available Channels

Icinga Notifications comes with multiple channels out of the box:

* _email_: Email submission via SMTP
* _rocketchat_: Rocket.Chat
* _webhook_: Configurable HTTP/HTTPS queries for your backend

Additional custom channels can be developed independently of Icinga Notifications,
following the [channel specification](10-Channels.md).

## Installation

To install Icinga Notifications see [Installation](02-Installation.md).

## License

Icinga Notifications and the Icinga Notifications documentation are licensed under the terms of the
[GNU General Public License Version 2](../LICENSE).
109 changes: 109 additions & 0 deletions doc/02-Installation.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
<!-- {% if index %} -->
# Installing Icinga Notifications

The recommended way to install Icinga Notifications is to use prebuilt packages
for all supported platforms from our official release repository.
Please follow the steps listed for your target operating system,
which guide you through setting up the repository and installing Icinga Notifications.

To upgrade an existing Icinga Notifications installation to a newer version,
see the [Upgrading](04-Upgrading.md) documentation for the necessary steps.

<!-- {% else %} -->
<!-- {% if not icingaDocs %} -->
## Installing the Package

The recommended way to install Icinga Notifications is to use prebuilt packages from our official release repository.
If the [repository](https://packages.icinga.com) is not configured yet, please add it first.
Then use your distribution's package manager to install the `icinga-notifications` package
or install [from source](02-Installation.md.d/From-Source.md).
<!-- {% endif %} --><!-- {# end if not icingaDocs #} -->

## Setting up the Database

A MySQL (≥5.5), MariaDB (≥10.1), or PostgreSQL (≥9.6) database is required to run Icinga Notifications.
Please follow the steps listed for your target database,
which guide you through setting up the database and user and importing the schema.

### Setting up a MySQL or MariaDB Database

If you use a version of MySQL < 5.7 or MariaDB < 10.2, the following server options must be set:

```
innodb_file_format=barracuda
innodb_file_per_table=1
innodb_large_prefix=1
```

Set up a MySQL database for Icinga Notifications:

```
# mysql -u root -p
CREATE DATABASE notifications;
CREATE USER 'notifications'@'localhost' IDENTIFIED BY 'CHANGEME';
GRANT ALL ON notifications.* TO 'notifications'@'localhost';
```

After creating the database, import the Icinga Notifications schema using the following command:

```
mysql -u root -p notifications < /usr/share/icinga-notifications/schema/mysql/schema.sql
```

### Setting up a PostgreSQL Database

Set up a PostgreSQL database for Icinga Notifications:

```
# su -l postgres
createuser -P notifications
createdb -E UTF8 --locale en_US.UTF-8 -T template0 -O notifications notifications
echo 'CREATE EXTENSION IF NOT EXISTS citext;' | psql notifications
```

The `CREATE EXTENSION` command requires the `postgresql-contrib` package.

Edit `pg_hba.conf`, insert the following before everything else:

```
local all notifications md5
host all notifications 0.0.0.0/0 md5
host all notifications ::/0 md5
```

To apply these changes, run `systemctl reload postgresql`.

After creating the database, import the Icinga Notifications schema using the following command:

```
psql -U icinga-notifications notifications < /usr/share/icinga-notifications/schema/pgsql/schema.sql
```

## Configuring Icinga Notifications

Icinga Notifications installs its configuration file to `/etc/icinga-notifications/config.yml`,
pre-populating most of the settings for a local setup. Before running Icinga Notifications,
adjust the database credentials and the Icinga Web 2 URL.
The configuration file explains general settings.
All available settings can be found under [Configuration](03-Configuration.md).

## Running Icinga Notifications

The `icinga-notifications` package automatically installs the necessary systemd unit files to run Icinga Notifications.
Please run the following command to enable and start its service:

```
systemctl enable --now icinga-notifications
```

## Installing Icinga Notifications Web

With Icinga 2, Icinga Notifications and the database fully set up, it is now time to install Icinga Notifications Web,
which connects to the database and allows configuring Icinga Notifications.

Please follow the
[Icinga Notifications Web documentation](https://icinga.com/docs/icinga-notifications-web/latest/doc/02-Installation/).

<!-- {% endif %} --><!-- {# end else if index #} -->
10 changes: 10 additions & 0 deletions doc/02-Installation.md.d/From-Source.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Installing Icinga Notifications from Source

Although the provided packages are highly recommended and are the only officially supported installation,
it may be necessary to build the software manually, e.g., if there are no packages available for the target platform.

Please follow the build instructions in the [development section](30-Development.md)
for more information about requirements and the `Makefile`-based build.

<!-- {% set from_source = True %} -->
<!-- {% include "02-Installation.md" %} -->
120 changes: 120 additions & 0 deletions doc/03-Configuration.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# Configuration

The configuration for Icinga Notifications is twofold.
The main configuration resides in the database,
shared between the Icinga Notifications daemon and Icinga Notifications Web.
However, as the Icinga Notifications daemon needs to know how to access this database and some further settings,
it needs its own configuration file as well.

This configuration is stored in `/etc/icinga-notifications/config.yml`.
See [config.example.yml](../config.example.yml) for an example configuration.

## Top Level Configuration

### HTTP API Configuration

The HTTP API listener can be used both for submission and for debugging purposes.

| Option | Description |
|----------------|----------------------------------------------------------------------|
| listen | Address to bind to, port included. (Example: `localhost:5680`) |
| debug-password | Password expected via HTTP Basic Authentication for debug endpoints. |

### Icinga Web 2

The `icingaweb2-url` is expected to point to the base directory of your Icinga Web 2 installation,
i.e., `https://example.com/icingaweb2/`, to be used for URL creation.

### Channels Directory

All available Icinga Notification channels should reside in the `channels-dir` directory.
For a package installation, the default will point to the correct location and must not be changed.

This directory should be `/usr/libexec/icinga-notifications/channels` on systems that follow the Filesystem Hierarchy Standard.
It may also be `/usr/lib/icinga-notifications/channels`, depending on the operating system conventions.

## Database Configuration

Connection configuration for the database to which Icinga Notifications synchronizes monitoring data.
This is also the database used in Icinga Notifications Web to view and work with the data.

| Option | Description |
|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
| type | **Optional.** Either `mysql` (default) or `pgsql`. |
| host | **Required.** Database host or absolute Unix socket path. |
| port | **Optional.** Database port. By default, the MySQL or PostgreSQL port, depending on the database type. |
| database | **Required.** Database name. |
| user | **Required.** Database username. |
| password | **Optional.** Database password. |
| tls | **Optional.** Whether to use TLS. |
| cert | **Optional.** Path to TLS client certificate. |
| key | **Optional.** Path to TLS private key. |
| ca | **Optional.** Path to TLS CA certificate. |
| insecure | **Optional.** Whether not to verify the peer. |
| options | **Optional.** List of low-level [database options](#database-options) that can be set to influence some Icinga Notifications internal default behaviours. |

### Database Options

Each of these configuration options are highly technical with thoroughly considered and tested default values that you
should only change when you exactly know what you are doing. You can use these options to influence the Icinga Notifications default
behaviour, how it interacts with databases, thus the defaults are usually sufficient for most users and do not need any
manual adjustments.

!!! important

Do not change the defaults if you do not have to!

| Option | Description |
|--------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|
| max_connections | **Optional.** Maximum number of database connections Icinga Notifications is allowed to open in parallel if necessary. Defaults to `16`. |
| max_connections_per_table | **Optional.** Maximum number of queries Icinga Notifications is allowed to execute on a single table concurrently. Defaults to `8`. |
| max_placeholders_per_statement | **Optional.** Maximum number of placeholders Icinga Notifications is allowed to use for a single SQL statement. Defaults to `8192`. |
| max_rows_per_transaction | **Optional.** Maximum number of rows Icinga Notifications is allowed to `SELECT`,`DELETE`,`UPDATE` or `INSERT` in a single transaction. Defaults to `8192`. |
| wsrep_sync_wait | **Optional.** Enforce [Galera cluster](#galera-cluster) nodes to perform strict cluster-wide causality checks. Defaults to `7`. |

## Logging Configuration

Configuration of the logging component used by Icinga Notifications.

| Option | Description |
|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| level | **Optional.** Specifies the default logging level. Can be set to `fatal`, `error`, `warn`, `info` or `debug`. Defaults to `info`. |
| output | **Optional.** Configures the logging output. Can be set to `console` (stderr) or `systemd-journald`. If not set, logs to systemd-journald when running under systemd, otherwise stderr. |
| interval | **Optional.** Interval for periodic logging defined as [duration string](#duration-string). Defaults to `"20s"`. |
| options | **Optional.** Map of component name to logging level in order to set a different logging level for each component instead of the default one. See [logging components](#logging-components) for details. |

### Logging Components

| Component | Description |
|-----------------|---------------------------------------------------------------------------|
| channel | Notification channels, their configuration and output. |
| database | Database connection status and queries. |
| icinga2 | Icinga 2 API communications, including the Event Stream. |
| incident | Incident management and changes. |
| listener | HTTP listener for event submission and debugging. |
| runtime-updates | Configuration changes through Icinga Notifications Web from the database. |

## Appendix

### Duration String

A duration string is a sequence of decimal numbers and a unit suffix, such as `"20s"`.
Valid units are `"ms"`, `"s"`, `"m"` and `"h"`.

### Galera Cluster

Icinga Notifications expects a more consistent behaviour from its database than a
[Galera cluster](https://mariadb.com/kb/en/what-is-mariadb-galera-cluster/) provides by default. To accommodate this,
Icinga Notifications sets the [wsrep_sync_wait](https://mariadb.com/kb/en/galera-cluster-system-variables/#wsrep_sync_wait) system
variable for all its database connections. Consequently, strict cluster-wide causality checks are enforced before
executing specific SQL queries, which are determined by the value set in the `wsrep_sync_wait` system variable.
By default, Icinga Notifications sets this to `7`, which includes `READ, UPDATE, DELETE, INSERT, REPLACE` query types and is
usually sufficient. Unfortunately, this also has the downside that every single Icinga Notifications query will be blocked until
the cluster nodes resynchronise their states after each executed query, and may result in degraded performance.

However, this does not necessarily have to be the case if, for instance, Icinga Notifications is only allowed to connect to a
single cluster node at a time. This is the case when a load balancer does not randomly route connections to all the
nodes evenly, but always to the same node until it fails, or if your database cluster nodes have a virtual IP address
fail over assigned. In such situations, you can set the `wsrep_sync_wait` system variable to `0` in the
`/etc/icinga-notifications/config.yml` file to disable it entirely, as Icinga Notifications doesn't have to wait for cluster
synchronisation then.
Loading

0 comments on commit 3ef3a03

Please sign in to comment.