Skip to content

Latest commit

 

History

History

logging-sample

README.adoc

Rico: Logging appender

This example shows how the logging appender of Rico can be used to add useful metadata to application logging. Next to this the sample shows how application logging can be stored in a central place.

Rico provides logging appender for [log4j2](https://logging.apache.org/log4j/2.x/) and [logback](http://logback.qos.ch/). By using such an appender all context information of Rico (see dev.rico.core.context.RicoApplicationContext for more information) will automatically be added to MDC (Mapped Diagnostic Context) of each log message.

Centralized logging

When having several applications and services in production the monitoring can become quite complex. By providing a centralized log management that stores all logs and provides analytic functionalities developers and administrators can find problems faster.

central log

In this sample 2 different tools (ELK stack & Graylog) can be used to provide a centralized log management. For both stacks you can find predefined docker-compose setups that can be used to easily start a running environment. Since both servers are configured to listen on same port they cannot be run simultaneously.

Next to storing logging a log management provides much more functionality. One important aspect is the support of custom fields or tags that can be used to analyze logging. The Rico context API provides an easy way to add metadata to an application. By using the Rico logging-appender such metadata will automatically be added to log information. Next to this the GELF standard is used in this sample to send the logging over a UDP connection to the used log manager.

logging appender

Using the Graylog stack

The sample provides docker containers to easily start a preconfigured Graylog instance that is used to store the log messages. The complete setup can be found in the docker-graylog folder. The docker-compose tool is needed to run the sample. If docker-compose is installed on your system you only need to execute the start.sh from docker-graylog folder to boot the graylog instance. Once this is done the web ui of Graylog can be accessed at http://localhost:9000.

Configuring Graylog

When opening the ui for the first time you need to login to Graylog (user: admin - passwort: admin).

Graylog supports different sources of log messages. Before it is able to receive any message you need to tell it where to listen.

  1. Visit to http://localhost:9000

  2. Login to Graylog (user: admin - passwort: admin)

  3. In the top menu select "System" > "Inputs"

  4. Select "GELF UDP" as the input type and click "Launch new input".

  5. In the popup select the "global" checkbox and give the input a title (i.e. GELF UDP)

  6. Click "save".

    graylog gelf input

  7. Click "Show received messages" to the right of the newly added input.

  8. Click on the little play symbol in the top right corner to enable auto refresh of the page.

  9. Launch the server-spring-log4j or the server-spring-logback example (mvn spring-boot:run).

  10. Click on a message to see all the details.

    graylog logging

The server application will send the complete logging to the Graylog server by using the GELF format. You can see the complete Rico context based metadata that is part of each logging message in the Graylog UI.

The server applications both provide a http endpoint that can be called to create additional logging. To call the endpoint just open http://localhost:8080/api/sample.

Using the ELK stack

The sample provides docker containers to easily start a preconfigured ELK stack that is used to store the log messages. The complete setup can be found in the docker-elk folder. The docker-compose tool is needed to run the sample. If docker-compose is installed on your system you only need to execute the start.sh from docker-elk folder to boot the ELK stack. Once this is done the web ui of Kibana can be accessed at http://localhost:5601.

INFO

If the ELK stack fails to start with the message max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] or similar than have a look at elasticsearch issue #111

Configuring Kibana

The ELK stack is ready to receive log messages without a configuration. In order to browse the messages a little configuration is needed:

  1. Launch the server-spring-log4j or the server-spring-logback example (mvn spring-boot:run).

  2. Visit http://localhost:5601.

  3. If asked to load sample data then chose "I want to se my own data".

  4. In the left-hand menu select "Discover".

  5. Click "Create Index Pattern".

  6. Enter * for the index pattern.

  7. Click "Next step".

  8. Chose @timestamp as the "Time filter field name".

  9. Click "Create index pattern".

  10. In the left-hand menu select "Discover".

Log4J, Logback or SLF4J

Rico uses SLF4J throughout its Java code. By using SLF4J Rico leaves the choice of logging framework to the application.

The Rico logging module extends the MDC (Mapped Diagnostic Context) with Rico and application specific information. In order to be able to use Rico logging the application must use a logging framework which supports MDC. At the time of writing this module only Logback and Log4j provide MDC support.

Thus Rico logging can only be used if the application is using either Logback or Log4j.