A scalable stack of Elasticsearch, Logstash, and Kibana for your own BOSH-managed infrastructure.
In v211.1.0 basic cluster security features were implemented using Security plugin from OpenDistro Elasticsearch implementation. For better handling of these features, a following changes was made:
- Additional Elasticsearch job has been colocated on Maintenance instance. This allows secure communication over localhost for all singletons also colocated there (all singletons have been unlinked from any remote Elasticsearch cluster, and bound to local one).
- Since using of Ls-rounter instance is not mandatory - it was moved to separate ops-file.
- Secure Elasticsearch node-to-node communication has been implemented using enable-tls ops-file.
- Secure ingesting logs is implemented using enable-ssl ops-file.
- Elasticsearch 6.x can use indices created in Elasticsearch 5.x, but not those created in Elasticsearch 2.x or before.
- Important: After upgrading running 5.x cluster to 6.x all existing indicies will be available for reading data. However, writing to these indicies is not possible. In order to write data immediatelly after upgrade you have to change index naming convention. As long as index names are usually based on current date, this change can be safely reverted in a day or so.
- For upgrade procedure from Elasticsearch 2.x please refer to v205.0.0 release notes.
- There is NO upgrade path from Elasticsearch 1.x to 2.x. Sorry :(
This repo contains Logsearch Core; which deploys an ELK cluster that can receive and parse logs via syslog that contain JSON.
Most users will want to combine Logsearch Core with a Logsearch Addon to customise their cluster for a particular type of logs. Its likely you want to be following an Addon installation guides - see below for a list of the common Addons:
Before starting deployment, make sure your BOSH environment is ready, and all BOSH_ evironment variables are set. We suggest you to use BBL tool to spin up the BOSH environment.
$ cd deployment
$ bosh -d logsearch deploy logsearch-deployment.yml
Adding new parsing rules:
logstash_parser:
filters: |
# Put your additional Logstash filter config here, eg:
json {
source => "@message"
remove_field => ["@message"]
}
- The latest stable, final release will be soon available on bosh.io
- develop - The develop branch in this repo is deployed to our test environments. It is occasionally broken - use with care!