Skip to content

Latest commit

 

History

History
119 lines (99 loc) · 2.88 KB

README.md

File metadata and controls

119 lines (99 loc) · 2.88 KB
Raw

Elasticsearch Anonymize Ingest Processor

Anonymize a field using by replacing values with a consistent HMAC function.

Anonymize Options

Name Required Default Description
field yes - The field to anonymize. The field in the document can be a string or an array.
target_field no field The field to assign the anonymize value to. If not defined field will be overwritten.
key no supersecrethere The key value used for the HMAC.
method no HmacSHA1 HMAC algorithm to use. See here for a complete list.
ignore_missing no true If true and field does not exist, the processor quietly exits without modifying the document

Usage

PUT _ingest/pipeline/anonymize-pipeline
{
  "description": "A pipeline to do whatever",
  "processors": [
    {
      "anonymize" : {
        "field"          : "my_field",
        "target_field"   : "hash",
        "key"            : "testkey",
        "method"         : "HmacSHA256",
        "ignore_missing" : true
      }
    }
  ]
}

PUT /my-index/_doc/1?pipeline=anonymize-pipeline
{
  "my_field" : ["my_value1", "my_value2","my_value3"]
}

GET /my-index/_doc/1
{
  "_index": "my-index",
  "_type": "_doc",
  "_id": "1",
  "_version": 1,
  "found": true,
  "_source": {
    "my_field": [
      "my_value1",
      "my_value2",
      "my_value3"
    ],
    "hash": [
      "a4b12fddb3fd1389dec57010bd7b8c68c0f613d07115b8a5b1fd166a981c93fa",
      "a9b22cef4d3cbd2dbb80f6209faa637d7d7ef96dbeb06bf10fac47eb23813859",
      "33510269cdd2ac8b819a85cc12cbd1a7948fa5abf695944c0250488334d2ce61"
    ]
  }
}

PUT _ingest/pipeline/anonymize-pipeline
{
  "description": "A pipeline to do whatever",
  "processors": [
    {
      "anonymize" : {
        "field" : "my_field"
      }
    }
  ]
}

PUT /my-index/_doc/2?pipeline=anonymize-pipeline
{
  "my_field" : "Some content"
}

GET /my-index/_doc/2
{
  "_index": "my-index",
  "_type": "_doc",
  "_id": "2",
  "_version": 1,
  "found": true,
  "_source": {
    "my_field": "5e7dc9df6c9bd2503167c70afbf1b52cb8a1c095",
  }
}

Configuration

No configuration required.

Setup

Remember to set the elasticsearchVersion parameter in your gradle.properties file.

First you need to create a zip distribution by running:

gradle clean check

This will produce a zip file in build/distributions.

After building the zip file, you can install it:

bin/elasticsearch-plugin install file:///path/to/ingest-anonymize/build/distribution/ingest-anonymize-x.y.z.zip

Java

Starting from elasticsearch 7.x, developers must use Java 14.

Bugs & TODO

  • There are always bugs
  • and todos...

Acknowledgements

Thanks to Alexander Reelsen for the project https://github.com/spinscale/cookiecutter-elasticsearch-ingest-processor.