Substation transfer not maintaining new line delimited json

[viraj-lunani]

I am using substation for an enrichment, however before creating the enrichment, we set up a basic config that just sends the logs to s3.

local sub = import "../../substation.libsonnet";
{
  concurrency: 2,
  transforms: [
      sub.tf.send.aws.s3(settings={
        bucket_name: "<bucket>",
        file_path: {
          prefix: "logs/gsuite/alerts/",
          uuid: true
        }
      }),
    ]
}

The logs come in as .json.gz and are writting to the bucket but when they are written to the bucket they are no longer newline delimited. How can I resolve this?

[jshlbrd]

@viraj-lunani I think this example will help -- you can use the auxiliary_transforms setting to apply a series of transforms to the data after it is batched but before it is written to S3. We have a pattern for JSON Lines that you can add (also seen in the example).