SC-20509 #4
SC-20509 #4
Conversation
Filtered some things we do not need from the Elastisearch exporter after doing some research
…s not very clean once things begin to work i will clean it up code. Also its hard to really differentiate work done into individual commits But what is going here is i am trying to follow STA pattern of shipping logs to SC Some of this code was copy and pasted from STA
Had to setup factory settings for the exporter
…some logging statements Will clean code up once i am done
exporter/sematextexporter/es.go
Outdated
| for i := 0; i < v.Len(); i++ { | ||
| doc := v.Index(i).Interface() | ||
| if docMap, ok := doc.(map[string]any); ok { | ||
| docMap["os.host"] = getHostname() |
There was a problem hiding this comment.
Instead of calling the getHostname func everytime lets save this in the client struct itself as a field similar to what the flinkClient does
logaf: high
exporter/sematextexporter/config.go
Outdated
| // LogMaxSize is the maximum size in megabytes of the log file before it gets rotated | ||
| LogMaxSize int `mapstructure:"logs_max_size"` | ||
| // WriteEvents determines if events are logged | ||
| WriteEvents atomic.Bool `mapstructure:"write_events"` |
There was a problem hiding this comment.
Question: Why is WriteEvents an atomic.Bool? Is it something that can be updated in multiple threads concurrently?
There was a problem hiding this comment.
So this was what was defined in the agent so I literally copied and pasted
There was a problem hiding this comment.
Read up on Atomic Counters in Go. It'll help you make a decision if this is really required or not
There was a problem hiding this comment.
Just read up on it and yes I believe its required
exporter/sematextexporter/es.go
Outdated
| func (c *client) Bulk(body any, config *Config) error { | ||
| if grp, ok := c.clients[config.LogsEndpoint]; ok { | ||
| bulkRequest := grp.client.Bulk() | ||
| if reflect.TypeOf(body).Kind() == reflect.Slice { |
There was a problem hiding this comment.
Is using reflect really necessary here?
There was a problem hiding this comment.
yes
But what would you suggest
I tried making changes and it broke my code
exporter/sematextexporter/es_test.go
Outdated
| "io" | ||
| "testing" | ||
|
|
||
| "github.com/olivere/elastic" |
There was a problem hiding this comment.
This library is deprecated now
There was a problem hiding this comment.
The "github.com/olivere/elastic" library?
AkhigbeEromo
changed the title
exporter/sematextexporter/es.go
Outdated
| "time" | ||
|
|
||
| json "github.com/json-iterator/go" | ||
| "github.com/olivere/elastic/v7" |
There was a problem hiding this comment.
Still using a deprecated lib. Replace with https://github.com/elastic/go-elasticsearch
exporter/sematextexporter/es.go
Outdated
|
|
||
| json "github.com/json-iterator/go" | ||
| "github.com/olivere/elastic/v7" | ||
| "github.com/sirupsen/logrus" |
There was a problem hiding this comment.
"github.com/sirupsen/logrus" is in maintenance mode. Lets use something that is being used by other exporters as well like "go.uber.org/zap". See them for the appropriate usage.
Adding logs support following STA pattern