Should jemalloc be enabled by default on builds?

[fbarbeira]

I'm building fluent-bit using documented instructions. I noticed that despite use of jemalloc is strongly suggested, on the Makefile it's disabled:

https://github.com/fluent/fluent-bit/blob/master/CMakeLists.txt#L114

On the binary provided by the debian/ubuntu repository it's actually enabled (FLB_HAVE_JEMALLOC flag is present):

root@06836bfad9a9:/# /opt/fluent-bit/bin/fluent-bit --version
Fluent Bit v2.2.0
Git commit:
root@06836bfad9a9:/# /opt/fluent-bit/bin/fluent-bit --help | grep 'Build Flags'
 Build Flags = FLB_HAVE_SYS_WAIT_H FLB_HAVE_IN_STORAGE_BACKLOG FLB_HAVE_CHUNK_TRACE FLB_HAVE_PARSER FLB_HAVE_RECORD_ACCESSOR FLB_HAVE_STREAM_PROCESSOR JSMN_PARENT_LINKS JSMN_STRICT FLB_HAVE_TLS FLB_HAVE_OPENSSL FLB_HAVE_METRICS FLB_HAVE_WASM FLB_HAVE_AWS FLB_HAVE_AWS_CREDENTIAL_PROCESS FLB_HAVE_SIGNV4 FLB_HAVE_SQLDB FLB_HAVE_TRACE FLB_HAVE_METRICS FLB_HAVE_HTTP_SERVER FLB_HAVE_SYSTEMD FLB_HAVE_SYSTEMD_SDBUS FLB_HAVE_FORK FLB_HAVE_GMTOFF FLB_HAVE_UNIX_SOCKET FLB_HAVE_LIBYAML FLB_HAVE_ATTRIBUTE_ALLOC_SIZE FLB_HAVE_PROXY_GO FLB_HAVE_JEMALLOC JEMALLOC_MANGLE FLB_HAVE_LIBBACKTRACE FLB_HAVE_REGEX FLB_HAVE_UTF8_ENCODER FLB_HAVE_LUAJIT FLB_HAVE_C_TLS FLB_HAVE_ACCEPT4 FLB_HAVE_INOTIFY FLB_HAVE_GETENTROPY FLB_HAVE_GETENTROPY_SYS_RANDOM
root@06836bfad9a9:/#

Other important builds had jemalloc enabled:

https://github.com/aws/aws-for-fluent-bit/blob/develop/scripts/dockerfiles/Dockerfile.main-release#L7

Another popular log managers discard jemalloc years ago giving their reasons:

https://vector.dev/highlights/2021-06-02-drop-jemalloc/

So I'm confused about this configuration. I mainly work with linux machines and use 'tail' inputs, so I'm not really sure about use jemalloc or default memory allocator. Principal reason is that jemalloc builds sometimes uses as twice the ram of the default memory allocator. Can anyone offer some guide?

[fbarbeira]

I ran a fluent-bit with jemalloc on a VM and consumes ~85MB. Then reboot machine, run exactly the same configuration on the same machine and now it consumes ~220MB. Why is this happening? is this related with system free memory?

[fbarbeira]

I try to compile same configuration on same machine with jemalloc disabled, it consumes ~43MB.

[littlejoyo]

hi，Do you directly start the amount of memory occupied, or in the case of log processing?

[fbarbeira]

I only use "tail" for input. I'm not sure if this is what you asking for.

[littlejoyo]

I also have a similar problem about memory, my fluent-bit is enabled jemalloc, I found that when fluent-bit processes logs, the memory usage will gradually increase, and when the traffic stops, I will find that the memory is not released immediately, is this part normal?
Is it related to jemalloc?