Bad File Descriptor Error

[sheshjee]

pgagroal[3999010]: (libev) epoll_wait: Bad file descriptor
systemd[1]: Started Process Core Dump (PID 3999011/UID 0).
systemd-coredump[3999018]: Resource limits disable core dumping for process 3999010 (pgagroal).
systemd-coredump[3999018]: Process 3999010 (pgagroal) of user 26 dumped core.
systemd[1]: systemd-coredump@10-3999011-0.service: Succeeded.
Please provide a solution to this error

[jesperpedersen]

What is your configuration ?

Does the core dump give you any additional information ?

Can you compile pgagroal yourself in debug mode, and reproduce the issue and get the information that way ?

[sheshjee]

I have 3 node patroni postgresql db cluster. 1 haproxy server. pgagroal is installed on db cluster. I can't set debug mode since it's production environment.

…

On Thu, Mar 7, 2024, 5:19 PM Jesper Pedersen ***@***.***> wrote: What is your configuration ? Does the core dump give you any additional information ? Can you compile pgagroal yourself in debug mode, and reproduce the issue and get the information that way ? — Reply to this email directly, view it on GitHub <#409 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BFMCG6ZITHERJ5ENUMJQ5FTYXBH6JAVCNFSM6AAAAABEKSHUXSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DOMBWGIZTC> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[jesperpedersen]

Configuration ?

And, can you get the core dump ?

[sheshjee]

[pgagroal] host = * port = 2345 log_type = file log_level = info log_path = /var/log/pgagroal/pgagroal.log max_connections = 900 idle_timeout = 600 validation = off unix_socket_dir = /tmp [primary] host = localhost port = 5432 Core dump is disabled.

…

On Thu, Mar 7, 2024 at 5:55 PM Jesper Pedersen ***@***.***> wrote: Configuration ? And, can you get the core dump ? — Reply to this email directly, view it on GitHub <#409 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BFMCG63GYGCNHKLCDB4EN4DYXBMDBAVCNFSM6AAAAABEKSHUXSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DOMBWGYZDI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- sheshjee

[jesperpedersen]

Can you create a test setup with debug enabled where the issue is shown as well ?

Or enable core dumps ?

[sheshjee]

I will try and update you. Meanwhile can you please check the configuration whether is it correct or any parameter is missing?

…

On Fri, Mar 8, 2024, 6:11 PM Jesper Pedersen ***@***.***> wrote: Can you create a test setup with debug enabled where the issue is shown as well ? Or enable core dumps ? — Reply to this email directly, view it on GitHub <#409 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BFMCG673VSYUDYPFULPJMEDYXGWYBAVCNFSM6AAAAABEKSHUXSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DOMJZGY4TM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[jesperpedersen]

The configuration should work

[sheshjee]

Any other alternative beside enabling core dump because it is a production setup, and I am not authorised to enable it. Can you pleas describe the relation between pgagroal and bad file descriptor error?

…

On Fri, Mar 8, 2024, 9:00 PM Jesper Pedersen ***@***.***> wrote: The configuration should work — Reply to this email directly, view it on GitHub <#409 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BFMCG67VDP26TQIJQI7MNP3YXHKRPAVCNFSM6AAAAABEKSHUXSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DOMRRGQYTM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[fluca1978]

Can you pleas describe the relation between pgagroal and bad file descriptor error?

At glance I think it is impossible, since we don't know what's happening.
Which version of pgagroal are you running?
Which operating system and libev version?
Moreover, is there a particular point in time when you experience this problem (e.g., when the connections increase or from the very beginning of your working setup)?

[sheshjee]

OS Version- Red Hat Enterprise Linux release 8.9 (Ootpa) libev version- libev-4.24-6.el8.x86_64 pgagroal 1.5.1 There is no particular point in time when the error occurred but we are guessing when the connection increases we are getting this error.

…

On Mon, Mar 11, 2024 at 5:42 PM Luca Ferrari ***@***.***> wrote: Can you pleas describe the relation between pgagroal and bad file descriptor error? At glance I think it is impossible, since we don't know what's happening. Which version of pgagroal are you running? Which operating system and libev version? Moreover, is there a particular point in time when you experience this problem (e.g., when the connections increase or from the very beginning of your working setup)? — Reply to this email directly, view it on GitHub <#409 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BFMCG655I6E533OZR3XGXQDYXWNTBAVCNFSM6AAAAABEKSHUXSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DONBVGA2DG> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- sheshjee

[sheshjee]

rpm -ivh libev-4.33-5.el9.x86_64.rpm
warning: libev-4.33-5.el9.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 12765c34d: NOKEY
Verifying... ################################# [100%]
Preparing... ################################# [100%]
Updating / installing...
1:libev-4.33-5.el9 ################################# [100%]

yum install pgagroal

pgagroal --version
pgagroal: /usr/lib64/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib64/libev.so.4)

[jesperpedersen]

4.24 was released on 2016-12-28, and 4.33 was released on 2020-03-18 - so 3+ years of bug fixes...

[sheshjee]

Agree, but how to resolve this error?
pgagroal: /usr/lib64/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib64/libev.so.4)

[jesperpedersen]

You are installing a EL9 version on a EL8 platform...

[sheshjee]

this is the problem that I want to tell you libev 4.24 is for the EL9 version not for the EL8 version.

[fluca1978]

Any chance you can at least increment log_level to debug in order to get logs around the problem?

[sheshjee]

Its production environment, I am not allowed to do so.
Can you tell me the root cause of this error? this is a random error that occurs anytime, we are guessing that when more transactions takes place, this error occurs.

[fluca1978]

Its production environment,

Yes, you already told us this, so you cannot change neither the dump settings, nor the pgagroal settings, nor connect a debugger or try another version.

Can you tell me the root cause of this error? this is a random error that occurs anytime, we are guessing that when more transactions takes place, this error occurs.

It could be, but so far we don't have any real data or evidence to work on, so we are not able to tell you anything.
Unless you provide us with logs, or a dump, or something else that can help us triggering the same error in our environment, we cannot help.
Could you replicate your setup and try to reproduce the same error in a staging/test machine? May be a machine where you are allowed to change settings in order to dig into details.

[fluca1978]

then the question again same.. if the configuration is good, libev is good then why Bad File Descriptor error is occurring?

Let me turn the question: why changing libev version if the ABI is the same? It could be that libev fixed some bug?
Despite my rant, we are suggesting to you a few ways to try to solve the problem, but there is nothing we are sure about since we don't have enough information.
Since you are facing the problem, you have to change something in order to try to fix it.

[sheshjee]

You have suggested enabling DEBUG mode or enabling core dumps. Right??