[FileBeat - syslog input] More lenient parsing of RFC3164 dates

[cyrille-leclerc]

Describe the enhancement:

Syslog RFC 3614 use an uncommon pattern for dates with single digit day of month requiring a padding with a space char (" ") instead of the common padding with a "0".

Some syslog clients are not strictly compliant with RFC 3164 and use a padding with "0" instead of " ". Supporting these minor violations of the standard would ease the usage of FileBeat syslog input. Syslog endpoints such as papertrail accept this violation of the RFC 3164 date format.

Example:

<34>Aug  7 22:14:15 mymachine su: 'su ... ## strictly compliant message: two space chars  between `Aug` and `7`
<34>Aug 07 22:14:15 mymachine su: 'su ... ## slightly NOT compliant message: one space char and one `0` char between `Aug` and `7`

Today, the parsing of the not strictly compliant message fails with the debug message:

2020-03-04T16:15:54.481+0100	ERROR	[syslog]	syslog/input.go:243	can't parse event as syslog rfc3164	{"message": "<34>Aug 07 22:14:15 mymachine su: 'su ..."}

RFC3164 - 4.1.2 HEADER Part of a syslog Packet

https://tools.ietf.org/html/rfc3164#section-4.1.2

The TIMESTAMP field is the local time and is in the format of "Mmm dd
   hh:mm:ss" (without the quote marks) where:

         Mmm is the English language abbreviation for the month of the
         year with the first character in uppercase and the other two
         characters in lowercase.  The following are the only acceptable
         values:

         Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec

         dd is the day of the month.  If the day of the month is less
         than 10, then it MUST be represented as a space and then the
         number.  For example, the 7th day of August would be
         represented as "Aug  7", with two spaces between the "g" and
         the "7".
         ...

Describe a specific use case for the enhancement or feature:

Supporting Syslog clients that suffer from this slight misunderstanding of the required date format of RFC 3164.
Example of syslog client making this mistake: jenkinsci/syslog-java-client#37

Problem discussed with @kvch

[elasticmachine]

Pinging @elastic/integrations-services (Team:Services)

[DWOF]

I'm attempting to use filebeat for the first time to ingest forcepoint web appliance logs. I believe that I am experiencing this same issue (I will know more when I can test on a double digit day)

Is there a workaround I can use while this fix gets tested/released?

[killswitch-GUI]

I'm attempting to use filebeat for the first time to ingest forcepoint web appliance logs. I believe that I am experiencing this same issue (I will know more when I can test on a double digit day)

Is there a workaround I can use while this fix gets tested/released?

Same issues here. Following

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[efd6]

There are two syslog parser packages in beats, one in libbeat/reader/syslog (since March this year) and an older one in filebeat/input/syslog (since 2018). The older one specifically accepts the slightly invalid format. The newer one does additional structuring of the result, but does not handle this variation. It might make sense to reduce this duplication and roll one of them into the other.

/cc @taylor-swanson

[efd6]

@cyrille-leclerc I believe this is now fixed with #31254. Can you confirm?

[elasticmachine]

Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)

[taylor-swanson]

Closing issue as the syslog input has been deprecated (#38277), although it does look like this issue was fixed with #31254.