Wrong packet is logged by Suricata for L7NP when enabling logging for rule

[antoninbas]

Describe the bug
According to https://github.com/antrea-io/antrea/blob/main/docs/antrea-l7-network-policy.md#logs, enableLogging can be set to true for a L7 NP rule, in which case Antrea will request Suricata to log packets that need to be dropped.
The expectation is that if an HTTP request is rejected by the default reject rule installed by Antrea, the offending packet / HTTP request will be logged for further troubleshooting.
However, the wrong packet is logged. Instead of logging the packet / request from the client, the TCP RST packet sent by Suricata is logged instead.

This can be reproduced easily by following instructions in https://github.com/antrea-io/antrea/blob/main/docs/antrea-l7-network-policy.md#logs, and enabling logging for the L7 NP rules.

The log shown in the documentation clearly illustrates the issue:

{
  "timestamp": "2023-03-09T20:00:28.225016+0000",
  "flow_id": 627175734391745,
  "in_iface": "antrea-l7-tap0",
  "event_type": "packet",
  "vlan": [
    1
  ],
  "src_ip": "10.10.1.4",
  "src_port": 80,
  "dest_ip": "10.10.1.5",
  "dest_port": 43352,
  "proto": "TCP",
  "packet": "/lhtPRglzmQvxnJoCABFAAAoUGYAAEAGFE4KCgEECgoBBQBQqVhIGzbi/odenlAUAfsR7QAA",
  "packet_info": {
    "linktype": 1
  }
}

Decoding the packet from base64 to hex gives us FE 58 6D 3D 18 25 CE 64 2F C6 72 68 08 00 45 00 00 28 50 66 00 00 40 06 14 4E 0A 0A 01 04 0A 0A 01 05 00 50 A9 58 48 1B 36 E2 FE 87 5E 9E 50 14 01 FB 11 ED 00 00. And using an hex packet decoder gives us 10.10.1.4 → 10.10.1.5 TCP 80 → 43352 [RST, ACK]. This is clearly a TCP RST packet, and we can also easily see that the src_ip matches the server IP, while the dest_ip matches the client IP.

Logging the TCP RST packet sent by Suricata is not very useful, and the HTTP GET request which triggered the RST should be logged instead.

Versions:
Antrea v2.1.0, ToT

Additional context
Just a quick note that the documentation is in the process of being updated as part of #6589, but while the example changes, the issue remains the same.

[antoninbas]

@qiyueyao do you know how we could resolve this?

[qiyueyao]

@qiyueyao do you know how we could resolve this?

This part of logging is done with tag keyword https://docs.suricata.io/en/latest/rules/tag.html. I just decoded the example there, it is a TCP packet. Perhaps it is due to specifying wrong value of scope, which is currently tag: session, 30, seconds in Antrea. I can test other values.
I think that could be the reason, otherwise it is most likely because we manually created the rules in Suricata based on the packets, then there might not be a quick solution. Despite the logs in Suricata, Antrea NP logging also logs the traffic with Redirect action if enableLogging: true which should contain the expected information.

[antoninbas]

I just decoded the example there, it is a TCP packet

The problem is not that it is a TCP packet, the problem is that it is the "wrong" TCP packet. It should be the packet which is actually dropped by Suricata.

Despite the logs in Suricata, Antrea NP logging also logs the traffic with Redirect action if enableLogging: true which should contain the expected information.

Do you mean that the packet is also logged to /var/log/antrea/networkpolicy/np.log? But AFAIK that will not include any payload information (HTTP request)?

[qiyueyao]

I just decoded the example there, it is a TCP packet

The problem is not that it is a TCP packet, the problem is that it is the "wrong" TCP packet. It should be the packet which is actually dropped by Suricata.

The example there shows 192.168.0.27 → 192.168.0.103 TCP 54634 → 22 [ACK] instead of [RST, ACK].

Despite the logs in Suricata, Antrea NP logging also logs the traffic with Redirect action if enableLogging: true which should contain the expected information.

Do you mean that the packet is also logged to /var/log/antrea/networkpolicy/np.log? But AFAIK that will not include any payload information (HTTP request)?

Oh yes they are logged there but without payload information. Also without final action if we need that information.