Message removed from SQS is still being retried by consumer

[ctataryn]

Scenario

I have setup an s3 bucket as well as a "Create Object" event that will notify my SQS queue when a new object is created in the bucket (see: https://docs.aws.amazon.com/AmazonS3/latest/userguide/ways-to-add-notification-config-to-bucket.html)

I have a consumer setup with the @SqsMessageHandler decorator to process the message.body.s3 event messages, however what I failed to do was to handle any other type of message that might wind up on the queue, such as the s3:TestEvent message that s3 will dump on the queue when it first sets up the notification linkage between your bucket event and your SQS queue.

The test message looks like this:

{
  "Service": "Amazon S3",
  "Event": "s3:TestEvent",
  "Time": "2023-10-23T16:28:49.589Z",
  "Bucket": "bucket-mc-bucket-face",
  "RequestId": "VZSZPEZ9ASQQW8Z7",
  "HostId": "xk2D4VzA/keQCyP7oJbt3qvR74ShJn5MYQ393tiIz/WQrKJttABDeiuOeqtS178bQButkULwY9E="
}

Problem

Because my handler would error out when it found the test event message on the queue, the message would be re-delivered continuously.

I went into AWS Web Console and polled for the message, then removed it manually. However, still, even between restarts of my NestJS app, the test event message would be reprocessed via the hanlder.

The only way I could get the message to stop being reprocessed is when I pushed a code change to handle the test event message instead of having it throw an error. Then I observed that no re-delivery of the test message was happening (i.e. handler wasn't being re-invoked).

Anyone know what's going on here? Is the underlying code of the SQS Consumer in this library (or one of its dependencies) keeping a local message bus state that doesn't respect manual message removal from AWS or something?