Skip to content

Commit

Permalink
initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
Ayache Khettar committed May 31, 2021
1 parent fd11bde commit f5d7822
Show file tree
Hide file tree
Showing 8 changed files with 191 additions and 192 deletions.
42 changes: 42 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: main

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

jobs:

build:
name: Build
runs-on: ubuntu-latest
steps:

- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
go-version: ^1.15
id: go

- name: Check out code into the Go module directory
uses: actions/checkout@v2

- name: Get dependencies
run: |
go get -v -t -d ./...
if [ -f Gopkg.toml ]; then
curl https://raw.githubusercontent.com/golang/dep/main/install.sh | sh
dep ensure
fi
- name: Build
run: go build -v .

- name: Test
run: |
go test -v .
env:
GITHUB_AUTH_TOKEN: ${{ secrets.TOKEN}}
GITHUB_EMAIL: ${{ secrets.EMAIL}}


24 changes: 24 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
on: release
name: Release
jobs:
goreleaser:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
-
name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.15
-
name: Run GoReleaser
uses: goreleaser/goreleaser-action@v2
with:
version: latest
args: release --rm-dist
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
47 changes: 46 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,46 @@
# sqs-pub
# SQS Messages replayer

## Overview

This is a simple tool that allows replaying messages from the AWS SQS DLQ. While the primary function of this tool is to replay messages present in the DLQ, it can also be used to push a message from one given queue to another regardless if the source queue is a DLQ or not.

## Usage

Below is the print of the following command: `./sqs-pub -help`, which describe the usage of this tool


```
USAGE
sqs-pub [-from queue1 - to queue2 -filter text1,text2,...]
FLAGS
-delete true delete messages from source after successfuly pushed to destination queue
-filters 10104211111292 comma separted text that can be used a message body filter
-from vf-cm-dev-marketplace-emea-bi-deadletter-queue sqs queue from where messages will be sourced from
-to vf-cm-dev-marketplace-emea-bi-orders sqs queue from where messages will be sourced from
```

## Flags
1. `delete`: by default this is set to true and the messages will be deleted from the soruce queue once successfully published to the destination queue.
2. `filters`: a comma separated list of keys that can be used to identify a given message is meant to be processed or not. This is a very simple filtering system, in subsequent releases timestamp or messageId can be used to filter out messages.
3. `from`: The source queue name (exp, DLQ)
4. `to`: The destionation queue name.
5. `dryrun`: By the default this flag is to false. If it is set to true then it will run the replay process but withouth sending the actual messages to the destination queue and reports will be generated as it has been a real dry. This flag is useful to get an picture of what is this tool attempt to do when run it for real.

## How to run

1. Set the AWS Credentials in the terminal

```
export AWS_ACCESS_KEY_ID="xxxxxxxV2M3L"
export AWS_SECRET_ACCESS_KEY="xxxxxxxxDK"
export AWS_SESSION_TOKEN="IQoJxxxxxxxxx"
```

2. Run the binary as follow
```
./sqs-pub -from=queue-name-source -to=queue-name-destination -filters=text1,text2
## License
[MIT](LICENSE)
Empty file removed failed.log
Empty file.
144 changes: 0 additions & 144 deletions filtered.log

This file was deleted.

5 changes: 3 additions & 2 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,11 @@ var (
)

func init() {
rootFlagSet.StringVar(&replayer.cfg.from, "from", "vf-cm-dev-marketplace-emea-bi-deadletter-queue", "sqs queue from where messages will be sourced from")
rootFlagSet.StringVar(&replayer.cfg.to, "to", "vf-cm-dev-marketplace-emea-bi-orders", "sqs queue from where messages will be sourced from")
rootFlagSet.StringVar(&replayer.cfg.from, "from", "queue-name-source", "sqs queue from where messages will be sourced from")
rootFlagSet.StringVar(&replayer.cfg.to, "to", "queue-name-destination", "sqs queue from where messages will be sourced from")
rootFlagSet.StringVar(&replayer.cfg.filters, "filters", "10104211111292", "comma separted text that can be used a message body filter")
rootFlagSet.BoolVar(&replayer.cfg.deleteFromSource, "delete", true, "delete messages from source after successfuly pushed to destination queue")
rootFlagSet.BoolVar(&replayer.cfg.dryrun, "dryrun", false, "a flag to run the replay in dry run mode.")
}

func main() {
Expand Down
Empty file removed processed.log
Empty file.
Loading

0 comments on commit f5d7822

Please sign in to comment.