Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DO NOT MERGE, pull request for testing #662

Closed
wants to merge 42 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
6fd26a8
github action and personal run test to check what kind of data gets p…
shivam-rawat-akto Oct 12, 2023
8b13c06
allow only test to run
shivam-rawat-akto Oct 12, 2023
f9be321
allow only test to run
shivam-rawat-akto Oct 12, 2023
21d0263
allow only test to run
shivam-rawat-akto Oct 12, 2023
cf07fa0
allow only test to run
shivam-rawat-akto Oct 12, 2023
d8e28a3
allow only test to run
shivam-rawat-akto Oct 12, 2023
6c61559
reverting staging.yml
shivam-rawat-akto Oct 12, 2023
db86132
github app settings and publishing comment on pull request
shivam-rawat-akto Oct 13, 2023
0b06ce3
github run on pr commit
shivam-rawat-akto Oct 13, 2023
959c8b9
github run on pr commit
shivam-rawat-akto Oct 13, 2023
66fcba1
staging changes for quick run
shivam-rawat-akto Oct 13, 2023
9ffa2b1
staging changes for quick run
shivam-rawat-akto Oct 13, 2023
77eb558
staging changes for quick run
shivam-rawat-akto Oct 13, 2023
43035d6
github app integration workflow
shivam-rawat-akto Oct 13, 2023
90602d8
reverting changes to staging.yml
shivam-rawat-akto Oct 13, 2023
5bba3f7
removing prod check for testing
shivam-rawat-akto Oct 13, 2023
ca5b063
hardcoding image tag for testing
shivam-rawat-akto Oct 13, 2023
eda6e7c
hardcoding image tag for testing
shivam-rawat-akto Oct 13, 2023
9a6c1a9
adding publishCommentApi in cicd
shivam-rawat-akto Oct 14, 2023
088287d
Merge remote-tracking branch 'origin/develop' into feature/github_com…
shivam-rawat-akto Oct 25, 2023
a0aa9eb
including github checks functionality
shivam-rawat-akto Oct 26, 2023
b34911f
changing test
shivam-rawat-akto Oct 26, 2023
edf42a5
allow github api's in cicd list
shivam-rawat-akto Oct 27, 2023
b0448e9
adding comment
shivam-rawat-akto Oct 27, 2023
c2a5bc2
Merge remote-tracking branch 'origin/master' into feature/github_comm…
shivam-rawat-akto Nov 1, 2023
a2e6b84
Merge remote-tracking branch 'origin/master' into feature/github_comm…
shivam-rawat-akto Nov 1, 2023
9e458b5
staging.yml changes
shivam-rawat-akto Nov 1, 2023
519dd95
staging.yml changes
shivam-rawat-akto Nov 1, 2023
2fe596a
changing version for run-test
shivam-rawat-akto Nov 1, 2023
5171a9c
removing akto build
shivam-rawat-akto Nov 1, 2023
ea52cb5
change version
shivam-rawat-akto Nov 1, 2023
03966f6
getter for testingRunResultSummaryHexId in StartTestAction
shivam-rawat-akto Nov 1, 2023
7fb4d6a
changing runscan version
shivam-rawat-akto Nov 1, 2023
f0e8184
changing runscan version
shivam-rawat-akto Nov 1, 2023
385007b
changing runscan version
shivam-rawat-akto Nov 2, 2023
2ac643b
changing runscan version
shivam-rawat-akto Nov 2, 2023
e0127b9
changing runscan version
shivam-rawat-akto Nov 2, 2023
9981eef
fix github comment
shivam-rawat-akto Nov 2, 2023
ce2f895
remove staging build process
shivam-rawat-akto Nov 2, 2023
8bd2c05
remove staging build process
shivam-rawat-akto Nov 3, 2023
3a19dca
remove staging build process
shivam-rawat-akto Nov 3, 2023
235eb0e
changing test id
shivam-rawat-akto Nov 3, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 26 additions & 5 deletions .github/scripts/akto-cicd.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,7 @@ echo "### Akto test summary" >> $GITHUB_STEP_SUMMARY
while true; do
current_time=$(date +%s)
elapsed_time=$((current_time - start_time))

if ((elapsed_time >= MAX_POLL_INTERVAL)); then
echo "Max poll interval reached. Exiting."
break
fi


current_time=$(date +%s)
recency_period=$((60 * 24 * 60 * 60))
Expand All @@ -36,13 +32,21 @@ while true; do
}")

state=$(echo "$response" | jq -r '.testingRunResultSummaries[0].state // empty')
testingRunSummaryhexId=$(echo "$response" | jq -r '.testingRunResultSummaries[0].hexId // empty')

if [[ "$state" == "COMPLETED" ]]; then
count=$(echo "$response" | jq -r '.testingRunResultSummaries[0].countIssues // empty')
high=$(echo "$response" | jq -r '.testingRunResultSummaries[0].countIssues.HIGH // empty')
medium=$(echo "$response" | jq -r '.testingRunResultSummaries[0].countIssues.MEDIUM // empty')
low=$(echo "$response" | jq -r '.testingRunResultSummaries[0].countIssues.LOW // empty')

publishGithubCommentResponse = $(curl -s "$AKTO_DASHBOARD_URL/api/publishGithubComments" \
--header 'content-type: application/json' \
--header "X-API-KEY: $AKTO_API_KEY" \
--data "{
\"testingRunSummaryHexId\": \"$testingRunSummaryhexId\"
}")

echo "[Results]($AKTO_DASHBOARD_URL/dashboard/testing/$AKTO_TEST_ID/results)" >> $GITHUB_STEP_SUMMARY
echo "HIGH: $high" >> $GITHUB_STEP_SUMMARY
echo "MEDIUM: $medium" >> $GITHUB_STEP_SUMMARY
Expand All @@ -55,11 +59,28 @@ while true; do
fi
break
elif [[ "$state" == "STOPPED" ]]; then
publishGithubCommentResponse = $(curl -s "$AKTO_DASHBOARD_URL/api/publishGithubComments" \
--header 'content-type: application/json' \
--header "X-API-KEY: $AKTO_API_KEY" \
--data "{
\"testingRunSummaryHexId\": \"$testingRunSummaryhexId\"
}")
echo "Test stopped" >> $GITHUB_STEP_SUMMARY
exit 1
break
else
echo "Waiting for akto test to be completed..."
sleep 5 # Adjust the polling interval as needed
fi

if ((elapsed_time >= MAX_POLL_INTERVAL)); then
publishGithubCommentResponse = $(curl -s "$AKTO_DASHBOARD_URL/api/publishGithubComments" \
--header 'content-type: application/json' \
--header "X-API-KEY: $AKTO_API_KEY" \
--data "{
\"testingRunSummaryHexId\": \"$testingRunSummaryhexId\"
}")
echo "Max poll interval reached. Exiting."
break
fi
done
72 changes: 37 additions & 35 deletions .github/workflows/staging.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ jobs:
node-version: '17'
- name: Convert github branch name to be compatible with docker tag name convention and generate tag name
id: docker_tag
run: echo "IMAGE_TAG=${{ github.event.inputs.Tag }}_$(echo ${{ github.ref_name }} | sed 's/[^a-zA-Z0-9]/_/g')" >> $GITHUB_OUTPUT
# run: echo "IMAGE_TAG=${{ github.event.inputs.Tag }}_$(echo ${{ github.ref_name }} | sed 's/[^a-zA-Z0-9]/_/g')" >> $GITHUB_OUTPUT
run: echo "IMAGE_TAG=akto_staging1_feature_github_comments_cicd_test" >> $GITHUB_OUTPUT
- name: Download Akto templates zip
working-directory: ./apps/dashboard/src/main/resources
run: wget -O test-library-master.zip https://github.com/akto-api-security/tests-library/archive/refs/heads/master.zip
Expand All @@ -64,30 +65,30 @@ jobs:
working-directory: ./apps/dashboard/web/polaris_web
run: npm install && export RELEASE_VERSION=${{steps.docker_tag.outputs.IMAGE_TAG}} && npm run build
- name: Configure AWS Credentials
if: ${{ github.event.inputs.Environment == 'prod' }}
# if: ${{ github.event.inputs.Environment == 'prod' }}
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{secrets.AWS_ACCESS_KEY_ID}}
aws-secret-access-key: ${{secrets.AWS_SECRET_ACCESS_KEY}}
aws-region: ap-south-1

- name: Deploy static site to S3 bucket
if: ${{ github.event.inputs.Environment == 'prod' }}
# if: ${{ github.event.inputs.Environment == 'prod' }}
run: aws s3 sync ./apps/dashboard/web/dist s3://dashboard-on-cdn/web/${{steps.docker_tag.outputs.IMAGE_TAG}}/dist --delete
- name: Deploy polaris site to S3 bucket
if: ${{ github.event.inputs.Environment == 'prod' }}
# if: ${{ github.event.inputs.Environment == 'prod' }}
run: aws s3 sync ./apps/dashboard/web/polaris_web/web/dist s3://dashboard-on-cdn/polaris_web/${{steps.docker_tag.outputs.IMAGE_TAG}}/dist --delete

- run: mvn package -Dakto-image-tag=${{ github.event.inputs.Tag }} -Dakto-build-time=$(eval "date +%s") -Dakto-release-version=${{steps.docker_tag.outputs.IMAGE_TAG}}
- name: DockerHub login
if: ${{ github.event.inputs.Environment == 'prod' }}
# if: ${{ github.event.inputs.Environment == 'prod' }}
env:
DOCKER_USERNAME: ${{secrets.DOCKER_USERNAME}}
DOCKER_PASSWORD: ${{secrets.DOCKER_PASSWORD}}
run: |
docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD
- name: Build, tag, and push the image to DockerHub
if: ${{ github.event.inputs.Environment == 'prod' }}
# if: ${{ github.event.inputs.Environment == 'prod' }}
id: build-image-dockerhub
env:
ECR_REGISTRY: aktosecurity
Expand All @@ -98,10 +99,10 @@ jobs:
IMAGE_TAG_TESTING_CLI: ${{ github.event.inputs.Deploy == 'true' && '-t aktosecurity/akto-api-testing-cli:flash' || '' }}
run: |
docker buildx create --use
# Build a docker container and push it to DockerHub
# Build a docker container and push it to DockerHub
cd apps/dashboard
docker buildx build --platform linux/arm64/v8,linux/amd64 -t $ECR_REGISTRY/$ECR_REPOSITORY-dashboard:$IMAGE_TAG $IMAGE_TAG_DASHBOARD . --push
echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY-dashboard:$IMAGE_TAG"
echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY-dashboard:$IMAGE_TAG"
cd ../testing
docker buildx build --platform linux/arm64/v8,linux/amd64 -t $ECR_REGISTRY/akto-api-testing:$IMAGE_TAG $IMAGE_TAG_TESTING . --push
echo "::set-output name=image::$ECR_REGISTRY/akto-api-testing:$IMAGE_TAG"
Expand All @@ -127,53 +128,54 @@ jobs:
restore-keys: ${{ runner.os }}-m2

deploy:
if: ${{ github.event.inputs.Environment == 'prod' && github.event.inputs.Deploy == 'true' }}
# if: ${{ github.event.inputs.Environment == 'prod' && github.event.inputs.Deploy == 'true' }}
needs: build
runs-on: ubuntu-latest
environment: Dev
permissions:
id-token: write
contents: read
steps:
- uses: actions/checkout@v2
- uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: ${{ secrets.IAMROLE_GITHUB }}
role-session-name: GitHub-Action-Role
aws-region: ${{ vars.AWS_REGION }}
- name: Deploy to flash
run: |
echo "Deploying branch ${{ env.GITHUB_REF }} to ${{ github.event.inputs.environment }}"
commit_hash=`git rev-parse HEAD`
aws deploy create-deployment --application-name flash-deploy --deployment-group-name flash-deployment-group --github-location repository=$GITHUB_REPOSITORY,commitId=$commit_hash --ignore-application-stop-failures
- uses: actions/checkout@v2
- uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: ${{ secrets.IAMROLE_GITHUB }}
role-session-name: GitHub-Action-Role
aws-region: ${{ vars.AWS_REGION }}
- name: Deploy to flash
run: |
echo "Deploying branch ${{ env.GITHUB_REF }} to ${{ github.event.inputs.environment }}"
commit_hash=`git rev-parse HEAD`
aws deploy create-deployment --application-name flash-deploy --deployment-group-name flash-deployment-group --github-location repository=$GITHUB_REPOSITORY,commitId=$commit_hash --ignore-application-stop-failures

test:
if: ${{ github.event.inputs.Environment == 'prod' && github.event.inputs.Deploy == 'true' }}
# if: ${{ github.event.inputs.Environment == 'prod' && github.event.inputs.Deploy == 'true' }}
needs: deploy
runs-on: ubuntu-latest
steps:

- uses: actions/checkout@v2

- name: Run testsigma end to end tests
env:
TESTSIGMA_API_KEY: ${{ secrets.TESTSIGMA_API_KEY }}
TESTSIGMA_TEST_PLAN_ID: 789
TS_DASHBOARD_URL: ${{vars.TS_DASHBOARD_URL}}
run: bash ./.github/scripts/akto-testsigma.sh

# - name: Run testsigma end to end tests
# env:
# TESTSIGMA_API_KEY: ${{ secrets.TESTSIGMA_API_KEY }}
# TESTSIGMA_TEST_PLAN_ID: 789
# TS_DASHBOARD_URL: ${{vars.TS_DASHBOARD_URL}}
# run: bash ./.github/scripts/akto-testsigma.sh
#
- name: Run Akto CI/CD tests
uses: akto-api-security/run-scan@v1.0.3
uses: shivam-rawat-akto/run-scan@v1.0.13
with:
AKTO_DASHBOARD_URL: https://flash.staging.akto.io
AKTO_API_KEY: ${{secrets.AKTO_API_KEY}}
AKTO_TEST_ID: 650d5bcb7bd5e15e3360553f
AKTO_API_KEY: ${{secrets.TEST_ACCOUNT_SECRET_KEY}}
AKTO_TEST_ID: 6544c94711a3300fd70210dd
GITHUB_COMMIT_ID: ${{github.event.pull_request.head.sha}}

- name: Check for Akto test completion
env:
AKTO_DASHBOARD_URL: https://flash.staging.akto.io
AKTO_API_KEY: ${{secrets.AKTO_API_KEY}}
AKTO_TEST_ID: 650d5bcb7bd5e15e3360553f
AKTO_DASHBOARD_URL: https://flash.staging.akto.io
AKTO_API_KEY: ${{secrets.TEST_ACCOUNT_SECRET_KEY}}
AKTO_TEST_ID: 6544c94711a3300fd70210dd
run: bash ./.github/scripts/akto-cicd.sh

- name: Check Vulnerable API's
Expand Down
6 changes: 6 additions & 0 deletions apps/dashboard/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,12 @@
<artifactId>commons-lang3</artifactId>
<version>3.12.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.kohsuke/github-api -->
<dependency>
<groupId>org.kohsuke</groupId>
<artifactId>github-api</artifactId>
<version>1.316</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.eclipse.jetty/jetty-servlets -->
<dependency>
<groupId>org.eclipse.jetty</groupId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,7 @@
import org.bson.conversions.Bson;
import org.bson.types.ObjectId;

import java.util.HashMap;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.*;
import java.util.stream.Collectors;

public class StartTestAction extends UserAction {
Expand Down Expand Up @@ -203,7 +197,8 @@ public String startTest() {
loggerMaker.infoAndAddToDb("CICD test triggered at " + Context.now(), LogDb.DASHBOARD);
summary.setMetadata(metadata);
}
TestingRunResultSummariesDao.instance.insertOne(summary);
testingRunResultSummaryHexId = Objects.requireNonNull(TestingRunResultSummariesDao.instance.insertOne(summary).getInsertedId())
.asObjectId().getValue().toHexString();
}

this.startTimestamp = 0;
Expand Down Expand Up @@ -612,6 +607,9 @@ public List<TestingRunResultSummary> getTestingRunResultSummaries() {
return this.testingRunResultSummaries;
}

public String getTestingRunResultSummaryHexId() {
return this.testingRunResultSummaryHexId;
}
public void setTestingRunResultSummaryHexId(String testingRunResultSummaryHexId) {
this.testingRunResultSummaryHexId = testingRunResultSummaryHexId;
}
Expand Down
Loading
Loading