Code

Threat detection#1783

Code

Draft

ag060 wants to merge 78 commits intomasterakto-api-security/akto:masterfrom threat-detection-v1akto-api-security/akto:threat-detection-v1Copy head branch name to clipboard

+20,698-2,805

Commits

Commits on Jan 2, 2025

• pushing malicious requests to kafka

  ag060

  committed
  b4fc250

• added flush kafka messages task

  ag060

  committed
  ba42c14

• added window based aggregation support for malicious requests

  ag060

  committed
  302a0d1

• using separate mongo for storing suspect data

  ag060

  committed
  4f8fa2f

• fixed kafka host

  ag060

  committed
  585a185

• fixed source ip generation logic

  ag060

  committed
  bd49fec

• not syncing writes and read for RedisWriteBackCache at the same time

  ag060

  committed
  dee4078

• using redis backed increment counter cache

  ag060

  committed
  63620f7

• added group id to aggregate notifier

  ag060

  committed
  c319b3c

• added notification cooldown support

  ag060

  committed
  a359662

• added start and end bucket ids with each detected alerts

  ag060

  committed
  6344fef

• added ttl for redis entries

  ag060

  committed
  e80b9e2

• added new collections for sample malicious requests and detected alerts

  ag060

  committed
  1001bd7

• addded cleanup sample malicious records task

  ag060

  committed
  ff95829

• refactor code

  ag060

  committed
  db2b77c

• refactor code

  ag060

  committed
  8667425

• refactor code

  ag060

  committed
  4913668

• add aggregation parse layer

  

  ayushaga14

  authored and

  ag060

  committed
  c6dc848

• grpc service for consuming malicious and smart events (#1748)

  Show description for ecc9765
  ag060

  committed
  ecc9765

• apply aggregations

  

  ayushaga14

  authored and

  ag060

  committed
  6e4655e

• removed db service and using mongo client everywhere

  ag060

  committed
  2ee9fe7

• moved clean up task to threat protection service

  ag060

  committed
  c437d16

• added authorization interceptor for grpc server

  ag060

  committed
  babb05a

• removed account_id field from consume event service requests (threat-detection-backend)

  Show description for 7408248
  ag060

  committed
  7408248

• added client side authentication support while saving malicious and smart events

  ag060

  committed
  1addc02

• added new api threat detection

  Show description for 5a9afa6
  ag060

  committed
  5a9afa6

• saving malicious sample data to postgres db

  ag060

  committed
  dd051b8

• abstracted out common kafka polling consumer task

  ag060

  committed
  7e23fd3

• added tasks for sending alerts to protection backend

  ag060

  committed
  c915686

• exposing single rpc method for recording alerts

  ag060

  committed
  d2a73b3

• updated proto definition

  ag060

  committed
  6cbd95d

• removed cleanup task from threat protection backend module

  ag060

  committed
  8b1bfe9

• updated kafka topic names

  ag060

  committed
  536ff02

• running flyway migrations programatically

  ag060

  committed
  a292b59

• refactored code

  ag060

  committed
  02cfa89

• added clean up for deleting all the malicious entries older than 7 days

  ag060

  committed
  67d5ebd

• added hibernate orm (#1786)

  ag060

  committed
  74a90b2

• threat protection producer consumer db writes

  

  ayushaga14

  authored and

  ag060

  committed
  b4ad6ed

• added proto generation to ci

  ag060

  committed
  08e3af2

• fix create mongo client params

  

  ayushaga14

  authored and

  ag060

  committed
  c82cb6d

• deleted generated proto files

  ag060

  committed
  0cbcb93

• added dashboard rpc service (#1808)

  ag060

  committed
  0ff337d

• updated proto definition to include malicious event type (SINGLE or AGGREGATED)

  ag060

  committed
  d4f3fff

• storing api_collection_id in postgres

  ag060

  committed
  23fb750

• fixed paths in ci for threat-detection module

  ag060

  committed
  b4ceae8

• moved kafka config from threat detection to utils

  ag060

  committed
  dba48ad

• moved consumer and producer on same machine for threat protection backend

  ag060

  committed
  f1787ca

• refactored dashboard proto messages

  ag060

  committed
  809c62a

• added check if buf is installed in proto-gen script

  ag060

  committed
  31fe6f0

• fixed MaliciousEventModel not populating data from mongo

  ag060

  committed
  78dda75

• refactored code

  ag060

  committed
  1f0b584

• disabled debug logging for hibernate

  ag060

  committed
  78a464a

• removed unused imports

  ag060

  committed
  e349f89

• disabled debug logging for hibernate

  ag060

  committed
  c4c0433

• committing kafka offsets manually now

  ag060

  committed
  d1b1039

• renamed threat detection backend service

  ag060

  committed
  e3ead06

• added health check server

  ag060

  committed
  2b02791

• building threat detection backend image in staging workflow

  ag060

  committed
  d2532c5

• refactored generated proto files

  ag060

  committed
  3b69b6c

• added health check for rpc and enabled reflection

  ag060

  committed
  dd09c8b

• added health service for grpc

  ag060

  committed
  159cd8a

• using vertx http server as threat detection backend (#1842)

  ag060

  committed
  54c8b9b

• reading akto threat detection backend url from env

  ag060

  committed
  8929c38

• added try catch

  ag060

  committed
  5b67f3b

• fixed sample not being to backend

  ag060

  committed
  9bbcd1f

• clearing cache as soon the alert is raised

  ag060

  committed
  c846c77

• sending sample malicious events to backend only once

  ag060

  committed
  89216b4

• use fastjson lib and increase template fetch interval

  

  ayushaga14

  authored and

  ag060

  committed
  edef904

• add log

  

  ayushaga14

  authored and

  ag060

  committed
  57eb219

• avoid reevaluating params for each filter check

  

  ayushaga14

  authored and

  ag060

  committed
  fe659be

• [temp commit] remove unneceesary build from stagiong workflow

  

  ayushaga14

  authored and

  ag060

  committed
  0d86956

• using proto message envelope for kafka transport in threat detection client (#1874)

  ag060

  committed
  89e44b3

• setting redis ttl while sync to redis instead of increment op

  ag060

  committed
  ac20f9e

• fixed merge conflicts in staging and prod workflows

  ag060

  committed
  eea86e7

Commits on Jan 3, 2025

• proto changes

  ayushaga14

  committed
  a74bf9a

• remove logs and headers loop

  ayushaga14

  committed
  fb3e1b6

• optimizations

  notshivansh

  committed
  eb9f899

Commits on Jan 4, 2025

• move httpresponse param proto generated files

  ayushaga14

  committed
  a12d198