Home

Welcome to the AWS-Digi wiki!

Project Description

AWS-Digi (Abuse-feed WebIron Source) is a ‘DNIF Open Source’ project which works on processing large volumes of real-time data inside DNIF, operating it, analyzing it, and generating alerts. This real-time data is the one which is delivered immediately after collection, e.g. Logs etc. DNIF also serves as an Analytical tool to query the data, to look for a particular event in the chain of events and to model the data by visualizing it.

Objective

Understanding DNIF Open Big Data Analytics Platform.
Analysis of Real-Time dataset through DQL (DNIF Query Language).
Generating alerts on DNIF console.
Visualizing the data through creating Dashboards and charts.

Platform

DNIF is a data platform that can collect, parse, enrich, index, balance, and analyze data in a continuously changing environment, helping enterprises take precautionary measures for cyber defense. It allows users to partition one data infrastructure and enable multiple teams to solve many challenges.

Understand DNIF platform and how it works: https://dnif.it/how-it-works.html

Installing and getting started with DNIF: https://dnif.it/docs/guides/getting-started/

Tools Used

Following are the tools used while running the project through DNIF :

Virtual Box
JetBrains: PyCharm Community Edition
Ubuntu 16.04 or above
Docker
Postman

Sources and Usage

Most of the resources listed below provide lists and/or APIs to obtain up-to-date (hopefully) information with regards to threats/attacks. All kinds of tools for parsing, creating and editing can be used for Threat Intelligence. But in this project, we used simple Python code to parse the data out of the data source. We used Python-based libraries like Beautiful Soup ver 4.0 in our code through which we were able to store the data in the required format i.e., the XLSX/JSON/CSV formats and further post the data to the platform used (DNIF).