-
Notifications
You must be signed in to change notification settings - Fork 4
Home
Welcome to the openargus clients wiki! Here we'll try to use the powers of GitHub to develop some new features.
In Argus 5.0 we moved some of ArgusPro's features to the open source, such as 128-bit Argus source id's, Argus events, expanded behavioral analytics, json processing, importing other flow data into the Argus processing system, enhanced content capture and processing, and new tunnel support.
Argus 5.0 is focused on generating argus data in as many points in the network as possible, including external and internal high speed links, workgroup edges, endpoints and wireless access points. This is important to addressing the cyber security challenges that enterprises face today. This involves granular visibility inside the enterprise, to support effective cyber detection and forensics. With increased network visibility inside the enterprise, there are new opportunities for sophisticated detections by correlating data from multiple points in the network at or near the same time.
Because Argus has already been ported to most endpoint operating systems and OpenWRT access points, we have a good start on getting a lot of sensors into an environment. As a part of improving visibility throughout the network, we're also going to import data from other flow systems. Argus already processes NetFlow and IPFIX records, but there are a lot of other flow data strategies out there. In particular, we'll want to import Zeek connection logs, as many organizations generate Zeek data, Google VPC flows, and possibly some of the single letter flows, like Qflow, Jflow, and maybe Kflow records.
The open source argus code is very portable, and runs in a number of operating systems, including Linux and it's variants, RHEL, Rocky, Ubuntu, Debian, Kali, FreeBSD, CentOS, Fedora, OpenSUSE, (and all of these subvariants), Windows, MacOS, AIX, SunOS, HPUX, Solaris, IRIX, CrayOS, VxWorks, PSoS, and OpenWRT, so we have a good start.
Argus 5.0 sensors run great in endpoints, and with a few changes to libpcap, we can attain < 0.5% avg CPU utilization for an argus daemon on most commercial endpoints (Windows, MacOS, Linux). There are specific features that are useful to achieve complete network accountability on endpoints, as there are a lot of interfaces types that exist that we all would like to monitor. BlueTooth interfaces, RadioTaps, USB devices, VPNs, even docker interfaces are fair game for monitoring in an endpoint. And of course there are a lot of different types of endpoints now ... cloud based VMs and containers are an important part of the mix.
To improve managing large numbers of endpoint sensors, argus supports using the hostuuid as the argus source id. With this feature, argus can be deployed as a zero-configuration daemon (no conf file mods needed), and to improve visibility on endpoints, argus will generally add the 'inf' to the flow key of every flow it monitors. This means that argus-clients should expect from endpoint argi flow data that has a 128-bit source id, and a 4-char interface identifier, where the flow was monitored.
128-bit ARGUS_MONITOR_ID's are pretty unwieldy. To make data processing easier, all ra* programs can use a RA_SRCID_ALIAS file to alias short names for the big uuid identifiers. The aliases are "node"s and can be printed, filtered, etc ...
[carter@red clients]$ ra -S localhost -s +node StartTime Dur Flgs Proto SrcAddr Sport Dir DstAddr Dport SrcPkts DstPkts SrcBytes DstBytes State Node 2024/06/22.15:22:38.842553 0.000000 e arp 192.168.1.139 who 192.168.1.82 1 0 60 0 INT red 2024/06/22.15:22:38.930476 2.028915 e arp 192.168.1.82 who 192.168.1.155 3 0 180 0 INT red 2024/06/22.15:22:38.930860 2.028531 e arp 192.168.1.82 who 192.168.1.114 3 0 180 0 INT red 2024/06/22.15:22:38.930966 2.028426 e arp 192.168.1.82 who 192.168.1.128 3 0 180 0 INT red 2024/06/22.15:22:38.931736 2.027706 e arp 192.168.1.82 who 192.168.1.153 3 0 180 0 INT red 2024/06/22.15:22:39.147142 0.000000 e udp 192.168.1.131.37889 -> 239.255.250.250.9131 1 0 150 0 INT red 2024/06/22.15:22:39.842539 4.001828 e udp 192.168.1.131.43475 -> 239.255.255.250.51200 3 0 366 0 REQ red 2024/06/22.15:22:43.050204 2.426843 man 0 0 487015 2 0 15 0 0 CON red 2024/06/22.15:22:40.843153 5.054052 man 0 0 4294560977 1 74 13 8027 0 CON red 2024/06/22.15:22:41.003790 0.000834 e arp 192.168.1.49 who 192.168.1.82 1 1 42 60 CON red 2024/06/22.15:22:41.023691 0.000008 e arp 192.168.1.82 who 192.168.1.49 1 1 60 42 CON red 2024/06/22.15:22:41.255548 0.000000 e udp 192.168.1.131.44622 -> 239.255.255.250.1900 1 0 188 0 REQ red
will try to bring the experiences we've had in porting Argus and in trying to come up with some basic zero-configuration strategies for getting complete network accountability.
Argus can natively read Netflow V 4,5 and flow-tools flow formats. And as of argus-clients.3.0.8.4 argus can convert json formatted Zeek conn.logs into Argus binary formats using our existing program raconvert.1 ... Json because we added json processing into the argus client library, but we can just as easily do non-json formats as well.
We extended raconvert.1 to take in a conversion map, using the '-f conversion.map' command-line option. And the specific support for converting zeek con logs is done through the support/Config/raconvert.zeek.conf file. This sample raconvert conversion map, should work for all the basic zeek conn.log variables, and as new are added, this file will need to be updated.
raconvert.1 can convert any json formatted string into a flow record, if it contains a minimum set of flow identifiers. Start time, an IP address or name, some metrics and optionally some metadata, is all that is needed.
This approach should work very well with Google VPC flow logs. If we can find some real examples of VPC flow logs, we can generate a raconvert.google.conf conversion map. Should be pretty easy ...