-
Notifications
You must be signed in to change notification settings - Fork 4
/
CHANGES
165 lines (130 loc) · 7.09 KB
/
CHANGES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
/*
* Argus-5.0 Client Software. Tools to read, analyze and manage Argus data.
* Copyright (c) 2000-2024 QoSient, LLC
* All rights reserved.
*
* This program is free software, released under the GNU General
* Public License; you can redistribute it and/or modify it under the terms
* of the GNU General Public License as published by the Free Software
* Foundation; either version 3, or any later version.
*
* Other licenses are available through QoSient, LLC.
* Inquire at info@qosient.com.
*
* This program is distributed WITHOUT ANY WARRANTY; without even the
* implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
* See the * GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*
*/
Argus-5.0 Data Support
Argus-5.0 is a public version of commercial argus, that provides enhanced
functionality and performance. Argus-5.0 is designed to be completely
backward compatible with the previous versions of open source argus and
its program names, configuration, etc should be similar when possible.
Product specific changes are provided with each product. When no
specific changes are mentioned, gargoyle should be perceived as
completely compatible, but enhanced.
Argus Data Support
Argus-5.0 represents a major change in argus data. The format
was completely updated to support 128-bit uuid argus source id's,
As a result there is no forward compatibility between argus-3.0 programs and
argus-5.0 data. Argus-5.0 programs are backward compatible, however, so
you can read and process your data with argus-5.0 programs.
We have made provisions for argus-5.0 programs to generate argus-3.x formatted
data, but that is configurable, and not the default behavior. So there is
explicit conversion that will be going on under the covers. There is no
compelling justification to convert your argus-3.x data, allowing you to leave
the original data unmodified.
Argus-5.0.0 and its clients provides for extended modes of transport of argus
data, earlier version of argus-clients cannot read these transport formats.
Argus-clients-5.0 provides extended capabilities for reading flow-tools
originated data, and bro/zeek data and converting them to argus-5.0 data
formats.
Argus-clients-5.0.0 is backward compatible with all prior releases of
argus data, fixing a large number of bugs, with regard to data represenation
and processing.
Architecture
The client programs evolved quite a bit between argus-2.0, argus-3.0 and argus-5.0.
With the addition of large scale argus data collection and distribution, using
radium(), and rastream(). Argus-clients-5.0 extends this architecture, providing
the ability to collect, distribute, archive, analyze, and process network flow
data, for argus data, flow-tools data, netflow v5-8 partial sflow data processing,
and zeek data conversion.
radium(), and rastream() are the principal programs that have been added to the
ra* family of programs. radium can connect to multiple sources of argus data,
whether they are streams or files of data, and can write out data to multiple targets,
supporting independant access control, authentication, and filtering per target.
What this means is that you can build a argus data distribution tree, to collect,
process and redistribute argus data.
Rastream() is known as a stream block processors (SBP). You want to collect data
from a set of argus data stream sources, and the data just keeps coming in.
When/how can you stop to process the data, say for real-time indexing, search
and/or processing? In the database world this is called 'stream block processing'.
rastream() reads in argus data, and output the data into a set of files
that make up a native OS filesystem based archive. rastream() extends
this capability by implementing a wa hold buffer to allow for input sorting,
and then based on command line options, rastream() can call scripts against the
files after a time period or an event. We use rastream() to periodically commit
data to an information system for indexing, searching, processing, compressing,
and then archive. Say every 5 minutes, on the second, rastream() will close
completed input files and then spawn any number of processes against those files.
With these programs, we have collected data from as many as 5K argus data
sources, and managed the data in a set of argus data respositories.
Argus-clients-3.0.6 provides new capabilities in this area, allowing
radium to "serve up" files that are generated by rastream().
See the manpages for radium.1 and ra.1.
The clients distribution has been restructred in argus-5.0.0. It is organized
into argus client Core Programs, and Examples.
Client Core Programs
ra - principal program that read, process, filter, and print argus data.
racount - no basic changes.
racluster - complete rewrite of argus aggregation strategies,
and replaces ragator.
radium - argus record collection and distribution program.
ranonymize - updated for new data types.
rasort - ported.
ratop - massive rewrite. Completely new program.
Client Example Programs
These programs provide examples in key areas of argus data processing
and management.
argus data environment
ratop - realtime argus data processing environment (curses based)
provides vi() like functionality for streaming and file based flow data,
supporting printing, searching, editing, sorting, writing argus data.
argus data processing
raconvert - ascii to binary data record conversion
raevent - non flow data printing
rafilteraddr - high performance filtering
ralabel - semantic enhancement / metadata tagging
rastream - stream block processing
rastrip - data compression
analytics
rahisto - frequency distribution analysis for argus data metrics
graphing
ragraph - time series graphing (rrd-tool based)
raplot - general plotting (gnuplot based)
storage management
ramysql - mysql based utilities
rasql - read native argus data from mysql database tables.
rasqlinsert - insert and read argus data from/to mysql data tables.
rasqltimeindex - generate argus data file time indexes for searching.
forensics
radump - decode captured user data
ragrep - regular expression matching from captured user data
raservices - user data analysis to determine used protocol
reporting
radark - scanner detection and reporting
rascan - scanner detection and reporting
rahosts - IP address inventory reporting
raips - IP address inventory reporting
rapath - print topology information derived from argus data
rapolicy - continuous access control policy verification
raports - application port usage
rarpwatch - arpwatch driven using argus data
ratimerange - argus data file time span
development
ratemplate - ra client development template when using the argus clients library.