| 1 |
/* |
| 2 |
* Gargoyle Client Software. Tools to read, analyze and manage Argus data. |
| 3 |
* Copyright (c) 2000-2024 QoSient, LLC |
| 4 |
* All rights reserved. |
| 5 |
* |
| 6 |
* THE ACCOMPANYING PROGRAM IS PROPRIETARY SOFTWARE OF QoSIENT, LLC, |
| 7 |
* AND CANNOT BE USED, DISTRIBUTED, COPIED OR MODIFIED WITHOUT |
| 8 |
* EXPRESS PERMISSION OF QoSIENT, LLC. |
| 9 |
* |
| 10 |
* QOSIENT, LLC DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS |
| 11 |
* SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY |
| 12 |
* AND FITNESS, IN NO EVENT SHALL QOSIENT, LLC BE LIABLE FOR ANY |
| 13 |
* SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES |
| 14 |
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER |
| 15 |
* IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, |
| 16 |
* ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF |
| 17 |
* THIS SOFTWARE. |
| 18 |
* |
| 19 |
*/ |
| 20 |
|
| 21 |
/* |
| 22 |
* ratod - tools used to develop time of day behavioral flow data |
| 23 |
* The essence of the effort is to generate the canonical day, |
| 24 |
* through statistical process of 1,000s of days, and comparing |
| 25 |
* data presentations. |
| 26 |
* |
| 27 |
* The first concept is to remove the time from the flows, so |
| 28 |
* for daily processing, we'll set the starting time to be |
| 29 |
* 0 by subtracting the starting second for the specific day, |
| 30 |
* and all other time is offset by this value. |
| 31 |
* |
| 32 |
* Then, we'll bin the data to generate 5m inventories. |
| 33 |
* While this maybe a lot of data, we'll do some experiments to |
| 34 |
* see what works ... |
| 35 |
* |
| 36 |
* |
| 37 |
* |
| 38 |
* Author: Carter Bullard [email protected] |
| 39 |
*/ |
| 40 |
|
| 41 |
NOTES |
| 42 |
|
| 43 |
OK, so currently the implementation provides rabins() behavior |
| 44 |
with time normalization to Jan 1, 1970. This provides the |
| 45 |
greatest flexibility to generate aggregated or unaggregated |
| 46 |
data. Right now we're doing Time of Whatever normalization. |
| 47 |
Using the time bins command line configuration data to |
| 48 |
specify arbitrary time period normalization. |
| 49 |
|
| 50 |
The easiest support for ToD processing is to stream data |
| 51 |
through a tool that only does time normalization, but |
| 52 |
putting the rabins() aggregation does provide most of |
| 53 |
what we need in a tool like this. |
| 54 |
|
| 55 |
Simple normalization of the time is not enough to provide usable |
| 56 |
data, as direction information is critical to making useful TOD |
| 57 |
flow run data. |
| 58 |
|
| 59 |
The concept to date is to provide time normalized 300sec resolved |
| 60 |
L2/L3 service oriented flow run data. This will be read in using |
| 61 |
the " -M rtime " options, so that it represents a TOD cache of |
| 62 |
expected network behavior. The flow records can span the entire |
| 63 |
day, if, for example, there are no idle times, or it can be broken |
| 64 |
down into 300 sec bins. |
| 65 |
|
| 66 |
NOTE - Racurses() will need to manage its timeout values based on |
| 67 |
" ltime " for this to work. |
| 68 |
|
| 69 |
|
| 70 |
Service oriented flow run data |
| 71 |
Normalizing a set of flows into a daily canonical flow run dataset, |
| 72 |
involes removing the ephemeral data, such as source port values. |
| 73 |
This demands that service direction be a part of the process, so |
| 74 |
as to minimize ephemeral collision (where ephemeral data is inadvertently |
| 75 |
introduced into the comparison sets). |
| 76 |
|
| 77 |
The goal is identify new flows that are outside the services box, |
| 78 |
as well as identify when expected flows do not show up. |
| 79 |
The primary motivator in the ratod() analytics. |
| 80 |
|