| 1 |
/* |
| 2 |
* Argus-5.0 Client Software. Tools to read, analyze and manage Argus data. |
| 3 |
* Copyright (c) 2000-2024 QoSient, LLC |
| 4 |
* All rights reserved. |
| 5 |
* |
| 6 |
* THE ACCOMPANYING PROGRAM IS PROPRIETARY SOFTWARE OF QoSIENT, LLC, |
| 7 |
* AND CANNOT BE USED, DISTRIBUTED, COPIED OR MODIFIED WITHOUT |
| 8 |
* EXPRESS PERMISSION OF QoSIENT, LLC. |
| 9 |
* |
| 10 |
* QOSIENT, LLC DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS |
| 11 |
* SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY |
| 12 |
* AND FITNESS, IN NO EVENT SHALL QOSIENT, LLC BE LIABLE FOR ANY |
| 13 |
* SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES |
| 14 |
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER |
| 15 |
* IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, |
| 16 |
* ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF |
| 17 |
* THIS SOFTWARE. |
| 18 |
* |
| 19 |
* |
| 20 |
* Author: Carter Bullard [email protected] |
| 21 |
*/ |
| 22 |
|
| 23 |
/* |
| 24 |
* $Id$ |
| 25 |
* $DateTime$ |
| 26 |
* $Change$ |
| 27 |
*/ |
| 28 |
|
| 29 |
This is an experimental effort around flow data processing - sorting. |
| 30 |
|
| 31 |
Generally, we want to work with very large files, and so reading |
| 32 |
in the complete set of data, and sorting the records in memory, and |
| 33 |
then writing those records out, (the algorithm used by rasort.1) doesn't |
| 34 |
scale with file size and nodal resources. |
| 35 |
|
| 36 |
In this example we investigate "external sorting" as defined by Knuth, |
| 37 |
where the data records are not in internal memory, but are left extant |
| 38 |
to the algorithmic context. |
| 39 |
|
| 40 |
So, here we are first going to do a system that generates an offset list |
| 41 |
that represents the sorted file. We don't have to write out the sorted |
| 42 |
file, we can write the list of offsets, and squirrel that away if that |
| 43 |
is a good thing. |
| 44 |
|
| 45 |
Using basically rasort.1 and rahisto.1 types of processing, we'll support |
| 46 |
a "-m field(s)" like option to support multi-key sorting strategies, based |
| 47 |
on ArgusFetchAlgorithmTable[x] lookup mechanisms. This can support everything |
| 48 |
except strings, array values, and IPv6 addresses. |