Debian Codegraph @

examples/raqsort/README

raw
1 /*
2 * Argus-5.0 Client Software. Tools to read, analyze and manage Argus data.
3 * Copyright (c) 2000-2024 QoSient, LLC
4 * All rights reserved.
5 *
6 * THE ACCOMPANYING PROGRAM IS PROPRIETARY SOFTWARE OF QoSIENT, LLC,
7 * AND CANNOT BE USED, DISTRIBUTED, COPIED OR MODIFIED WITHOUT
8 * EXPRESS PERMISSION OF QoSIENT, LLC.
9 *
10 * QOSIENT, LLC DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS
11 * SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
12 * AND FITNESS, IN NO EVENT SHALL QOSIENT, LLC BE LIABLE FOR ANY
13 * SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
14 * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER
15 * IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
16 * ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF
17 * THIS SOFTWARE.
18 *
19 *
20 * Author: Carter Bullard [email protected]
21 */
22
23 /*
24 * $Id$
25 * $DateTime$
26 * $Change$
27 */
28
29 This is an experimental effort around flow data processing - sorting.
30
31 Generally, we want to work with very large files, and so reading
32 in the complete set of data, and sorting the records in memory, and
33 then writing those records out, (the algorithm used by rasort.1) doesn't
34 scale with file size and nodal resources.
35
36 In this example we investigate "external sorting" as defined by Knuth,
37 where the data records are not in internal memory, but are left extant
38 to the algorithmic context.
39
40 So, here we are first going to do a system that generates an offset list
41 that represents the sorted file. We don't have to write out the sorted
42 file, we can write the list of offsets, and squirrel that away if that
43 is a good thing.
44
45 Using basically rasort.1 and rahisto.1 types of processing, we'll support
46 a "-m field(s)" like option to support multi-key sorting strategies, based
47 on ArgusFetchAlgorithmTable[x] lookup mechanisms. This can support everything
48 except strings, array values, and IPv6 addresses.