Debian Codegraph @

README

raw
1 NAME
2
3 Catmandu::Stat - Catmandu modules for working with statistical data
4
5 SYNOPSIS
6
7 # Calculate statistics on the availabity of the ISBN fields in the dataset
8 cat data.json | catmandu convert JSON to Stat --fields isbn
9
10 # Preprocess data and calculate statistics
11 catmandu convert MARC to Stat --fix 'marc_map(020a,isbn)' --fields isbn < data.mrc
12
13 # Or in fix files
14
15 # Calculate the mean of foo. E.g. foo => [1,2,3,4]
16 stat_mean(foo) # foo => '2.5'
17
18 # Calculate the median of foo. E.g. foo => [1,2,3,4]
19 stat_median(foo) # foo => '2.5'
20
21 # Calculate the standard deviation of foo. E.g. foo => [1,2,3,4]
22 stat_stddev(foo) # foo => '1.12'
23
24 # Calculate the variance of foo. E.g. foo => [1,2,3,4]
25 stat_variance(foo) # foo => '1.25'
26
27 MODULES
28
29 * Catmandu::Exporter::Stat
30
31 * Catmandu::Fix::stat_mean
32
33 * Catmandu::Fix::stat_median
34
35 * Catmandu::Fix::stat_stddev
36
37 * Catmandu::Fix::stat_variance
38
39 EXAMPLES
40
41 The Catmandu::Stat distribution includes a CSV file on the Sacramento
42 crime rate in January 2006, "t/SacramentocrimeJanuary2006.csv" also
43 available at
44 http://samplecsvs.s3.amazonaws.com/SacramentocrimeJanuary2006.csv
45
46 To view statistics on the fields available in this file type:
47
48 $ catmandu convert CSV to Stat < t/SacramentocrimeJanuary2006.csv
49
50 | name | count | zeros | zeros% | min | max | mean | variance | stdev | uniq~ | uniq% | entropy |
51 |---------------|-------|-------|--------|-----|-----|------|----------|-------|-------|-------|-----------|
52 | # | 7584 | | | | | | | | | | |
53 | address | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 5425 | 71.5 | 12.4/12.4 |
54 | beat | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 20 | 0.3 | 4.3/12.9 |
55 | cdatetime | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 5071 | 66.9 | 12.3/12.3 |
56 | crimedescr | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 305 | 4.0 | 5.6/12.6 |
57 | district | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 6 | 0.1 | 2.6/12.9 |
58 | grid | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 537 | 7.1 | 7.8/9.9 |
59 | latitude | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 5288 | 69.7 | 12.4/12.4 |
60 | longitude | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 5295 | 69.8 | 12.4/12.4 |
61 | ucr_ncic_code | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 88 | 1.2 | 4.1/12.9 |
62
63 The file has 7584 rows where and all the fields address to
64 ucr_ncic_code contain values. Each field has only one value (no arrays
65 available in the CSV file). The are 5492 unique addresses in the CSV
66 file. The district field has the lowest entropy, most of its values are
67 shared among many rows.
68
69 SEE ALSO
70
71 Catmandu, Catmandu::Breaker,
72
73 AUTHOR
74
75 Patrick Hochstenbach, <patrick.hochstenbach at ugent.be>
76
77 LICENSE AND COPYRIGHT
78
79 This program is free software; you can redistribute it and/or modify it
80 under the terms of either: the GNU General Public License as published
81 by the Free Software Foundation; or the Artistic License.
82
83 See http://dev.perl.org/licenses/ for more information.
84