| 1 |
NAME |
| 2 |
|
| 3 |
Catmandu::Stat - Catmandu modules for working with statistical data |
| 4 |
|
| 5 |
SYNOPSIS |
| 6 |
|
| 7 |
# Calculate statistics on the availabity of the ISBN fields in the dataset |
| 8 |
cat data.json | catmandu convert JSON to Stat --fields isbn |
| 9 |
|
| 10 |
# Preprocess data and calculate statistics |
| 11 |
catmandu convert MARC to Stat --fix 'marc_map(020a,isbn)' --fields isbn < data.mrc |
| 12 |
|
| 13 |
# Or in fix files |
| 14 |
|
| 15 |
# Calculate the mean of foo. E.g. foo => [1,2,3,4] |
| 16 |
stat_mean(foo) # foo => '2.5' |
| 17 |
|
| 18 |
# Calculate the median of foo. E.g. foo => [1,2,3,4] |
| 19 |
stat_median(foo) # foo => '2.5' |
| 20 |
|
| 21 |
# Calculate the standard deviation of foo. E.g. foo => [1,2,3,4] |
| 22 |
stat_stddev(foo) # foo => '1.12' |
| 23 |
|
| 24 |
# Calculate the variance of foo. E.g. foo => [1,2,3,4] |
| 25 |
stat_variance(foo) # foo => '1.25' |
| 26 |
|
| 27 |
MODULES |
| 28 |
|
| 29 |
* Catmandu::Exporter::Stat |
| 30 |
|
| 31 |
* Catmandu::Fix::stat_mean |
| 32 |
|
| 33 |
* Catmandu::Fix::stat_median |
| 34 |
|
| 35 |
* Catmandu::Fix::stat_stddev |
| 36 |
|
| 37 |
* Catmandu::Fix::stat_variance |
| 38 |
|
| 39 |
EXAMPLES |
| 40 |
|
| 41 |
The Catmandu::Stat distribution includes a CSV file on the Sacramento |
| 42 |
crime rate in January 2006, "t/SacramentocrimeJanuary2006.csv" also |
| 43 |
available at |
| 44 |
http://samplecsvs.s3.amazonaws.com/SacramentocrimeJanuary2006.csv |
| 45 |
|
| 46 |
To view statistics on the fields available in this file type: |
| 47 |
|
| 48 |
$ catmandu convert CSV to Stat < t/SacramentocrimeJanuary2006.csv |
| 49 |
|
| 50 |
| name | count | zeros | zeros% | min | max | mean | variance | stdev | uniq~ | uniq% | entropy | |
| 51 |
|---------------|-------|-------|--------|-----|-----|------|----------|-------|-------|-------|-----------| |
| 52 |
| # | 7584 | | | | | | | | | | | |
| 53 |
| address | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 5425 | 71.5 | 12.4/12.4 | |
| 54 |
| beat | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 20 | 0.3 | 4.3/12.9 | |
| 55 |
| cdatetime | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 5071 | 66.9 | 12.3/12.3 | |
| 56 |
| crimedescr | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 305 | 4.0 | 5.6/12.6 | |
| 57 |
| district | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 6 | 0.1 | 2.6/12.9 | |
| 58 |
| grid | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 537 | 7.1 | 7.8/9.9 | |
| 59 |
| latitude | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 5288 | 69.7 | 12.4/12.4 | |
| 60 |
| longitude | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 5295 | 69.8 | 12.4/12.4 | |
| 61 |
| ucr_ncic_code | 7584 | 0 | 0.0 | 1 | 1 | 1 | 0.0 | 0.0 | 88 | 1.2 | 4.1/12.9 | |
| 62 |
|
| 63 |
The file has 7584 rows where and all the fields address to |
| 64 |
ucr_ncic_code contain values. Each field has only one value (no arrays |
| 65 |
available in the CSV file). The are 5492 unique addresses in the CSV |
| 66 |
file. The district field has the lowest entropy, most of its values are |
| 67 |
shared among many rows. |
| 68 |
|
| 69 |
SEE ALSO |
| 70 |
|
| 71 |
Catmandu, Catmandu::Breaker, |
| 72 |
|
| 73 |
AUTHOR |
| 74 |
|
| 75 |
Patrick Hochstenbach, <patrick.hochstenbach at ugent.be> |
| 76 |
|
| 77 |
LICENSE AND COPYRIGHT |
| 78 |
|
| 79 |
This program is free software; you can redistribute it and/or modify it |
| 80 |
under the terms of either: the GNU General Public License as published |
| 81 |
by the Free Software Foundation; or the Artistic License. |
| 82 |
|
| 83 |
See http://dev.perl.org/licenses/ for more information. |
| 84 |
|