| 1 |
<!DOCTYPE html> |
| 2 |
<html class="writer-html5" lang="en"> |
| 3 |
<head> |
| 4 |
<meta charset="utf-8" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" /> |
| 5 |
|
| 6 |
<meta name="viewport" content="width=device-width, initial-scale=1.0" /> |
| 7 |
<title>Categorical Variables — HARP 1.29 documentation</title> |
| 8 |
<link rel="stylesheet" type="text/css" href="../_static/pygments.css" /> |
| 9 |
<link rel="stylesheet" type="text/css" href="../_static/css/theme.css" /> |
| 10 |
<link rel="stylesheet" type="text/css" href="../_static/custom.css" /> |
| 11 |
|
| 12 |
|
| 13 |
<!--[if lt IE 9]> |
| 14 |
<script src="../_static/js/html5shiv.min.js"></script> |
| 15 |
<![endif]--> |
| 16 |
|
| 17 |
<script src="../_static/jquery.js"></script> |
| 18 |
<script src="../_static/_sphinx_javascript_frameworks_compat.js"></script> |
| 19 |
<script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script> |
| 20 |
<script src="../_static/doctools.js"></script> |
| 21 |
<script src="../_static/sphinx_highlight.js"></script> |
| 22 |
<script src="../_static/js/theme.js"></script> |
| 23 |
<link rel="index" title="Index" href="../genindex.html" /> |
| 24 |
<link rel="next" title="Datetime values" href="datetime.html" /> |
| 25 |
<link rel="prev" title="Axis variables" href="axis_variables.html" /> |
| 26 |
</head> |
| 27 |
|
| 28 |
<body class="wy-body-for-nav"> |
| 29 |
<div class="wy-grid-for-nav"> |
| 30 |
<nav data-toggle="wy-nav-shift" class="wy-nav-side"> |
| 31 |
<div class="wy-side-scroll"> |
| 32 |
<div class="wy-side-nav-search" > |
| 33 |
|
| 34 |
|
| 35 |
|
| 36 |
<a href="../index.html" class="icon icon-home"> |
| 37 |
HARP |
| 38 |
</a> |
| 39 |
<div class="version"> |
| 40 |
1.29 |
| 41 |
</div> |
| 42 |
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu"> |
| 43 |
<ul class="current"> |
| 44 |
<li class="toctree-l1"><a class="reference internal" href="../install.html">Installation</a></li> |
| 45 |
<li class="toctree-l1 current"><a class="reference internal" href="index.html">Conventions</a><ul class="current"> |
| 46 |
<li class="toctree-l2"><a class="reference internal" href="datatypes.html">Data types</a></li> |
| 47 |
<li class="toctree-l2"><a class="reference internal" href="dimensions.html">Dimensions</a></li> |
| 48 |
<li class="toctree-l2"><a class="reference internal" href="global_attributes.html">Global attributes</a></li> |
| 49 |
<li class="toctree-l2"><a class="reference internal" href="variable_attributes.html">Variable attributes</a></li> |
| 50 |
<li class="toctree-l2"><a class="reference internal" href="variables.html">Variables</a></li> |
| 51 |
<li class="toctree-l2"><a class="reference internal" href="variable_names.html">Variable names</a></li> |
| 52 |
<li class="toctree-l2"><a class="reference internal" href="axis_variables.html">Axis variables</a></li> |
| 53 |
<li class="toctree-l2 current"><a class="current reference internal" href="#">Categorical Variables</a></li> |
| 54 |
<li class="toctree-l2"><a class="reference internal" href="datetime.html">Datetime values</a></li> |
| 55 |
<li class="toctree-l2"><a class="reference internal" href="filenames.html">File naming</a></li> |
| 56 |
<li class="toctree-l2"><a class="reference internal" href="netcdf3.html">netCDF-3</a></li> |
| 57 |
<li class="toctree-l2"><a class="reference internal" href="hdf4.html">HDF4</a></li> |
| 58 |
<li class="toctree-l2"><a class="reference internal" href="hdf5.html">HDF5/netCDF-4</a></li> |
| 59 |
<li class="toctree-l2"><a class="reference internal" href="compatibility.html">Compatibility with other standards</a></li> |
| 60 |
<li class="toctree-l2"><a class="reference internal" href="collocation_result.html">Collocation result file</a></li> |
| 61 |
</ul> |
| 62 |
</li> |
| 63 |
<li class="toctree-l1"><a class="reference internal" href="../algorithms/index.html">Algorithms</a></li> |
| 64 |
<li class="toctree-l1"><a class="reference internal" href="../operations.html">Operations</a></li> |
| 65 |
<li class="toctree-l1"><a class="reference internal" href="../ingestions/index.html">Ingestion definitions</a></li> |
| 66 |
<li class="toctree-l1"><a class="reference internal" href="../libharp.html">C library</a></li> |
| 67 |
<li class="toctree-l1"><a class="reference internal" href="../idl.html">IDL interface</a></li> |
| 68 |
<li class="toctree-l1"><a class="reference internal" href="../matlab.html">MATLAB interface</a></li> |
| 69 |
<li class="toctree-l1"><a class="reference internal" href="../python.html">Python interface</a></li> |
| 70 |
<li class="toctree-l1"><a class="reference internal" href="../R.html">R interface</a></li> |
| 71 |
<li class="toctree-l1"><a class="reference internal" href="../tools.html">Command line tools</a></li> |
| 72 |
</ul> |
| 73 |
|
| 74 |
</div> |
| 75 |
</div> |
| 76 |
</nav> |
| 77 |
|
| 78 |
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" > |
| 79 |
<i data-toggle="wy-nav-top" class="fa fa-bars"></i> |
| 80 |
<a href="../index.html">HARP</a> |
| 81 |
</nav> |
| 82 |
|
| 83 |
<div class="wy-nav-content"> |
| 84 |
<div class="rst-content"> |
| 85 |
<div role="navigation" aria-label="Page navigation"> |
| 86 |
<ul class="wy-breadcrumbs"> |
| 87 |
<li><a href="../index.html" class="icon icon-home" aria-label="Home"></a></li> |
| 88 |
<li class="breadcrumb-item"><a href="index.html">Conventions</a></li> |
| 89 |
<li class="breadcrumb-item active">Categorical Variables</li> |
| 90 |
<li class="wy-breadcrumbs-aside"> |
| 91 |
</li> |
| 92 |
</ul> |
| 93 |
<hr/> |
| 94 |
</div> |
| 95 |
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article"> |
| 96 |
<div itemprop="articleBody"> |
| 97 |
|
| 98 |
<section id="categorical-variables"> |
| 99 |
<h1>Categorical Variables</h1> |
| 100 |
<p>HARP has special support for categorical variables, which are variables that can be represented by an enumeration type.</p> |
| 101 |
<p>Categorical variables are represented in HARP using integer data types that provide an index into a fixed list of |
| 102 |
enumeration labels. These enumeration labels are stored as metadata together with a variable. For <code class="docutils literal notranslate"><span class="pre">int8</span></code> data the |
| 103 |
labels are stored as a <code class="docutils literal notranslate"><span class="pre">flag_meanings</span></code> attribute (together with a <code class="docutils literal notranslate"><span class="pre">flag_values</span></code> attribute) in netCDF3, HDF4, and |
| 104 |
HDF5 files.</p> |
| 105 |
<p>HARP does not impose a rule on which integer value maps to a specific enumeration label. The integer values are |
| 106 |
just plain indices into the enumeration label list, and the list can be sorted in any way.</p> |
| 107 |
<p>Any integer value in a categorical variable that is outside the range of the available enumeration labels |
| 108 |
(<code class="docutils literal notranslate"><span class="pre">value</span> <span class="pre"><</span> <span class="pre">0</span></code> or <code class="docutils literal notranslate"><span class="pre">value</span> <span class="pre">>=</span> <span class="pre">N</span></code>) will be treated as an invalid value (<code class="docutils literal notranslate"><span class="pre">valid_min</span></code> and <code class="docutils literal notranslate"><span class="pre">valid_max</span></code> should therefore |
| 109 |
always be set to <code class="docutils literal notranslate"><span class="pre">0</span></code> and <code class="docutils literal notranslate"><span class="pre">N-1</span></code> respectively for these variables). Invalid values are represented by an empty string |
| 110 |
as label.</p> |
| 111 |
<p>Operations in HARP on categorical variables will need to primarily use the enumeration label <cite>names</cite> |
| 112 |
(and not the <cite>integer values</cite>). Any standardization (such as for the variable names) on the categories for |
| 113 |
categorical variables in HARP will thus only be on enumeration names and not on their numbers.</p> |
| 114 |
<p>In HARP we distinguish category types and category values. The name of the category type <code class="docutils literal notranslate"><span class="pre"><cattype></span></code> is used in the |
| 115 |
variable name and the names of the category values <code class="docutils literal notranslate"><span class="pre"><catval></span></code> are the enumeration labels. Variables for category |
| 116 |
types should generally be called <code class="docutils literal notranslate"><span class="pre"><cattype>_type</span></code>. For instance, if we want to classify clouds we would have |
| 117 |
<code class="docutils literal notranslate"><span class="pre"><cattype>=cloud</span></code> and a variable name <code class="docutils literal notranslate"><span class="pre">cloud_type</span></code>, and for land use classification we would have <code class="docutils literal notranslate"><span class="pre">land_type</span></code>.</p> |
| 118 |
<p>The enumeration labels for a variable should ideally be fully distinct categories. This means that if you have land |
| 119 |
types <code class="docutils literal notranslate"><span class="pre">A</span></code>, <code class="docutils literal notranslate"><span class="pre">B</span></code>, and <code class="docutils literal notranslate"><span class="pre">C</span></code>, you should try to avoid also having land types for <code class="docutils literal notranslate"><span class="pre">A_and_B</span></code>, <code class="docutils literal notranslate"><span class="pre">B_and_C</span></code>, |
| 120 |
<code class="docutils literal notranslate"><span class="pre">A_and_B_and_C</span></code>, etc. |
| 121 |
A <code class="docutils literal notranslate"><span class="pre"><cattype>_type</span></code> variable should ideally indicate just a single dominant applicable category. |
| 122 |
To deal with mixed presence of multiple categories one can use variables such as <code class="docutils literal notranslate"><span class="pre"><catval>_flag</span></code> and |
| 123 |
<code class="docutils literal notranslate"><span class="pre"><catval>_fraction</span></code>.</p> |
| 124 |
<p>For instance, for a <code class="docutils literal notranslate"><span class="pre">land_type</span></code> variable we could have <code class="docutils literal notranslate"><span class="pre">forest</span></code> as enumeration label, but then also have variables |
| 125 |
<code class="docutils literal notranslate"><span class="pre">forest_flag</span></code> and <code class="docutils literal notranslate"><span class="pre">forest_fraction</span></code>.</p> |
| 126 |
<p>A <code class="docutils literal notranslate"><span class="pre"><catval>_flag</span></code> variable in HARP should always be a binary/dichotomous variable (it should contain true/false, 1/0 |
| 127 |
values). A flag variable denotes whether something is present/applicable/true or not. A flag variable will therefore |
| 128 |
generally always have an <code class="docutils literal notranslate"><span class="pre">int8</span></code> data type and is not treated as categorical variable (i.e. it will not have |
| 129 |
enumeration labels). Important to note is that <code class="docutils literal notranslate"><span class="pre"><*>_flag</span></code> variables in HARP should not be used as bitfield variables |
| 130 |
(unlike the <code class="docutils literal notranslate"><span class="pre"><*>_validity</span></code> variables).</p> |
| 131 |
<p>A <code class="docutils literal notranslate"><span class="pre"><catval>_fraction</span></code> variable can be used to define the fraction of the area that has the given <code class="docutils literal notranslate"><span class="pre"><catval></span></code> |
| 132 |
classification. This fraction is a floating point value between 0 and 1.</p> |
| 133 |
<p>Note that many other data product formats use bitfields to indicate applicability of multiple classification categories |
| 134 |
in order to save storage space. However, the aim of HARP is not to provide a compact data format, but to have data |
| 135 |
variables that can be readily used by algorithms. For this reason, combining multiple information elements in a single |
| 136 |
variable is discouraged within the HARP conventions.</p> |
| 137 |
</section> |
| 138 |
|
| 139 |
|
| 140 |
</div> |
| 141 |
</div> |
| 142 |
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer"> |
| 143 |
<a href="axis_variables.html" class="btn btn-neutral float-left" title="Axis variables" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a> |
| 144 |
<a href="datetime.html" class="btn btn-neutral float-right" title="Datetime values" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a> |
| 145 |
</div> |
| 146 |
|
| 147 |
<hr/> |
| 148 |
|
| 149 |
<div role="contentinfo"> |
| 150 |
<p>© Copyright 2015-2026 S[&]T, The Netherlands.</p> |
| 151 |
</div> |
| 152 |
|
| 153 |
|
| 154 |
|
| 155 |
</footer> |
| 156 |
</div> |
| 157 |
</div> |
| 158 |
</section> |
| 159 |
</div> |
| 160 |
<script> |
| 161 |
jQuery(function () { |
| 162 |
SphinxRtdTheme.Navigation.enable(true); |
| 163 |
}); |
| 164 |
</script> |
| 165 |
|
| 166 |
</body> |
| 167 |
</html> |