/************************************************************************/ /* */ /* Copyright 2011-2012 by Ullrich Koethe */ /* */ /* This file is part of the VIGRA computer vision library. */ /* The VIGRA Website is */ /* http://hci.iwr.uni-heidelberg.de/vigra/ */ /* Please direct questions, bug reports, and contributions to */ /* ullrich.koethe@iwr.uni-heidelberg.de or */ /* vigra@informatik.uni-hamburg.de */ /* */ /* Permission is hereby granted, free of charge, to any person */ /* obtaining a copy of this software and associated documentation */ /* files (the "Software"), to deal in the Software without */ /* restriction, including without limitation the rights to use, */ /* copy, modify, merge, publish, distribute, sublicense, and/or */ /* sell copies of the Software, and to permit persons to whom the */ /* Software is furnished to do so, subject to the following */ /* conditions: */ /* */ /* The above copyright notice and this permission notice shall be */ /* included in all copies or substantial portions of the */ /* Software. */ /* */ /* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND */ /* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES */ /* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND */ /* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT */ /* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, */ /* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING */ /* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR */ /* OTHER DEALINGS IN THE SOFTWARE. */ /* */ /************************************************************************/ #ifndef VIGRA_ACCUMULATOR_HXX #define VIGRA_ACCUMULATOR_HXX #ifdef _MSC_VER #pragma warning (disable: 4503) #endif #include "accumulator-grammar.hxx" #include "config.hxx" #include "metaprogramming.hxx" #include "bit_array.hxx" #include "static_assert.hxx" #include "mathutil.hxx" #include "utilities.hxx" #include "multi_iterator_coupled.hxx" #include "matrix.hxx" #include "multi_math.hxx" #include "eigensystem.hxx" #include "histogram.hxx" #include "polygon.hxx" #ifdef WITH_LEMON #include "polytope.hxx" #endif #include "functorexpression.hxx" #include "labelimage.hxx" #include "multi_labeling.hxx" #include #include namespace vigra { /** \defgroup FeatureAccumulators Feature Accumulators The namespace vigra::acc provides the function \ref vigra::acc::extractFeatures() along with associated statistics functors and accumulator classes. Together, they provide a framework for efficient compution of a wide variety of statistical features, both globally for an entire image, and locally for each region defined by a label array. Many different statistics can be composed out of a small number of fundamental statistics and suitable modifiers. The user simply selects the desired statistics by means of their tags (see below), and a template meta-program automatically generates an efficient functor that computes exactly those statistics. The function \ref acc::extractFeatures() "extractFeatures()" scans the data in as few passes as the selected statstics permit (usually one or two passes are sufficient). Statistics are computed by accurate incremental algorithms, whose internal state is maintained by accumulator objects. The state is updated by passing data to the accumulator one sample at a time. Accumulators are grouped within an accumulator chain. Dependencies between accumulators in the accumulator chain are automatically resolved and missing dependencies are inserted. For example, to compute the mean, you also need to count the number of samples. This allows accumulators to offload some of their computations on other accumulators, making the algorithms more efficient. Each accumulator only sees data in the appropriate pass through the data, called its "working pass". \#include \ Basic statistics: - PowerSum (computes @f$ \sum_i x_i^N @f$) - AbsPowerSum (computes @f$ \sum_i |x_i|^N @f$) - Skewness, UnbiasedSkewness - Kurtosis, UnbiasedKurtosis - Minimum, Maximum - FlatScatterMatrix (flattened upper-triangular part of scatter matrix) - 4 histogram classes (see \ref histogram "below") - StandardQuantiles (0%, 10%, 25%, 50%, 75%, 90%, 100%) - ArgMinWeight, ArgMaxWeight (store data or coordinate where weight assumes its minimal or maximal value) - CoordinateSystem (identity matrix of appropriate size) Modifiers: (S is the statistc to be modified) - Normalization
DivideByCount S/Count
RootDivideByCount sqrt( S/Count )
DivideUnbiased S/(Count-1)
RootDivideUnbiased     sqrt( S/(Count-1) )
- Data preparation:
Central substract mean before computing S
Principal project onto PCA eigenvectors
Whitened     scale to unit variance after PCA
Coord compute S from pixel coordinates rather than from pixel values
Weighted compute weighted version of S
Global compute S globally rather than per region (per region is default if labels are given)
Aliases for many important features are implemented (mainly as typedef FullName Alias). The alias names are equivalent to full names. Below are some examples for supported alias names. A full list of all available statistics and alias names can be found in the namespace reference vigra::acc. These examples also show how to compose statistics from the fundamental statistics and modifiers:
Alias Full Name
Count PowerSum<0>
Sum PowerSum<1>
SumOfSquares PowerSum<2>
Mean DivideByCount>
RootMeanSquares   RootDivideByCount>
Moment DivideByCount>
Variance DivideByCount>>
StdDev RootDivideByCount>>
Covariance DivideByCount
RegionCenter Coord
CenterOfMass Weighted>
There are a few rules for composing statistics: - modifiers can be specified in any order, but are internally transformed to standard order: Global or Coord<> modifiers, see below): \code #include #include #include using namespace vigra::acc; typedef double DataType; int size = 1000; vigra::MultiArray<2, DataType> data(vigra::Shape2(size, size)); AccumulatorChain > a; std::cout << "passes required: " << a.passesRequired() << std::endl; extractFeatures(data.begin(), data.end(), a); std::cout << "Mean: " << get(a) << std::endl; std::cout << "Variance: " << get(a) << std::endl; \endcode The \ref acc::AccumulatorChain object contains the selected statistics and their dependencies. Statistics have to be wrapped with \ref acc::Select. The statistics are computed with the acc::extractFeatures function and the statistics can be accessed with acc::get . Rules and notes: - order of statistics in Select<> is arbitrary - up to 20 statistics in Select<>, but Select<> can be nested - dependencies are automatically inserted - duplicates are automatically removed - extractFeatures() does as many passes through the data as necessary - each accumulator only sees data in the appropriate pass (its "working pass") The Accumulators can also be used with vector-valued data (vigra::RGBValue, vigra::TinyVector, vigra::MultiArray or vigra::MultiArrayView): \code typedef vigra::RGBValue DataType; AccumulatorChain > a; ... \endcode To compute weighted statistics (Weighted<>) or statistics over coordinates (Coord<>), the accumulator chain can be used with several coupled arrays, one for the data and another for the weights and/or the labels. "Coupled" means that statistics are computed over the corresponding elements of the involved arrays. This is internally done by means of \ref CoupledScanOrderIterator and \ref vigra::CoupledHandle which provide simultaneous access to several arrays (e.g. weight and data) and corresponding coordinates. The types of the coupled arrays are best specified by means of the helper class \ref vigra::CoupledArrays : \code vigra::MultiArray<3, RGBValue > data(...); vigra::MultiArray<3, double> weights(...); AccumulatorChain, double>, Select<...> > a; \endcode This works likewise for label images which are needed for region statistics (see below). The indxx of the array holding data, weights, or labels respectively can be specified inside the Select wrapper. These index specifiers are: (INDEX is of type int) - DataArg: data are in array 'INDEX' (default INDEX=1) - LabelArg: labels are in array 'INDEX' (default INDEX=2) - WeightArg: weights are in array 'INDEX' (default INDEX=rightmost index) Pixel coordinates are always at index 0. To collect statistics, you simply pass all arrays to the extractFeatures() function: \code using namespace vigra::acc; vigra::MultiArray<3, double> data(...), weights(...); AccumulatorChain, // two 3D arrays for data and weights Select, WeightArg<2>, // in which array to look (coordinates are always arg 0) Mean, Variance, //statistics over values Coord, Coord, //statistics over coordinates, Weighted, Weighted, //weighted values, Weighted > > > //weighted coordinates. a; extractFeatures(data, weights, a); \endcode This even works for a single array, which is useful if you want to combine values with coordinates. For example, to find the location of the minimum element in an array, you interpret the data as weights and select the Coord statistic (note that the version of extractFeatures() below only works in conjunction with CoupledArrays, despite the fact that there is only one array involved): \code using namespace vigra::acc; vigra::MultiArray<3, double> data(...); AccumulatorChain, Select, // we interprete the data as weights Coord > > // and look for the coordinate with minimal weight a; extractFeatures(data, a); std::cout << "minimum is at " << get >(a) << std::endl; \endcode To compute region statistics, you use \ref acc::AccumulatorChainArray. Regions are defined by means of a label array whose elements specify the region ID of the corresponding point. Therefore, you will always need at least two arrays here, which are again best specified using the CoupledArrays helper: \code using namespace vigra::acc; vigra::MultiArray<3, double> data(...); vigra::MultiArray<3, int> labels(...); AccumulatorChainArray, Select, LabelArg<2>, // in which array to look (coordinates are always arg 0) Mean, Variance, //per-region statistics over values Coord, Coord, //per-region statistics over coordinates Global, Global > > //global statistics a; a.ignoreLabel(0); //statistics will not be computed for region 0 (e.g. background) extractFeatures(data, labels, a); int regionlabel = ...; std::cout << get(a, regionlabel) << std::endl; //get Mean of region with label 'regionlabel' \endcode In some application it will be known only at run-time which statistics have to be computed. An Accumulator with run-time activation is provided by the \ref acc::DynamicAccumulatorChain class. One specifies a set of statistics at compile-time and from this set one can activate the needed statistics at run-time: \code using namespace vigra::acc; vigra::MultiArray<2, double> data(...); DynamicAccumulatorChain > a; // at compile-time activate(a); //at run-time a.activate("Minimum"); //same as activate(a) (alias names are not recognized) extractFeatures(data.begin(), data.end(), a); std::cout << "Mean: " << get(a) << std::endl; //ok //std::cout << "Maximum: " << get(a) << std::endl; // run-time error because Maximum not activated \endcode Likewise, for run-time activation of region statistics, use \ref acc::DynamicAccumulatorChainArray. Accumulator merging (e.g. for parallelization or hierarchical segmentation) is possible for many accumulators: \code using namespace vigra::acc; vigra::MultiArray<2, double> data(...); AccumulatorChain > a, a1, a2; extractFeatures(data.begin(), data.end(), a); //process entire data set at once extractFeatures(data.begin(), data.begin()+data.size()/2, a1); //process first half extractFeatures(data.begin()+data.size()/2, data.end(), a2); //process second half a1 += a2; // merge: a1 now equals a0 (within numerical tolerances) \endcode Not all statistics can be merged (e.g. Principal usually cannot, except for some important specializations). A statistic can be merged if the "+=" operator is supported (see the documentation of that particular statistic). If the accumulator chain only requires one pass to collect the data, it is also possible to just apply the extractFeatures() function repeatedly: \code using namespace vigra::acc; vigra::MultiArray<2, double> data(...); AccumulatorChain > a; extractFeatures(data.begin(), data.begin()+data.size()/2, a); // this works because extractFeatures(data.begin()+data.size()/2, data.end(), a); // all statistics only need pass 1 \endcode More care is needed to merge coordinate-based statistics. By default, all coordinate statistics are computed in the local coordinate system of the current region of interest. That is, the upper left corner of the ROI has the coordinate (0, 0) by default. This behavior is not desirable when you want to merge coordinate statistics from different ROIs: then, all accumulators should use the same coordinate system, usually the global system of the entire dataset. This can be achieved by the setCoordinateOffset() function. The following code demonstrates this for the RegionCenter statistic: \code using namespace vigra; using namespace vigra::acc; MultiArray<2, double> data(width, height); MultiArray<2, int> labels(width, height); AccumulatorChainArray, Select, LabelArg<2>, RegionCenter> > a1, a2; // a1 is responsible for the left half of the image. The local coordinate system of this ROI // happens to be identical to the global coordinate system, so the offset is zero. Shape2 origin(0,0); a1.setCoordinateOffset(origin); extractFeatures(data.subarray(origin, Shape2(width/2, height)), labels.subarray(origin, Shape2(width/2, height)), a1); // a2 is responsible for the right half, so the offset of the local coordinate system is (width/2, 0) origin = Shape2(width/2, 0); a2.setCoordinateOffset(origin); extractFeatures(data.subarray(origin, Shape2(width, height)), labels.subarray(origin, Shape2(width, height)), a2); // since both accumulators worked in the same global coordinate system, we can safely merge them a1.merge(a2); \endcode When you compute region statistics in ROIs, it is sometimes desirable to use a local region labeling in each ROI. In this way, the labels of each ROI cover a consecutive range of numbers starting with 0. This can save a lot of memory, because AccumulatorChainArray internally uses dense arrays -- accumulators will be allocated for all labels from 0 to the maxmimum label, even when many of them are unused. This is avoided by a local labeling. However, this means that label 1 (say) may refer to two different regions in different ROIs. To adjust for this mismatch, you can pass a label mapping to merge() that provides a global label for each label of the accumulator to be merged. Thus, each region on the right hand side will be merged into the left-hand-side accumulator with the given global label. For example, let us assume that the left and right half of the image contain just one region and background. Then, the accumulators of both ROIs have the label 0 (background) and 1 (the region). Upon merging, the region from the right ROI should be given the global label 2, whereas the background should keep its label 0. This is achieved like this: \code std::vector labelMapping(2); labelMapping[0] = 0; // background keeps label 0 labelMapping[1] = 2; // local region 1 becomes global region 2 a1.merge(a2, labelMapping); \endcode \anchor histogram Four kinds of histograms are currently implemented:
IntegerHistogram Data values are equal to bin indices
UserRangeHistogram User provides lower and upper bounds for linear range mapping from values to indices.
AutoRangeHistogram Range mapping bounds are defiend by minimum and maximum of the data (2 passes needed!)
GlobalRangeHistogram   Likewise, but use global min/max rather than region min/max as AutoRangeHistogram will
- The number of bins is specified at compile time (as template parameter int BinCount) or at run-time (if BinCount is zero at compile time). In the first case the return type of the accumulator is TinyVector (number of bins cannot be changed). In the second case, the return type is MultiArray<1, double> and the number of bins must be set before seeing data (see example below). - If UserRangeHistogram is used, the bounds for the linear range mapping from values to indices must be set before seeing data (see below). - Options can be set by passing an instance of HistogramOptions to the accumulator chain (same options for all histograms in the chain) or by directly calling the appropriate member functions of the accumulators. - Merging is supported if the range mapping of the histograms is the same. - Histogram accumulators have two members for outliers (left_outliers, right_outliers). With the StandardQuantiles class, histogram quantiles (0%, 10%, 25%, 50%, 75%, 90%, 100%) are computed from a given histgram using linear interpolation. The return type is TinyVector . \anchor acc_hist_options Usage: \code using namespace vigra::acc; typedef double DataType; vigra::MultiArray<2, DataType> data(...); typedef UserRangeHistogram<40> SomeHistogram; //binCount set at compile time typedef UserRangeHistogram<0> SomeHistogram2; // binCount must be set at run-time typedef AutoRangeHistogram<0> SomeHistogram3; typedef StandardQuantiles Quantiles3; AccumulatorChain > a; //set options for all histograms in the accumulator chain: vigra::HistogramOptions histogram_opt; histogram_opt = histogram_opt.setBinCount(50); //histogram_opt = histogram_opt.setMinMax(0.1, 0.9); // this would set min/max for all three histograms, but range bounds // shall be set automatically by min/max of data for SomeHistogram3 a.setHistogramOptions(histogram_opt); // set options for a specific histogram in the accumulator chain: getAccumulator(a).setMinMax(0.1, 0.9); // number of bins must be set before setting min/max getAccumulator(a).setMinMax(0.0, 1.0); extractFeatures(data.begin(), data.end(), a); vigra::TinyVector hist = get(a); vigra::MultiArray<1, double> hist2 = get(a); vigra::TinyVector quant = get(a); double right_outliers = getAccumulator(a).right_outliers; \endcode */ /** \brief Efficient computation of object statistics. This namespace contains the accumulator classes, fundamental statistics and modifiers. See \ref FeatureAccumulators for examples of usage. */ namespace acc { /****************************************************************************/ /* */ /* infrastructure */ /* */ /****************************************************************************/ /// \brief Wrapper for MakeTypeList that additionally performs tag standardization. template struct Select : public MakeTypeList< typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type, typename StandardizeTag::type > {}; // enable nesting of Select<> expressions template struct StandardizeTag, Select > { typedef typename Select::type type; }; struct AccumulatorBegin { typedef Select<> Dependencies; static std::string name() { return "AccumulatorBegin (internal)"; // static const std::string n("AccumulatorBegin (internal)"); // return n; } template struct Impl : public BASE {}; }; struct AccumulatorEnd; struct DataArgTag; struct WeightArgTag; struct LabelArgTag; struct CoordArgTag; struct LabelDispatchTag; template struct HandleArgSelector; // find the correct handle in a CoupledHandle struct Error__Global_statistics_are_only_defined_for_AccumulatorChainArray; /** \brief Specifies index of labels in CoupledHandle. LabelArg tells the acc::AccumulatorChainArray which index of the Handle contains the labels. (Note that coordinates are always index 0) */ template class LabelArg { public: typedef Select<> Dependencies; static std::string name() { return std::string("LabelArg<") + asString(INDEX) + "> (internal)"; // static const std::string n = std::string("LabelArg<") + asString(INDEX) + "> (internal)"; // return n; } template struct Impl : public BASE { typedef LabelArgTag Tag; typedef void value_type; typedef void result_type; static const int value = INDEX; static const unsigned int workInPass = 0; }; }; template class CoordArg { public: typedef Select<> Dependencies; static std::string name() { return std::string("CoordArg<") + asString(INDEX) + "> (internal)"; // static const std::string n = std::string("CoordArg<") + asString(INDEX) + "> (internal)"; // return n; } template struct Impl : public BASE { typedef CoordArgTag Tag; typedef void value_type; typedef void result_type; static const int value = INDEX; static const unsigned int workInPass = 0; }; }; template struct AccumulatorBase; template struct LookupTag; template struct LookupDependency; #ifndef _MSC_VER // compiler bug? (causes 'ambiguous overload error') template typename LookupTag::reference getAccumulator(A & a); template typename LookupDependency::result_type getDependency(A const & a); #endif namespace acc_detail { /****************************************************************************/ /* */ /* internal tag handling meta-functions */ /* */ /****************************************************************************/ // we must make sure that Arg tags are at the end of the chain because // all other tags potentially depend on them template struct PushArgTagToTail { typedef T type; }; #define VIGRA_PUSHARGTAG(TAG) \ template \ struct PushArgTagToTail, TAIL> > \ { \ typedef typename Push > >::type type; \ }; VIGRA_PUSHARGTAG(DataArg) VIGRA_PUSHARGTAG(WeightArg) VIGRA_PUSHARGTAG(CoordArg) VIGRA_PUSHARGTAG(LabelArg) #undef VIGRA_PUSHARGTAG // Insert the dependencies of the selected functors into the TypeList and sort // the list such that dependencies come after the functors using them. Make sure // that each functor is contained only once. template struct AddDependencies; template struct AddDependencies > { typedef typename AddDependencies::type TailWithDependencies; typedef typename StandardizeDependencies::type HeadDependencies; typedef typename AddDependencies::type TransitiveHeadDependencies; typedef TypeList HeadWithDependencies; typedef typename PushUnique::type UnsortedDependencies; typedef typename PushArgTagToTail::type type; }; template <> struct AddDependencies { typedef void type; }; // Helper class to activate dependencies at runtime (i.e. when activate(accu) is called, // activate() must also be called for Tag's dependencies). template struct ActivateDependencies; template struct ActivateDependencies > { template static void exec(ActiveFlags & flags) { LookupTag::type::activateImpl(flags); ActivateDependencies::template exec(flags); } template static void exec(ActiveFlags & flags, GlobalFlags & gflags) { LookupTag::type::template activateImpl(flags, gflags); ActivateDependencies::template exec(flags, gflags); } }; template struct ActivateDependencies, TAIL> > { template static void exec(ActiveFlags & flags, GlobalFlags & gflags) { LookupTag, Chain>::type::activateImpl(gflags); ActivateDependencies::template exec(flags, gflags); } }; template <> struct ActivateDependencies { template static void exec(ActiveFlags &) {} template static void exec(ActiveFlags &, GlobalFlags &) {} }; template struct SeparateGlobalAndRegionTags; template struct SeparateGlobalAndRegionTags > { typedef SeparateGlobalAndRegionTags Inner; typedef TypeList RegionTags; typedef typename Inner::GlobalTags GlobalTags; }; template struct SeparateGlobalAndRegionTags, TAIL> > { typedef SeparateGlobalAndRegionTags Inner; typedef typename Inner::RegionTags RegionTags; typedef TypeList GlobalTags; }; template struct SeparateGlobalAndRegionTags, TAIL> > { typedef SeparateGlobalAndRegionTags Inner; typedef TypeList, typename Inner::RegionTags> RegionTags; typedef TypeList, typename Inner::GlobalTags> GlobalTags; }; template struct SeparateGlobalAndRegionTags, TAIL> > { typedef SeparateGlobalAndRegionTags Inner; typedef TypeList, typename Inner::RegionTags> RegionTags; typedef TypeList, typename Inner::GlobalTags> GlobalTags; }; template struct SeparateGlobalAndRegionTags, TAIL> > { typedef SeparateGlobalAndRegionTags Inner; typedef TypeList, typename Inner::RegionTags> RegionTags; typedef TypeList, typename Inner::GlobalTags> GlobalTags; }; template struct SeparateGlobalAndRegionTags, TAIL> > { typedef SeparateGlobalAndRegionTags Inner; typedef TypeList, typename Inner::RegionTags> RegionTags; typedef TypeList, typename Inner::GlobalTags> GlobalTags; }; template <> struct SeparateGlobalAndRegionTags { typedef void RegionTags; typedef void GlobalTags; }; /****************************************************************************/ /* */ /* helper classes to handle tags at runtime via strings */ /* */ /****************************************************************************/ template struct CollectAccumulatorNames; template struct CollectAccumulatorNames > { template static void exec(BackInsertable & a, bool skipInternals=true) { if(!skipInternals || HEAD::name().find("internal") == std::string::npos) a.push_back(HEAD::name()); CollectAccumulatorNames::exec(a, skipInternals); } }; template <> struct CollectAccumulatorNames { template static void exec(BackInsertable &, bool /* skipInternals */ = true) {} }; template struct ApplyVisitorToTag; template struct ApplyVisitorToTag > { template static bool exec(Accu & a, std::string const & tag, Visitor const & v) { static std::string * name = VIGRA_SAFE_STATIC(name, new std::string(normalizeString(HEAD::name()))); if(*name == tag) { v.template exec(a); return true; } else { return ApplyVisitorToTag::exec(a, tag, v); } } }; template <> struct ApplyVisitorToTag { template static bool exec(Accu &, std::string const &, Visitor const &) { return false; } }; struct ActivateTag_Visitor { template void exec(Accu & a) const { a.template activate(); } }; struct TagIsActive_Visitor { mutable bool result; template void exec(Accu & a) const { result = a.template isActive(); } }; /****************************************************************************/ /* */ /* histogram initialization functors */ /* */ /****************************************************************************/ template struct SetHistogramBincount { template static void exec(Accu &, HistogramOptions const &) {} }; template