Debian Codegraph @

armci/examples/features/aggregation/README

raw
1 Introduction
2 ============
3
4 ARMCI has support for aggregating messages via its aggregate non-blocking
5 handles. Any ARMCI non-blocking handle can be explicitly made as an aggregate
6 handle and all subsequent messages that use the same handle are aggregated and
7 sent as a single message. The details of the implementation can be found in
8 the cluster 2003 paper from the ARMCI publications website.
9
10 Description
11 ============
12
13 The implicit aggregation of data transfers is implemented using the
14 generalized I/O vector operations available in ARMCI. This interface enables
15 the representation of a data transfer as a combination of multiple sets of
16 equally sized contiguous data segments. When the first call involving
17 aggregate nonblocking handle is executed, the library starts building a vector
18 descriptor stored in one of the preallocated internal buffers. The actual data
19 transfer takes place when the user calls wait operation or the buffer storing
20 the vector descriptor fills up.
21
22 Function Definitions
23 ====================
24
25 ARMCI_SET_AGGREGATE_HANDLE (armci_hdl_t* handle)
26 ------------------------------------------------
27
28 handle - Pointer to a desciptor associated with a particular non-blocking
29 transfer.
30
31 PURPOSE: Mark a handle as aggregate. This will allow ARMCI to combine
32 nonblocking operations that use that particular handle and process them as a
33 single operation. In the initial implementation only contiguous puts or gets
34 could use aggregate handle. Specifying the same handle for a mix of put anmd
35 get calls is not allowed i.e., only multiple put or only multiple get calls
36 can use the same handle.
37
38 ARMCI_UNSET_AGGREGATE_HANDLE (armci_hdl_t* handle)
39 --------------------------------------------------
40
41 handle - Pointer to a desciptor associated with a particular non-blocking
42 transfer.
43
44 PURPOSE: Clears a handle that has been marked as aggregate.
45
46 Sample Programs
47 ===============
48
49 1. Sparse matrix-vector multiplication
50
51 Sparse matrix-vector multiplication is one of the common computational
52 kernels, for example in solving linear systems using conjugate gradient
53 method. It is described as Ax = b, where A is an nxn nonsingular sparse
54 matrix, b is an n-dimensional vector, and x is an n-dimensional vector of
55 unknowns. In this benchmark, one of the sparse matrices (Figure 8a) from
56 the Harwell-Boeing collection is used [9] to test the matrix-vector
57 multiplication. The sparse matrix size is 41092 and has 1683902 (~.1%)
58 nonzero elements.
59
60 The experiments were conducted on the Linux cluster (dual node, 1GHz
61 Itanium-2, Myrinet-2000 interconnect) at PNNL. Sparse matrix-vector
62 multiplication was done with aggregation enabled and disabled. The sparse
63 matrix and the vector are distributed among processors. Instead of
64 gathering the entire vector, each process caches the vector elements
65 corresponding to the non-zero element columns of its locally owned part of
66 the matrix. When aggregation is enabled, all the get calls corresponding to
67 a single processor are aggregated into a single request, thus reducing the
68 overall latency and improving the data transfer rate.
69
70 This sample program is in the sparse_matvecmul directory. There is an input
71 required for this program. The method to obtain the input is given in the
72 sparse_matvecmul directory in a README file