| 1 |
%% |
| 2 |
%% |
| 3 |
|
| 4 |
\chapter{Bacula Memory Management} |
| 5 |
\label{blb:TheChapterStart7} |
| 6 |
\index{Management!Bacula Memory} |
| 7 |
\index{Bacula Memory Management} |
| 8 |
|
| 9 |
\section{General} |
| 10 |
\index{General} |
| 11 |
|
| 12 |
This document describes the memory management routines that are used in Bacula |
| 13 |
and is meant to be a technical discussion for developers rather than part of |
| 14 |
the user manual. |
| 15 |
|
| 16 |
Since Bacula may be called upon to handle filenames of varying and more or |
| 17 |
less arbitrary length, special attention needs to be used in the code to |
| 18 |
ensure that memory buffers are sufficiently large. There are four |
| 19 |
possibilities for memory usage within {\bf Bacula}. Each will be described in |
| 20 |
turn. They are: |
| 21 |
|
| 22 |
\begin{bitemize} |
| 23 |
\item Statically allocated memory. |
| 24 |
\item Dynamically allocated memory using malloc() and free(). |
| 25 |
\item Non-pooled memory. |
| 26 |
\item Pooled memory. |
| 27 |
\end{bitemize} |
| 28 |
|
| 29 |
\subsection{Statically Allocated Memory} |
| 30 |
\index{Statically Allocated Memory} |
| 31 |
\index{Memory!Statically Allocated} |
| 32 |
|
| 33 |
Statically allocated memory is of the form: |
| 34 |
|
| 35 |
\footnotesize |
| 36 |
\begin{bVerbatim} |
| 37 |
char buffer[MAXSTRING]; |
| 38 |
\end{bVerbatim} |
| 39 |
\normalsize |
| 40 |
|
| 41 |
The use of this kind of memory is discouraged except when you are 100\% sure |
| 42 |
that the strings to be used will be of a fixed length. One example of where |
| 43 |
this is appropriate is for {\bf Bacula} resource names, which are currently |
| 44 |
limited to 127 characters (MAX\_NAME\_LENGTH). Although this maximum size may |
| 45 |
change, particularly to accommodate Unicode, it will remain a relatively small |
| 46 |
value. |
| 47 |
|
| 48 |
\subsection{Dynamically Allocated Memory} |
| 49 |
\index{Dynamically Allocated Memory} |
| 50 |
\index{Memory!Dynamically Allocated} |
| 51 |
|
| 52 |
Dynamically allocated memory is obtained using the standard malloc() routines. |
| 53 |
As in: |
| 54 |
|
| 55 |
\footnotesize |
| 56 |
\begin{bVerbatim} |
| 57 |
char *buf; |
| 58 |
buf = malloc(256); |
| 59 |
\end{bVerbatim} |
| 60 |
\normalsize |
| 61 |
|
| 62 |
This kind of memory can be released with: |
| 63 |
|
| 64 |
\footnotesize |
| 65 |
\begin{bVerbatim} |
| 66 |
free(buf); |
| 67 |
\end{bVerbatim} |
| 68 |
\normalsize |
| 69 |
|
| 70 |
It is recommended to use this kind of memory only when you are sure that you |
| 71 |
know the memory size needed and the memory will be used for short periods of |
| 72 |
time -- that is it would not be appropriate to use statically allocated |
| 73 |
memory. An example might be to obtain a large memory buffer for reading and |
| 74 |
writing files. When {\bf SmartAlloc} is enabled, the memory obtained by |
| 75 |
malloc() will automatically be checked for buffer overwrite (overflow) during |
| 76 |
the free() call, and all malloc'ed memory that is not released prior to |
| 77 |
termination of the program will be reported as Orphaned memory. |
| 78 |
|
| 79 |
\subsection{Pooled and Non-pooled Memory} |
| 80 |
\index{Memory!Pooled and Non-pooled} |
| 81 |
\index{Pooled and Non-pooled Memory} |
| 82 |
|
| 83 |
In order to facility the handling of arbitrary length filenames and to |
| 84 |
efficiently handle a high volume of dynamic memory usage, we have implemented |
| 85 |
routines between the C code and the malloc routines. The first is called |
| 86 |
``Pooled'' memory, and is memory, which once allocated and then released, is |
| 87 |
not returned to the system memory pool, but rather retained in a Bacula memory |
| 88 |
pool. The next request to acquire pooled memory will return any free memory |
| 89 |
block. In addition, each memory block has its current size associated with the |
| 90 |
block allowing for easy checking if the buffer is of sufficient size. This |
| 91 |
kind of memory would normally be used in high volume situations (lots of |
| 92 |
malloc()s and free()s) where the buffer length may have to frequently change |
| 93 |
to adapt to varying filename lengths. |
| 94 |
|
| 95 |
The non-pooled memory is handled by routines similar to those used for pooled |
| 96 |
memory, allowing for easy size checking. However, non-pooled memory is |
| 97 |
returned to the system rather than being saved in the Bacula pool. This kind |
| 98 |
of memory would normally be used in low volume situations (few malloc()s and |
| 99 |
free()s), but where the size of the buffer might have to be adjusted |
| 100 |
frequently. |
| 101 |
|
| 102 |
\paragraph*{Types of Memory Pool:} |
| 103 |
|
| 104 |
Currently there are three memory pool types: |
| 105 |
|
| 106 |
\begin{bitemize} |
| 107 |
\item PM\_NOPOOL -- non-pooled memory. |
| 108 |
\item PM\_FNAME -- a filename pool. |
| 109 |
\item PM\_MESSAGE -- a message buffer pool. |
| 110 |
\item PM\_EMSG -- error message buffer pool. |
| 111 |
\end{bitemize} |
| 112 |
|
| 113 |
\paragraph*{Getting Memory:} |
| 114 |
|
| 115 |
To get memory, one uses: |
| 116 |
|
| 117 |
\footnotesize |
| 118 |
\begin{bVerbatim} |
| 119 |
void *get_pool_memory(pool); |
| 120 |
\end{bVerbatim} |
| 121 |
\normalsize |
| 122 |
|
| 123 |
where {\bf pool} is one of the above mentioned pool names. The size of the |
| 124 |
memory returned will be determined by the system to be most appropriate for |
| 125 |
the application. |
| 126 |
|
| 127 |
If you wish non-pooled memory, you may alternatively call: |
| 128 |
|
| 129 |
\footnotesize |
| 130 |
\begin{bVerbatim} |
| 131 |
void *get_memory(size_t size); |
| 132 |
\end{bVerbatim} |
| 133 |
\normalsize |
| 134 |
|
| 135 |
The buffer length will be set to the size specified, and it will be assigned |
| 136 |
to the PM\_NOPOOL pool (no pooling). |
| 137 |
|
| 138 |
\paragraph*{Releasing Memory:} |
| 139 |
|
| 140 |
To free memory acquired by either of the above two calls, use: |
| 141 |
|
| 142 |
\footnotesize |
| 143 |
\begin{bVerbatim} |
| 144 |
void free_pool_memory(void *buffer); |
| 145 |
\end{bVerbatim} |
| 146 |
\normalsize |
| 147 |
|
| 148 |
where buffer is the memory buffer returned when the memory was acquired. If |
| 149 |
the memory was originally allocated as type PM\_NOPOOL, it will be released to |
| 150 |
the system, otherwise, it will be placed on the appropriate Bacula memory pool |
| 151 |
free chain to be used in a subsequent call for memory from that pool. |
| 152 |
|
| 153 |
\paragraph*{Determining the Memory Size:} |
| 154 |
|
| 155 |
To determine the memory buffer size, use: |
| 156 |
|
| 157 |
\footnotesize |
| 158 |
\begin{bVerbatim} |
| 159 |
size_t sizeof_pool_memory(void *buffer); |
| 160 |
\end{bVerbatim} |
| 161 |
\normalsize |
| 162 |
|
| 163 |
\paragraph*{Resizing Pool Memory:} |
| 164 |
|
| 165 |
To resize pool memory, use: |
| 166 |
|
| 167 |
\footnotesize |
| 168 |
\begin{bVerbatim} |
| 169 |
void *realloc_pool_memory(void *buffer); |
| 170 |
\end{bVerbatim} |
| 171 |
\normalsize |
| 172 |
|
| 173 |
The buffer will be reallocated, and the contents of the original buffer will |
| 174 |
be preserved, but the address of the buffer may change. |
| 175 |
|
| 176 |
\paragraph*{Automatic Size Adjustment:} |
| 177 |
|
| 178 |
To have the system check and if necessary adjust the size of your pooled |
| 179 |
memory buffer, use: |
| 180 |
|
| 181 |
\footnotesize |
| 182 |
\begin{bVerbatim} |
| 183 |
void *check_pool_memory_size(void *buffer, size_t new-size); |
| 184 |
\end{bVerbatim} |
| 185 |
\normalsize |
| 186 |
|
| 187 |
where {\bf new-size} is the buffer length needed. Note, if the buffer is |
| 188 |
already equal to or larger than {\bf new-size} no buffer size change will |
| 189 |
occur. However, if a buffer size change is needed, the original contents of |
| 190 |
the buffer will be preserved, but the buffer address may change. Many of the |
| 191 |
low level Bacula subroutines expect to be passed a pool memory buffer and use |
| 192 |
this call to ensure the buffer they use is sufficiently large. |
| 193 |
|
| 194 |
\paragraph*{Releasing All Pooled Memory:} |
| 195 |
|
| 196 |
In order to avoid orphaned buffer error messages when terminating the program, |
| 197 |
use: |
| 198 |
|
| 199 |
\footnotesize |
| 200 |
\begin{bVerbatim} |
| 201 |
void close_memory_pool(); |
| 202 |
\end{bVerbatim} |
| 203 |
\normalsize |
| 204 |
|
| 205 |
to free all unused memory retained in the Bacula memory pool. Note, any memory |
| 206 |
not returned to the pool via free\_pool\_memory() will not be released by this |
| 207 |
call. |
| 208 |
|
| 209 |
\paragraph*{Pooled Memory Statistics:} |
| 210 |
|
| 211 |
For debugging purposes and performance tuning, the following call will print |
| 212 |
the current memory pool statistics: |
| 213 |
|
| 214 |
\footnotesize |
| 215 |
\begin{bVerbatim} |
| 216 |
void print_memory_pool_stats(); |
| 217 |
\end{bVerbatim} |
| 218 |
\normalsize |
| 219 |
|
| 220 |
an example output is: |
| 221 |
|
| 222 |
\footnotesize |
| 223 |
\begin{bVerbatim} |
| 224 |
Pool Maxsize Maxused Inuse |
| 225 |
0 256 0 0 |
| 226 |
1 256 1 0 |
| 227 |
2 256 1 0 |
| 228 |
\end{bVerbatim} |
| 229 |
\normalsize |