Debian Codegraph @

docs/src/cdist-type.rst

raw
1 cdist type
2 ==========
3
4 Description
5 -----------
6 Types are the main component of cdist and define functionality. If you
7 use cdist, you'll write a type for every functionality you would like
8 to use.
9
10 Synopsis
11 --------
12
13 .. code-block:: sh
14
15 __TYPE ID --parameter value [--parameter value ...]
16 __TYPE --parameter value [--parameter value ...] (for singletons)
17
18
19 How to use a type
20 -----------------
21
22 You can use types from the initial manifest or the type manifest like a
23 normal shell command:
24
25 .. code-block:: sh
26
27 # Creates empty file /etc/cdist-configured
28 __file /etc/cdist-configured --type file
29
30 # Ensure tree is installed
31 __package tree --state installed
32
33 A list of supported types can be found in the `cdist reference <cdist-reference.html>`_ manpage.
34
35
36 Singleton types
37 ---------------
38 If a type is flagged as a singleton, it may be used only
39 once per host. This is useful for types which can be used only once on a
40 system. Singleton types do not take an object name as argument.
41
42
43 Example:
44
45 .. code-block:: sh
46
47 # __issue type manages /etc/issue
48 __issue
49
50 # Probably your own type - singletons may use parameters
51 __myfancysingleton --colour green
52
53
54 Config types
55 ------------
56 By default types are used with config command. These are types that are not
57 flagged by any known command flag. If a type is marked then it will be skipped
58 with config command.
59
60
61 Install types
62 -------------
63 If a type is flagged with 'install' flag then it is used only with install command.
64 With other commands, i.e. config, these types are skipped if used.
65
66
67 Nonparallel types
68 -----------------
69 If a type is flagged with 'nonparallel' flag then its objects cannot be run in parallel
70 when using -j option. Example of such a type is __package_dpkg type where dpkg itself
71 prevents to be run in more than one instance.
72
73
74 Deprecated types
75 -----------------
76 If a type is flagged with 'deprecated' marker then it is considered deprecated.
77 When it is used cdist writes warning line. If 'deprecated' marker has content
78 then this content is printed as a deprecation messages, e.g.:
79
80 .. code-block:: sh
81
82 $ ls -l deprecated
83 -rw-r--r-- 1 darko darko 71 May 20 18:30 deprecated
84 $ cat deprecated
85 This type is deprecated. It will be removed in the next minor release.
86 $ echo '__foo foo' | ./bin/cdist config -i - 185.203.112.26
87 WARNING: 185.203.112.26: Type __foo is deprecated: This type is deprecated. It will be removed in the next minor release.
88
89 If 'deprecated' marker has no content then general message is printed, e.g.:
90
91 .. code-block:: sh
92
93 $ ls -l deprecated
94 -rw-r--r-- 1 darko darko 0 May 20 18:36 deprecated
95 $ echo '__bar foo' | ./bin/cdist config -i - 185.203.112.26
96 WARNING: 185.203.112.26: Type __bar is deprecated.
97
98
99 How to write a new type
100 -----------------------
101 A type consists of
102
103 - parameter (optional)
104 - manifest (optional)
105 - singleton (optional)
106 - explorer (optional)
107 - gencode (optional)
108 - nonparallel (optional)
109
110 Types are stored below cdist/conf/type/. Their name should always be prefixed with
111 two underscores (__) to prevent collisions with other executables in $PATH.
112
113 To implement a new type, create the directory **cdist/conf/type/__NAME**.
114
115 Type manifest and gencode can be written in any language. They just need to be
116 executable and have a proper shebang. If they are not executable then cdist assumes
117 they are written in shell so they are executed using '/bin/sh -e' or 'CDIST_LOCAL_SHELL'.
118
119 For executable shell code it is suggested that shebang is '#!/bin/sh -e'.
120
121 For creating type skeleton you can use helper script
122 `cdist-new-type <man1/cdist-new-type.html>`_.
123
124
125 Defining parameters
126 -------------------
127 Every type consists of required, optional and boolean parameters, which must
128 each be declared in a newline separated file in **parameter/required**,
129 **parameter/required_multiple**, **parameter/optional**,
130 **parameter/optional_multiple** and **parameter/boolean**.
131 Parameters which are allowed multiple times should be listed in
132 required_multiple or optional_multiple respectively. All other parameters
133 follow the standard unix behaviour "the last given wins".
134 If either is missing, the type will have no required, no optional, no boolean
135 or no parameters at all.
136
137 Default values for optional parameters can be predefined in
138 **parameter/default/<name>**.
139
140 Example:
141
142 .. code-block:: sh
143
144 echo servername >> cdist/conf/type/__nginx_vhost/parameter/required
145 echo logdirectory >> cdist/conf/type/__nginx_vhost/parameter/optional
146 echo loglevel >> cdist/conf/type/__nginx_vhost/parameter/optional
147 mkdir cdist/conf/type/__nginx_vhost/parameter/default
148 echo warning > cdist/conf/type/__nginx_vhost/parameter/default/loglevel
149 echo server_alias >> cdist/conf/type/__nginx_vhost/parameter/optional_multiple
150 echo use_ssl >> cdist/conf/type/__nginx_vhost/parameter/boolean
151
152
153 Using parameters
154 ----------------
155 The parameters given to a type can be accessed and used in all type scripts
156 (e.g manifest, gencode, explorer). Note that boolean parameters are
157 represented by file existence. File exists -> True,
158 file does not exist -> False
159
160 Example: (e.g. in cdist/conf/type/__nginx_vhost/manifest)
161
162 .. code-block:: sh
163
164 # required parameter
165 servername="$(cat "$__object/parameter/servername")"
166
167 # optional parameter
168 if [ -f "$__object/parameter/logdirectory" ]; then
169 logdirectory="$(cat "$__object/parameter/logdirectory")"
170 fi
171
172 # optional parameter with predefined default
173 loglevel="$(cat "$__object/parameter/loglevel")"
174
175 # boolean parameter
176 if [ -f "$__object/parameter/use_ssl" ]; then
177 # file exists -> True
178 # do some fancy ssl stuff
179 fi
180
181 # parameter with multiple values
182 if [ -f "$__object/parameter/server_alias" ]; then
183 for alias in $(cat "$__object/parameter/server_alias"); do
184 echo $alias > /some/where/useful
185 done
186 fi
187
188
189 Deprecated parameters
190 ---------------------
191 To deprecate type parameters one can declare a file for each deprecated
192 parameter under **parameter/deprecated** directory.
193
194 When such parameter is used cdist writes warning line with deprecation message.
195 If such file has content then this content is printed as deprecation message.
196 If there is no content then generic parameter deprecation message is printed.
197
198 Example:
199
200 .. code-block:: sh
201
202 $ ls parameter/deprecated/
203 eggs spam
204 $ cat parameter/deprecated/eggs
205 eggs parameter is deprecated, please use multiple egg parameter.
206 $ cat parameter/deprecated/spam
207 $ echo '__foo foo --foo foo --eggs eggs' | ./bin/cdist config -i - 185.203.112.26
208 WARNING: 185.203.112.26: eggs parameter of type __foo is deprecated: eggs parameter is deprecated, please use multiple egg parameter.
209 $ echo '__foo foo --foo foo --eggs eggs --spam spam' | ./bin/cdist config -i - 185.203.112.26
210 WARNING: 185.203.112.26: spam parameter of type __foo is deprecated.
211 WARNING: 185.203.112.26: eggs parameter of type __foo is deprecated: eggs parameter is deprecated, please use multiple egg parameter.
212
213
214 Input from stdin
215 ----------------
216 Every type can access what has been written on stdin when it has been called.
217 The result is saved into the **stdin** file in the object directory.
218
219 Example use of a type: (e.g. in cdist/conf/type/__archlinux_hostname)
220
221 .. code-block:: sh
222
223 __file /etc/rc.conf --source - << eof
224 ...
225 HOSTNAME="$__target_host"
226 ...
227 eof
228
229 If you have not seen this syntax (<< eof) before, it may help you to read
230 about "here documents".
231
232 In the __file type, stdin is used as source for the file, if - is used for source:
233
234 .. code-block:: sh
235
236 if [ -f "$__object/parameter/source" ]; then
237 source="$(cat "$__object/parameter/source")"
238 if [ "$source" = "-" ]; then
239 source="$__object/stdin"
240 fi
241 ....
242
243
244 Stdin inside a loop
245 ~~~~~~~~~~~~~~~~~~~
246 Since cdist saves type's stdin content in the object as **$__object/stdin**,
247 so it can be accessed in manifest and gencode-* scripts, this can lead to
248 unexpected behavior. For example, suppose you have some type with the following
249 in its manifest:
250
251 .. code-block:: sh
252
253 if [ -f "$__object/parameter/foo" ]
254 then
255 while read -r l
256 do
257 __file "$l"
258 echo "$l" >&2
259 done < "$__object/parameter/foo"
260 fi
261
262 and init manifest:
263
264 .. code-block:: sh
265
266 __foo foo --foo a --foo b --foo c
267
268 You expect that manifest stderr content is:
269
270 .. code-block:: sh
271
272 a
273 b
274 c
275
276 and that files *a*, *b* and *c* are created. But all you get in manifest stderr
277 is:
278
279 .. code-block:: sh
280
281 a
282
283 and only *a* file is created.
284
285 When redirecting parameter *foo* file content to while's stdin that means that all
286 commands in while body have this same stdin. So when *__file* type gets executed,
287 cdist saves its stdin which means it gets the remaining content of parameter *foo*
288 file, i.e.:
289
290 .. code-block:: sh
291
292 b
293 c
294
295 The solution is to make sure that your types inside such loops get their stdin
296 from somewhere else, e.g. for the above problem *__file* type can get empty
297 stdin from */dev/null*:
298
299 .. code-block:: sh
300
301 if [ -f "$__object/parameter/foo" ]
302 then
303 while read -r l
304 do
305 __file "$l" < /dev/null
306 echo "$l" >&2
307 done < "$__object/parameter/foo"
308 fi
309
310
311 Writing the manifest
312 --------------------
313 In the manifest of a type you can use other types, so your type extends
314 their functionality. A good example is the __package type, which in
315 a shortened version looks like this:
316
317 .. code-block:: sh
318
319 os="$(cat "$__global/explorer/os")"
320 case "$os" in
321 archlinux) type="pacman" ;;
322 debian|ubuntu) type="apt" ;;
323 gentoo) type="emerge" ;;
324 *)
325 echo "Don't know how to manage packages on: $os" >&2
326 exit 1
327 ;;
328 esac
329
330 __package_$type "$@"
331
332 As you can see, the type can reference different environment variables,
333 which are documented in `cdist reference <cdist-reference.html>`_.
334
335 Always ensure the manifest is executable, otherwise cdist will not be able
336 to execute it. For more information about manifests see `cdist manifest <cdist-manifest.html>`_.
337
338
339 Singleton - one instance only
340 -----------------------------
341 If you want to ensure that a type can only be used once per target, you can
342 mark it as a singleton: Just create the (empty) file "singleton" in your type
343 directory:
344
345 .. code-block:: sh
346
347 touch cdist/conf/type/__NAME/singleton
348
349 This will also change the way your type must be called:
350
351 .. code-block:: sh
352
353 __YOURTYPE --parameter value
354
355 As you can see, the object ID is omitted, because it does not make any sense,
356 if your type can be used only once.
357
358
359 Install - type with install command
360 -----------------------------------
361 If you want a type to be used with install command, you must mark it as
362 install: create the (empty) file "install" in your type directory:
363
364 .. code-block:: sh
365
366 touch cdist/conf/type/__install_NAME/install
367
368 With other commands, i.e. config, it will be skipped if used.
369
370
371 Nonparallel - only one instance can be run at a time
372 ----------------------------------------------------
373 If objects of a type must not or cannot be run in parallel when using -j
374 option, you must mark it as nonparallel: create the (empty) file "nonparallel"
375 in your type directory:
376
377 .. code-block:: sh
378
379 touch cdist/conf/type/__NAME/nonparallel
380
381 For example, package types are nonparallel types.
382
383
384 The type explorers
385 ------------------
386 If a type needs to explore specific details, it can provide type specific
387 explorers, which will be executed on the target for every created object.
388
389 The explorers are stored under the "explorer" directory below the type.
390 It could for instance contain code to check the md5sum of a file on the
391 client, like this (shortened version from the type __file):
392
393 .. code-block:: sh
394
395 if [ -f "$__object/parameter/destination" ]; then
396 destination="$(cat "$__object/parameter/destination")"
397 else
398 destination="/$__object_id"
399 fi
400
401 if [ -e "$destination" ]; then
402 md5sum < "$destination"
403 fi
404
405
406 Writing the gencode script
407 --------------------------
408 There are two gencode scripts: **gencode-local** and **gencode-remote**.
409 The output of gencode-local is executed locally, whereas
410 the output of gencode-remote is executed on the target.
411 The gencode scripts can make use of the parameters, the global explorers
412 and the type specific explorers.
413
414 If the gencode scripts encounters an error, it should print diagnostic
415 messages to stderr and exit non-zero. If you need to debug the gencode
416 script, you can write to stderr:
417
418 .. code-block:: sh
419
420 # Debug output to stderr
421 echo "My fancy debug line" >&2
422
423 # Output to be saved by cdist for execution on the target
424 echo "touch /etc/cdist-configured"
425
426 Notice: if you use __remote_copy or __remote_exec directly in your scripts
427 then for IPv6 address with __remote_copy execution you should enclose IPv6
428 address in square brackets. The same applies to __remote_exec if it behaves
429 the same as ssh for some options where colon is a delimiter, as for -L ssh
430 option (see :strong:`ssh`\ (1) and :strong:`scp`\ (1)).
431
432
433 Variable access from the generated scripts
434 ------------------------------------------
435 In the generated scripts, you have access to the following cdist variables
436
437 - __object
438 - __object_id
439
440 but only for read operations, means there is no back copy of this
441 files after the script execution.
442
443 So when you generate a script with the following content, it will work:
444
445 .. code-block:: sh
446
447 if [ -f "$__object/parameter/name" ]; then
448 name="$(cat "$__object/parameter/name")"
449 else
450 name="$__object_id"
451 fi
452
453
454 Environment variable usage idiom
455 --------------------------------
456 In type scripts you can support environment variables with default values if
457 environment variable is unset or null by using **${parameter:-[word]}**
458 parameter expansion.
459
460 Example using mktemp in a portable way that supports TMPDIR environment variable.
461
462 .. code-block:: sh
463
464 tempfile=$(mktemp "${TMPDIR:-/tmp}/cdist.XXXXXXXXXX")
465
466
467 Log level in types
468 ------------------
469 cdist log level can be accessed from __cdist_log_level variable.One of:
470
471 +----------------+-----------------+
472 | Log level | Log level value |
473 +================+=================+
474 | OFF | 60 |
475 +----------------+-----------------+
476 | ERROR | 40 |
477 +----------------+-----------------+
478 | WARNING | 30 |
479 +----------------+-----------------+
480 | INFO | 20 |
481 +----------------+-----------------+
482 | VERBOSE | 15 |
483 +----------------+-----------------+
484 | DEBUG | 10 |
485 +----------------+-----------------+
486 | TRACE | 5 |
487 +----------------+-----------------+
488
489
490 It is available for initial manifest, explorer, type manifest,
491 type explorer, type gencode.
492
493
494 Detecting dry run
495 -----------------
496
497 If ``$__cdist_dry_run`` environment variable is set, then it's dry run.
498
499 It is available for initial manifest, explorer, type manifest,
500 type explorer, type gencode.
501
502
503 Hints for typewriters
504 ----------------------
505 It must be assumed that the target is pretty dumb and thus does not have high
506 level tools like ruby installed. If a type requires specific tools to be present
507 on the target, there must be another type that provides this tool and the first
508 type should create an object of the specific type.
509
510 If your type wants to save temporary data, that may be used by other types
511 later on (for instance \__file), you can save them in the subdirectory
512 "files" below $__object (but you must create it yourself).
513 cdist will not touch this directory.
514
515 If your type contains static files, it's also recommended to place them in
516 a folder named "files" within the type (again, because cdist guarantees to
517 never ever touch this folder).
518
519
520 How to include a type into upstream cdist
521 -----------------------------------------
522 If you think your type may be useful for others, ensure it works with the
523 current master branch of cdist and have a look at `cdist hacking <cdist-hacker.html>`_ on
524 how to submit it.