Debian Codegraph @

manuals/en/main/kubernetes.tex

raw
1 \chapter{Plugins}
2 \def\kubernetes{Kubernetes~}
3 \section{Kubernetes Plugin}
4 \label{kubernetesplugin}
5 \subsection{Features Summary}
6 \begin{bitemize}
7 \item \kubernetes cluster resources configuration backup
8 \item Ability to restore single \kubernetes configuration resource
9 \item Ability to restore \kubernetes resource configuration to local directory
10 \item \kubernetes Persistent Volumes data backup and restore
11 \item Ability to restore \kubernetes Persistent Volumes data to local directory
12 \item Ability to use new \kubernetes CSI driver snapshot features to perform Persistent Volume data backup using Cloned or Snapshot techniques
13 \item Ability to execute user defined commands on required Pod containers to prepare and clean data backup
14 \item Configure \kubernetes workload backup requirements directly with Pod annotations
15 \end{bitemize}
16
17
18 \subsection{Glossary}
19 \begin{description}
20 \item[Docker] is a computer program that performs operating-system-level
21 virtualization, also known as "containerization".
22 \item[Kubernetes] is an open-source system for automating deployment, scaling, and
23 management of containerized applications. It groups containers that make up an application
24 into logical units for easy management and discovery.
25 \item[Container] is a runnable instance of an image and refers to an operating system
26 feature in which the kernel allows the existence of multiple isolated user-space instances.
27 \item[Kubernetes Resource] is a configuration entity which describe how a particular component
28 should be executed and maintained. It includes core components like Pod, Service,
29 Persistent Volume Claim or extended components like Replica Set, Deployment or
30 Stateful Set.
31 \item[K8S] is short for Kubernetes and is used interchangeably.
32 \item[Persistent Volume] (PV) is a piece of storage in the cluster that has been
33 dynamically provisioned or provisioned by an administrator who's life cycle is independent of
34 any individual Pod that uses the PV.
35 \item[Persistent Volume Claim] (PVC) is a request for storage by a user which could be
36 consumed by Pod. This resource associates a PV with a Pod.
37 \item[Bacula Backup Proxy Pod] is a Kubernetes service dedicated to PVC Data archive backup
38 and restore operations which acts as a proxy application between Persistent Volume data and
39 a Bacula File Daemon with the Kubernetes Plugin. This application requires a dedicated
40 bacula-backup docker image to be deployed.
41 \item[CSI] is the Container Storage Interface. CSI is a standard for exposing arbitrary block and
42 file storage systems to containerized workloads on Container Orchestration Systems (COs)
43 like Kubernetes. Using CSI third-party storage providers can write and deploy plugins exposing
44 new storage systems in Kubernetes without ever having to touch the core Kubernetes code.
45 \end{description}
46
47 Red Hat, Red Hat Enterprise Linux are trademarks
48 of Red Hat, Inc., registered in the United States and other countries.
49
50 \subsection{\kubernetes Backup Overview}
51
52 Containers are very light system level virtualization with less overhead
53 because programs in virtual partitions use the operating system's normal
54 system call interface and do not need to be subjected to emulation or
55 run in an intermediate virtual machine. \kubernetes manages a set of
56 containers to create a flexible execution environment for applications and
57 services.
58
59 \smallskip
60 The Bacula \kubernetes plugin will save all the important
61 Kubernetes resources which make up the application or service. This includes the
62 following namespaced objects:
63
64 \label{kubernetesresources}
65 \begin{bitemize}
66 \item{Config Map}
67 \item{Daemon Set}
68 \item{Deployment}
69 \item{Endpoint}
70 \item{Limit Range}
71 \item{Pod}
72 \item{Persistent Volume Claim}
73 \item{Pod Template}
74 \item{Replica Set}
75 \item{Replication Controller}
76 \item{Resource Quota}
77 \item{Secret}
78 \item{Service}
79 \item{Service Account}
80 \item{Stateful Set}
81 \item{PVC Data Archive}\footnote{The PVC Data is not exact Kubernetes Object
82 but represents archive of real data existed on selected PVC.}
83 \end{bitemize}
84
85 and non namespaced objects:
86
87 \begin{bitemize}
88 \item{Namespace}
89 \item{Persistent Volume}
90 \item{Storage Class}
91 \end{bitemize}
92
93 All namespaced objects which belong to the particular namespace are grouped
94 together for easy backup data browsing and recovery.
95
96 \smallskip
97 Users and service accounts can be authorized to access the server API. This process goes
98 through authentication, authorization and admission control. To be able to successfully backup
99 the Kubernetes resources, it is required to have a user or service account with the correct permissions
100 and rights to be successfully authenticated and authorized to access the API server and resources to
101 be backed up.
102
103 \smallskip
104 For resource configuration backups, the user or service account must be able to read
105 and list the resources. In the case of PVC Data backup, it is also required that the user
106 or service account can create and delete pods because the plugin will need to create and delete
107 the Bacula Backup Proxy Pod during the backup.
108
109 \smallskip
110 Please see the Kubernetes documentation for more details.
111
112 \subsection{\kubernetes Persistent Volume Claim Backup}
113
114 All Pods in Kubernetes are ephemeral and may be destroyed manually or
115 by operations from controllers. Pods do not store data locally because
116 stored data would be destroyed with a pod's life cycle management, so data
117 is saved on Persistent Volumes using Persistent Volume Claim objects to
118 control Volume Space availability.
119
120 \smallskip
121 This brings a new challenge to data backup. Fortunately most of the challenges
122 found here are similar to standard bare metal or virtualized environments.
123 As with bare metal and virtual machine environments, data stored in databases
124 should be protected with dedicated Bacula plugins that take advantage
125 of the database backup routines.
126
127 \bigskip
128 \textit{Please refer to the appropriate Bacula plugin whitepapers for more details
129 on database backups.}
130 \bigskip
131
132 On the other hand, most non-database applications store data as simple
133 flat files we can backup as-is without forcing complicated transactions or data
134 consistency procedures. This use case is handled directly with the \kubernetes plugin
135 using a dedicated Bacula Backup Proxy Pod executed in the cluster.
136
137 \smallskip
138 If the container application is more complex, it is possible to execute
139 commands inside the container to quiesce the application.
140 \begin{bitemize}
141 \item before the snapshot
142 \item after the snapshot
143 \item after the backup of the container
144 \end{bitemize}
145
146 A problem with command execution can abort the backup of the container with the
147 \texttt{run.*.failonerror} annotation. You can find detailed description of this feature
148 at \vref{csisnapshot}.
149
150 \smallskip
151 A Bacula Backup Proxy Pod is a service executed automatically by the \kubernetes plugin
152 which manages secure access to Persistent Volume data for the plugin. It is
153 executed on the \kubernetes cluster infrastructure and requires a network connection to
154 the \kubernetes plugin for data exchange on backup and restore operations. No external
155 cluster service like \texttt{NodePort}, \texttt{LoadBalancer}, \texttt{Ingress} or
156 \texttt{Host Based Networking} configuration is required to use this feature.
157
158 It is also not required to permanently deploy and run this service on the cluster itself
159 as it is executed on demand. The Bacula Backup Proxy Pod does not consume any
160 valuable compute resources outside of the backup window. You can even operate your
161 \kubernetes backup solution (Bacula Edition service with \kubernetes plugin)
162 directly from your on-premise backup infrastructure to backup a public Kubernetes
163 cluster (it requires a simple port forwarding firewall rule) or use public backup
164 infrastructure to backup on-premise Kubernetes cluster(s).
165 Support for these varied architecture modes is built into the \kubernetes plugin.
166 It is designed to be a \textit{One-Click} solution for \kubernetes backups.
167
168 \smallskip
169
170 It is possible to backup and restore any PVC data including PVCs not attached
171 to any running Kubernetes Pod.
172
173 \subsection{CSI Snapshot Support}
174 \label{csisnapshot}
175
176 Kubernetes CSI Snapshot functionality support together with a bunch of other features
177 was added. Starting from this version you can use CSI Snapshots to acquire a consistent
178 data view of selected Volume. Additionally, you can configure remote command execution on
179 a selected Container of the Pod. You can configure command execution just before or after
180 a Pod backup and just after snapshot creation.
181
182 Our plugin uses the volume clone or the snapshot api when doing a volume
183 snapshot. CSI drivers may or may not have implemented the volume cloning or
184 snapshot functionalities.
185
186 \subsection{Kubernetes Pod Annotations}
187
188 The \textbf{CSI Snapshot Support} feature described in \vref{csisnapshot} comes with
189 a configuration of Volume data backup using Kubernetes Pod annotations. This feature allows
190 you to define what volumes (PVC Data) to backup, where and what commands to execute, and
191 how to react to some failures to achieve the best results from data snapshot functionality.
192 You can select which volumes mounted at the Pod you want to backup, the
193 preferred backup mode for the Pod, and what commands you want to execute.
194
195 \smallskip
196 The supported annotations are:
197
198 \begin{description}
199 \item[\texttt{bacula/backup.mode:[snapshot|standard]}] defines how to handle
200 PVC Data backups. If not defined, the default is \texttt{snapshot}.
201
202 \item[\texttt{bacula/backup.volumes:<pvcname[,pvcname2...]>}] defines what
203 volumes will be backed up from this Pod. Multiple PVC names may be selected
204 as a comma separated list. This annotation is required if you want to backup
205 volumes on this Pod. Any other annotations are optional.
206
207 \item[\texttt{bacula/run.before.job.container.command:<container>/<command>}]
208 defines what command to execute on which container of the Pod before any snapshot
209 and data backup of this Pod occurs. An asterisk (\texttt{*}) as \texttt{<container>} name means
210 to execute \texttt{<command>} on all containers.
211
212 \item[\texttt{bacula/run.before.job.failjobonerror:[yes|no]}] defines whether or not to fail the job
213 when the exit code of the executed command is not zero. The default is yes.
214
215 \item[\texttt{bacula/run.after.job.container.command:<container>/<command>}]
216 defines what command to execute on which container of the Pod after all snapshot
217 and data backup of this Pod. An asterisk (\texttt{*}) as \texttt{<container>} name means
218 to execute \texttt{<command>} on all containers.
219
220 \item[\texttt{bacula/run.after.job.failjobonerror:[yes|no]}] defines whether or not to fail the job
221 when exit code of the executed command is not zero. The default is no.
222
223 \item[\texttt{bacula/run.after.snapshot.container.command:<container>/<command>}]
224 defines what command to execute on which container of the Pod after all snapshot
225 creations and just before any data backup.
226 An asterisk (\texttt{*}) as \texttt{<container>} name means to execute \texttt{<command>}
227 on all containers.
228
229 \item[\texttt{bacula/run.after.snapshot.failjobonerror:[yes|no]}] defines to fail the job
230 when exit code of the executed command is not zero. The default is no.
231 \end{description}
232
233 Pod annotations is an extension to the current PVC Data backup feature available
234 with the \texttt{pvcdata=\ldots} plugin parameter as described in \vref{backupparameters}.
235 This is an independent function which may be used together with the functionality
236 described above, especially since both use the same data archive stream handling with
237 Bacula Backup Pod.
238
239 \smallskip
240 All you need to use a new feature is to configure selected Pod annotations and make sure
241 that the backup for a required Kubernetes namespace is properly configured. There is no
242 need for any plugin configuration modifications. A Pod's volumes will be backed up automatically.
243
244 \subsection{Examples}
245
246 Below you can find some examples how to configure Bacula annotations in Kubernetes Pods.
247
248 \smallskip
249 In the example below you will use a simple Linux command \texttt{sync} to synchronize
250 cached writes to persistent storage before volume snapshot.
251
252 \begin{verbatim}
253 apiVersion: v1
254 kind: Pod
255 metadata:
256 name: app1
257 namespace: default
258 annotations:
259 bacula/backup.mode: snapshot
260 bacula/run.before.job.container.command: "*/sync -f /data1; sync -f /data2"
261 bacula/run.before.job.failjobonerror: "no"
262 bacula/backup.volumes: "pvc1, pvc2"
263 spec:
264 containers:
265 - image: ubuntu:latest
266 name: test-container
267 volumeMounts:
268 - name: pvc1
269 mountPath: /data1
270 - name: pvc2
271 mountPath: /data2
272 volumes:
273 - name: pvc1
274 persistentVolumeClaim:
275 claimName: pvc1
276 - name: pvc2
277 persistentVolumeClaim:
278 claimName: pvc2
279 \end{verbatim}
280
281 In the example below you will use PostgreSQL's database data files quiesce feature to perform consistent
282 backup with snapshot\footnote{The final PostgreSQL backup solution requires more configuration and preparation
283 which was skipped in this example to make it clear}.
284
285 The first command (\textit{run.before.job.container.command}) freezes writes to database files
286 and the second (\textit{run.after.snapshot.container.command}) resumes standard database operation
287 as soon as PVC snapshot becomes ready.
288
289 \begin{verbatim}
290 apiVersion: v1
291 kind: Pod
292 metadata:
293 name: postgresql13
294 namespace: default
295 annotations:
296 bacula/backup.mode: standard
297 bacula/run.before.job.container.command: "*//bin/startpgsqlbackup.sh"
298 bacula/run.after.snapshot.container.command: "*//bin/stoppgsqlbackup.sh"
299 bacula/run.after.snapshot.failjobonerror: "yes"
300 bacula/backup.volumes: "pgsql"
301 spec:
302 containers:
303 - image: postgresql:13
304 name: postgresql-server
305 volumeMounts:
306 - name: pgsql
307 mountPath: /var/lib/pgsql
308 volumes:
309 - name: pgsql
310 persistentVolumeClaim:
311 claimName: pgsql-volume
312 \end{verbatim}
313
314 \subsection{Run Container Command annotation}
315
316 All flavors of the Run Container Command parameters are remotely executed using the Kubernetes Pod
317 remote execution API. Every command is prepared to execute with a standard Linux shell \texttt{/bin/sh}.
318 This requires that a container image has to have the specified shell available. Using command shell
319 execution gives flexibility to command execution or even allows for preparation of small scripts without
320 additional container image customization.
321
322 \subsection{Build}
323
324 To enable the build of the Kubernetes plugin, add the following options to
325 your \texttt{./configure} command line:
326
327 \begin{bVerbatim}
328 ./configure --enable-kubernetes-plugin [... other options]
329 \end{bVerbatim}
330
331 The Kubernetes plugin is written in Python, to ensure the stability of the Python environment,
332 the default installation process will use Cython to convert the Python program to an executable.
333
334 \subsection{Installation}
335
336 The Bacula File Daemon and the Kubernetes plugin can be installed outside of the
337 Kubernetes cluster on a server which has access to the Kubernetes API, or inside a protected
338 cluster in a Container / Pod. The Kubernetes plugin can be installed on different
339 operating systems and distributions, so the Bacula File Daemon
340 for the correct operating system and platform has to be used.
341
342 There is no need, or in some solutions (when \texttt{K8S} is a cloud service
343 like \texttt{GKE} or \texttt{EKS}), is it even a possible to install a Bacula File Daemon
344 and the Kubernetes Plugin on a Kubernetes Master Server (\texttt{etcd}, \texttt{control pane}).
345
346 The \texttt{--enable-kubernetes-plugin} command line option of the \texttt{./configure} script
347 will allow to install the kubernetes plugin with \texttt{make install}
348
349 The following packages are required to build and use the kubernetes plugin:
350 \begin{itemize}
351 \item Cython (to bundel python dependencies)
352 \end{itemize}
353
354 Note that it should be possible to use directly python files rather than Cython.
355
356 \subsection{File Daemon Configuration}
357
358 The \texttt{Plugin Directory} directive of \texttt{File Daemon}
359 resource in \texttt{/opt/bacula/etc/bacula-fd.conf} must
360 point to where the \texttt{kubernetes-fd.so} plugin file is installed. The
361 standard Bacula plugin directory is \texttt{/opt/bacula/plugins}
362
363 \begin{verbatim}
364 FileDaemon {
365 Name = bacula-fd
366 Plugin Directory = /opt/bacula/plugins
367 ...
368 }
369 \end{verbatim}
370
371 \subsection{Bacula Backup Proxy Pod Image}
372 \label{baculatarimage}
373 For Kubernetes PVC Data backup functionality, the Bacula Kubernetes plugin
374 requires a dedicated Bacula Backup Proxy Pod which is automatically deployed
375 using an image that is available in the bacula/scripts/kubernetes-bacula-backup
376 directory.
377
378 This image should be installed manually on your local Docker images
379 registry service which is available on your Kubernetes cluster as a source
380 for application images.
381
382 Installation of the image can be performed with the following example commands:
383
384 \begin{verbatim}
385 # cd bacula-13.0.0/scripts/kubernetes-bacula-backup
386 # make
387 # docker load -i bacula-backup-<timestamp>.tar
388 # docker image tag bacula-backup:<timestamp> <registry>/bacula-backup:<timestamp>
389 # docker push <registry>/bacula-backup:<timestamp>
390 \end{verbatim}
391
392 Where \texttt{<timestamp>} is the image version generated with the
393 above package and \texttt{<registry>} is the location of your Docker images registry
394 service. The exact procedure depends on your Kubernetes cluster deployment, so please
395 verify the above steps before attempting to run the docker commands.
396
397 You can use any registry service available for
398 your cluster, public or private, i.e. \texttt{gcr.io/}.
399
400 \smallskip
401 Depending on your cluster configuration it may be necessary to set the
402 \texttt{baculaimage=<name>} plugin parameter (described at \vref{baculaimageparam})
403 to define which repository and container image to use. The default for this parameter
404 is \texttt{bacula-backup:<timestamp>} which may not be correct for your deployment.
405
406 Another example where you will need to modify the Bacula Backup Proxy Pod
407 Image is in the case where your registry requires authentication. Please see the
408 section \vref{advancedpoddeployment} for more details.
409
410 \section{Backup and Restore Operations}
411
412 \subsection{Backup}
413
414 The plugin can backup a number of Kubernetes Resources including: Deployments,
415 Pods, Services or Persistent Volume Claims, check chapter~\vref{kubernetesresources}
416 for a complete list.
417
418 \smallskip
419 The backup will create a single (\texttt{.yaml}) file for any Kubernetes Resource which
420 is saved. For PVC Data backup functionality the \kubernetes plugin generates a data archive
421 as a single \texttt{<pvcname>.tar} archive file.
422 The resources are organized inside the Bacula catalog to facilitate browsing
423 and restore operations. In the Bacula catalog, Kubernetes resources are represented as follows:
424
425 \begin{bitemize}
426 \item \texttt{/@kubernetes/namespaces/<namespace>.yaml} - Namespace definition
427 \item \texttt{/@kubernetes/namespaces/<namespace>/<resource>/<name>.yaml}
428 - Resource definitions in the namespace
429 \item \texttt{/@kubernetes/namespaces/<namespace>/persistentvolumeclaim/<pvcname>.tar}
430 - PVC Data backup in the selected namespace
431 \item \texttt{/@kubernetes/persistentvolumes/<pvname>.yaml} - Persistent Volume definition
432 \item \texttt{/@kubernetes/storageclass/<scname>.yaml} - Storage Class definition
433 \end{bitemize}
434
435 All supported Kubernetes Resources will be saved if no filter options are set.
436 You may limit which resources are saved using filtering options described in
437 chapter~\vref{backupparameters}.
438 By default, if no filter options are set, all supported Kubernetes Resources will be saved.
439 To see the Kubernetes Resources that may be filtered, a listing mode is available. This mode is described in chapter~\vref{listing}.
440
441 \subsection{Restore}
442
443 The \kubernetes plugin provides two targets for restore operations:
444 \begin{bitemize}
445 \item Restore to a Kubernetes cluster
446 \item Restore to a local directory
447 \end{bitemize}
448
449
450 \subsubsection{Restore to a \kubernetes Cluster}
451
452 To use this restore method, the \texttt{where=/}
453 parameter of a Bacula \texttt{restore} command is used. You can select
454 any supported Kubernetes Resource to restore, or batch restore the whole
455 namespace or even multiple namespaces. If you select a single resource to restore
456 it will be restored as is without any dependent objects. In most cases, for
457 (\texttt{Config Maps}, \texttt{Secrets}, \texttt{Services}, etc.) this is fine and
458 restore will always succeed.
459 On the other hand, compound objects (\texttt{Pods}, \texttt{Deployments}, etc.)
460 won't be ready unless all dependencies are resolved during the restore.
461 In this case you should make sure that you select all required resources to restore.
462
463 \smallskip
464 In Kubernetes, a successful resource restore doesn't necessarily result in the service
465 successfully coming online. In some cases further monitoring and investigation
466 will be required. For example:
467
468 \begin{bitemize}
469 \item \textit{Container image is unavailable}.
470 \item \textit{Volume Claim cannot provision new volume}.
471 \item \textit{Untrusted Image Repository}.
472 \item \textit{Infrastructure limits exceeded, i.e. a number of Pods or Memory allocations}.
473 \item \textit{etc\ldots}
474 \end{bitemize}
475
476 All example cases above must be resolved by the \kubernetes administrator.
477 When all issues are resolved, the resource should automatically come online. If not, it may
478 be necessary to repeat a restore to redeploy the Resource configuration.
479
480 \smallskip
481 The \kubernetes plugin does not wait for a Resource to come online during restore.
482 It checks the Resource creation or replace operation status and reports any errors
483 in the job log. The only exception to this is PVC Data restore, when the \kubernetes
484 plugin will wait for a successful archive data restore. This operation is always
485 executed at the end of the namespace recovery (when pvcdata is restored with other
486 k8s objects) and should wait for proper PVC mount.
487
488 \subsubsection{Restore to a Local Directory}
489
490 To use this mode, the \texttt{where=/some/path} Bacula \texttt{restore}
491 parameter is set to a full path on a server where the Bacula File Daemon and \kubernetes plugin
492 are installed. If the path does not exist, it will be created by the \kubernetes
493 plugin. With this restore mode you can restore any saved Kubernetes Resource
494 including PVC Data archive file to a location on disk.
495
496 \subsection{Plugin Configuration}
497
498 The plugin is configured using \texttt{Plugin Parameters} defined
499 in a \texttt{FileSets} -> \texttt{Include} section of the Bacula Enterprise
500 Edition Director configuration.
501
502 \subsection{Generic Plugin Parameters}
503 \label{genericparameters}
504
505 The following \kubernetes plugin parameters affect any type of Job
506 (Backup, Estimate, or Restore).
507
508 \begin{description}
509 \item[\texttt{abort\_on\_error[=<0 or 1>]}] specifies whether or
510 not the plugin should abort execution (and the Bacula Job) if a
511 fatal error occurs during a Backup, Estimate, or Restore
512 operation.
513
514 This parameter is optional.
515
516 \item[\texttt{config=</path/to/file>}] points to the location of
517 the config file which defines how to connect to the Kubernetes cluster.
518 You can read more about this \textit{kubeconfig} file at:
519 \url{https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/}
520 If this directive is omitted and no other access method is configured then
521 a default config file location will be used - \texttt{\$HOME/.kube/config}.
522
523 This parameter is optional.
524
525 \item[\texttt{incluster}] if this directive is defined then a standard in-cluster
526 access to the Kubernetes API will be used. This option should be used only
527 when the Bacula File Daemon and \kubernetes plugin are deployed as a Kubernetes
528 cluster service in a Pod. In this case, Kubernetes itself will provide the
529 required access credentials. This option won't work when executed outside of a Kubernetes
530 cluster and services.
531
532 This parameter is optional.
533
534 \item[\texttt{host=<url-k8s-api>}] defines a Kubernetes API url. This option
535 is used only with \texttt{token} parameter described below. If this option is specified,
536 then both parameters are required.
537
538 This parameter is optional.
539
540 \item[\texttt{token=<bearer-token>}] defines a \texttt{Bearertoken} used
541 for authorization to the Kubernetes API available at \texttt{host=<url-k8s-api>}.
542 This option is used only with \texttt{host} parameter described above.
543 You can read more about this type of authentication at:
544 \url{https://swagger.io/docs/specification/authentication/bearer-authentication/}
545
546 This parameter is optional.
547
548 \item[\texttt{verify\_ssl[=<0 or 1>]}] specifies whether or not the plugin
549 should verify a Kubernetes API certificate when the connection uses SSL/TLS.
550 If set to \texttt{verify\_ssl=0} then verification will be disabled.
551 This is useful when connecting to a Kubernetes API server with a self-signed
552 certificate. The default behavior if this parameter if not set
553 is to perform proper certificate validation.
554
555 This parameter is optional.
556
557 \item[\texttt{ssl\_ca\_cert=</path/to/file>}] specifies a file with the CA certificate
558 used to customize the Kubernetes API server identity to verify the peer.
559
560 This parameter is optional.
561
562 \item[\texttt{timeout=<seconds>}] specifies the number of seconds for various network
563 operations. Examples include: Waiting for Bacula Backup Proxy Pod connection
564 or Kubernetes API operations, waiting for Pod execution or removal. The default is
565 600 seconds. The minimum timeout you may set is 1 second. When an invalid value is set
566 the default will be used.
567
568 This parameter is optional.
569
570 \item[\texttt{debug[=1]}] specifies that the plugin backend will generate an execution
571 debug file at location \texttt{/bacula/working/backendplugin/}. This file can help with troubleshooting
572 the job execution if something goes wrong. If not defined then no debug file will be generated.
573
574 This parameter is optional.
575 \end{description}
576
577 \subsection{Estimate and Backup Plugin Parameters}
578 \label{backupparameters}
579 These plugin parameters are relevant only for Backup and Estimate jobs:
580
581 \begin{description}
582 \item[\texttt{namespace=<name>}] specifies a Kubernetes namespace
583 name which you want to backup. Multiple \texttt{namespace=<name>}
584 parameters are allowed if you want to backup multiple namespaces.
585 If a namespace with \texttt{name} can not be found its backup will be silently
586 ignored. If this parameter is not set, all namespaces will be saved.
587
588 This parameter is optional.
589
590 \item[\texttt{persistentvolume=<name>}] specifies a Kubernetes persistent
591 volume configuration you want to backup. Multiple \texttt{persistentvolume=<name>}
592 parameters are allowed if you want to backup multiple volumes.
593 You can use standard shell wildcard pattern matching to select multiple volumes
594 using a single \texttt{persistentvolume} parameter.
595 If a persistent volume with \texttt{name} can not be found its backup will be silently
596 ignored. If this parameter is not set, all persistent volume configurations will be saved
597 unless \texttt{pvconfig=0} is set as described below.
598
599 This parameter is optional.
600
601 \item[\texttt{pvconfig=[0|1]}] this option is used to disable persistent volume
602 configuration backups when \texttt{pvconfig=0} is set. The default is to
603 backup persistent volume configurations.
604
605 This parameter is optional.
606
607 \item[\texttt{storageclass=<name>}] specifies a Kubernetes Storage Class
608 configuration to be backed up. Multiple \texttt{storageclass=<name>}
609 parameters are allowed if you want to backup multiple resources.
610 You can use standard shell wildcard pattern matching to select multiple volumes
611 using a single \texttt{storageclass} parameter.
612 If a storage class with \texttt{name} can not be found, its backup will be silently
613 ignored. If this parameter is not set, all storage class configuration will be saved
614 unless \texttt{scconfig=0} is set as described below.
615
616 This parameter is optional.
617
618 \item[\texttt{scconfig=[0|1]}] this option is used to disable Storage Class
619 configuration backups when \texttt{scconfig=0} is set. The default is to
620 backup Storage Class resource configurations.
621
622 This parameter is optional.
623
624 \item[\texttt{pvcdata[=<pvcname>]}] specifies a Kubernetes Persistent
625 Volume Claim name you want to make a PVC Data archive for. As all PVCs are
626 namespaced objects, to use this option you should select a required
627 namespace with \texttt{namespace=<name>} parameter. If you define a simple
628 \texttt{pvcdata} parameter without the equals sign (\textit{"="}) and subsequent
629 value, all detected persistent volume claims will be selected for the PVC Data
630 archive backup.
631
632 This parameter is optional.
633 \end{description}
634
635 If none of the parameters above are specified, then all available
636 Namespaces and Persistent Volume Configurations will be backed up.
637 However, the plugin does not force a PVC Data archive backup in this case.
638
639 \subsection{Backup and Restore Plugin Parameters}
640 \label{rbackupparameters}
641 These plugin parameters are relevant only for Backup and Restore
642 jobs when the PVC Data archive functionality is used:
643
644 \begin{description}
645 \item[\texttt{fdaddress=<IP or name>}] is the IP address or host name where the \kubernetes
646 plugin should listen for incoming connections from a Bacula Backup Proxy Pod. This
647 parameter, combined with \texttt{fdport=<number>} below will define a socket to listen on.
648 The default is to listen on all available interfaces (\texttt{0.0.0.0}) which should be
649 fine in most cases.
650
651 This parameter is optional if \texttt{pluginhost=<IP or name>} is defined.
652
653 \item[\texttt{fdport=<number>}] is a port number on which \kubernetes plugin should
654 listen for incoming connections from a Bacula Backup Proxy Pod. This parameter, combined with
655 \texttt{fdaddress=<IP or name>} above will define a socket to listen on.
656 The default is to listen on port \texttt{9104}.
657
658 This parameter is optional.
659
660 \item[\texttt{pluginhost=<IP or name>}] is the entry point address where a Bacula Backup
661 Proxy Pod should connect during backup or restore operations. The name should be
662 resolvable on the \kubernetes cluster, otherwise an IP address must be used here.
663 This with \texttt{pluginport=<number>} parameter below will define the exact service entry point.
664 The default is to use the value from \texttt{fdaddress=<IP or name>} parameter above.
665
666 This parameter is required when \texttt{fdaddress=<IP or name>} is not defined. Otherwise
667 it is optional.
668
669 \item[\texttt{pluginport=<number>}] is the port number for service entry point address
670 where the Bacula Backup Proxy Pod should connect during backup or restore operations.
671 This, combined with the \texttt{pluginhost=<IP or name>} parameter above define the
672 exact service entry point.
673 The default is to use the value from the \texttt{fdaddress=<IP or name>} parameter above.
674 When neither is defined the default \texttt{9104} port number will be used.
675
676 This parameter is optional.
677
678 \item[\texttt{fdcertfile=<path>}] is the SSL certificate file location for
679 the \kubernetes plugin endpoint service data connection for a Bacula Backup Proxy Pod.
680 The certificate and key (see \texttt{fdkeyfile=<path>} below) files are required for
681 proper Bacula Backup Proxy Pod operations. You can create and use any valid SSL
682 certificate including a self-signed one. You can even just use the certificate generated
683 during the \kubernetes plugin installation located at \texttt{/opt/bacula/etc/bacula-backup.cert}
684 The default is to use the certificate generated during plugin installation.
685
686 This parameter is optional.
687
688 \item[\texttt{fdkeyfile=<path>}] is an SSL private key file location for the SSL
689 certificate defined by \texttt{fdcertfile=<path>} above. An unencrypted private key
690 must be used for this purpose. The default is to use a private key
691 file created during plugin installation at \texttt{/opt/bacula/etc/bacula-backup.key}
692 when \texttt{fdcertfile=<path>} above is not defined (the default). Otherwise the plugin
693 will refer to the same certificate file from \texttt{fdcertfile=<path>} and use it as
694 a \texttt{.pem} container.
695
696 This parameter is optional.
697
698 \label{baculaimageparam}
699 \item[\texttt{baculaimage=<name>}] is a Bacula Backup Proxy Pod container
700 image registry location for your cluster as described in the image installation chapter
701 \vref{baculatarimage}. In most cases this parameter will consist of your
702 Docker images registry location with the tag of the required image.
703 The default for this parameter is \texttt{bacula-backup:<timestamp>} which
704 may not match your cluster configuration.
705
706 This parameter is optional.
707
708 \item[\texttt{imagepullpolicy=<Always|Never|ifNotPresent>}] sets the Kubernetes
709 Pod image pull policy during Bacula Backup Proxy Pod container deployment.
710 You can configure Kubernetes to \texttt{Always} or \texttt{Never} pull the image
711 from the repository, or pull it only when not available with \texttt{ifNotPresent}.
712 The option value is case insensitive. If this parameter is not defined, the default
713 \texttt{ifNotPresent} will be used.
714
715 This parameter is optional.
716 \end{description}
717
718 \subsection{Restore Plugin Parameters}
719
720 During restore, the \kubernetes plugin will use the same parameters which were
721 set for the backup job and saved in the catalog. During restore, you may change any of the
722 parameters described in chapter \vref{genericparameters} and \vref{rbackupparameters}.
723 In addition to the options used for backups, the \texttt{outputformat} option can be
724 used during restore. This option specifies the file format when restoring to a local filesystem.
725 You can choose the restore output in \texttt{JSON} or \texttt{YAML}.
726 If not defined the restored files will be saved in \texttt{YAML}.
727
728 \begin{description}
729
730 \item[\texttt{outputformat: <JSON or YAML>}] specifies the
731 file format when restoring to a local filesystem as described above.
732
733 This parameter is optional.
734 \end{description}
735
736 \subsection{FileSet Examples}
737
738 In the example below, all Kubernetes Namespaces, Resources and Persistent
739 Volume Configurations will be backed up using the default Kubernetes API
740 access credentials.
741
742 \begin{verbatim}
743 FileSet {
744 Name = FS_Kubernetes_All
745 Include {
746 Plugin = "kubernetes:"
747 }
748 }
749 \end{verbatim}
750
751 In this example, we will backup a single Kubernetes Namespace using the Bearer
752 Token authorization method.
753
754 \begin{verbatim}
755 FileSet {
756 Name = FS_Kubernetes_plugintest
757 Include {
758 Plugin = "kubernetes: host=http://10.0.0.1/k8s/clusters/test \
759 token=kubeconfig-user:cbhssdxq8vv8hrcw8jdxs2 namespace=plugintest"
760 }
761 }
762 \end{verbatim}
763
764 The same example as above, but with a Persistent Volume:
765
766 \begin{verbatim}
767 FileSet {
768 Name = FS_Kubernetes_mcache1
769 Include {
770 Plugin = "kubernetes: host=http://10.0.0.1/k8s/clusters/test \
771 token=kubeconfig-user:cbhssdxq8vv8hrcw8jdxs2 \
772 namespace=plugintest persistentvolume=myvol"
773 }
774 }
775 \end{verbatim}
776
777 This example backs up a single Namespace and all detected PVCs in
778 this Namespace using a defined listening and entry point address and the default connection port:
779
780 \begin{verbatim}
781 FileSet {
782 Name = FS_Kubernetes_test_namespace
783 Include {
784 Plugin = "kubernetes: namespace=test pvcdata fdaddress=10.0.10.10"
785 }
786 }
787 \end{verbatim}
788
789 The same example as above, but using different listening and entry point addresses as
790 may be found when the service is behind a firewall using port forwarding features:
791
792 \begin{verbatim}
793 FileSet {
794 Name = FS_Kubernetes_test_namespace_through_firewall
795 Include {
796 Plugin = "kubernetes: namespace=test pvcdata=plugin-storage fdaddress=10.0.10.10 \
797 pluginhost=backup.example.com pluginport=8080"
798 }
799 }
800 \end{verbatim}
801
802 This example shows PVC Data archive backup with the Bacula File Daemon inside a
803 Kubernetes cluster:
804
805 \begin{verbatim}
806 FileSet {
807 Name = FS_Kubernetes_incluster
808 Include {
809 Plugin = "kubernetes: incluster namespace=test pvcdata \
810 pluginhost=backup.bacula.svc.cluster.local"
811 }
812 }
813 \end{verbatim}
814
815 The configuration above is designed for use in situations where the Bacula server components
816 are located on-premise and behind a firewall with no external ports allowed in, but must back
817 up data on an external \kubernetes cluster.
818
819 \newpage
820 \section{Restore examples}
821
822 \subsection{Restore to \kubernetes Cluster}
823
824 To restore Kubernetes resources to a Kubernetes cluster, the administrator
825 should execute the restore command and specify the \texttt{where}
826 parameter as in this example:
827 \begin{verbatim}
828 * restore where=/
829 \end{verbatim}
830
831 \noindent and then set any other required restore plugin parameters
832 for the restore.
833
834 \begin{verbatim}
835 * restore where=/
836 ...
837 $ cd /@kubernetes/namespaces/plugintest/configmaps/
838 cwd is: /@kubernetes/namespaces/plugintest/configmaps/
839 $ ls
840 plugintest-configmap.yaml
841 $ add *
842 1 file marked.
843 $ done
844 Bootstrap records written to /opt/bacula/working/bacula-devel-dir.restore.1.bsr
845
846 The Job will require the following (*=>InChanger):
847 Volume(s) Storage(s) SD Device(s)
848 ===========================================================================
849
850 Vol005 File1 FileChgr1
851
852 Volumes marked with "*" are in the Autochanger.
853
854
855 1 file selected to be restored.
856
857 Run Restore job
858 JobName: RestoreFiles
859 Bootstrap: /opt/bacula/working/bacula-devel-dir.restore.1.bsr
860 Where: /
861 Replace: Always
862 FileSet: Full Set
863 Backup Client: bacula-devel-fd
864 Restore Client: bacula-devel-fd
865 Storage: File1
866 When: 2019-09-30 12:39:13
867 Catalog: MyCatalog
868 Priority: 10
869 Plugin Options: *None*
870 OK to run? (yes/mod/no): mod
871 Parameters to modify:
872 1: Level
873 2: Storage
874 3: Job
875 4: FileSet
876 5: Restore Client
877 6: When
878 7: Priority
879 8: Bootstrap
880 9: Where
881 10: File Relocation
882 11: Replace
883 12: JobId
884 13: Plugin Options
885 Select parameter to modify (1-13): 13
886 Automatically selected : kubernetes: config=/home/radekk/.kube/config
887 Plugin Restore Options
888 config: radekk/.kube/config (*None*)
889 host: *None* (*None*)
890 token: *None* (*None*)
891 username: *None* (*None*)
892 password: *None* (*None*)
893 verify_ssl: *None* (True)
894 ssl_ca_cert: *None* (*None*)
895 outputformat: *None* (RAW)
896 Use above plugin configuration? (yes/mod/no): mod
897 You have the following choices:
898 1: config (K8S config file)
899 2: host (K8S API server URL/Host)
900 3: token (K8S Bearertoken)
901 4: verify_ssl (K8S API server cert verification)
902 5: ssl_ca_cert (Custom CA Certs file to use)
903 6: outputformat (Output format when saving to file (JSON, YAML))
904 7: fdaddress (The address for listen to incoming backup pod data)
905 8: fdport (The port for opening socket for listen)
906 9: pluginhost (The endpoint address for backup pod to connect)
907 10: pluginport (The endpoint port to connect)
908 Select parameter to modify (1-8): 1
909 Please enter a value for config: /root/.kube/config
910 Plugin Restore Options
911 config: /root/.kube/config (*None*)
912 host: *None* (*None*)
913 token: *None* (*None*)
914 verify_ssl: *None* (True)
915 ssl_ca_cert: *None* (*None*)
916 outputformat: *None* (RAW)
917 fdaddress: *None* (*FDAddress*)
918 fdport: *None* (9104)
919 pluginhost: *None* (*FDAddress*)
920 pluginport: *None* (9104)
921 Use above plugin configuration? (yes/mod/no): yes
922 Job queued. JobId=1084
923 \end{verbatimx}
924
925 The plugin does not wait for Kubernetes Resources to become ready and online in the same way
926 as the \texttt{kubectl} or the \texttt{oc} commands.
927
928 \subsection{Restore to a Local Directory}
929 \label{localdirrestore}
930 It is possible to restore any Kubernetes Resource(s) to file without
931 loading them into a cluster. To do so, the \texttt{where} restore option should point
932 to the local directory:
933
934 \begin{verbatim}
935 * restore where=/tmp/bacula/restores
936 ...
937 $ cd /@kubernetes/namespaces/
938 cwd is: /@kubernetes/namespaces/
939 $ ls
940 bacula/
941 cattle-system/
942 default/
943 graphite/
944 ingress/
945 plugintest/
946 $ add plugintest
947 25 files marked.
948 $ done
949 Bootstrap records written to /opt/bacula/working/bacula-devel-dir.restore.2.bsr
950
951 The Job will require the following (*=>InChanger):
952 Volume(s) Storage(s) SD Device(s)
953 ===========================================================================
954
955 Vol005 File1 FileChgr1
956
957 Volumes marked with "*" are in the Autochanger.
958
959
960 25 files selected to be restored.
961
962 Run Restore job
963 JobName: RestoreFiles
964 Bootstrap: /opt/bacula/working/bacula-devel-dir.restore.2.bsr
965 Where: /tmp/bacula/restores
966 Replace: Always
967 FileSet: Full Set
968 Backup Client: bacula-devel-fd
969 Restore Client: bacula-devel-fd
970 Storage: File1
971 When: 2019-09-30 12:58:16
972 Catalog: MyCatalog
973 Priority: 10
974 Plugin Options: *None*
975 OK to run? (yes/mod/no): mod
976 Parameters to modify:
977 1: Level
978 2: Storage
979 3: Job
980 4: FileSet
981 5: Restore Client
982 6: When
983 7: Priority
984 8: Bootstrap
985 9: Where
986 10: File Relocation
987 11: Replace
988 12: JobId
989 13: Plugin Options
990 Select parameter to modify (1-13): 13
991 Automatically selected : kubernetes: config=/home/radekk/.kube/config debug=1
992 Plugin Restore Options
993 config: *None* (*None*)
994 host: *None* (*None*)
995 token: *None* (*None*)
996 verify_ssl: *None* (True)
997 ssl_ca_cert: *None* (*None*)
998 outputformat: *None* (RAW)
999 fdaddress: *None* (*FDAddress*)
1000 fdport: *None* (9104)
1001 pluginhost: *None* (*FDAddress*)
1002 pluginport: *None* (9104)
1003 Use above plugin configuration? (yes/mod/no): mod
1004 You have the following choices:
1005 1: config (K8S config file)
1006 2: host (K8S API server URL/Host)
1007 3: token (K8S Bearertoken)
1008 4: verify_ssl (K8S API server cert verification)
1009 5: ssl_ca_cert (Custom CA Certs file to use)
1010 6: outputformat (Output format when saving to file (JSON, YAML))
1011 7: fdaddress (The address for listen to incoming backup pod data)
1012 8: fdport (The port for opening socket for listen)
1013 9: pluginhost (The endpoint address for backup pod to connect)
1014 10: pluginport (The endpoint port to connect)
1015 Select parameter to modify (1-8): 8
1016 Please enter a value for outputformat: JSON
1017 Plugin Restore Options
1018 config: *None* (*None*)
1019 host: *None* (*None*)
1020 token: *None* (*None*)
1021 verify_ssl: *None* (True)
1022 ssl_ca_cert: *None* (*None*)
1023 outputformat: *None* (RAW)
1024 fdaddress: *None* (*FDAddress*)
1025 fdport: *None* (9104)
1026 pluginhost: *None* (*FDAddress*)
1027 pluginport: JSON (9104)
1028 Use above plugin configuration? (yes/mod/no): yes
1029 Run Restore job
1030 JobName: RestoreFiles
1031 Bootstrap: /opt/bacula/working/bacula-devel-dir.restore.2.bsr
1032 Where: /tmp/bacula/restores
1033 Replace: Always
1034 FileSet: Full Set
1035 Backup Client: bacula-devel-fd
1036 Restore Client: bacula-devel-fd
1037 Storage: File1
1038 When: 2019-09-30 12:58:16
1039 Catalog: MyCatalog
1040 Priority: 10
1041 Plugin Options: User specified
1042 OK to run? (yes/mod/no):
1043 Job queued. JobId=1085
1044 \end{verbatim}
1045
1046 Output format conversion at restore time will format all data in a human readable format.
1047 You can find an example of this restore below.
1048
1049 \begin{verbatim}
1050 # cat /tmp/bacula/restores/namespaces/plugintest/plugintest.json
1051 {
1052 "apiVersion": "v1",
1053 "kind": "Namespace",
1054 "metadata": {
1055 "annotations": {
1056 "field.cattle.io/projectId": "c-hb9ls:p-bm6cw",
1057 "lifecycle.cattle.io/create.namespace-auth": "true"
1058 },
1059 "cluster_name": null,
1060 "creation_timestamp": "2019-09-25T16:31:03",
1061 "deletion_grace_period_seconds": null,
1062 "deletion_timestamp": null,
1063 "finalizers": [
1064 "controller.cattle.io/namespace-auth"
1065 ],
1066 "generate_name": null,
1067 "generation": null,
1068 "initializers": null,
1069 "labels": {
1070 "field.cattle.io/projectId": "p-bm6cw"
1071 },
1072 "name": "plugintest",
1073 "namespace": null,
1074 "owner_references": null,
1075 "resource_version": "11622",
1076 "self_link": "/api/v1/namespaces/plugintest",
1077 "uid": "dd873930-dfb1-11e9-aad0-022014368e80"
1078 },
1079 "spec": {
1080 "finalizers": [
1081 "kubernetes"
1082 ]
1083 },
1084 "status": {
1085 "phase": "Active"
1086 }
1087 }
1088 \end{verbatim}
1089
1090 The supported output transformations are: \texttt{JSON} and \texttt{YAML}.
1091
1092 \subsection{Restore PVC Data Archive}
1093
1094 Here we describe functionalities and requirements related to pvcdata restore.
1095
1096 \subsubsection{Local Directory Restore}
1097
1098 The procedure to restore a PVC Data archive file to a local directory is basically the same as restoring
1099 the Kubernetes Resource configuration file as described in \vref{localdirrestore}. However,
1100 output transformation is unavailable and ignored when restoring PVC data. Restore of this data will create a
1101 \texttt{tar} archive file you can manually inspect and use.
1102
1103 \subsubsection{Restore to PVC}
1104
1105 This procedure is similar to the one described in PVC Data backup and uses the same Bacula Backup
1106 Proxy Pod image. During restore, the plugin uses the same endpoint configuration parameters
1107 so it is not necessary to setup it again. If your endpoint parameters have changed you
1108 can update them using Bacula plugin restore options modification as in example below:
1109
1110 \begin{verbatim}
1111 *restore select all done where=/
1112 (...)
1113 OK to run? (yes/mod/no): mod
1114 Parameters to modify:
1115 1: Level
1116 2: Storage
1117 3: Job
1118 4: FileSet
1119 5: Restore Client
1120 6: When
1121 7: Priority
1122 8: Bootstrap
1123 9: Where
1124 10: File Relocation
1125 11: Replace
1126 12: JobId
1127 13: Plugin Options
1128 Select parameter to modify (1-13): 13
1129 Automatically selected : kubernetes: namespace=plugintest pvcdata pluginhost=example.com
1130 Plugin Restore Options
1131 config: *None* (*None*)
1132 host: *None* (*None*)
1133 token: *None* (*None*)
1134 verify_ssl: *None* (True)
1135 ssl_ca_cert: *None* (*None*)
1136 outputformat: *None* (RAW)
1137 fdaddress: *None* (*FDAddress*)
1138 fdport: *None* (9104)
1139 pluginhost: *None* (*FDAddress*)
1140 pluginport: *None* (9104)
1141 Use above plugin configuration? (yes/mod/no): mod
1142 You have the following choices:
1143 1: config (K8S config file)
1144 2: host (K8S API server URL/Host)
1145 3: token (K8S Bearertoken)
1146 4: verify_ssl (K8S API server cert verification)
1147 5: ssl_ca_cert (Custom CA Certs file to use)
1148 6: outputformat (Output format when saving to file (JSON, YAML))
1149 7: fdaddress (The address for listen to incoming backup pod data)
1150 8: fdport (The port for opening socket for listen)
1151 9: pluginhost (The endpoint address for backup pod to connect)
1152 10: pluginport (The endpoint port to connect)
1153 Select parameter to modify (1-10): 9
1154 Please enter a value for pluginhost: newbackup.example.com
1155 Plugin Restore Options
1156 config: *None* (*None*)
1157 host: *None* (*None*)
1158 token: *None* (*None*)
1159 verify_ssl: *None* (True)
1160 ssl_ca_cert: *None* (*None*)
1161 outputformat: *None* (RAW)
1162 fdaddress: *None* (*FDAddress*)
1163 fdport: *None* (9104)
1164 pluginhost: newbackup.example.com (*FDAddress*)
1165 pluginport: *None* (9104)
1166 Use above plugin configuration? (yes/mod/no): yes
1167 \end{verbatim}
1168
1169 You can restore all data available from the backup archive for a selected Persistent Volume Claim
1170 and all data will be overwritten, ignoring the \texttt{Replace} job parameter. Please take note of this
1171 behavior, which may change in the future.
1172
1173
1174 \subsection{Other}
1175
1176 \subsubsection{Resource listing}
1177 \label{listing}
1178
1179 The Bacula Enterprise \kubernetes plugin supports the "plugin listing"
1180 feature of Bacula Enterprise Edition 8.x or newer. This mode allows the
1181 plugin to display some useful information about available Kubernetes
1182 resources such as:
1183
1184 \begin{itemize}
1185 \item List of Kubernetes namespaces
1186 \item List of Kubernetes persistent volumes
1187 \item List of Kubernetes storage class resources
1188 \end{itemize}
1189
1190 The feature uses the special \texttt{.ls} command with a
1191 \texttt{plugin=<plugin>} parameter.
1192
1193 \bigskip
1194 The command requires the following parameters to be set:
1195 \begin{description}
1196 \item[\texttt{client=<client>}] A Bacula Client name with the \kubernetes plugin installed.
1197 \item[\texttt{plugin=<plugin>}] A plugin name, which would be
1198 \texttt{kubernetes:} in this case, with optional plugin parameters
1199 as described in section~\vref{genericparameters}.
1200 \item[\texttt{path=<path>}] An object path to display.
1201 \end{description}
1202
1203 \bigskip
1204 The supported values for a \texttt{path=<path>} parameter are:
1205 \begin{description}
1206 \item[\texttt{/}] to display Object types available to list
1207 \item[\texttt{namespaces}] to display a list of Namespaces
1208 \item[\texttt{persistentvolumes}] to display a list of Persistent Volumes
1209 \item[\texttt{storageclass}] to display a list of Storage Class Resources
1210 \item[\texttt{namespaces/<name>/pvcdata}] to display all available Persistent
1211 Volume Claims available in Namespace \texttt{<name>}
1212 \end{description}
1213
1214 \bigskip
1215 To display available Object types, follow the following command
1216 example:
1217
1218 \begin{verbatim}
1219 *.ls plugin=kubernetes: client=kubernetes-fd path=/
1220 Connecting to Client kubernetes-fd at localhost:9102
1221 drwxr-x--- 1 root root 2018-09-28 14:32:20 /namespaces
1222 drwxr-x--- 1 root root 2018-09-28 14:32:20 /persistentvolumes
1223 drwxr-x--- 1 root root 2018-09-28 14:32:20 /storageclass
1224 2000 OK estimate files=2 bytes=0
1225 \end{verbatim}
1226
1227 To display the list of all available Kubernetes namespaces, the following command
1228 example can be used:
1229
1230 \begin{verbatim}
1231 *.ls plugin=kubernetes: client=kubernetes-fd path=namespaces
1232 Connecting to Client kubernetes-fd at localhost:9102
1233 drwxr-xr-x 1 root root 2019-09-25 16:39:56 /namespaces/default
1234 drwxr-xr-x 1 root root 2019-09-25 16:39:56 /namespaces/kube-public
1235 drwxr-xr-x 1 root root 2019-09-25 16:39:56 /namespaces/kube-system
1236 drwxr-xr-x 1 root root 2019-09-25 16:46:19 /namespaces/cattle-system
1237 drwxr-xr-x 1 root root 2019-09-27 13:04:01 /namespaces/plugintest
1238 2000 OK estimate files=5 bytes=0
1239 \end{verbatim}
1240
1241 To display the list of available Persistent Volume Claims which could be used for PVC Data
1242 archive feature selection, you can use the following example command for the \texttt{mysql}
1243 namespace:
1244
1245 \begin{verbatim}
1246 *.ls client=bacula-devel-fd plugin="kubernetes:" path=/namespaces/mysql/pvcdata
1247 Connecting to Client kubernetes-fd at localhost:9102
1248 -rw-r----- 1 root root 2019-10-16 14:29:38 /namespaces/mysql/pvcdata/mysql-mysql
1249 2000 OK estimate files=1 bytes=0
1250 \end{verbatim}
1251
1252 To display the list of all available Persistent Volumes, the following command
1253 example can be used:
1254
1255 \begin{verbatim}
1256 *.ls plugin=kubernetes: client=kubernetes-fd path=persistentvolumes
1257 Connecting to Client kubernetes-fd at localhost:9102
1258 -rw-r----- 1073741824 2019-09-25 /persistentvolumes/pvc-bfaebd0d-dfad-11e9-a2cc-42010a8e0174
1259 -rw-r----- 1073741824 2019-09-25 /persistentvolumes/pvc-b1a49497-dfad-11e9-a2cc-42010a8e0174
1260 -rw-r----- 1073741824 2019-09-25 /persistentvolumes/pvc-949cb638-dfad-11e9-a2cc-42010a8e0174
1261 -rw-r----- 1073741824 2019-09-25 /persistentvolumes/pvc-9313388c-dfad-11e9-a2cc-42010a8e0174
1262 -rw-r----- 10737418240 2019-09-24 /persistentvolumes/myvolume
1263 2000 OK estimate files=5 bytes=15,032,385,536
1264 \end{verbatim}
1265
1266 The volume lists display a Volume Storage size which does not reflect
1267 the actual configuration size during backup.
1268
1269 To display the list of all defined Storage Class Resources, the following command
1270 example can be used:
1271
1272 \begin{verbatim}
1273 *.ls plugin=kubernetes: client=kubernetes-fd path=storageclass
1274 Connecting to Client kubernetes-fd at localhost:9102
1275 -rw-r----- 1024 2020-07-27 13:39:48 /storageclass/local-storage
1276 -rw-r----- 1024 2020-07-23 16:14:13 /storageclass/default-postgresql-1
1277 -rw-r----- 1024 2020-07-24 11:47:02 /storageclass/local-storage-default
1278 -rw-r----- 1024 2020-07-23 12:00:02 /storageclass/standard
1279 2000 OK estimate files=4 bytes=4,096
1280 \end{verbatim}
1281
1282
1283 % disabled until prepared
1284 %\subsection{Best practice}
1285
1286 \subsection{Advanced Bacula Backup Proxy Pod Deployment}
1287 \label{advancedpoddeployment}
1288
1289 \textit{WARNING: This is an advanced topic related to Kubernetes clusters. You should !!NOT!!
1290 try to implement or customize the Bacula \kubernetes plugin behavior unless you REALLY
1291 know what you are doing.}
1292 \bigskip
1293
1294 You can customize the service parameters used for deploying Bacula backup Pods
1295 dedicated to Persistent Volume Claim data backups to suit your needs. The plugin uses
1296 the following Pod service deployment YAML template to execute the proxy operation
1297 pod on the cluster.
1298
1299 \begin{verbatim}
1300 apiVersion: v1
1301 kind: Pod
1302 metadata:
1303 name: {podname}
1304 namespace: {namespace}
1305 labels:
1306 app: {podname}
1307 spec:
1308 hostname: {podname}
1309 {nodenameparam}
1310 containers:
1311 - name: {podname}
1312 resources:
1313 limits:
1314 cpu: "1"
1315 memory: "64Mi"
1316 requests:
1317 cpu: "100m"
1318 memory: "16Mi"
1319 image: {image}
1320 env:
1321 - name: PLUGINMODE
1322 value: "{mode}"
1323 - name: PLUGINHOST
1324 value: "{host}"
1325 - name: PLUGINPORT
1326 value: "{port}"
1327 - name: PLUGINTOKEN
1328 value: "{token}"
1329 imagePullPolicy: {imagepullpolicy}
1330 volumeMounts:
1331 - name: {podname}-storage
1332 mountPath: /{mode}
1333 restartPolicy: Never
1334 volumes:
1335 - name: {podname}-storage
1336 persistentVolumeClaim:
1337 claimName: {pvcname}
1338 \end{verbatim}
1339
1340 The above template uses a number of predefined placeholders which will be replaced
1341 by corresponding variables during Pod execution preparation. To customize proxy
1342 Pod deployment you can change or tune template variables or the template body.
1343 Below is a list of all supported variables with short descriptions and requirement
1344 conditions.
1345
1346 \begin{description}
1347 \item[\texttt{podname}] - This is the predefined Pod name used by a plugin.
1348 This variable is required and cannot be customized.
1349 \item[\texttt{namespace}] - This is a Namespace name for which the PVC
1350 Data backup is performed.
1351 This variable is required and cannot be customized.
1352 \item[\texttt{nodenameparam}] - This is a placeholder for Cluster node name
1353 parameter (\texttt{nodeName: \ldots}) used to mount an existing Volume Claim
1354 for backup or restore if required for the selected PVC. In most cases this variable is
1355 required and cannot be customized.
1356 \item[\texttt{image}] - This is a Pod container image name to execute.
1357 You can customize or omit this variable as long as you provide
1358 a container image name required by the cluster.
1359 \item[\texttt{mode}] - This is an operation mode for the Proxy Pod.
1360 The supported values are: \texttt{backup} and \texttt{restore}.
1361 This variable is required and cannot be customized.
1362 \item[\texttt{host}] - This is an endpoint address which corresponds to the
1363 \texttt{pluginport=\ldots} Kubernetes plugin parameter.
1364 This variable is required. You can customize or omit this variable as long
1365 as you provide a value for the \texttt{PLUGINHOST} container environment.
1366 \item[\texttt{port}] - This is an endpoint address which corresponds to the
1367 \texttt{pluginport=\ldots} Kubernetes plugin parameter.
1368 This variable is required. You can customize or omit this variable as long
1369 as you provide a value for the \texttt{PLUGINPORT} container environment.
1370 \item[\texttt{token}] - This is an Authorization Token (randomly generated).
1371 This variable is required and cannot be customized.
1372 \item[\texttt{pvcname}] - This is the name of the PVC for backup or restore
1373 operations.
1374 This variable is required and cannot be customized.
1375 \end{description}
1376 You can create the required file: \texttt{/opt/bacula/scripts/bacula-backup.yaml} or point
1377 to the custom one using \texttt{\$DEFAULTPODYAML} environment variable.
1378
1379 \subsection{Limitations}
1380
1381 \begin{bitemize}
1382 \item Only full level backups are possible. This is a Kubernetes limitation.
1383 \item You can perform a single PVC Data backup or restore with a single Bacula File
1384 Daemon installation associated with single \texttt{fdaddress=<name>}.
1385 This limitation may be removed in a future release of the \kubernetes plugin.
1386 \end{bitemize}
1387
1388 \subsection{Common Problems}
1389
1390 In this section we describe the common problems you may encounter when you first deploy the
1391 Bacula \kubernetes plugin or when you are not very familiar with this plugin.
1392
1393 \begin{bitemize}
1394 \item You have set the \texttt{incluster} parameter in your Job but you have the following error:
1395 \begin{verbatim}
1396 Error: kubernetes: incluster error: Service host/port is not set.
1397 \end{verbatim}
1398 This means you are running the Bacula File Daemon and \kubernetes plugin not in a Pod, or
1399 Kubernetes does not provide default service access in your installation.
1400 \smallskip
1401 In the latter case you should use a standard Kubernetes access method in a prepared kubeconfig file.
1402 \end{bitemize}