| 1 |
\chapter{Bacula Git Usage} |
| 2 |
\label{blb:_GitChapterStart} |
| 3 |
\index{Git} |
| 4 |
\index{Git!Repo} |
| 5 |
|
| 6 |
This chapter is intended to help you use the Git source code |
| 7 |
repositories to obtain, modify, and submit Bacula source code. |
| 8 |
|
| 9 |
|
| 10 |
\section{Bacula Git repositories} |
| 11 |
\index{Git} |
| 12 |
As of September 2009, the Bacula source code has been split into |
| 13 |
three Git repositories. One is a repository that holds the |
| 14 |
main Bacula source code with directories {\bf bacula}, {\bf gui}, |
| 15 |
and {\bf regress}. The second repository contains |
| 16 |
the directories {\bf docs} directory, and the third repository |
| 17 |
contains the {\bf rescue} directory. All three repositories are |
| 18 |
hosted by UKFast. |
| 19 |
|
| 20 |
Previously everything was in a single SVN repository. |
| 21 |
We have split the SVN repository into three because Git |
| 22 |
offers significant advantages for ease of managing and integrating |
| 23 |
developer's changes. However, one of the disadvantages of Git is that you |
| 24 |
must work with the full repository, while SVN allows you to checkout |
| 25 |
individual directories. If we put everything into a single Git |
| 26 |
repository it would be far bigger than most developers would want |
| 27 |
to checkout, so we have separted the docs and rescue into their own |
| 28 |
repositories, and moved only the parts that are most actively |
| 29 |
worked on by the developers (bacula, gui, and regress) to a the |
| 30 |
Git Bacula repository. |
| 31 |
|
| 32 |
Each major version of Bacula uses a specific branch name rather than the |
| 33 |
default git ``master'' branch. For example, the version 11.0.1 code will be |
| 34 |
accessible via the branch Branch-11.0. In this section, the word ``master'' |
| 35 |
refers to the current active development branch. |
| 36 |
|
| 37 |
Bacula developers must now have a certain knowledege of Git. |
| 38 |
|
| 39 |
\section{Git Usage} |
| 40 |
\index{Git Usage} |
| 41 |
|
| 42 |
Please note that if you are familiar with SVN, Git is similar, |
| 43 |
(and better), but there can be a few surprising differences that |
| 44 |
can be very confusing (nothing worse than converting from CVS to SVN). |
| 45 |
|
| 46 |
The main Bacula Git repo contains the subdirectories {\bf bacula}, {\bf gui}, |
| 47 |
and {\bf regress}. With Git it is not possible to pull only a |
| 48 |
single directory, because of the hash code nature of Git, you |
| 49 |
must take all or nothing. |
| 50 |
|
| 51 |
For developers, the most important thing to remember about Git and |
| 52 |
the bacula.org repository is not to "force" a {\bf push} to the |
| 53 |
repository. Doing so, can possibly rewrite |
| 54 |
the Git repository history and cause a lot of problems for the |
| 55 |
project. |
| 56 |
|
| 57 |
You can get a full copy of the Bacula Git repository with the |
| 58 |
following command: |
| 59 |
|
| 60 |
\begin{bVerbatim} |
| 61 |
git clone http://git.bacula.org/bacula.git bacula |
| 62 |
git checkout -b Branch-11.0 origin/Branch-11.0 |
| 63 |
\end{bVerbatim} |
| 64 |
|
| 65 |
This will put a read-only copy into the directory {\bf bacula} |
| 66 |
in your current directory, and {\bf bacula} will contain |
| 67 |
the subdirectories: {\bf bacula}, {\bf gui}, and {\bf regress}. |
| 68 |
Obviously you can use any name an not just {\bf bacula}. In fact, |
| 69 |
once you have the repository in say {\bf bacula}, you can copy the |
| 70 |
whole directory to another place and have a fully functional |
| 71 |
git repository. |
| 72 |
|
| 73 |
The above command needs to be done only once. Thereafter, you can: |
| 74 |
|
| 75 |
\begin{bVerbatim} |
| 76 |
cd bacula |
| 77 |
git pull # refresh my repo with the latest code |
| 78 |
\end{bVerbatim} |
| 79 |
|
| 80 |
As of August 2009, the size of the repository ({\bf bacula} in the above |
| 81 |
example) will be approximately 55 Megabytes. However, if you build |
| 82 |
from source in this directory and do a lot of updates and regression |
| 83 |
testing, the directory could become several hundred megabytes. |
| 84 |
|
| 85 |
\subsection{Learning Git} |
| 86 |
\index{Learning Git} |
| 87 |
If you want to learn more about Git, we recommend that you visit:\\ |
| 88 |
\bref{http://book.git-scm.com/}{book.git-scm.com/}. |
| 89 |
|
| 90 |
Some of the differences between Git and SVN are: |
| 91 |
\begin{bitemize} |
| 92 |
\item Your main Git directory is a full Git repository to which you can |
| 93 |
and must commit. In fact, we suggest you commit frequently. |
| 94 |
\item When you commit, the commit goes into your local Git |
| 95 |
database. You must use another command to write it to the |
| 96 |
bacula.org repository (see below). |
| 97 |
\item The local Git database is kept in the directory {\bf .git} at the |
| 98 |
top level of the directory. |
| 99 |
\item All the important Git configuration information is kept in the |
| 100 |
file {\bf .git/config} in ASCII format that is easy to manually edit. |
| 101 |
\item When you do a {\bf commit} the changes are put in {\bf .git} |
| 102 |
rather but not in the main bacula.org repository. |
| 103 |
\item You can push your changes to the external repository using |
| 104 |
the command {\bf git push} providing you have write permission |
| 105 |
on the repository. |
| 106 |
\item We restrict developers just learning git to have read-only |
| 107 |
access until they feel comfortable with git before giving them |
| 108 |
write access. |
| 109 |
\item You can download all the current changes in the external repository |
| 110 |
and merge them into your {\bf master} branch using the command |
| 111 |
{\bf git pull}. |
| 112 |
\item The command {\bf git add} is used to add a new file to the |
| 113 |
repository AND to tell Git that you want a file that has changed |
| 114 |
to be in the next commit. This has lots of advantages, because |
| 115 |
a {\bf git commit} only commits those files that have been |
| 116 |
explicitly added. Note with SVN {\bf add} is used only |
| 117 |
to add new files to the repo. |
| 118 |
\item You can add and commit all files modifed in one command |
| 119 |
using {\bf git commit -a}. |
| 120 |
\item This extra use of {\bf add} allows you to make a number |
| 121 |
of changes then add only a few of the files and commit them, |
| 122 |
then add more files and commit them until you have committed |
| 123 |
everything. This has the advantage of allowing you to more |
| 124 |
easily group small changes and do individaual commits on them. |
| 125 |
By keeping commits smaller, and separated into topics, it makes |
| 126 |
it much easier to later select certain commits for backporting. |
| 127 |
\item If you {\bf git pull} from the main repository and make |
| 128 |
some changes, and before you do a {\bf git push} someone |
| 129 |
else pushes changes to the Git repository, your changes will |
| 130 |
apply to an older version of the repository you will probably |
| 131 |
get an error message such as: |
| 132 |
|
| 133 |
\begin{bVerbatim} |
| 134 |
git push |
| 135 |
To [email protected]:bacula/bacula.git |
| 136 |
! [rejected] Branch-11.0 -> Branch-11.0 (non-fast forward) |
| 137 |
error: failed to push some refs to '[email protected]:bacula/bacula.git' |
| 138 |
\end{bVerbatim} |
| 139 |
|
| 140 |
which is Git's way of telling you that the main repository has changed |
| 141 |
and that if you push your changes, they will not be integrated properly. |
| 142 |
This is very similar to what happens when you do an "svn update" and |
| 143 |
get merge conflicts. |
| 144 |
As we have noted above, you should never ask Git to force the push. |
| 145 |
See below for an explanation of why. |
| 146 |
\item To integrate (merge) your changes properly, you should always do |
| 147 |
a {\bf git pull} just prior to doing a {\bf git push}. |
| 148 |
\item If Git is unable to merge your changes or finds a conflict it |
| 149 |
will tell you and you must do conflict resolution, which is much |
| 150 |
easier in Git than in SVN. |
| 151 |
\item Resolving conflicts is described below in the {\bf github} section. |
| 152 |
\end{bitemize} |
| 153 |
|
| 154 |
\section{Step by Step Modifying Bacula Code} |
| 155 |
Suppose you want to download Bacula source code, build it, make |
| 156 |
a change, then submit your change to the Bacula developers. What |
| 157 |
would you do? |
| 158 |
|
| 159 |
\begin{bitemize} |
| 160 |
\item Tell git who you are:\\ |
| 161 |
\begin{bVerbatim} |
| 162 |
git config --global user.name "First-name Last-name" |
| 163 |
git config --global user.email "[email protected]" |
| 164 |
\end{bVerbatim} |
| 165 |
|
| 166 |
Where you put your real name and your email address. Since |
| 167 |
this is global, you only need to do it once on any given |
| 168 |
machine regardless of how many git repos you work with. |
| 169 |
|
| 170 |
\item Download the Source code:\\ |
| 171 |
\begin{bVerbatim} |
| 172 |
git clone http://git.bacula.org/bacula.git bacula |
| 173 |
\end{bVerbatim} |
| 174 |
|
| 175 |
\item Configure and Build Bacula:\\ |
| 176 |
\begin{bVerbatim} |
| 177 |
./configure (all-your-normal-options) |
| 178 |
make |
| 179 |
\end{bVerbatim} |
| 180 |
|
| 181 |
\item Create a branch to work on: |
| 182 |
\begin{bVerbatim} |
| 183 |
cd bacula/bacula |
| 184 |
git checkout -b bugfix Branch-11.0 |
| 185 |
\end{bVerbatim} |
| 186 |
|
| 187 |
\item Edit, build, Test, ...\\ |
| 188 |
\begin{bVerbatim} |
| 189 |
edit file jcr.h |
| 190 |
make |
| 191 |
test |
| 192 |
\end{bVerbatim} |
| 193 |
|
| 194 |
Note: if you forget to create a working branch prior to making changes, and you |
| 195 |
make them on the development branch, this is no problem providing that you |
| 196 |
create the working branch before your first commit. So assuming that you have |
| 197 |
edited ``master'' instead of your bugfix branch, you can simply: |
| 198 |
|
| 199 |
\begin{bVerbatim} |
| 200 |
git checkout -b bugfix master |
| 201 |
\end{bVerbatim} |
| 202 |
|
| 203 |
and a new bugfix branch will be created and checked out. |
| 204 |
You can then proceed to committing to your bugfix branch as |
| 205 |
described in the next step. |
| 206 |
|
| 207 |
\item commit your work: |
| 208 |
\begin{bVerbatim} |
| 209 |
git commit -am "Short comment on what I did" |
| 210 |
\end{bVerbatim} |
| 211 |
|
| 212 |
\item Possibly repeat the above two items |
| 213 |
|
| 214 |
\item Switch back to the master branch:\\ |
| 215 |
\begin{bVerbatim} |
| 216 |
git checkout master |
| 217 |
\end{bVerbatim} |
| 218 |
|
| 219 |
\item Pull the latest changes:\\ |
| 220 |
\begin{bVerbatim} |
| 221 |
git pull |
| 222 |
\end{bVerbatim} |
| 223 |
|
| 224 |
\item Get back on your bugfix branch:\\ |
| 225 |
\begin{bVerbatim} |
| 226 |
git checkout bugfix |
| 227 |
\end{bVerbatim} |
| 228 |
|
| 229 |
\item Merge your changes and correct any conflicts:\\ |
| 230 |
\begin{bVerbatim} |
| 231 |
git rebase master bugfix |
| 232 |
\end{bVerbatim} |
| 233 |
|
| 234 |
\item Fix any conflicts:\\ |
| 235 |
You will be notified if there are conflicts. The first |
| 236 |
thing to do is: |
| 237 |
|
| 238 |
\begin{bVerbatim} |
| 239 |
git diff |
| 240 |
\end{bVerbatim} |
| 241 |
|
| 242 |
This will produce a diff of only the files having a conflict. |
| 243 |
Fix each file in turn. When it is fixed, the diff for that file |
| 244 |
will go away. |
| 245 |
|
| 246 |
For each file fixed, you must do the same as SVN, inform git with: |
| 247 |
|
| 248 |
\begin{bVerbatim} |
| 249 |
git add (name-of-file-no-longer-in-conflict) |
| 250 |
\end{bVerbatim} |
| 251 |
|
| 252 |
\item When all files are fixed do: |
| 253 |
\begin{bVerbatim} |
| 254 |
git rebase --continue |
| 255 |
\end{bVerbatim} |
| 256 |
|
| 257 |
\item If you find that it is impossible to reconcile the two |
| 258 |
branches or you made a mistake in correcting and adding files, |
| 259 |
before you enter the: |
| 260 |
\begin{bVerbatim} |
| 261 |
git rebase --continue |
| 262 |
\end{bVerbatim} |
| 263 |
you can instead enter: |
| 264 |
|
| 265 |
\begin{bVerbatim} |
| 266 |
git rebase --abort |
| 267 |
\end{bVerbatim} |
| 268 |
|
| 269 |
which will essentially cancel the the original git rebase and reset |
| 270 |
everything to the beginning with no changes to your bugfix branch. |
| 271 |
|
| 272 |
\item When you have completed the rebase and |
| 273 |
are ready to send a patch, do the following:\\ |
| 274 |
\begin{bVerbatim} |
| 275 |
git checkout bugfix |
| 276 |
git format-patch -M master |
| 277 |
\end{bVerbatim} |
| 278 |
Look at the files produced. They should be numbered 0001-xxx.patch |
| 279 |
where there is one file for each commit you did, number sequentially, |
| 280 |
and the xxx is what you put in the commit comment. |
| 281 |
|
| 282 |
\item If the patch files are good, send them by email to the developers |
| 283 |
as attachments. |
| 284 |
|
| 285 |
\item Then you can continue working on your code if you want, or |
| 286 |
start another branch with a new project. |
| 287 |
|
| 288 |
\item If you continue working on your bugfix branch, you should |
| 289 |
do a {\bf git rebase master} from time to time, and when |
| 290 |
your changes are committed to the repo, you will be automatically |
| 291 |
synchronized. So that the next {\bf git format-patch} will produce |
| 292 |
only the changes you made since the last format-patch you sent |
| 293 |
to the developers. |
| 294 |
|
| 295 |
|
| 296 |
\end{bitemize} |
| 297 |
|
| 298 |
|
| 299 |
|
| 300 |
\subsection{More Details} |
| 301 |
|
| 302 |
Normally, you will work by creating a branch of the development branch of your |
| 303 |
repository, make your modifications, then make sure it is up to date, and finally |
| 304 |
create format-patch patches or push it to the bacula.org. Assuming |
| 305 |
you call the Bacula repository {\bf bacula}, you might use the following |
| 306 |
commands: |
| 307 |
|
| 308 |
\begin{bVerbatim} |
| 309 |
cd bacula |
| 310 |
git checkout bacula |
| 311 |
git pull |
| 312 |
git checkout -b newbranch bacula |
| 313 |
(edit, ...) |
| 314 |
git add <file-edited> |
| 315 |
git commit -m "<comment about commit>" |
| 316 |
... |
| 317 |
\end{bVerbatim} |
| 318 |
|
| 319 |
When you have completed working on your branch, you will do: |
| 320 |
|
| 321 |
\begin{bVerbatim} |
| 322 |
cd bacula |
| 323 |
git checkout newbranch # ensure I am on my branch |
| 324 |
git pull # get latest source code |
| 325 |
git rebase master # merge my code |
| 326 |
\end{bVerbatim} |
| 327 |
|
| 328 |
If you have completed your edits before anyone has modified the repository, |
| 329 |
the {\bf git rebase master} will report that there was nothing to do. Otherwise, |
| 330 |
it will merge the changes that were made in the repository before your changes. |
| 331 |
If there are any conflicts, Git will tell you. Typically resolving conflicts with |
| 332 |
Git is relatively easy. You simply make a diff: |
| 333 |
|
| 334 |
\begin{bVerbatim} |
| 335 |
git diff |
| 336 |
\end{bVerbatim} |
| 337 |
|
| 338 |
Then edit each file that was listed in the {\bf git diff} to remove the |
| 339 |
conflict, which will be indicated by lines of: |
| 340 |
|
| 341 |
\begin{bVerbatim} |
| 342 |
other text |
| 343 |
\end{bVerbatim} |
| 344 |
|
| 345 |
where {\bf text} is what is in the Bacula repository, and {\bf other text} |
| 346 |
is what you have changed. |
| 347 |
|
| 348 |
Once you have eliminated the conflict, the {\bf git diff} will show nothing, |
| 349 |
and you must do a: |
| 350 |
|
| 351 |
\begin{bVerbatim} |
| 352 |
git add <file-with-conflicts-fixed> |
| 353 |
\end{bVerbatim} |
| 354 |
|
| 355 |
Once you have fixed all the files with conflicts in the above manner, you enter: |
| 356 |
|
| 357 |
\begin{bVerbatim} |
| 358 |
git rebase --continue |
| 359 |
\end{bVerbatim} |
| 360 |
|
| 361 |
and your rebase will be complete. |
| 362 |
|
| 363 |
If for some reason, before doing the --continue, you want to abort the rebase and return to what you had, you enter: |
| 364 |
|
| 365 |
\begin{bVerbatim} |
| 366 |
git rebase --abort |
| 367 |
\end{bVerbatim} |
| 368 |
|
| 369 |
Finally to make a set of patch files |
| 370 |
|
| 371 |
\begin{bVerbatim} |
| 372 |
git format-patch -M master |
| 373 |
\end{bVerbatim} |
| 374 |
|
| 375 |
When you see your changes have been integrated and pushed to the |
| 376 |
main repo, you can delete your branch with: |
| 377 |
|
| 378 |
\begin{bVerbatim} |
| 379 |
git checkout master |
| 380 |
git branch -D newbranch |
| 381 |
\end{bVerbatim} |
| 382 |
|
| 383 |
|
| 384 |
\section{Forcing Changes} |
| 385 |
If you want to understand why it is not a good idea to force a |
| 386 |
push to the repository, look at the following picture: |
| 387 |
|
| 388 |
\bimageH{git-edit-commit}{Git Edit Commit}{} |
| 389 |
|
| 390 |
The above graphic has three lines of circles. Each circle represents |
| 391 |
a commit, and time runs from the left to the right. The top line |
| 392 |
shows the repository just before you are going to do a push. Note the |
| 393 |
point at which you pulled is the circle on the left, your changes are |
| 394 |
represented by the circle labeled {\bf Your mods}. It is shown below |
| 395 |
to indicate that the changes are only in your local repository. Finally, |
| 396 |
there are pushes A and B that came after the time at which you pulled. |
| 397 |
|
| 398 |
If you were to force your changes into the repository, Git would place them |
| 399 |
immediately after the point at which you pulled them, so they would |
| 400 |
go before the pushes A and B. However, doing so would rewrite the history |
| 401 |
of the repository and make it very difficult for other users to synchronize |
| 402 |
since they would have to somehow wedge their changes at some point before the |
| 403 |
current HEAD of the repository. This situation is shown by the second line of |
| 404 |
pushes. |
| 405 |
|
| 406 |
What you really want to do is to put your changes after Push B (the current |
| 407 |
HEAD). This is shown in the third line of pushes. The best way to accomplish |
| 408 |
this is to work in a branch, pull the repository so you have your master equal |
| 409 |
to HEAD (in first line), then to rebase your branch on the current master and |
| 410 |
then commit it. The exact commands to accomplish this are shown in the next |
| 411 |
couple of sections. |