| 1 |
debian-cd |
| 2 |
========= |
| 3 |
|
| 4 |
Copyright 1999-2001 Raphaƫl Hertzog <[email protected]> and others, |
| 5 |
2004-2019 Steve McIntyre <[email protected]> |
| 6 |
This set of tools is licensed under the General Public License version |
| 7 |
2 or any later version. You can find it in |
| 8 |
/usr/share/common-licenses/GPL on a Debian GNU system. |
| 9 |
|
| 10 |
Some of the ideas here (a loooong time ago) came from Steve McIntyre's |
| 11 |
slink_cd script. |
| 12 |
|
| 13 |
Thanks also to all the contributors on the debian-cd mailing list. |
| 14 |
|
| 15 |
What is needed? |
| 16 |
=============== |
| 17 |
Software: |
| 18 |
- the apt-get (>= 0.3.11.1) tool |
| 19 |
- perl (>= 5.004) |
| 20 |
- bash (or another POSIX shell) |
| 21 |
- make |
| 22 |
- cpp |
| 23 |
- mkisofs/genisoimage |
| 24 |
- dpkg-dev |
| 25 |
- wget |
| 26 |
- the perl Digest::MD5 module |
| 27 |
- the perl Compress::Zlib module |
| 28 |
- lynx (for text version of README.html) and todos from sysutils |
| 29 |
to convert docs to DOS format (although you can rip that out, too) |
| 30 |
- if you want to generate jigdo files: jigdo-file (see below) |
| 31 |
Other: |
| 32 |
- apt-utils (for apt-ftparchive) |
| 33 |
- lots of free space on your disks |
| 34 |
- a Debian mirror, on a partition where you can write. |
| 35 |
If you can't write on it then you may try to use a symlink farm, |
| 36 |
but it's not the recommended way to build Debian CDs. |
| 37 |
|
| 38 |
|
| 39 |
------------------------------- |
| 40 |
- GENERATING DEBIAN CD IMAGES - |
| 41 |
------------------------------- |
| 42 |
|
| 43 |
|
| 44 |
For the people that don't have time, here's the quick explanation |
| 45 |
================================================================= |
| 46 |
|
| 47 |
Edit the CONF.sh and change the PATHs for the mirror and so on. |
| 48 |
$ sensible-editor CONF.sh |
| 49 |
$ . CONF.sh |
| 50 |
$ make distclean |
| 51 |
$ make status |
| 52 |
$ make official_images |
| 53 |
|
| 54 |
However, you really should consider reading further for more information. |
| 55 |
You can also take a look at build.sh and build_all.sh for an automated |
| 56 |
way of building CD images. |
| 57 |
|
| 58 |
The script easy-build.sh offers the easiest way to build a specific image |
| 59 |
or set of images, but is still very flexible and powerful. It is the |
| 60 |
recommended tool for building test images and for people new to debian-cd. |
| 61 |
See the file README.easy-build for further info. |
| 62 |
|
| 63 |
How to build a CD set - step by step |
| 64 |
==================================== |
| 65 |
|
| 66 |
If you haven't already, change to the /usr/share/debian-cd/ directory |
| 67 |
(or, alternatively, set the variable BASEDIR in CONF.sh to point |
| 68 |
there). |
| 69 |
|
| 70 |
The process of building a CD is composed of the following steps: |
| 71 |
|
| 72 |
- first configure what is needed in CONF.sh and source it in your shell: |
| 73 |
|
| 74 |
$ . CONF.sh |
| 75 |
|
| 76 |
The exported environment variables will be used by all the |
| 77 |
tools involved here (Makefiles, perl scripts, shell scripts). |
| 78 |
|
| 79 |
If you want to build CD images for more than one arch, you will |
| 80 |
have to build them one after the other (you may use a shell |
| 81 |
script for this). |
| 82 |
|
| 83 |
Note that the temporary dir must be on the same device as the |
| 84 |
mirror because debian-cd uses hardlinks for generating an image |
| 85 |
tree. If you can't do this, you'll have to use a symlink farm. |
| 86 |
The symlink farm is explained at the end of this README. |
| 87 |
|
| 88 |
Keep in mind that the environment variables will stay in the |
| 89 |
environment after your debian-cd run, and may interfere with |
| 90 |
other program using the same variables (e.g. kernel-package). |
| 91 |
So if you want to be 100% safe, run debian-cd in a |
| 92 |
separate shell that you can leave after you're done. |
| 93 |
|
| 94 |
- then we clean everything that may still be there from previous runs: |
| 95 |
|
| 96 |
$ make distclean |
| 97 |
|
| 98 |
- then we initialize the temporary directory used for the build: |
| 99 |
|
| 100 |
$ make status |
| 101 |
|
| 102 |
If this has failed then this will be automatically launched: |
| 103 |
|
| 104 |
$ make correctstatus |
| 105 |
|
| 106 |
Note however that "make status" should never fail if it is |
| 107 |
used for building a CD set for the stable release... |
| 108 |
|
| 109 |
- now you can decide what you want on your CDs |
| 110 |
|
| 111 |
Note that task files are always taken from the subdirectory in ./tasks/ |
| 112 |
that matches the CODENAME environment variable. At the beginning of a |
| 113 |
build these "static" task files are copied to the working directory. |
| 114 |
During the build some additional task files - that are referenced from |
| 115 |
the static task files - are generated automatically using scripts from |
| 116 |
the ./tools directory. |
| 117 |
|
| 118 |
Examples: |
| 119 |
|
| 120 |
$ make packagelists TASK=Debian-generic COMPLETE=1 |
| 121 |
|
| 122 |
or |
| 123 |
|
| 124 |
$ make packagelists TASK=Debian-kde COMPLETE=0 |
| 125 |
|
| 126 |
or |
| 127 |
|
| 128 |
$ export NONFREE=1; make packagelists TASK=your-task-here COMPLETE=1 |
| 129 |
|
| 130 |
or for something like an official image for the USA (without non-free): |
| 131 |
|
| 132 |
$ make packagelists COMPLETE=1 |
| 133 |
|
| 134 |
.... take a look at the file tasks/* to see the options you can have :) |
| 135 |
|
| 136 |
You can change the behaviour of this command with the following |
| 137 |
variables: |
| 138 |
|
| 139 |
- if NONFREE is set, then packages from the components defined in |
| 140 |
NONFREE_COMPONENTS will be allowed (NONFREE and NONFREE_COMPONENTS |
| 141 |
must be exported to all sub-shells) |
| 142 |
- if EXTRANONFREE is set, then similarly non-free packages will be |
| 143 |
included on an extra CD (the last CD in fact). Don't use NONFREE |
| 144 |
and EXTRANONFREE at the same time! (EXTRANONFREE must be exported |
| 145 |
to all sub-shells) |
| 146 |
- if FORCE_FIRMWARE=1, then firmware packages are included on the |
| 147 |
first disc (most useful if NONFREE_COMPONENTS is set and exported |
| 148 |
to all sub-shells) |
| 149 |
- if COMPLETE is set, all packages that are not listed in the |
| 150 |
selected task file will be included at the end |
| 151 |
- setting INSTALLER_CD will use an appropriate task file for |
| 152 |
building small CDs (businesscard and netinst) |
| 153 |
|
| 154 |
- now, we'll start making temporary trees: |
| 155 |
|
| 156 |
$ make image-trees |
| 157 |
|
| 158 |
This will first work sort the list of packages for each architecture |
| 159 |
into order so that standard, required, important and base packages |
| 160 |
are placed first, then other packages will be added in the order |
| 161 |
given modulo dependency ordering. Once the sorted list is created, |
| 162 |
the different architecture lists will be merged (if more than one |
| 163 |
architecture is selected). |
| 164 |
|
| 165 |
Then the code will start laying out temporary directory trees for |
| 166 |
the CDs. In order, this includes the following steps: |
| 167 |
|
| 168 |
- Creating an empty directory layout |
| 169 |
- Generating an image label and volume ID and other metadata such as |
| 170 |
debian-installer information files |
| 171 |
- Add documentation and installation/upgrade tools |
| 172 |
- Add Release files and other archive metadata |
| 173 |
- Make the image bootable for the selected architecture(s) |
| 174 |
- Start generating the md5sum.txt file |
| 175 |
|
| 176 |
If you want to use boot-floppies built by yourself you can add |
| 177 |
a parameter BOOTDISKS=<dir> which specifies the directory where |
| 178 |
they are. Note that $BOOTDISKS/current must be a symlink to the |
| 179 |
real directory and it must follow the same setup as the FTP |
| 180 |
mirror. Your boot-floppies must also be on the same partition as |
| 181 |
your mirror and temporary dir (hardlinks are used here too). |
| 182 |
|
| 183 |
Once the disc tree has all of this start data, we start filling the |
| 184 |
directory trees with packages from the sorted list. The size of the |
| 185 |
image to be created is set using DISKTYPE in CONF.sh; if the |
| 186 |
standard sizes do not match what you're after, use DISKTYPE=CUSTOM |
| 187 |
and specify your own size using CUSTOMSIZE. The algorithm is simple |
| 188 |
for adding packages: |
| 189 |
|
| 190 |
- link the package into the temporary disc tree |
| 191 |
- append the metadata to the appropriate Packages or Sources file |
| 192 |
- add md5sum information for the added file(s) to the md5sum.txt file |
| 193 |
|
| 194 |
This continues until the temporary tree grows one package *too |
| 195 |
large* for the selected image size. At that point, we roll back the |
| 196 |
last set of changes associated with that package. Then: |
| 197 |
|
| 198 |
- check if the disc contains all the packages needed to install a base system |
| 199 |
- finish off the Release file, using the checksums of the |
| 200 |
Packages/Sources files we generated |
| 201 |
- finish off the md5sum.txt file |
| 202 |
|
| 203 |
Next, we continue to the next disc tree, using the same process: |
| 204 |
start it, copy packages in until they overflow, roll back and |
| 205 |
finish. And repeat. Each time a package is found to be too large to |
| 206 |
fit inside an image, it will be kept back and will (obviously) be |
| 207 |
the first package placed into the next disc tree. |
| 208 |
|
| 209 |
- now we can create the images: |
| 210 |
|
| 211 |
$ make images |
| 212 |
|
| 213 |
If you don't have enough space for all images, you can generate |
| 214 |
only one image (of the second CD for example) with: |
| 215 |
|
| 216 |
$ make image CD=2 |
| 217 |
|
| 218 |
Note: here we use "make images", but you could as well use |
| 219 |
"make official_images" since the latter is the same as the former |
| 220 |
with some dependencies on targets that you already launched |
| 221 |
(make bootable packages sources). |
| 222 |
|
| 223 |
- if you want to generate checkum files with the checksums of the |
| 224 |
images you can do it with: |
| 225 |
|
| 226 |
$ make imagesums |
| 227 |
|
| 228 |
Official images |
| 229 |
=============== |
| 230 |
|
| 231 |
If you use make official_images you're building CD images that have |
| 232 |
the same properties than official CD images but they still doesn't |
| 233 |
have the name of "Official Images". The name of the images is given |
| 234 |
by setting the OFFICIAL and DEBVERSION environment variable (check |
| 235 |
CONF.sh). |
| 236 |
|
| 237 |
Please never ever use the "Official" name for a CD image that you |
| 238 |
built yourself. The only images that can be called "Official" are the |
| 239 |
ones built by Debian itself and which are provided on Debian's |
| 240 |
servers. |
| 241 |
|
| 242 |
The default configuration shipped with this package will automatically |
| 243 |
name the images "Unofficial". CD will work exactly in the same way |
| 244 |
with all Debian tools, only the label is different. That means you |
| 245 |
can use build.sh and build_all.sh to build your "Unofficial" images |
| 246 |
without modifying anything. |
| 247 |
|
| 248 |
|
| 249 |
Local packages |
| 250 |
============== |
| 251 |
|
| 252 |
If you provide some custom made packages and you want to put them on |
| 253 |
Debian CD set you can do it. Simply put your packages in |
| 254 |
$MIRROR/dists/$CODENAME/local/binary-$ARCH/<section>. |
| 255 |
The organization of this sub-tree is the same than what you can find |
| 256 |
in the main, contrib or non-free part. You may use different section |
| 257 |
names if you want. Be sure to create Packages files (and Sources.gz if you |
| 258 |
include sources). |
| 259 |
|
| 260 |
You can also put your local packages under $MIRROR/pool/local (just a new |
| 261 |
facility for people who don't want packages under dists/). To include local |
| 262 |
packages, the LOCAL environment variable must be set to "1" while building |
| 263 |
the CDs. |
| 264 |
|
| 265 |
You can also set the LOCALDEBS environment variable, and it will be used |
| 266 |
instead of MIRROR when looking for local packages. |
| 267 |
|
| 268 |
|
| 269 |
Additional targets |
| 270 |
================== |
| 271 |
|
| 272 |
Jigdo |
| 273 |
----- |
| 274 |
You may also want to make the CD images available in jigdo format. |
| 275 |
Jigsaw Download, the successor to the Pseudo-Image Kit. |
| 276 |
See https://www.debian.org/CD/jigdo-cd/ for more information on jigdo. |
| 277 |
|
| 278 |
Set the DOJIGDO and related variables in CONF.sh. This is no separate target |
| 279 |
for jigdo, merely a modification of the "images" targets. You can choose |
| 280 |
only .iso generation (default), only .jigdo generation (for highly reduced |
| 281 |
disk usage), or both .iso and .jigdo generation. |
| 282 |
|
| 283 |
To generate the jigdo files and templates, you need an |
| 284 |
appropriately-patched version of mkisofs/genisoimage, as shipped in |
| 285 |
Debian Etch. |
| 286 |
|
| 287 |
The checksums files generated by the "imagesums" target will contain |
| 288 |
the checksums of all generated images, regardless of the DOJIGDO |
| 289 |
setting. If no full iso image is available, the checksums will be |
| 290 |
extracted from the .template files. A note in the Makefile shows how |
| 291 |
the original file size can be extracted from the .template in a |
| 292 |
similar way. |
| 293 |
|
| 294 |
About jigdo "fallback servers": |
| 295 |
|
| 296 |
jigdo works by downloading individual packages and other files from a |
| 297 |
normal Debian mirror, and using them to regenerate a CD/DVD image. |
| 298 |
However, the content of Debian mirrors changes over time, files are |
| 299 |
added and removed. But jigdo must have access to all files needed for |
| 300 |
the image it has to regenerate, even those that have been removed from |
| 301 |
the normal Debian mirrors. |
| 302 |
|
| 303 |
A fallback server contains a backup of the Debian FTP space for the |
| 304 |
moment the .jigdo files were generated. This backup is made available |
| 305 |
under a certain URL which is written to the .jigdo files. jigdo will |
| 306 |
*only* revert to the fallback server after an unsuccessful attempt to |
| 307 |
retrieve a file from the normal user-selected Debian mirror, so the |
| 308 |
bandwidth requirements are modest. |
| 309 |
|
| 310 |
A fallback is even necessary for .jigdo files of the stable release, |
| 311 |
because some files (typically documentation or boot floppies) can |
| 312 |
change at any time. |
| 313 |
|
| 314 |
debian-cd allows you to automatically create a directory on disc which |
| 315 |
is suitable for use as a fallback mirror. It is populated with hard |
| 316 |
links to the archive contents. In CONF.sh, simply supply as |
| 317 |
JIGDOFALLBACKPATH the name of the directory, and as JIGDOFALLBACKURLS |
| 318 |
the URLs under which it will be made available. |
| 319 |
|
| 320 |
About the hook system |
| 321 |
===================== |
| 322 |
|
| 323 |
A hook script can be executed at different times during the CD build |
| 324 |
process to customise your CDs. You can specify the script by setting |
| 325 |
the various HOOK variables in CONF.sh; look there for more information |
| 326 |
about what hook points are available. |
| 327 |
|
| 328 |
About the symlink farm |
| 329 |
====================== |
| 330 |
|
| 331 |
If you don't have write access on the disk where you have the mirror |
| 332 |
or if for another reason hardlink cannot be used, you can try to |
| 333 |
use a symlink farm. Instead of having real files, your temporary tree |
| 334 |
will be filled with symlinks that mkhybrid will change into files when |
| 335 |
it will build the image. You'll need to use a special options. You |
| 336 |
have 2 lines of options in CONF.sh as examples. |
| 337 |
|
| 338 |
I've never tested the symlink farm ... it may well generate unusable |
| 339 |
images. Don't use it for ISO images that will used by many users. |
| 340 |
|
| 341 |
Note that you will also need a patched mkhybrid that does support the |
| 342 |
-F option. Have a look here about it : |
| 343 |
https://www.chiark.greenend.org.uk/~stevem/DebianCD/ |