| 1 |
.. _imap-developer-major-releasing: |
| 2 |
|
| 3 |
===================================== |
| 4 |
Releasing Cyrus IMAP - major releases |
| 5 |
===================================== |
| 6 |
|
| 7 |
.. contents:: |
| 8 |
|
| 9 |
These instructions describe the process of turning what was the master |
| 10 |
branch into a new release series, and making the first release from it. |
| 11 |
|
| 12 |
For normal point releases, see :ref:`imap-developer-releasing` |
| 13 |
|
| 14 |
Prerequisites |
| 15 |
============= |
| 16 |
|
| 17 |
Same as for normal releases. |
| 18 |
|
| 19 |
Feature freeze |
| 20 |
============== |
| 21 |
|
| 22 |
This is the period where new features are not being merged to the master |
| 23 |
branch. It usually starts at the start of December, and continues until |
| 24 |
the new series has its own cyrus-imapd-x.y branch, which is usually forked |
| 25 |
in early January. |
| 26 |
|
| 27 |
Once the new series has its own branch, any bug fixes will need to be |
| 28 |
developed against master and then backported to the new branch. In |
| 29 |
contrast, bug fixes that happen during the feature freeze only need to land |
| 30 |
once, so take advantage of this window to focus on those. |
| 31 |
|
| 32 |
The feature freeze also reduces races between the release manager doing the |
| 33 |
various tasks of a release and other developers merging changes. |
| 34 |
|
| 35 |
Once the new series has its own branch, normal feature development can resume |
| 36 |
on master -- developing features for the *next* major release. |
| 37 |
|
| 38 |
Make sure master is good |
| 39 |
======================== |
| 40 |
|
| 41 |
With the master branch checked out and up to date: |
| 42 |
|
| 43 |
1. Ensure your git repository is clean, using something like |
| 44 |
``git clean -xfd``. Note that this command will destroy any uncommitted |
| 45 |
work you might have, so make sure your ducks are in line before proceeding. |
| 46 |
2. Generate a configure script: ``autoreconf -i -s`` |
| 47 |
3. Generate everything else: ``./configure --enable-maintainer-mode`` (you do |
| 48 |
not need any other options at this stage). |
| 49 |
4. Run ``make distcheck``. This will generate a distribution tarball, and |
| 50 |
test it in various ways. It takes about 10-15 mins to run, depending on |
| 51 |
your hardware. If you usually build Cyrus with a script that sets PATH etc, |
| 52 |
you will need to provide the same environment at this step. For example, |
| 53 |
ellie uses an alias like this for this step: |
| 54 |
|
| 55 |
``alias distcheck="PATH=/usr/local/cyruslibs/bin:$PATH make distcheck"``. |
| 56 |
|
| 57 |
If ``make distcheck`` fails, you are not ready to proceed -- fix the |
| 58 |
problems, get them tested and committed, then restart this testing. |
| 59 |
5. ``make distcheck`` can only test so much (it doesn't know about cunit or |
| 60 |
cassandane), so you also need to check the tarball against those. |
| 61 |
|
| 62 |
i. The tarball will be called something like |
| 63 |
``cyrus-imapd-3.0.0-rc2-23-g0241b22.tar.gz`` |
| 64 |
(this corresponds to the ``git describe`` output). |
| 65 |
ii. Extract it: ``tar xfz cyrus-imapd-*.tar.gz`` |
| 66 |
(substitute version for ``*``). |
| 67 |
iii. Change into the directory: ``cd cyrus-imapd-*`` |
| 68 |
iv. Configure it: ``./configure [...]`` (provide the same arguments and |
| 69 |
environment that you would when building for Cassandane at any other |
| 70 |
time). |
| 71 |
v. Compile it: ``make -j4`` -- it should build correctly. |
| 72 |
vi. Run the unit tests: ``make -j4 check`` -- they should pass. |
| 73 |
vii. Install it to your Cassandane prefix: ``make install`` |
| 74 |
viii. Change into the `cassandane` directory within the extracted source |
| 75 |
(not the git source!): ``cd cassandane`` |
| 76 |
ix. Build Cassandane's binary components: ``make -j4`` |
| 77 |
x. Run Cassandane: ``./testrunner.pl`` |
| 78 |
xi. If any of this fails, get it fixed and merged, then redo this testing |
| 79 |
|
| 80 |
|
| 81 |
Forking the new series branch |
| 82 |
============================= |
| 83 |
|
| 84 |
You will find (e.g. with ``git describe`` when viewing the master branch) that |
| 85 |
the master branch has a version with an odd number in the second field, e.g. |
| 86 |
3.\ **7**\ . The new series branch should be named one number higher than |
| 87 |
this, making it an even number. Thus, if master is currently 3.7, then the new |
| 88 |
series will be 3.8 (and then master will become 3.9). |
| 89 |
|
| 90 |
1. Make sure your repository and master branch are up to date |
| 91 |
2. Checkout the master branch: ``git checkout master`` |
| 92 |
3. Create and check out the new series branch: |
| 93 |
``git checkout -b cyrus-imapd-<series>`` |
| 94 |
4. Edit `docsrc/conf.py`. Update all the versioning information to say that: |
| 95 |
|
| 96 |
- this is version `<series>.0-alpha0` |
| 97 |
- the current stable version is `<series>.0-alpha0` (i.e. this one) |
| 98 |
- the previous stable version is whatever the current stable version |
| 99 |
used to be |
| 100 |
- the latest development version is the next odd number up from what it used |
| 101 |
to be, as a `.0-alpha0` -- that is, if it used to be `3.7.something`, |
| 102 |
it is now `3.9.0-alpha0`. |
| 103 |
- (these are all lies right now, but they will become true as we go) |
| 104 |
- find `html_theme_options` and update the option that configures which |
| 105 |
branch to show for the build status badge to be this branch, not "master" |
| 106 |
- Also add a suitable entry to the `extlinks` table near the bottom of the |
| 107 |
file. |
| 108 |
|
| 109 |
5. Update `docsrc/index.rst` to state that this is the stable version, not |
| 110 |
the development one. It's easiest to just copy and update the text from the |
| 111 |
previous stable version of this file. |
| 112 |
6. Add release notes infrastructure: |
| 113 |
|
| 114 |
a. Make the directories for the new series: |
| 115 |
``mkdir -p docsrc/imap/download/release-notes/<series>/x`` |
| 116 |
(note the `x`, it's important for some historical reason) |
| 117 |
b. Make the directories for the new dev version: |
| 118 |
``mkdir -p docsrc/imap/download/release-notes/<dev>/x`` |
| 119 |
c. Create `docsrc/imap/download/release-notes/<version>/index.rst` |
| 120 |
for each of these, with stub contents. It's easiest to just |
| 121 |
copy and update from an older one. |
| 122 |
d. Add stub release notes for alpha0. This will be a file called |
| 123 |
`docsrc/imap/download/release-notes/<series>/x/<series>.0-alpha0.rst`. |
| 124 |
If we've been doing dev snapshots from master, start by copying the |
| 125 |
release notes from the most recent one of those. If we haven't, then |
| 126 |
you will be starting with a blank document, in which case it's easiest |
| 127 |
to copy the release notes file from the previous major release, delete |
| 128 |
all the bullet points (leaving just the headings), and fix all the |
| 129 |
numbers. |
| 130 |
|
| 131 |
7. Update `README.md`: |
| 132 |
|
| 133 |
- It will be claiming to be the development version, but this is now (or |
| 134 |
will be) the stable version, so update that. If in doubt, mimic what |
| 135 |
the old stable branch's copy says. This is another set of lies that will |
| 136 |
become true as we go. |
| 137 |
- Search through the whole document for version numbers, and update them |
| 138 |
sensibly for the future reality. Do this mindfully, not with a batch |
| 139 |
find-replace. |
| 140 |
- The stable "build status" badge at the very top should reference the real |
| 141 |
stable version for now. This gets shown on GitHub rather than our own |
| 142 |
site, so it can't lie. |
| 143 |
- This is also a good time for a careful review of the contents of this |
| 144 |
file. Fix anything that's out of date, missing, etc. |
| 145 |
|
| 146 |
8. Make sure your RST changes are good: ``make doc-html``. Pay attention |
| 147 |
to any errors or warnings (they will be coloured). There will be some |
| 148 |
you can clearly ignore, such as glob patterns for future release notes |
| 149 |
that don't exist yet, but do your best to deal with everything else. |
| 150 |
The generated documentation will be under the `doc/html/` directory -- |
| 151 |
examine it in your browser to make sure all your formatting and such makes |
| 152 |
sense. |
| 153 |
9. XXX maybe missing some stuff here still? |
| 154 |
|
| 155 |
You can double check your work by looking at what changed last time a new |
| 156 |
stable series was forked: |
| 157 |
``git show --format=fuller cyrus-imapd-<oldstable>.0-alpha0``. |
| 158 |
Also look a few commits forward, in case the previous releaser |
| 159 |
missed steps before tagging, and had to catch them up later. |
| 160 |
|
| 161 |
Once you're satisfied that you've done everything that needed doing here: |
| 162 |
|
| 163 |
1. Commit all these changes. A single commit is good, we would like this to |
| 164 |
be the very first commit after the fork point. |
| 165 |
2. Create a signed, annotated tag declaring that this is now alpha0 of the .0 |
| 166 |
release of the new series: |
| 167 |
``git tag -s cyrus-imapd-<series>.0-alpha0`` |
| 168 |
3. You will be prompted to enter a commit message for the tag (this is |
| 169 |
what makes it an "annotated" tag). Ellie uses something like "not a real |
| 170 |
release, but need a tag for versioning". |
| 171 |
4. You will also be prompted to enter the pass phrase for your GPG key, do it. |
| 172 |
5. Push the new branch: ``git push ci cyrus-imapd-<series>`` (assuming your |
| 173 |
remote is named "ci") |
| 174 |
6. Push the new tag: ``git push ci cyrus-imapd-<series>.0-alpha0`` |
| 175 |
|
| 176 |
Fastmail specific: also push the new branch and tag to the Fastmail repo. |
| 177 |
|
| 178 |
Updating the master branch |
| 179 |
========================== |
| 180 |
|
| 181 |
You now need to make similar, but not identical, changes to the master branch, |
| 182 |
too. |
| 183 |
|
| 184 |
1. Check out the master branch: ``git checkout master`` |
| 185 |
2. Edit `docsrc/conf.py`: Make all the same changes as you did before, except |
| 186 |
that: |
| 187 |
|
| 188 |
- version and release should reflect that this is the development version, |
| 189 |
not the new stable version |
| 190 |
- XXX anything else? |
| 191 |
|
| 192 |
3. Create the release notes directories and populate their stub index files. |
| 193 |
Note that in this case you're doing both the new series stubs, and the new |
| 194 |
dev series stubs. You need to do both, because someday this will be a |
| 195 |
stable version, and the website will need all the historical release notes. |
| 196 |
4. Remove all files except the template from `changes/next/`. These will be |
| 197 |
new features in the new release, which means they're no longer new on the |
| 198 |
master branch. An exception is if there are changes currently on master |
| 199 |
that will be reverted from the new branch after forking -- in that case, |
| 200 |
don't delete those changes files from master. More on this later. |
| 201 |
5. Update `README.md`. |
| 202 |
6. XXX probably steps missing here too |
| 203 |
7. Make sure the RST changes are good: ``make doc-html``, pay attention |
| 204 |
to errors and warnings. |
| 205 |
|
| 206 |
You may think you can do this by cherry-picking your commit from the new |
| 207 |
release branch and then amending it with the dev version differences... and you |
| 208 |
can, but do so very cautiously, because the differences between these branches |
| 209 |
are important. |
| 210 |
|
| 211 |
You can double check your work by looking at what changed *on master* last time |
| 212 |
a new series forked. As before, look a few commits ahead too, in case the |
| 213 |
previous releaser missed steps before tagging. |
| 214 |
|
| 215 |
Once you're satisfied that you've done everything that needed doing here: |
| 216 |
|
| 217 |
1. Commit all these changes. A single commit is good, we would like this to |
| 218 |
be the very first commit after the fork point. |
| 219 |
2. Create a signed, annotated tag declaring that this is now alpha0 of the .0 |
| 220 |
"release" of the new development version: |
| 221 |
``git tag -s cyrus-imapd-<dev>.0-alpha0`` |
| 222 |
3. You will be prompted to enter a commit message for the tag (this is |
| 223 |
what makes it an "annotated" tag). Ellie uses something like "not a real |
| 224 |
release, but need a tag for versioning". |
| 225 |
4. You will also be prompted to enter the pass phrase for your GPG key, do it. |
| 226 |
5. Push the new commit |
| 227 |
6. Push the new tag: ``git push ci cyrus-imapd-<dev>.0-alpha0`` |
| 228 |
|
| 229 |
**Once this step is done, the feature freeze can end.** |
| 230 |
|
| 231 |
Fastmail specific: also push the updated master branch and new tag to the |
| 232 |
Fastmail repo. This ensures our builds will also start using the new version |
| 233 |
number once they update past the fork point. |
| 234 |
|
| 235 |
Github updates |
| 236 |
============== |
| 237 |
|
| 238 |
On Github, have a look at the branch protection settings that apply to the |
| 239 |
current stable branch. Apply the same protections to the new branch. |
| 240 |
|
| 241 |
Create labels for the new series and new dev series. Give them pleasant |
| 242 |
colours and sensible descriptions. |
| 243 |
|
| 244 |
- <series> |
| 245 |
- backport-to-<series> |
| 246 |
- <dev> |
| 247 |
|
| 248 |
Also update the description of the label for the old master version number. |
| 249 |
|
| 250 |
Revert anything that's not yet ready |
| 251 |
==================================== |
| 252 |
|
| 253 |
If there are commits on master that need to remain on master, but are not |
| 254 |
yet ready for release for some reason, this is a good point to revert those |
| 255 |
commits on the new branch only. Any `changes/next` files from these commits |
| 256 |
should remain on master, or be copied back to master if they were accidentally |
| 257 |
deleted earlier. |
| 258 |
|
| 259 |
This doesn't and shouldn't happen often. |
| 260 |
|
| 261 |
Tell the website builder about the new branch |
| 262 |
============================================= |
| 263 |
|
| 264 |
The website is automatically rebuilt by a script, which needs to be updated |
| 265 |
to know about the new series. |
| 266 |
|
| 267 |
1. Clone ``[email protected]:cyrusimap/cyrusimap.github.io.git``, or ensure the |
| 268 |
clone you already have is up to date |
| 269 |
2. Update ``build-cyrus-site`` to know about the new version. You do this by |
| 270 |
adding a ``cyrus_branch()`` call to the ``%sources`` hash. Do NOT change |
| 271 |
the line for current stable at this stage. We'll change that later, once |
| 272 |
the real release has been published. In the meantime, we want the top level |
| 273 |
and stable sections to continue to be built from the existing stable branch, |
| 274 |
but we want the website for the new branch to be available at ``/x.y``. |
| 275 |
3. You can check your work by comparing your changes to previous commits |
| 276 |
4. Commit and push your changes (or submit a PR). There is a Github Action |
| 277 |
on this repository that rebuilds the website once per hour, using the |
| 278 |
current version of this script, so your changes will take effect at the |
| 279 |
next run after they are merged to master. |
| 280 |
5. You should now be able to access a version of the website built from the |
| 281 |
new branch at `https://www.cyrusimap.org/<version>/`. Check that in your |
| 282 |
browser, make sure it reports the correct new versions. |
| 283 |
|
| 284 |
First beta |
| 285 |
========== |
| 286 |
|
| 287 |
This work mostly happens on the new branch. |
| 288 |
|
| 289 |
1. Check through `lib/imapoptions` for options with `"UNRELEASED"` in any of |
| 290 |
their version fields. |
| 291 |
|
| 292 |
- Replace these with the version number of the eventual actual (non-beta) |
| 293 |
release. For example, if we're starting the 3.8 series, this will be |
| 294 |
"3.8.0". That is to say, the first real release that these changes will |
| 295 |
appear in. |
| 296 |
- If any have been missed, there will be warnings (in yellow) when trying |
| 297 |
to (re)generate `lib/imapopts.c`. You can run |
| 298 |
``touch lib/imapoptions && make lib/imapopts.c`` to check |
| 299 |
- Commit this change, and also cherry-pick it onto `master`. |
| 300 |
|
| 301 |
2. Copy the stub release notes that you made for `<series>.0-alpha0` into a new |
| 302 |
document for `<series>.0-beta1`. |
| 303 |
3. Review the contents of all the `changes/next/*` files. Flesh out the new |
| 304 |
release notes document accordingly. (Compare previous `...-beta1` release |
| 305 |
notes to get a sense of the tone and level of detail.) |
| 306 |
4. Review `docsrc/imap/download/upgrade.rst`, also with reference to the |
| 307 |
`changes/next/*` files. Make any necessary updates. We expect people |
| 308 |
upgrading to the new version to follow these instructions, so they'd better |
| 309 |
be as complete and correct as we can get them. |
| 310 |
5. Review `docsrc/imap/rfc-support.rst`, also with reference to the |
| 311 |
`changes/next/*` files, and make any necessary updates. Also compare this |
| 312 |
file with the version of it on the stable branch. Check for any changes |
| 313 |
that don't have an accompanying `changes/next` file, and if there are any, |
| 314 |
also add suitable release notes and/or upgrade documentation for those. |
| 315 |
6. Check your RST changes: ``make doc-html`` |
| 316 |
7. Once the documentation updates have been finalised, the `changes/next/*` |
| 317 |
files (except for the template) should be removed -- they are no longer |
| 318 |
changes. The history is a bit easier to read later if you commit the |
| 319 |
doc updates and the removal of the changes files in the same commit. |
| 320 |
8. Follow the :ref:`imap-developer-releasing` instructions to get |
| 321 |
`cyrus-imapd-<series>.0-beta1` released. |
| 322 |
|
| 323 |
Subsequent betas |
| 324 |
================ |
| 325 |
|
| 326 |
Monitor Github and the mailing lists for bug reports against the previous beta. |
| 327 |
Make fixes against master, then backport them to the new series branch. |
| 328 |
|
| 329 |
Periodically make new beta releases, as bugs are found and fixed. |
| 330 |
|
| 331 |
Remember that until the real release is really released, the release notes |
| 332 |
contain the changes since the previous *stable* version. This means each of |
| 333 |
the betas will start with copying the previous beta's release notes and |
| 334 |
adding any new details, without removing what was already there. |
| 335 |
|
| 336 |
Release candidates |
| 337 |
================== |
| 338 |
|
| 339 |
After a while, the flow of bug reports and fixes will dry up, and so we start |
| 340 |
cutting "release candidates" instead. These are effectively identical to |
| 341 |
betas, except we call them -rc1, -rc2, etc instead. The change in name |
| 342 |
reflects our increased confidence in the software and documentation. |
| 343 |
|
| 344 |
Release |
| 345 |
======= |
| 346 |
|
| 347 |
Oh boy, we've come a long way, haven't we! |
| 348 |
|
| 349 |
For this one, we've got a little more housekeeping to do. |
| 350 |
|
| 351 |
1. Follow the :ref:`imap-developer-releasing` normal release process as |
| 352 |
previous, again copy-and-updating the release notes from the last release |
| 353 |
candidate, except this time you're actually doing `<series>.0`, with no |
| 354 |
alpha, beta, or rc qualifiers. Don't send the announcement email just |
| 355 |
yet though. |
| 356 |
2. Remember how we lied about the new version being the stable release? |
| 357 |
We only did that on the new branch and master, though. `docsrc/conf.py` |
| 358 |
on each of the existing branches will still be announcing old version |
| 359 |
numbers in the "rst_prolog" section. Go through the old branches and |
| 360 |
update each's `docsrc/conf.py` to contain the same lie. Commit and push |
| 361 |
these as you go. |
| 362 |
3. Remember the `build-cyrus-site` script from the cyrusimap.github.io |
| 363 |
repository? Go and remove the ``[ '/<version>', '/', '/stable ]`` slug from |
| 364 |
the previous stable version and add it (changing the version!) to the new |
| 365 |
one. Once this is pushed, the next website rebuild will make it all true. |
| 366 |
4. Once the website is fully updated, send that announcement email. |
| 367 |
|
| 368 |
Post-release |
| 369 |
============ |
| 370 |
|
| 371 |
From now on, just follow the normal release process to make point releases. |
| 372 |
Release notes for point releases describe the difference between this point |
| 373 |
release and the previous, and are much more specific than those of major |
| 374 |
releases. |
| 375 |
|