Skip to content

Release Management

Jump to bottom Edit New page
Zach Hoffman edited this page Nov 5, 2021 · 13 revisions

Release Schedule

Traffic Control strives for quarterly releases. For example:

Quarter Release Version Scope Release Manager Start Date End Date
Q1 4.1 X John Doe January 1st March 31st
Q2 5.0 X Jane Doe April 1st June 30th
Q3 TBD X TBD July 1st September 30th
Q4 TBD X TBD October 1st December 31st

Software Prerequisites

Before beginning the process of creating a release candidate for the first time, ensure you have the following software.

  • gpg
  • svn
  • b2sum - Comes with most Linux distributions by default, Mac/OSX users should install the coreutils HomeBrew package: brew install coreutils
  • sha512sum - Comes with most Linux distributions by default, Mac/OSX users should install the coreutils HomeBrew package: brew install coreutils
  • perl version 5.x

Pre-Release

On the start date, the designated RM should start the conversation (via the dev and users mailing lists) about what the goals for the upcoming release will be. Based on this email conversation, the RM can optionally use issues and/or milestone(s) to capture the scope for the release. At a minimum, most larger items should be identified. If the scope of the release is large or contains breaking changes to one or more components, then a major release is probably proposed. If the release is smaller, then a minor release is probably proposed.

Two weeks prior to the end date, the designated RM announces the upcoming release in the #traffic-control channel of the Apache Software Foundation Slack as well as the dev and users mailing lists.

On the end date, the designated RM cuts the release branch from master (5.0.x for example) and builds the first release candidate (RC0) and starts the process of getting the release approved.

If this is your first release as RM, you must place your GPG Public Key into the KEYS file via SVN in https://dist.apache.org/repos/dist/release/trafficcontrol like so:

svn co -N https://dist.apache.org/repos/dist/release/trafficcontrol
$EDITOR trafficcontrol/KEYS
svn status # to make sure the KEYS file shows as modified
svn commit -m "Update KEYS file for Release Manager"

For information on exporting a key in the same format seen in that KEYS file, refer to the documentation for your GPG implementation (hint: Linux users probably have GnuPG).

Release Process Overview

The following steps assume a fictional release of 3.0.0 is already in production and the 3.1.0 release is about to become a formal release. The typical flow is to build an "RC" that can be voted on by the community, which means if there are bugs or enhancements that need to get into the release, then subsequent RC tags will ensue. Only fixes that correct problems are allowed through "Cherry Picks" for the RC: no enhancements or new features.

Once the community all votes "+1" on the RC (at least 3 "+1" votes from PMC members are required - which are referred to as "binding votes"), then a "Release" tag+build of the source tarball is created and gpg-signed for public consumption.

Release Management Steps

For each RC, the RM should follow these steps.

1. Prepare Release Build and Branch

  1. Update the CHANGELOG.md file on the master branch, commit, and push.

    • The current top-level "[unreleased]" section header should be changed to the version number of the release for which the candidate is being made and the date at which its branch was cut e.g. "## [3.1.0] - 2018-10-30"
    • At the bottom, update the "[unreleased]" link to e.g. "[3.1.0]: https://github.com/apache/trafficcontrol/compare/RELEASE-3.0.0...RELEASE-3.1.0". This link shows a "diff" between the new and prior versions.

  2. Update Traffic Router pom.xml files on the master branch to the new version. For an example, see commit c1d901e

  3. Verify that the VERSION file in the repository's root on the master branch matches the version you want to release. If it doesn't already match, then update it.

  4. Create a new branch for the new release (if one does not already exist) e.g. for the 3.1.0 release:

    # Note this assumes the name of the https://github.com/apache/trafficcontrol
# remote is "upstream" - substitute for whatever name you use.
git fetch --prune upstream
git checkout upstream/master
git checkout -b 3.1.x
git push -u upstream 3.1.x

  5. On the newly created branch, update references to the "master" tag on the builder images to the version branch you want to cut e.g. 3.1.x (if cutting the 3.1.0 release) in infrastructure/docker/build/docker-compose.yml.

  6. Run the misc/release.pl script to cut the new release branch off master e.g. for the 0th RC of version 3.1.0:

    # Note this should be run from the repository's root.

# This will print a summary and ask for confirmation that you
# want to continue with the release.
./misc/release.pl --gpg-key="$KEYID" --release-no=RELEASE-3.1.0-RC0 cut

    Where KEYID contains the ID of the GPG key to be used to sign the new tag.

    This script will automate the following high level steps:

    1. Checkout the repository to /tmp/trafficcontrol
    2. Create a new release branch, or use an existing branch for managing the release if one already exists
    3. Tag the release with the release nnumber option given (e.g. RELEASE-3.1.0-RC0)
    4. Update the VERSION file according to the parameters given and commit it
    5. Push the new release (or tag the existing) upstream to the public Apache Traffic Control repository

  7. Once the tag is pushed, verify it like so:

    # Note this assumes the name of the https://github.com/apache/trafficcontrol
# remote is "upstream" - substitute for whatever name you use.
git fetch --tags upstream
git tag -v RELEASE-3.1.0-RC0

    This should print something like:

    remote: Enumerating objects: 708, done.
remote: Counting objects: 100% (333/333), done.
remote: Total 708 (delta 332), reused 332 (delta 332), pack-reused 375
Receiving objects: 100% (708/708), 115.10 KiB | 990.00 KiB/s, done.
Resolving deltas: 100% (483/483), completed with 190 local objects.
From github.com:apache/trafficcontrol
* [new tag]             RELEASE-3.1.0-RC0 -> RELEASE-3.1.0-RC0
object 66199be84c02ea62a952168ce8c9e68e0d2db79a
type commit
tag RELEASE-3.0.1-RC0
tagger Derek Gelinas <[email protected]> 1539188455 -0400

Release 3.1.0
gpg: Signature made Wed Oct 10 12:20:55 2018 EDT
gpg:                using RSA key FE2B6AD8A1A214D9FF0AE5D6209FA95905BE71D9
gpg: Good signature from "Derek Gelinas <[email protected]>" [ultimate]

    If the key is trusted and has a good signature, then you can move on to creating the release candidate. If the signature is not good, you need to ensure that your key is signed and trusted. One RM had a trusted key, but recently reinstalled his Operating System, which resulted in his key not showing as trusted. To resolve this, he ran the following commands:

    $ gpg --edit-key 05be71d9
gpg (GnuPG/MacGPG2) 2.2.10; Copyright (C) 2018 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

sec  rsa4096/209FA95905BE71D9
     created: 2018-04-24  expires: 2022-04-24  usage: SC
     trust: full          validity: unknown
ssb  rsa4096/767A5938418CB1
     created: 2018-04-24  expires: 2022-04-24  usage: E
[ unknown] (1). Derek Gelinas <[email protected]>

gpg> trust
sec  rsa4096/209FA95905BE71D9
     created: 2018-04-24  expires: 2022-04-24  usage: SC
     trust: full          validity: unknown
ssb  rsa4096/767A5938418CB1
     created: 2018-04-24  expires: 2022-04-24  usage: E
[ unknown] (1). Derek Gelinas <[email protected]>

Please decide how far you trust this user to correctly verify other users' keys
(by looking at passports, checking fingerprints from different sources, etc.)

  1 = I don't know or won't say
  2 = I do NOT trust
  3 = I trust marginally
  4 = I trust fully
  5 = I trust ultimately
  m = back to the main menu

Your decision? 5
Do you really want to set this key to ultimate trust? (y/N) y

sec  rsa4096/209FA95905BE71D9
     created: 2018-04-24  expires: 2022-04-24  usage: SC
     trust: ultimate      validity: unknown
ssb  rsa4096/767A5938418CB1
     created: 2018-04-24  expires: 2022-04-24  usage: E
[ unknown] (1). Derek Gelinas <[email protected]>
Please note that the shown key validity is not necessarily correct
unless you restart the program.

gpg> quit
$ git push --delete origin RELEASE-3.1.0-RC1

    At this point, it was necessary to recreate the tag for the RC.

2. Create Release Candidate

Now everything is in place so that a RC can be crafted from the new tag and branch.

  1. Build the new RC from the branch that was created for it, probably using pkg:

    # Note this should be run from the repository's root.
./pkg -v source weasel

    Built tarballs from GitHub Actions artifacts cannot be substituted for building source here.

  2. Verify that the dist/weasel.txt file generated by the build is an empty plaintext file. If it is not empty, its contents will detail licensing problems that must be addressed before continuing.

  3. Sign the RC tarball that was created in the dist/ directory with the same GPG key used to sign the release tag - e.g. for the 0th RC of the 3.1.0 release:

    gpg --armor --detach-sig apache-trafficcontrol-3.1.0.tar.gz
b2sum apache-trafficcontrol-3.1.0.tar.gz > apache-trafficcontrol-3.1.0.tar.gz.b2
sha512sum apache-trafficcontrol-3.1.0.tar.gz > apache-trafficcontrol-3.1.0.tar.gz.sha512

  4. Post the new RC to the Apache distribution repo under "development". Once the vote passes, the tarball should be renamed and uploaded under "release". Nothing else other than the filename may change- the release signatures must match the voted on signatures exactly.

    # For approved releases:
# svn co -N https://dist.apache.org/repos/dist/release/trafficcontrol/
# For release candidates:
svn co -N https://dist.apache.org/repos/dist/dev/trafficcontrol
cd trafficcontrol
mkdir -p 3.1.0/RC0
mv /path/to/apache-trafficcontrol-3.1.0.tar.gz.* ./3.1.0/RC0 # Should be 3 files: tgz, b2, sha512, asc
svn add  3.1.0 # (or svn add RC0 if 3.1.0 is already added)
svn status   # Confirm files are added to working area
svn commit --username "$ASF_USERNAME" -m "apache-trafficcontrol-3.1.0-RC0"

    Where ASF_USERNAME contains the Apache Software Foundation username of the RM.

  5. The Apache Traffic Control PMC must vote and approve on a release candidate. Three "binding" (PMC Member) votes are needed for release. Non-PMC votes do not count toward the three vote requirement - but are encouraged nonetheless to increase confidence in the release, and also to show involvement in the community for hopeful new committers.

Vote Email Template
To: [email protected]
Subject: [VOTE] Release Apache Traffic Control {{ATC Version}}-RC{{RC Number}}

Hello All,

I've prepared a release for {{ATC Version}}-RC{{RC Number}}. Changes since RELEASE-{{Previous ATC Version}}:

	https://github.com/apache/trafficcontrol/compare/RELEASE-{{Previous ATC Version}}...RELEASE-{{ATC Version}}-RC{{RC Number}}
	https://github.com/apache/trafficcontrol/blob/RELEASE-{{ATC Version}}-RC{{RC Number}}/CHANGELOG.md

The artifacts are available for download at:

	https://dist.apache.org/repos/dist/dev/trafficcontrol/{{ATC Version}}/RC{{RC Number}}/

BLAKE2 checksum:
{{BLAKE2 Checksum}}

SHA512 checksum:
{{SHA512 Checksum}}

This corresponds to git refs:

	Hash: {{Commit Hash}}
	Tag: RELEASE-{{ATC Version}}-RC{{RC Number}}

Which can be verified with the following command:

	$ git tag -v RELEASE-{{ATC Version}}-RC{{RC Number}}

All code signing keys are available here:

	https://dist.apache.org/repos/dist/release/trafficcontrol/KEYS

Make sure you refresh from a key server to get all relevant signatures.

The vote is open until 22:00 UTC on Wednesday, September 8 and passes if a majority of at least 3 +1 PMC votes are cast.
	[ ] +1 approve
	[ ] +0 no opinion
	[ ] -1 disapprove (and reason why)

Thanks!
{{Your Name}} {{Your Email Address}}

3. Get the vote result

  1. Go to the dev mailing list archives
  2. Click on the vote email
  3. Click the yellow "permalink" button
  4. Copy the URL to which that takes you
  5. Send a vote result email if the vote passed - if not, address the issues and create a new RC (no email required to notify of a failed vote).
Vote Result Email Template
To: [email protected]
Subject: [RESULT][VOTE] Release Apache Traffic Control {{version}}-RC{{RC Number}}

Thanks to all who voted!

The release has PASSED with the following PMC votes:

+1 {{voter's name}} (binding)
+1 {{voter's name}} (binding)
+1 {{voter's name}} (binding)

Voting results can be found here: {{Vote email thread permalink}}

I will proceed to publish the release and send ANNOUNCE.

On behalf of Apache Traffic Control, thank you!

Regards,
{{Your Name}} {{Your Email Address}}

4. Finalizing a Release

  1. Confirm one last time that there are no critical issues.

  2. Create or Update release notes (GitHub) using the version notes in CHANGELOG.md.

  3. Create a new release on Github using the misc/release.pl script.

  4. Update the release repository at http://dist.apache.org/repos/dist/release/trafficcontrol/ via SVN. You will want to remove any releases whose branches are no longer under development (supported by the ATC security policy). These will automatically be mirrored, so your actual download links on the website will point to https://www.apache.org/dyn/closer.cgi/trafficcontrol/{{release}}/{{file}}.

    svn co http://dist.apache.org/repos/dist/release/trafficcontrol/
cd trafficcontrol
mkdir 3.1.0
cp /path/to/apache-trafficcontrol-3.1.0.tar.gz* ./3.1.0/ # should be four files: .tar.gz, .tar.gz.asc, .tar.gz.b2, and .tar.gz.sha512
svn add 3.1.0
svn delete 3.0.2 # if necessary
svn status
svn commit -m "apache-trafficcontrol-3.1.0"

    Update the news, releases, and default profiles on the project website via the apache/trafficcontrol-website repository, on the asf-site branch.

    Activate the documentation build on ReadTheDocs - or ask someone with access to do so for you (e.g. ocket8888) or to give you access so that you can do it yourself. If doing it yourself:

    1. Login to ReadTheDocs or register for an account.

      1. Contact one of the PMC members to add your user to the Traffic Control docs project.
      2. Click Versions from the top menu.
      3. Find your version in the list of inactive versions, click "Edit", and activate the version. The version should be built and made available in a few minutes.

  5. Send the announcement email, detailed below.

  6. Update the SECURITY.md file on master for the latest release. Generally, we support the latest minor version of the latest two major versions. For example, see PR #4805.

Announcement Email Template
To: [email protected]; [email protected]; [email protected]
Subject: [ANNOUNCE] Release Apache Traffic Control {{version}}

The Apache Traffic Control team is proud to announce the release of Apache Traffic Control {{version}},
which contains various new features and bug fixes.

Apache Traffic Control allows you to build a large scale content delivery network using open source.
Built around Apache Traffic Server as the caching software, Traffic Control implements all the core
functions of a modern CDN.

More details regarding Apache Traffic Control can be found at:

https://trafficcontrol.apache.org/

The release artifacts, along with release notes, can be found here:

https://trafficcontrol.apache.org/releases/


Thanks!

The Apache Traffic Control Team
Add a custom footer
Add a custom sidebar
Clone this wiki locally