Refactor generate release notes script for enhanced release process

[danehlim]

Summary

Refactor generate release notes script (i.e., generate-release-notes.sh) to fit with the enhanced release process being worked on in branch feature/shinkansen.

NOTE: This pull request will be merged into branch feature/shinkansen.

Implementation details

• Take in options/flags instead of having set arguments with a specific order for AL2 NVIDIA version, AL2 CUDA version, and AL1 containerd version
• Make AL2 NVIDIA version and AL2 CUDA version options optional, as with the enhanced release process AL2 GPU AMIs may not necessarily be released during every AMI release
• Define new option/flag --exclude-ami to specify what AMI variants are excluded in a given release, as with the enhanced release process any given AMI variant may not necessarily be included
• Only generate release notes for AMI variants that are included as part of the release
• Take into account information / file structure changes made to release variables as part of release.auto.pkvars.hcl refactor: creating 1 file per AMI platform #167
• Update script usage information according to the changes made
• For better readability and organization, define a main function and group commands living outside of a function into more granular functions

Testing

Manually pulled in the most recent release variables information from branch main and then tested:

• generating release notes for 20231219 release using ./generate-release-notes.sh --al2-gpu-nvidia-ver 535.129.03 --al2-gpu-cuda-ver 12.2.2 --al1-containerd-ver 1.4.13 command and comparing the generated release notes against the expected release notes using diff. No differences were found

Generated output

### Source AMI release notes
---
* [Amazon Linux 2023 release notes](https://docs.aws.amazon.com/linux/al2023/release-notes/relnotes.html)
* [Amazon Linux 2 release notes](https://docs.aws.amazon.com/AL2/latest/relnotes/relnotes-al2.html)
* [Amazon Linux release notes](https://aws.amazon.com/amazon-linux-ami/2018.03-release-notes)

### Changelog
---
https://github.com/aws/amazon-ecs-ami/blob/main/CHANGELOG.md#20231219

### Amazon ECS-optimized Amazon Linux 2023 AMI
---
#### AMD64
- AMI name: al2023-ami-ecs-hvm-2023.0.20231219-kernel-6.1-x86_64
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: al2023-ami-minimal-2023.3.20231218.0-kernel-6.1-x86_64
- Distribution al2023 release: 2023.3.20231218

#### ARM64
- AMI name: al2023-ami-ecs-hvm-2023.0.20231219-kernel-6.1-arm64
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: al2023-ami-minimal-2023.3.20231218.0-kernel-6.1-arm64
- Distribution al2023 release: 2023.3.20231218

#### Neuron
- AMI name: al2023-ami-ecs-neuron-hvm-2023.0.20231219-kernel-6.1-x86_64
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: al2023-ami-minimal-2023.3.20231218.0-kernel-6.1-x86_64
- Distribution al2023 release: 2023.3.20231218

### Amazon ECS-optimized Amazon Linux 2 AMI
---
#### AMD64 (Kernel 4.14)
- AMI name: amzn2-ami-ecs-hvm-2.0.20231219-x86_64-ebs
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: amzn2-ami-minimal-hvm-2.0.20231218.0-x86_64-ebs

#### ARM64 (Kernel 4.14)
- AMI name: amzn2-ami-ecs-hvm-2.0.20231219-arm64-ebs
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: amzn2-ami-minimal-hvm-2.0.20231218.0-arm64-ebs

#### Neuron (Kernel 4.14)
- AMI name: amzn2-ami-ecs-inf-hvm-2.0.20231219-x86_64-ebs
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: amzn2-ami-minimal-hvm-2.0.20231218.0-x86_64-ebs

#### GPU (Kernel 4.14)
- AMI name: amzn2-ami-ecs-gpu-hvm-2.0.20231219-x86_64-ebs
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- NVIDIA driver version: 535.129.03
- CUDA version: 12.2.2
- Source AMI name: amzn2-ami-minimal-hvm-2.0.20231218.0-x86_64-ebs

#### AMD64 (Kernel 5.10)
- AMI name: amzn2-ami-ecs-kernel-5.10-hvm-2.0.20231219-x86_64-ebs
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: amzn2-ami-minimal-hvm-2.0.20231218.0-x86_64-ebs

#### ARM64 (Kernel 5.10)
- AMI name: amzn2-ami-ecs-kernel-5.10-hvm-2.0.20231219-arm64-ebs
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: amzn2-ami-minimal-hvm-2.0.20231218.0-arm64-ebs

### Amazon ECS-optimized Amazon Linux AMI
---
The Amazon ECS-optimized Amazon Linux AMI is deprecated as of April 15, 2021. After that date, Amazon ECS will continue providing critical and important security updates for the AMI but will not add support for new features.

- AMI name: amzn-ami-2018.03.20231219-amazon-ecs-optimized
- ECS Agent version: [1.51.0](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.51.0)
- Docker version: 20.10.13
- Containerd version: 1.4.13
- Source AMI name: amzn-ami-minimal-hvm-2018.03.0.20231218.0-x86_64-ebs

• generating release notes using the --exclude-ami option/flag and confirming that the excluded AMI variants specified using that option are not included in the generated output

Example test

Input

./generate-release-notes.sh --al2-gpu-nvidia-ver 000.00.00 --al2-gpu-cuda-ver 00.0.0 --al1-containerd-ver 0.0.0 --exclude-ami al2023neu,al2inf

Output

### Source AMI release notes
---
* [Amazon Linux 2023 release notes](https://docs.aws.amazon.com/linux/al2023/release-notes/relnotes.html)
* [Amazon Linux 2 release notes](https://docs.aws.amazon.com/AL2/latest/relnotes/relnotes-al2.html)
* [Amazon Linux release notes](https://aws.amazon.com/amazon-linux-ami/2018.03-release-notes)

### Changelog
---
https://github.com/aws/amazon-ecs-ami/blob/main/CHANGELOG.md#20231219

### Amazon ECS-optimized Amazon Linux 2023 AMI
---
#### AMD64
- AMI name: al2023-ami-ecs-hvm-2023.0.20231219-kernel-6.1-x86_64
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: al2023-ami-minimal-2023.3.20231218.0-kernel-6.1-x86_64
- Distribution al2023 release: 2023.3.20231218

#### ARM64
- AMI name: al2023-ami-ecs-hvm-2023.0.20231219-kernel-6.1-arm64
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: al2023-ami-minimal-2023.3.20231218.0-kernel-6.1-arm64
- Distribution al2023 release: 2023.3.20231218

### Amazon ECS-optimized Amazon Linux 2 AMI
---
#### AMD64 (Kernel 4.14)
- AMI name: amzn2-ami-ecs-hvm-2.0.20231219-x86_64-ebs
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: amzn2-ami-minimal-hvm-2.0.20231218.0-x86_64-ebs

#### ARM64 (Kernel 4.14)
- AMI name: amzn2-ami-ecs-hvm-2.0.20231219-arm64-ebs
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: amzn2-ami-minimal-hvm-2.0.20231218.0-arm64-ebs

#### GPU (Kernel 4.14)
- AMI name: amzn2-ami-ecs-gpu-hvm-2.0.20231219-x86_64-ebs
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- NVIDIA driver version: 000.00.00
- CUDA version: 00.0.0
- Source AMI name: amzn2-ami-minimal-hvm-2.0.20231218.0-x86_64-ebs

#### AMD64 (Kernel 5.10)
- AMI name: amzn2-ami-ecs-kernel-5.10-hvm-2.0.20231219-x86_64-ebs
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: amzn2-ami-minimal-hvm-2.0.20231218.0-x86_64-ebs

#### ARM64 (Kernel 5.10)
- AMI name: amzn2-ami-ecs-kernel-5.10-hvm-2.0.20231219-arm64-ebs
- ECS Agent version: [1.79.2](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.79.2)
- Docker version: 20.10.25
- Containerd version: 1.6.19
- Source AMI name: amzn2-ami-minimal-hvm-2.0.20231218.0-arm64-ebs

### Amazon ECS-optimized Amazon Linux AMI
---
The Amazon ECS-optimized Amazon Linux AMI is deprecated as of April 15, 2021. After that date, Amazon ECS will continue providing critical and important security updates for the AMI but will not add support for new features.

- AMI name: amzn-ami-2018.03.20231219-amazon-ecs-optimized
- ECS Agent version: [1.51.0](https://github.com/aws/amazon-ecs-agent/releases/tag/v1.51.0)
- Docker version: 20.10.13
- Containerd version: 0.0.0
- Source AMI name: amzn-ami-minimal-hvm-2018.03.0.20231218.0-x86_64-ebs

• incorrect usage (i.e., not passing in AL2 GPU NVIDIA version when AL2 GPU release is required and/or AL2 GPU CUDA version is provided, not passing in AL2 GPU CUDA version when AL2 GPU release is required and/or AL2 GPU NVIDIA version is provided, not passing in AL1 containerd version when AL1 release is required, etc.)

Example test

Input

./generate-release-notes.sh --al2-gpu-cuda-ver 00.0.00 --exclude-ami al2023,al2023arm,al2023neu,al1

Output

Error: AL2 GPU CUDA version or AL2 GPU NVIDIA version is empty when releasing AL2 GPU

Usage:
  ./generate-release-notes.sh

Options:
--al2-gpu-nvidia-ver  (Optional) AL2 GPU NVIDIA version. If specified, then --al2-gpu-cuda-ver option is also required to be specified.
--al2-gpu-cuda-ver    (Optional) AL2 GPU CUDA version. If specified, then  --al2-gpu-nvidia optin is also required to be specified.
--al1-containerd-ver  (Optional) AL1 containerd version.
--exclude-ami         (Optional) comma separated list of AMI variants that are excluded in the release.

Example:
  ./generate-release-notes.sh --al2-gpu-nvidia-ver 000.00.00 --al2-gpu-cuda-ver 00.0.0 --al1-containerd-ver 0.0.0 --exclude-ami al2023neu,al2inf

New tests cover the changes: no

Description for the changelog

Refactor generate release notes script for enhanced release process

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[mye956]

nit: Perhaps we can include a main function. It's a bit hard to trace through the code and might be more helpful in debugging. Thoughts?

[mye956]

nit (non-blocking): Could we also keep all of the functions grouped up for better readability? (i.e. get_ami_details)

[danehlim]

@mye956:

nit: Perhaps we can include a main function. It's a bit hard to trace through the code and might be more helpful in debugging. Thoughts?
nit (non-blocking): Could we also keep all of the functions grouped up for better readability? (i.e. get_ami_details)

Thanks for your comment! That's reasonable, the main reason I had not included these were to follow the current convention / layout of the script and keep any such enhancements in scope of a separate pull request. However, upon second thought, it shouldn't take too much effort to include in scope of this PR defining a main function and group the commands living outside of a function into more granular functions. I will add these changes in.

[mye956]

lgtm, thanks!