Roll out nightly and stable release packages #400
Assignees
Comments
|
Researching how to promote versions from dev/rc --> stable, without needing to rebuild wheels:
|
|
Nightly releases are now documented here: https://github.com/nod-ai/SHARK-Platform/blob/main/docs/nightly_releases.md |
|
We're making good progress on this. Latest detailed task list (in order of rollout): Publish
|
marbre
pushed a commit
that referenced
this issue
Nov 8, 2024
Progress on #400. Switching the versions to be earlier than the planned 3.0.0 will let us publish a 2.9.0 test release to PyPI and then increment the major version once we are comfortable with the new release process. I plan on deleting the existing 3.0.0 binaries that have been pushed to the "dev-wheels" release. (Yes, pre-releases are a thing: https://packaging.python.org/en/latest/specifications/version-specifiers/#pre-releases, but we want to trial the full release process)
|
I've been stuck all day trying to update iree-turbine to use the new iree-base-compiler and iree-base-runtime packages: iree-org/iree-turbine#258. There are new test failures that look meaningful. Without that change, the iree-turbine package will still depend on the old IREE package names, so users will download both We might actually be able to proceed with the redundant installs for a little bit, but I'm not sure what the implications will be. |
|
Figured out all the test failures and pushed iree-turbine 2.9.0: |
|
I'm going to call it for the day. Next steps for Monday:
Once we have that initially working, we can also add the shark-platform meta package that directly pins the versions of each package that we want, then do the same build -> publish to dev wheels -> push to pypi process for that package. |
ScottTodd
added a commit
that referenced
this issue
Nov 11, 2024
…#477) Progress on #400 This lets us take a release from https://github.com/nod-ai/SHARK-Platform/releases/tag/dev-wheels and "promote it" for publishing to https://pypi.org/ by turning the version from `X.Y.ZrcYYYYMMDD` to simply `X.Y.Z`. The script is forked from the one in IREE I added last week: iree-org/iree#19067. More steps like https://iree.dev/developers/general/release-management/#promoting-a-candidate-to-stable and scripting like https://github.com/iree-org/iree/blob/main/build_tools/python_deploy/pypi_deploy.sh will follow.
|
Initial 2.9.0 releases of all packages have been published to PyPI. More work remains to get the process running smoothly, but we should have enough for testing using standard pip commands with no extra package index or source builds. |
ScottTodd
added a commit
that referenced
this issue
Nov 15, 2024
Progress on #400. This scripting allows us to publish .whl files from https://github.com/nod-ai/SHARK-Platform/releases/tag/dev-wheels to [PyPI](https://pypi.org/). Here are the basic steps: 1. Download wheels for a specific pre-release (e.g. `2.9.1rc20241114`) 2. Edit the versions in the downloaded wheels to remove the `rcYYYYMMDD` suffix 3. Build the `shark-ai` meta package using the versions in those whls (NOTE: currently this uses the versions _in the source tree_, see below) 4. Upload all wheels to PyPI Logs of this running: https://gist.github.com/ScottTodd/9c7418d5bbbebc8aea72a39bc2ac37b0 (to push 2.9.1 to PyPI) The new `pypi_deploy.sh` script is based on the similar script we maintain in IREE: https://github.com/iree-org/iree/blob/main/build_tools/python_deploy/pypi_deploy.sh . The `README.md` file is also forked from IREE. Known sharp edges with the publishing process: * This currently mixes local information (versions from the source tree where this is running) with remote information (versions from the nightly release packages). Publishing nightly meta packages will let us simplify and make this safer to run from arbitrary working trees * The local build of the shark-ai meta package copies _all_ files in `shark-ai/build_tools/wheelhouse/` to the working directory that gets sent to `twine upload *`. If there are preexisting files in that directory they will be published. * The step that downloads releases from GitHub uses `pip download` so it can leverage PyPI's version resolution logic, but I couldn't figure out a way to download 3.13t wheels without running a 3.13t python interpreter. I happen to have 3.13t installed on my system, but we shouldn't require that in the scripting for all release engineers. We could try using the `gh` tool as in IREE, if we properly filter the download to just the version we want: https://github.com/iree-org/iree/blob/9eaa4ef7d6b439d8c444b533beddd82146578e25/build_tools/python_deploy/pypi_deploy.sh#L69-L72 (that has one release per nightly, while here we have a single release shared between all nightlies) --------- Co-authored-by: Marius Brehler <[email protected]>
|
Steps for 3.0.0 release next week:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Tasks
Target: this week (by Nov 1)
This will get us
pip install shark-platform -f https://github.com/nod-ai/SHARK-Platform/releases/expanded_assets/dev-wheelsTarget: next week (by Nov 8)
candidate/rcfrom names/versions)This will get us
pip install shark-platformBacklog
Wider support for Python versions
The text was updated successfully, but these errors were encountered: