exago: Add v1.2.0 and patches for builds without python or tests. #41350

cameronrutherford · 2023-11-30T02:26:59Z

Closes xsdk-project/xsdk-issues#224 with back ports for previous exago versions that had python support.

Versions earlier than 1.0.0 do not have python or issue with testing variant.

cc @balay and @v-dobrev for xSDK

spackbot-app · 2023-11-30T02:27:10Z

@ryandanehy can you review this PR?

This PR modifies the following package(s), for which you are listed as a maintainer:

exago

cameronrutherford · 2023-11-30T02:36:53Z

@spackbot help

spackbot-app · 2023-11-30T02:36:55Z

You can interact with me in many ways!

@spackbot hello: say hello and get a friendly response back!
@spackbot help or @spackbot commands: see this message
@spackbot run pipeline or @spackbot re-run pipeline: to request a new run of the GitLab CI pipeline
@spackbot rebuild everything: to run a pipeline rebuilding all specs from source.
@spackbot fix style if you have write and would like me to run spack style --fix for you.
@spackbot maintainers or @spackbot request review: to look for and assign reviewers for the pull request.

I'll also help to label your pull request and assign reviewers!
If you need help or see there might be an issue with me, open an issue here

cameronrutherford · 2023-11-30T02:37:16Z

@spackbot fix style

spackbot-app · 2023-11-30T02:37:19Z

Let me see if I can fix that for you!

spackbot-app · 2023-11-30T02:38:31Z

I was able to run spack style --fix for you!

spack style --fix

==> Running style checks on spack
  selected: isort, black, flake8, mypy
==> Modified files
  var/spack/repos/builtin/packages/exago/package.py
==> Running isort checks
  isort checks were clean
==> Running black checks
reformatted var/spack/repos/builtin/packages/exago/package.py
All done! ✨ 🍰 ✨
1 file reformatted.
  black checks were clean
==> Running flake8 checks
  flake8 checks were clean
==> Running mypy checks
Success: no issues found in 606 source files
  mypy checks were clean
==> spack style checks were clean

Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request. If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.

I've updated the branch with style fixes.

v-dobrev · 2023-11-30T15:11:45Z

There are some CI failures. Maybe it is because you added a patch for version 1.2.0 but there's no such version defined in the exago package? The error message from Spack is confusing:

==> Error: Can't extrapolate a URL for version 1.1.0 because package exago defines no URLs

since version 1.1.0 is explicitly defined -- maybe it means 1.2.0.

cameronrutherford · 2023-11-30T16:00:50Z

There are some CI failures. Maybe it is because you added a patch for version 1.2.0 but there's no such version defined in the exago package? The error message from Spack is confusing:
==> Error: Can't extrapolate a URL for version 1.1.0 because package exago defines no URLs
since version 1.1.0 is explicitly defined -- maybe it means 1.2.0.

lol I don't know how we went this long with that release not being in our package...

Thanks for the help debugging :) ExaGO v1.2.0 has been added to our package.

v-dobrev · 2023-11-30T19:16:41Z

Hmm, the ci/gitlab-ci check is still failing with the same error:

==> Error: Can't extrapolate a URL for version 1.1.0 because package exago defines no URLs

It looks like my guess for the reason of the failure was incorrect. I'm not sure what the issue is ...

balay · 2023-11-30T20:26:05Z

Its a strange error

cc: @eugeneswalker

Perhaps you can try adding in url as the message says..

+    url = "https://github.com/pnnl/ExaGO/archive/refs/tags/v1.0.0.tar.gz"

cameronrutherford · 2023-11-30T21:04:57Z

Its a strange error

cc: @eugeneswalker

Perhaps you can try adding in url as the message says..
+    url = "https://github.com/pnnl/ExaGO/archive/refs/tags/v1.0.0.tar.gz"

Adding this as the URL to the git version I assume you mean? In addition to the tag and commit sha?

balay · 2023-12-01T14:11:01Z

This error comes up with xsdk build as well.

https://gitlab.com/xsdk-project/spack-xsdk/-/jobs/5655279511

==> Error: Can't extrapolate a URL for version 1.1.0 because package exago defines no URLs

I try a manual build - and see the same thing

balay@pj01:~/spack$ ./bin/spack install exago
<snip>
==> Error: Can't extrapolate a URL for version 1.1.0 because package exago defines no URLs

balay · 2023-12-01T14:26:52Z

Ok - the issue here is version is a key word in spack - and can't be reused as a var name..

diff --git a/var/spack/repos/builtin/packages/exago/package.py b/var/spack/repos/builtin/packages/exago/package.py
index fa54c61b3c..139531197e 100644
--- a/var/spack/repos/builtin/packages/exago/package.py
+++ b/var/spack/repos/builtin/packages/exago/package.py
@@ -182,7 +182,7 @@ class Exago(CMakePackage, CudaPackage, ROCmPackage):
         depends_on("camp {0}".format(rocm_dep), when="+raja {0}".format(rocm_dep))
 
     # CMake patches to support ~python and ~testing
-    for version in [
+    for ver in [
         "1.6.0",
         "1.5.1",
         "1.5.0",
@@ -194,7 +194,7 @@ class Exago(CMakePackage, CudaPackage, ROCmPackage):
         "1.1.1",
         "1.1.0",
     ]:
-        patch("exago-{0}.patch".format(version), when="@{0}".format(version))
+        patch("exago-{0}.patch".format(ver), when="@{0}".format(ver))
 
     flag_handler = build_system_flags

var/spack/repos/builtin/packages/exago/package.py

balay · 2023-12-01T15:00:03Z

~~spack install [email protected] gives:~~

Sorry was building from a bad snapshot..

balay · 2023-12-01T15:27:11Z

var/spack/repos/builtin/packages/exago/package.py

+    # CMake patches to support ~python and ~testing
+    for ver in [
+        "1.6.0",
+        "1.5.1",
+        "1.5.0",
+        "1.4.1",
+        "1.4.0",
+        "1.3.0",
+        "1.2.0",
+        "1.1.2",
+        "1.1.1",
+        "1.1.0",
+    ]:
+        patch("exago-{0}.patch".format(ver), when="@{0}".format(ver))


Probably good to remove duplicate patches from here.

Suggested change

# CMake patches to support ~python and ~testing

for ver in [

"1.6.0",

"1.5.1",

"1.5.0",

"1.4.1",

"1.4.0",

"1.3.0",

"1.2.0",

"1.1.2",

"1.1.1",

"1.1.0",

]:

patch("exago-{0}.patch".format(ver), when="@{0}".format(ver))

# CMake patches to support ~python and ~testing

patch("exago-1.6.0.patch", when="@1.6.0")

patch("exago-1.5.0.patch", when="@1.5.0:1.5.1")

patch("exago-1.3.0.patch", when="@1.3.0:1.4.1")

patch("exago-1.1.0.patch", when="@1.1.0:1.2.0")

using exago-1.5.0.patch with [email protected] works for me.

==> Installing exago-1.5.1-tifh5k45ktj56fnib52gf5udrc2qqzh6 [39/39] ==> No binary for exago-1.5.1-tifh5k45ktj56fnib52gf5udrc2qqzh6 found: installing from source ==> Applied patch /home/balay/spack/var/spack/repos/builtin/packages/exago/exago-1.5.0.patch ==> exago: Executing phase: 'cmake' ==> exago: Executing phase: 'build' ==> exago: Executing phase: 'install' ==> exago: Successfully installed exago-1.5.1-tifh5k45ktj56fnib52gf5udrc2qqzh6 Stage: 1m 8.27s. Cmake: 2.87s. Build: 0.86s. Install: 0.26s. Post-install: 0.05s. Total: 1m 12.54s [+] /home/balay/spack/opt/spack/linux-fedora39-skylake/gcc-13.2.1/exago-1.5.1-tifh5k45ktj56fnib52gf5udrc2qqzh6

using exago-1.3.0.patch with [email protected] also works - but has subsequent build errors.

==> Installing exago-1.4.1-7vpnhzzxsrth7ahzm4pwdvgvpgybcdtf [34/34] ==> No binary for exago-1.4.1-7vpnhzzxsrth7ahzm4pwdvgvpgybcdtf found: installing from source ==> Applied patch /home/balay/spack/var/spack/repos/builtin/packages/exago/exago-1.3.0.patch ==> exago: Executing phase: 'cmake' ==> exago: Executing phase: 'build' ==> Error: ProcessError: Command exited with status 2: 'make' '-j12' 6 errors found in build log: 434 [ 78%] Linking CXX executable tcopflow 435 cd /home/balay/spack/spack-stage/spack-stage-exago-1.4.1-7vpnhzzxsrth7ahzm4pwdvgvpgybcdtf/spack-build-7vpnhzz/applications && /usr/bin/cmake -E cmake_link_script CMakeFiles/app_tcopflow.dir/link.txt --verbose=1 436 /software/mpich-4.1.1/bin/mpic++ -O3 -DNDEBUG CMakeFiles/app_tcopflow.dir/tcopflow_main.cpp.o -o tcopflow -Wl,-rpath,/home/balay/spack/opt/spack/linux-fedora39-skylake/gcc-13.2.1/petsc-3.16.6-diy2horf5ykuk5kbohcixnvkkxegfehn /lib:/home/balay/spack/opt/spack/linux-fedora39-skylake/gcc-13.2.1/netlib-lapack-3.11.0-mbofv6x6fzhx6yragjt3crpcjhrwocp7/lib64::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::: ../src/tcopflow/libexago_tcopflow.a ../src/opflow/libexago_opflow.a /usr/lib/gcc/x86_64-redhat-linux/13/libgomp.so /usr/lib64/libpthread.a /home/balay/spack/opt/spack/linux-fedora39-skylake/gcc-13.2.1/hiop-1.0.1- i7dp5ulk4wfb5iqhmgjab7wowocdhszr/lib/libhiop.a ../src/pflow/libexago_pflow.a ../src/ps/libexago_ps.a ../src/utils/libexago_utils.a ../tpl/spdlog/libspdlog.a /home/balay/spack/opt/spack/linux-fedora39-skylake/gcc-13.2.1/petsc- 3.16.6-diy2horf5ykuk5kbohcixnvkkxegfehn/lib/libpetsc.so /home/balay/spack/opt/spack/linux-fedora39-skylake/gcc-13.2.1/netlib-lapack-3.11.0-mbofv6x6fzhx6yragjt3crpcjhrwocp7/lib64/liblapack.so /home/balay/spack/opt/spack/linux- fedora39-skylake/gcc-13.2.1/netlib-lapack-3.11.0-mbofv6x6fzhx6yragjt3crpcjhrwocp7/lib64/libblas.so 437 make[2]: Leaving directory '/home/balay/spack/spack-stage/spack-stage-exago-1.4.1-7vpnhzzxsrth7ahzm4pwdvgvpgybcdtf/spack-build-7vpnhzz' 438 [ 78%] Built target app_tcopflow 439 In file included from /home/balay/spack/spack-stage/spack-stage-exago-1.4.1-7vpnhzzxsrth7ahzm4pwdvgvpgybcdtf/spack-src/src/scopflow/solver/hiop/scopflow_hiop.cpp:5: >> 440 /home/balay/spack/spack-stage/spack-stage-exago-1.4.1-7vpnhzzxsrth7ahzm4pwdvgvpgybcdtf/spack-src/src/scopflow/solver/hiop/scopflow_hiop.h:34:10: error: conflicting return type specified for 'virtual size_t SCOPFLOWHIOPInterfa ce::get_num_rterms() const' 441 34 | size_t get_num_rterms() const; 442 | ^~~~~~~~~~~~~~

so I re-tried a build of all versions [with the current PR changes]

balay@petsc-gpu-02:/scratch/balay/spack.z$ ./bin/spack -j64 install [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] |& tee build.log

The following version built fine

balay@petsc-gpu-02:/scratch/balay/spack.z$ ./bin/spack find exago -- linux-ubuntu22.04-zen4 / [email protected] -------------------------- [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] ==> 6 installed packages

And others failed.
build.log.txt

BTW: With my above suggested change [to 4 patch files] - the same versions are build (and with similar build errors for the other versions)

balay@petsc-gpu-02:/scratch/balay/spack.zz$ ./bin/spack find exago -- linux-ubuntu22.04-zen4 / [email protected] -------------------------- [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] ==> 6 installed packages

build2.log.txt

I pushed the above change to remove duplicate patch-files.

Perhaps exago-1.6.0.patch is a commit in the exago repo? If so - it can be changed to a URL from patch-file.

I'm not sure if [email protected] [email protected] [email protected] [email protected] failures should hold up this PR..

will go ahead and merge this in - so that 1.5.0 fix is in.

And I guess [email protected] [email protected] [email protected] [email protected] fixes - if any can be in a subsequent PR.

@balay did you create an issue for those exago build failures / can you make one in ExaGO?

I don't quite understand why you are seeing that error, but that would suggest we have the wrong HiOp versions listed as dependencies for those ExaGO versions, or we need to have a patch in ExaGO / HiOp to address build failures in those versions.

Either way thank you for merging, and apologies for the slow reply as I was taking some time off work.

@cameronrutherford, you can grab the logs from here [and steps to reproduce] - and file in your issue tracker [as needed]

I don't need fixes for those versions though..

pnnl/ExaGO#101

…ack#41350) Co-authored-by: Satish Balay <[email protected]>

exago: Add patches for builds without python or tests.

dc618b8

spackbot-app bot added patch update-package labels Nov 30, 2023

spackbot-app bot requested a review from pelesh November 30, 2023 02:27

cameronrutherford mentioned this pull request Nov 30, 2023

ExaGO build failures in xSDK CI xsdk-project/xsdk-issues#224

Closed

[@spackbot] updating style on behalf of cameronrutherford

ade4b42

cameronrutherford requested review from balay and v-dobrev November 30, 2023 02:42

exago: add version 1.2.0

0fdf996

spackbot-app bot added the new-version label Nov 30, 2023

cameronrutherford changed the title ~~exago: Add patches for builds without python or tests.~~ exago: Add v1.2.0 and patches for builds without python or tests. Nov 30, 2023

balay reviewed Dec 1, 2023

View reviewed changes

var/spack/repos/builtin/packages/exago/package.py Outdated Show resolved Hide resolved

var/spack/repos/builtin/packages/exago/package.py Outdated Show resolved Hide resolved

Apply suggestions from code review

542f820

balay added the xSDK label Dec 1, 2023

balay reviewed Dec 1, 2023

View reviewed changes

remov duplicate patches

937c3ac

alalazo assigned alalazo and balay Dec 11, 2023

balay approved these changes Dec 12, 2023

View reviewed changes

balay merged commit c673b92 into spack:develop Dec 12, 2023
13 checks passed

cameronrutherford mentioned this pull request Dec 12, 2023

ExaGO 1.2.0, 1.3.0, 1.4.0 and 1.4.1 fail to build with spack patch pnnl/ExaGO#101

Open

3 tasks

RikkiButler20 pushed a commit to RikkiButler20/spack that referenced this pull request Jan 31, 2024

exago: Add v1.2.0 and patches for builds without python or tests. (sp…

fb96f27

…ack#41350) Co-authored-by: Satish Balay <[email protected]>

kwryankrattiger pushed a commit to kwryankrattiger/spack that referenced this pull request Jul 9, 2024

exago: Add v1.2.0 and patches for builds without python or tests. (sp…

9017195

…ack#41350) Co-authored-by: Satish Balay <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

exago: Add v1.2.0 and patches for builds without python or tests. #41350

exago: Add v1.2.0 and patches for builds without python or tests. #41350

cameronrutherford commented Nov 30, 2023

spackbot-app bot commented Nov 30, 2023

cameronrutherford commented Nov 30, 2023

spackbot-app bot commented Nov 30, 2023

cameronrutherford commented Nov 30, 2023

spackbot-app bot commented Nov 30, 2023

spackbot-app bot commented Nov 30, 2023

v-dobrev commented Nov 30, 2023

cameronrutherford commented Nov 30, 2023 •

edited

Loading

v-dobrev commented Nov 30, 2023

balay commented Nov 30, 2023

cameronrutherford commented Nov 30, 2023

balay commented Dec 1, 2023

balay commented Dec 1, 2023

balay commented Dec 1, 2023 •

edited

Loading

balay Dec 1, 2023

balay Dec 1, 2023

balay Dec 9, 2023 •

edited

Loading

balay Dec 12, 2023

cameronrutherford Dec 12, 2023

balay Dec 12, 2023

cameronrutherford Dec 12, 2023

exago: Add v1.2.0 and patches for builds without python or tests. #41350

exago: Add v1.2.0 and patches for builds without python or tests. #41350

Conversation

cameronrutherford commented Nov 30, 2023

spackbot-app bot commented Nov 30, 2023

cameronrutherford commented Nov 30, 2023

spackbot-app bot commented Nov 30, 2023

cameronrutherford commented Nov 30, 2023

spackbot-app bot commented Nov 30, 2023

spackbot-app bot commented Nov 30, 2023

v-dobrev commented Nov 30, 2023

cameronrutherford commented Nov 30, 2023 • edited Loading

v-dobrev commented Nov 30, 2023

balay commented Nov 30, 2023

cameronrutherford commented Nov 30, 2023

balay commented Dec 1, 2023

balay commented Dec 1, 2023

balay commented Dec 1, 2023 • edited Loading

balay Dec 1, 2023

Choose a reason for hiding this comment

balay Dec 1, 2023

Choose a reason for hiding this comment

balay Dec 9, 2023 • edited Loading

Choose a reason for hiding this comment

balay Dec 12, 2023

Choose a reason for hiding this comment

cameronrutherford Dec 12, 2023

Choose a reason for hiding this comment

balay Dec 12, 2023

Choose a reason for hiding this comment

cameronrutherford Dec 12, 2023

Choose a reason for hiding this comment

cameronrutherford commented Nov 30, 2023 •

edited

Loading

balay commented Dec 1, 2023 •

edited

Loading

balay Dec 9, 2023 •

edited

Loading