[BUG] Continuous integration sometimes generates false "test failed" notifications

[hdwalters]

I recently received a random "Cargo & Clippy Test" email on my glob function branch, directing me to https://github.com/amber-lang/amber/actions/runs/10872830349/job/30168293449. This complains about a failed unit test, which I haven't touched:

---- tests::stdlib::test_stdlib_src_tests_stdlib_abs_ab stdout ----
thread 'tests::stdlib::test_stdlib_src_tests_stdlib_abs_ab' panicked at src/tests/mod.rs:18:28:
assertion `left == right` failed
  left: ""
 right: "1\n1"

I do not see how this unit test could possibly fail, as it doesn't access the file system or anything else external:

$ cat src/tests/stdlib/abs.ab 
import * from "std/math"

// Output 
// 1
// 1

main {
    echo abs(1);
    echo abs(-1);
}

[b1ek]

probably related to bash version

[hdwalters]

probably related to bash version

If that were the problem, surely it would succeed or fail consistently? I would expect the CI server to do something like spinning up a Docker instance, to run tests in a consistent environment.

[Mte90]

So it is a bug that happens sometimes.
Re run the jobs fixes it, I think that #381 could fix it but that PR is blocked because there is an issue in escaping with bash generated.

[Mte90]

I think that now postprocessor was fixed this issue is gone.
Just keep the ticket open for a while to close it later.

[hdwalters]

I think that now postprocessor was fixed this issue is gone. Just keep the ticket open for a while to close it later.

Please don't close it just yet; it seems to be an intermittent problem. I've occasionally had test failures on my own PC when I run cargo test, but these go away if I run cargo build first, especially when I'm switching Git branches.

[hdwalters]

Also, surely the absence of a postprocessor binary on the CI server would not cause intermittent failures as described above; it would either always succeed, or always fail.