Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some issues with the Editor Tests #966

Open
DeeDeeG opened this issue Mar 28, 2024 · 0 comments
Open

Some issues with the Editor Tests #966

DeeDeeG opened this issue Mar 28, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@DeeDeeG
Copy link
Member

DeeDeeG commented Mar 28, 2024

A couple of issues have been noticed in the Editor Tests.

  1. As mentioned in Improve "Project does not exists" popup #954 (comment), there are some silent fails possible, where an individual test will fail, the retry will happen but with an empty set of tests to run, for some reason, which "succeeds" with no failures and 100% specs skipped, which exits "cleanly", so CI thinks the run was good and "passes".

We should Ensure the re-run actually runs the intended tests... rather than having an empty set of tests to run and skipping 100% of tests, thereby false-passing the CI run. Or at least have the script error out if an error was detected and the set of tests we are about to re-run is malformed as empty.

  1. A false pass in this test?
AtomEnvironment > saving and loading > it serializes the project state with all the options supplied in saveState [1924:0328/191144.391928:INFO:CONSOLE(0)] "Uncaught (in promise) Error: ENOENT: no such file or directory, lstat '/private/var/folders/rd/9ccg_bdj4v949y62zjrcv9100000gn/T/dir2-2024228-1927-c8ixq1.py8ie'", source: file:///Users/user/pulsar/static/index.html (0)
[1924:0328/191144.392056:INFO:CONSOLE(0)] "Uncaught (in promise) Error: ENOENT: no such file or directory, lstat '/private/var/folders/rd/9ccg_bdj4v949y62zjrcv9100000gn/T/dir2-2024228-1927-c8ixq1.py8ie'", source: file:///Users/user/pulsar/static/index.html (0)
[1924:0328/191144.392144:INFO:CONSOLE(0)] "Uncaught (in promise) Error: ENOENT: no such file or directory, lstat '/private/var/folders/rd/9ccg_bdj4v949y62zjrcv9100000gn/T/dir1-2024228-1927-a6vtc6.egd1f'", source: file:///Users/user/pulsar/static/index.html (0)
[1924:0328/191144.392224:INFO:CONSOLE(0)] "Uncaught (in promise) Error: ENOENT: no such file or directory, lstat '/private/var/folders/rd/9ccg_bdj4v949y62zjrcv9100000gn/T/dir1-2024228-1927-a6vtc6.egd1f'", source: file:///Users/user/pulsar/static/index.html (0)
[pass]

(This looks more like it should be returning a "fail" result rather than a "pass" result, IMO? (Presumably there is some test assertion that coincidentally passes if nothing happens, and despite the run erroring out, the assertion passes?) Would be great to get this test properly running and true-passing, not false passing, if it is indeed having problems as it appears to be at a glance.)

  1. More? Worth doing a review of how well the editor tests are working.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant