Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to Debian Bookworm #193

Merged
merged 3 commits into from
Sep 16, 2023
Merged

Update to Debian Bookworm #193

merged 3 commits into from
Sep 16, 2023

Conversation

MarkKoz
Copy link
Member

@MarkKoz MarkKoz commented Sep 16, 2023

Mount /dev/shm in the container by setting ipc to "private". This is the same as "none" (the previous value) with the only difference being that shm is mounted. This is needed for integration tests to pass.

The integration tests always relied on shared memory due to their use of multiprocessing. They managed to work because glibc used to fall back to /tmp if /dev/shm wasn't available. However, newer versions of glibc, which Debian Bookworm now uses, removed that fallback behaviour.

Mount /dev/shm in the container by setting ipc to "private". This is
the same as "none" (the previous value) with the only difference being
that shm is mounted. This is needed for integration tests to pass.

The integration tests always relied on shared memory due to their use of
multiprocessing. They managed to work because glibc used to fall back to
/tmp if /dev/shm wasn't available. However, newer versions of glibc,
which Debian Bookworm now uses, removed that fallback behaviour.
@MarkKoz MarkKoz added the area: dependencies Related to package dependencies and management label Sep 16, 2023
@MarkKoz MarkKoz enabled auto-merge (rebase) September 16, 2023 02:20
@coveralls
Copy link

coveralls commented Sep 16, 2023

Coverage Status

coverage: 90.96% (+0.01%) from 90.947% when pulling 64e9ab5 on feat/deps/debian-bookworm into f763dba on main.

Copy link
Member

@ChrisLovering ChrisLovering left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Runs fine. Tested with it integrated tot he bot, works fine too. However ruining the tests locally, I get this failure.

FAIL: test_file_parsing_timeout (tests.test_nsjail.NsJailTests.test_file_parsing_timeout)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/snekbox/tests/test_nsjail.py", line 247, in test_file_parsing_timeout
    self.assertEqual(result.returncode, None)
AssertionError: 0 != None

Edit: I dound the issue after some further digging.

The test above is failing due to a race condition. There's a race between the timeout and the max file size. If your PC is fast enough, you'll parse the number of files needed to hit the max file size limit before the timeout of 1s is hit, so the test fails.

The only way I've found to make this pass on my PC is to double memfs_instance_size on that test.

This will be helpful in testing, since some machines are able to
process quite a lot in one second, which was the previous lowest
timeout.
Updating to Bookworm may have increased performance of the file
processing. In any case, this test started failing intermittently on
when running on a local machine. Lower the timeout so even fast
machines will hit the timeout.
@MarkKoz MarkKoz merged commit bc70851 into main Sep 16, 2023
7 checks passed
Copy link
Member

@ChrisLovering ChrisLovering left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works well, thanks Mark!

@MarkKoz MarkKoz deleted the feat/deps/debian-bookworm branch September 16, 2023 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: dependencies Related to package dependencies and management
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants