Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pyenv to uv based python and venv #303

Merged
merged 79 commits into from
Sep 3, 2024
Merged

Conversation

coder351
Copy link
Contributor

@coder351 coder351 commented Aug 10, 2024

resolves #240 and #30

UV based python uses optimized build
CONFIG_ARGS = "'--build=x86_64-unknown-linux-gnu' '--host=x86_64-unknown-linux-gnu' '--prefix=/install' '--with-openssl=/tools/deps' '--with-system-expat' '--with-system-libmpdec' '--without-ensurepip' '--with-readline=editline' '--enable-shared' '--enable-optimizations' '--with-lto`

UV based python is set to 3.12.3.
PyQt5 uses prebuilt wheels.

TODOs

  • I see uv has the optimizations enabled by default, but can we explicitly enable them so we're not relying on defaults?
  • PyQt5: either need a script to build/pull the wheel or build/install in the Docker build again
  • final test on device
  • openpilot runs
  • python, python3, pip, pip3, scons all work in a shell logged in as comma

@coder351
Copy link
Contributor Author

@adeebshihadeh The remaining Todos and comments are resolved.

Copy link
Contributor

@adeebshihadeh adeebshihadeh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyQt5 needs to be cleaned up still, other than that LGTM

@coder351
Copy link
Contributor Author

@adeebshihadeh I have the refactored PyQt5 to rebuild each time. The build time was 5mins which includes the docker cache.

userspace/compile-pyqt5.sh Outdated Show resolved Hide resolved
userspace/compile-pyqt5.sh Outdated Show resolved Hide resolved
userspace/compile-pyqt5.sh Outdated Show resolved Hide resolved
@robin-reckmann
Copy link
Contributor

robin-reckmann commented Aug 28, 2024

pip install does currently not work. ERROR: Could not install packages due to an OSError: [Errno 30] Read-only file system: '/usr/local/venv/lib/python3.12/site-packages/<package_name>.

Customs shims are not loaded properly.

@robin-reckmann
Copy link
Contributor

Also uv does currently not have a shim. I think we should have one if we ever intend to use uv on device.

@coder351
Copy link
Contributor Author

@robin-reckmann I checked the build logs and compile-pyqt5.sh ran without any errors. I have added back the custom shims export path in userspace/files/profile.

export PYENV_ROOT="/usr/local/pyenv"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
export XDG_DATA_HOME="/usr/local"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is also not necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@coder351
Copy link
Contributor Author

@robin-reckmann PR is ready if no other issues or concerns please confirm for @adeebshihadeh.

@adeebshihadeh
Copy link
Contributor

This should've been a straightforward PR, but it's taken a lot of people's time now. Bounty is unlocked, @robin-reckmann you want to pick it up?

@andiradulescu andiradulescu reopened this Sep 3, 2024
@adeebshihadeh adeebshihadeh merged commit 503688b into commaai:master Sep 3, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Switch from pyenv to uv
4 participants