Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

run() fails when the venv path is too long #20

Open
wosc opened this issue May 12, 2020 · 2 comments
Open

run() fails when the venv path is too long #20

wosc opened this issue May 12, 2020 · 2 comments
Labels
bug Something isn't working important Things that should be done earlier rather than later

Comments

@wosc
Copy link
Collaborator

wosc commented May 12, 2020

Currently, run() directly exec's the setuptools-generated entrypoint script, which in turn finds the correct python by including the full path to the {env_dir}/bin/python executable as a shebang line.

Shebang lines are limited to 127 characters, which is very easy to exceed with appenv, since the venv hash already takes up 64.

I propose to specify the python executable in appenv itself, instead of relying on the shebang line, like this:

diff --git a/batou b/batou
index 8f03df7..455def7 100755
--- a/batou
+++ b/batou
@@ -200,7 +200,10 @@ def run(argv, meta_args):
     env_dir = _prepare(meta_args)
     # Allow called programs to find out where the wrapper lives
     os.environ['APPENV_BASEDIR'] = meta_args.base
-    os.execv(os.path.join(env_dir, 'bin', meta_args.appname), argv)
+    interpreter = os.path.join(env_dir, 'bin', 'python')
+    entrypoint = os.path.join(env_dir, 'bin', meta_args.appname)
+    argv[0] = interpreter
+    argv.insert(1, entrypoint)
+    os.execv(interpreter, argv)
 

This works fine with batou, but argv[0] is not preserved (since at least under Debian we must set argv[0] to the venv/bin/python, otherwise the venv is not recognized and the spawned python runs with the system site-packages), so programs that are interested in that will have trouble. Not sure if that's something we have to worry about, though.

@ctheune
Copy link
Contributor

ctheune commented May 13, 2020

Ok, so one thing I'm up for is to to actually reduce the venv hash to 8 characters, similar to git's short hash. This should still be reliable enough according to 'the internet' and 'stackoverflow'. I'm with that. It's not cryptographically secure but I don't think that matters here as we're basically using it for content addressing.

Did you try the most current version of appenv? I'm running batou on debian machines and think I have fixed the site-packages issue already through a pth hack... or does that fail for you?

@ctheune ctheune added bug Something isn't working important Things that should be done earlier rather than later labels May 13, 2020
@wosc
Copy link
Collaborator Author

wosc commented May 14, 2020

This occurs with dc208ac, which is the current master, I'm afraid. Here's a reproduction recipe:

mkdir -p /tmp/appenv/work/workspace/somewhat-long-jenkins-job
cd /tmp/appenv/work/workspace/somewhat-long-jenkins-job
curl -o batou https://raw.githubusercontent.com/flyingcircusio/appenv/dc208ac9df334962b013001c60cd93fb88949c98/src/appenv.py
chmod +x batou
echo 'batou==2.0b11' > requirements.txt
./batou appenv-update-lockfile
./batou --help

which then yields

Creating venv ...
Ensuring pip ...
Installing batou ...
Traceback (most recent call last):
  File "./batou", line 312, in <module>
    main()
  File "./batou", line 308, in main
    meta_args.func(argv, meta_args)
  File "./batou", line 203, in run
    os.execv(os.path.join(env_dir, 'bin', meta_args.appname), argv)
OSError: [Errno 8] Exec format error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working important Things that should be done earlier rather than later
Projects
None yet
Development

No branches or pull requests

2 participants