How to scan a file system containing multiple Python projects for license information

[goneall]

Question

I'm working on a project where I'm scanning repositories for security vulnerabilities and license information using Trivy.

For NPM and Python packages using PIP, the packages need to be installed prior to scanning for the licenses to be picked up.

For NPM, I can pre-process the directories and install the packages.

For Python, I reviewed the code and it looks like the package either needs to be installed globally or the system environment variable VIRTUAL_ENV needs to be set to the directory for a single virtual environment.

Since I'm dealing with multiple Python projects, there will be multiple virtual environments.

Any ideas on how I can pre-process or post-process to get a single SBOM with all of the Python project information?

One thought was to scan each project individually setting the VIRTUAL_ENV variable and produce one SBOM per project then running a top level trivy scan scanning for SBOMs. I'm wondering if there may be a more efficient / reliable approach.

Reference: lfscanning/scaffold#76

Target

Filesystem

Scanner

License

Output Format

SPDX

Mode

Standalone

Operating System

Linux Ubuntu

Version

No response

[knqyf263]

I tried to find a way to detect a virtual environment without activating it, but no luck.
https://stackoverflow.com/questions/22003769/get-virtualenvs-bin-folder-path-from-script

Also, the virtualenv path differs depending on tools.

root@6e6d7714820b:/myproject# poetry env info

Virtualenv
Python:         3.12.7
Implementation: CPython
Path:           /root/.cache/pypoetry/virtualenvs/myproject-tjKdtegc-py3.12
Executable:     /root/.cache/pypoetry/virtualenvs/myproject-tjKdtegc-py3.12/bin/python
Valid:          True

Base
Platform:   linux
OS:         posix
Python:     3.12.7
Path:       /usr/local
Executable: /usr/local/bin/python3.12

I wonder if there is a uniform way of identifying the path of the project's virtualenv. Currently, the only way I can think of is to scan each project after activating virtualenv and merge the SBOMs as you said.

@DmitriyLewen Do you have any ideas?

[DmitriyLewen]

When i added licenses for pip - i already thought about this case, but didn't find solution 😞

But I looked again - I think I know 1 workaround:
we check <pkg-name>-<version>.dist-info/METADATA" (

trivy/pkg/fanal/analyzer/language/python/pip/pip.go

Line 113 in e9b43f8

pkgDir := fmt.Sprintf("%s-%s.dist-info", pkgName, pkgVer)

) files.
You can create temp dir and copy METADATA files from all virtual envs to this directory.
You will use this temp dir as $VIRTUAL_ENV for Trivy scan.

In this case Trivy can detect all required licenses from this directory.

[goneall]

Thanks @DmitriyLewen - I'll try this out. From looking at the code, it looks like this should work.