Remove repository-service-tuf dependency from dev.txt

[facutuesca]

While working on #15871, which includes adding sigstore as a dependency to warehouse (in order to be able to verify attestations), the following dependency conflict came up:

The latest version of repository-service-tuf (included in requirements/dev.txt) pins tuf==3.1.0 as a dependency.
Whereas sigstore requires tuf~=4.0.0, creating a conflict.

The current PR adding sigstore as a dependency is currently a draft due to having to comment out the repository-service-tuf dependency to avoid the conflict.

While the conflict mentioned above was solved by repository-service-tuf==0.12.0b1, this newly released version has another conflict with warehouse, this time with securesystemslib:

 repository-service-tuf 0.12.0b1 has requirement securesystemslib[crypto]<1.0.0,>=0.31.0, but you have securesystemslib 1.0.0.

warehouse depends on securesystemslib==1.0.0 via boto3, whereas repository-service-tuf requires <1.0.0:

    # via boto3
securesystemslib==1.0.0 \
    --hash=sha256:50f5053e274066502da7785dfd12b21e61131ca6e8b57ecedd2da0d1e9cd66c1 \
    --hash=sha256:a6d118c24eae8227a1cf2d9c173f47956709958f601eeaa38e86f6505a31455e

This PR removes repository-service-tuf from the dev.txt dependencies.

cc @woodruffw @di @kairoaraujo

[woodruffw]

LGTM -- assuming this doesn't break anything, I think we're fine to remove this and re-add it when rstuf upgrades their tuf subdependency.

(One thing I just thought of -- make inittuf might use this. Can you see if that breaks with these changes?)

[facutuesca]

(One thing I just thought of -- make inittuf might use this. Can you see if that breaks with these changes?)

Ah yeah indeed, make inittuf now fails due to not finding rstuf:

docker compose run --rm web rstuf admin ceremony -b -u -f dev/rstuf/bootstrap.json --api-server http://rstuf-api
[+] Creating 4/0
 ✔ Container warehouse-redis-1          Running                                                                                                                                                                                 0.0s 
 ✔ Container warehouse-db-1             Running                                                                                                                                                                                 0.0s 
 ✔ Container warehouse-elasticsearch-1  Running                                                                                                                                                                                 0.0s 
 ✔ Container warehouse-stripe-1         Running                                                                                                                                                                                 0.0s 
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "rstuf": executable file not found in $PATH: unknown
make: *** [inittuf] Error 1

[woodruffw]

Gotcha -- given that it's not anywhere on the critical local development path, maybe that failure is acceptable. CC @di for thoughts 🙂

(Another more invasive option here would be to create an rstuf-dev.txt file, and create a separate base development image for it. But I don't think this is work the effort, given that this removal should be temporary until tuf gets updated within rstuf.)

[kairoaraujo]

Hi @woodruffw and @facutuesca, now the rstuf supports python-tuf 4.0.0

[woodruffw]

Hi @woodruffw and @facutuesca, now the rstuf supports python-tuf 4.0.0

Cool, we'll update here (@facutuesca and I are at a company offsite so it'll be a few days).

[facutuesca]

Hi @woodruffw and @facutuesca, now the rstuf supports python-tuf 4.0.0

@kairoaraujo @woodruffw
The newly released repository-service-tuf==0.12.0b1 has another conflict, this time with securesystemslib:

 repository-service-tuf 0.12.0b1 has requirement securesystemslib[crypto]<1.0.0,>=0.31.0, but you have securesystemslib 1.0.0.

warehouse depends on securesystemslib==1.0.0 via boto3, whereas repository-service-tuf requires <1.0.0:

    # via boto3
securesystemslib==1.0.0 \
    --hash=sha256:50f5053e274066502da7785dfd12b21e61131ca6e8b57ecedd2da0d1e9cd66c1 \
    --hash=sha256:a6d118c24eae8227a1cf2d9c173f47956709958f601eeaa38e86f6505a31455e

I'm updating the description to reflect this new conflict.

[woodruffw]

warehouse depends on securesystemslib==1.0.0 via boto3, whereas repository-service-tuf requires <1.0.0:

FWICT, this is slightly wrong 🙂 -- securesystemslib==1.0.0 comes from tuf:

https://github.com/theupdateframework/python-tuf/blob/0ac86c67ad9d5d229cddb04789592fbc1371caa2/pyproject.toml#L49

...which is a transitive dependency of sigstore.

So this is still a conflict, but the conflict is because tuf and repository-service-tuf depend on different versions of securesystemslib.

The newly released repository-service-tuf==0.12.0b1 has another conflict, this time with securesystemslib:

@kairoaraujo is it possible to get this resolved? As-is, we're still going to need to remove the repository-service-tuf dependency unless it can be made compatible with Warehouse's other dependencies.

[lukpueh]

is it possible to get this resolved? As-is, we're still going to need to remove the repository-service-tuf dependency unless it can be made compatible with Warehouse's other dependencies.

We definitely need to migrate repository-service-tuf (i.e. the RSTUF CLI) to securesystemslib 1.0.0 soon, but it might take some time.

To resolve the conflict here quickly, I suggest to temporarily replicate the cli code use by make inittuf in Warehouse and remove the dependency. It is only posting a static file to the RSTUF API. I can submit a PR later today.

[woodruffw]

To resolve the conflict here quickly, I suggest to temporarily replicate the cli code use by make inittuf in Warehouse and remove the dependency. It is only posting a static file to the RSTUF API. I can submit a PR later today.

Sounds good, thank you!

[facutuesca]

Fixed by #16098