Security: restrict custom HTTP request headers to specific URL patterns

[sanmai-NL]

Currently, the --header CLI and configuration parameter does not distinguish between target URLs. If the user were to supply a crucial secret in there, in order to make some URLs accessible during CI, then this secret would leak to all other hosts for which URLs are found.

Solution design

Rather than implementing URL/header mapping logic in Lychee, I propose to separate this concern into an, if you so choose, external tool like a proxy. If Lychee were to have proxying support, complex logic, mappings, analysis, flows, etc. can be configured through a proxy. If common use cases are documented in a how-to style within the Lychee docs, the value for the user would not be much less than with a native implementation by Lychee (one could argue, much more indeed).

[sanmai-NL]

An example of a tool that could support this functionality: https://docs.mitmproxy.org/stable/overview-features/#modify-headers

[mre]

I like the modify_headers syntax you linked to. We could add something like this.
Just to clarify, this doesn't require a proxy, but rather a way to pass these headers to reqwest, our HTTP request client, right?