Proxy-Authorization added

[dtalkachou]

I think it's an important feature because not always people have an unauthorized proxy.

We had the following error before:

scrapy.core.downloader.handlers.http11.TunnelError: Could not open CONNECT tunnel with proxy <proxy> [{'status': 407, 'reason': b'Proxy Authentication Required'}]

Please, update PyPI after merging this request.

[dkarelov]

Thanks, confirm it is working!

[8W9aG]

Thanks for this, can confirm its a big issue, this is forcing me to use forks rather than the main package

[dtalkachou]

@StasDeep, is it possible to merge and close this PR. Seems like it's an actual problem for some engineers.

[StanHENI]

@dtalkachou I don't have merge permissions on this repository.
@kmike ?

[kmike]

Unfortunately I lost permissions myself :( We should probably fork this repo to another place, likely scrapy-plugins.

[dtalkachou]

@kmike, Is there anybody else who has permissions?

[divtiply]

Proxy-Authorization header is set by Scrapy built-in HttpProxyMiddleware. @dtalkachou do you have it enabled?

[dtalkachou]

Seems like I didn't have it enabled.

Yes, after code reading I think that HTTPPROXY_ENABLED = True should solve the problem. If somebody checks this approach, let me know. And provide more details for developers who will have the same problem.