"Lighter" repetitive HTTP GETs

[lud0v1c]

Describe The Problem To Be Solved

When performing a concurrent download, some hosts might be listed several times, as many vendors list several packs/pdsc files. Initiating multiple fast HTTP connections and GET requests might be seen as a DoS or threat by them, potentially blacklisting the user or denying the download.

(Optional): Suggest A Solution

If the same base domain/URL is detected, all downloads after the second one should have a random or user set waiting period between the next downloads of this host. For simplicity's sake maybe set it to a random range and make it configurable via a global flag.