error 429, but requests executes the request correctly [duplicate]

[AndrejedaI]

I previously asked this question, they told me that when responding, the server returns a response with the Retry-After header, where after what time can I make a request again. But then why does the requests library make a request for the same resource, with the same headers and ip, an unlimited* number of times and the server does not return Retry-After in response. What could this be connected with?

import asyncio
import aiohttp
import requests


headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:130.0) Gecko/20100101 Firefox/130.0',
    'Accept': '*/*',
    'Accept-Language': 'ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3',
    'Content-Type': 'text/plain;charset=UTF-8',
    'Origin': 'https://app.uniswap.org',
    'Retry-After': '5'
}

data = '{"tokenInChainId":1,"tokenIn":"ETH","tokenOutChainId":1,"tokenOut":"0xe41d2489571d322189246dafa5ebde1f4699f498","amount":"1000000000000000000","sendPortionEnabled":true,"type":"EXACT_INPUT","intent":"quote","configs":[{"useSyntheticQuotes":false,"routingType":"DUTCH_V2"},{"enableUniversalRouter":true,"protocols":["V2","V3","MIXED"],"routingType":"CLASSIC","enableFeeOnTransferFeeFetching":true}],"useUniswapX":true,"slippageTolerance":"0.5"}'


def sync_main():
    for i in range(10):
        s = requests.Session()
        s.headers.update(headers)
        response = s.post('https://interface.gateway.uniswap.org/v2/quote', data=data)
        print("requests ->", response.status_code, response.headers)


async def async_main():
    async with aiohttp.ClientSession(headers=headers) as session:
        for i in range(10):
            async with session.request("POST",
                                       'https://interface.gateway.uniswap.org/v2/quote',
                                       data=data) as response:
                print("aiohttp ->", response.status, response.headers)


if __name__ == '__main__':
    asyncio.run(async_main())
    sync_main()

By the way, I made exactly the same request on urlib3, the result is the same as requests, there are no Retry-After headers in the response, which allows you to make an unlimited* number of requests. The solution to simply delay for the time specified in the Retry-After header does not suit me (as other libraries allow you to easily make an unlimited* number of requests)

Python 3.12
aiohttp==3.10.6
requests==2.32.3

[Dreamsorcerer]

Install netcat and run nc -l 8000, then send the request to http://localhost:8000. Compare the request sent from requests and the one sent from aiohttp. Then you'll know what the actual difference is in the request.

[AndrejedaI]

aiohttp:

POST / HTTP/1.1
Host: localhost:8000
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:130.0) Gecko/20100101 Firefox/130.0
Accept: */*
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
Content-Type: text/plain;charset=UTF-8
Origin: https://app.uniswap.org
Accept-Encoding: gzip, deflate
Content-Length: 443

{"tokenInChainId":1,"tokenIn":"ETH","tokenOutChainId":1,"tokenOut":"0xe41d2489571d322189246dafa5ebde1f4699f498","amount":"1000000000000000000","sendPortionEnabled":true,"type":"EXACT_INPUT","intent":"quote","configs":[{"useSyntheticQuotes":false,"routingType":"DUTCH_V2"},{"enableUniversalRouter":true,"protocols":["V2","V3","MIXED"],"routingType":"CLASSIC","enableFeeOnTransferFeeFetching":true}],"useUniswapX":true,"slippageTolerance":"0.5"}

requests:

POST / HTTP/1.1
Host: localhost:8000
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:130.0) Gecko/20100101 Firefox/130.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
Content-Type: text/plain;charset=UTF-8
Origin: https://app.uniswap.org
Content-Length: 443

{"tokenInChainId":1,"tokenIn":"ETH","tokenOutChainId":1,"tokenOut":"0xe41d2489571d322189246dafa5ebde1f4699f498","amount":"1000000000000000000","sendPortionEnabled":true,"type":"EXACT_INPUT","intent":"quote","configs":[{"useSyntheticQuotes":false,"routingType":"DUTCH_V2"},{"enableUniversalRouter":true,"protocols":["V2","V3","MIXED"],"routingType":"CLASSIC","enableFeeOnTransferFeeFetching":true}],"useUniswapX":true,"slippageTolerance":"0.5"}

the only difference is in the connection header, but after adding it nothing has changed, aiohttp still returns 429 after 5-7 requests

[AndrejedaI]

I even changed the order of the headers so that they completely coincide with requests but still there is no difference

[AndrejedaI]

for example, the httpx library returns 200, again despite the fact that the headers, ip, cookies (there are none) are the same.

async def async_httpx_main():
    for i in range(10):
        async with httpx.AsyncClient(headers=headers) as client:
            r = await client.post('https://interface.gateway.uniswap.org/v2/quote',
                                       json=json.loads(data))
            print("httpx ->", r.status_code)

There's clearly something wrong with aiohttp...

[Dreamsorcerer]

If it's literally sending the same bytes as other libraries, then clearly there's not something wrong with aiohttp...

The only other difference (the obvious one for rate limiting) is that it might be sending requests faster and thus hitting the rate limit.

[Dreamsorcerer]

Actually, testing it myself, the retry period is 1 hour. So, there must be something in the server that is banning the connection. There's nothing for us to fix, but if you can figure out the difference, then maybe you can circumvent their stupid firewall. If the requests are identical over HTTP, then I can only imagine the difference is in the TLS layer (you could try custom SSLContexts).

[Dreamsorcerer]

OK, figured it out for you. Honestly, such a stupid bloody firewall...

    c = ssl.create_default_context()
    c.minimum_version = ssl.TLSVersion.TLSv1_2
    c.maximum_version = ssl.TLSVersion.TLSv1_3

This (a very sensible default) trips the firewall. It seems that accepting a connection with multiple valid TLS versions with an upper limit of 1.3 (despite the server itself doing this) is unacceptable. So, the difference might be that requests and httpx don't support TLS 1.3?

Any of these work fine:

    c = ssl.create_default_context()
    c.minimum_version = ssl.TLSVersion.TLSv1_2
    c.maximum_version = ssl.TLSVersion.TLSv1_2

    c = ssl.create_default_context()
    c.minimum_version = ssl.TLSVersion.TLSv1_3
    c.maximum_version = ssl.TLSVersion.TLSv1_3

    c = ssl.create_default_context()
    c.minimum_version = ssl.TLSVersion.TLSv1_1
    c.maximum_version = ssl.TLSVersion.TLSv1_2

[sywhb]

Thanks @Dreamsorcerer! Worked like a charm 👍