wps_forward_bot
🚀 New issue to wemake-services/django-modern-rest by @milssky
📝 Changes in benchmarking (#1505)
While working toward #207, I ran into the need for reliable benchmarking. We currently have four workers benchmarked using ab, which does not properly support HTTP/1.1. In particular, ab turns out to handle the -k flag incorrectly: it sends HTTP/1.0 requests with a Connection: Keep-Alive header, and different servers handle these requests differently. As a result, Uvicorn and Granian are effectively being tested under different conditions.
Furthermore, when multiple workers are used, the benchmark measures more than just the performance of the request-response lifecycle. The load generator creates a limited number of TCP connections, which the operating system distributes among the worker processes. Each keep-alive connection then remains bound to a single worker. This distribution may be uneven and can vary between runs, significantly affecting the resulting RPS.
Therefore, the current results cannot be used to evaluate the impact of individual parts of Django or a future RSGI implementation.
I see two possible approaches:
1. Fix the existing benchmark:
• replace ab with a load generator that properly supports HTTP/1.1, such as hey;
• run all servers with a single worker;
• add Granian running in ASGI mode and soon add Granian in RSGO mode;
• use the same concurrency level, duration, and other parameters for every server.
2. Keep the existing benchmark unchanged and add a separate benchmark for the request-response lifecycle:
• use a single worker for Uvicorn, Granian, and Gunicorn;
• use hey as the primary load generator;
• run all servers under identical conditions;
Let’s discuss which direction we should take.
#feature #help_wanted #django_modern_rest
sent via relator
1 · 447 ·