Details
-
Bug
-
Status: Stalled (View Workflow)
-
Major
-
Resolution: Unresolved
-
BB v1.08
-
None
-
None
Description
Discussion on Zulip: https://mariadb.zulipchat.com/#narrow/channel/236699-Buildbot/topic/pageing.20through.20pending.20builders.20causes.20503.20response.20on.20ajax/with/528997147
Description:
When expanding triggers under a build, requests to the /buildsets api will be sent proportional to the number of triggered builds. These requests are most probably sent async by javascript, thus arriving in very short intervals so the burst directive becomes effective.
Background:
buildbot nginx configuration implements rate limiting and burst which can greatly limit the number of honored requests per unique uri path
i.e.
limit_req_zone $request_uri zone=bb:10m rate=30r/m; |
which creates a bucket per unique uri path.
Issue:
How to reproduce:
Simulation on buildbot.dev.mariadb.org
- Case 1: send 20 async requests with current config - Expected : half are rejected
- Case 2: for the / location increase the bursts to 40, restart nginx and retest with 50 async requests. Expected: ~ 10 are rejected.
Case 1:
--- Request Results ---
|
HTTP 200: 11
|
HTTP 503: 9
|
Case 2:
--- Request Results ---
|
HTTP 200: 41
|
HTTP 503: 9
|
import asyncio |
import aiohttp |
from collections import Counter |
|
|
# CONFIGURATION
|
URL = "https://buildbot.dev.mariadb.org" |
TOTAL_REQUESTS = 50 |
|
|
async def send_request(session, i): |
try: |
async with session.get(URL) as response:
|
return response.status |
except Exception as e: |
print(f"Request {i} failed: {e}") |
return "error" |
|
|
async def main(): |
counter = Counter() |
|
|
async with aiohttp.ClientSession() as session:
|
tasks = [send_request(session, i) for i in range(TOTAL_REQUESTS)] |
responses = await asyncio.gather(*tasks) |
|
|
for status in responses: |
counter[str(status)] += 1 |
|
|
print("\n--- Request Results ---") |
for code, count in sorted(counter.items()): |
print("http code: " + str(code) + ": " + str(count)) |
|
|
if __name__ == "__main__": |
asyncio.run(main())
|
|
|
|
