Every request takes a connection from a pool. The size of the pool is the ceiling on concurrency, and the ceiling is the number that decides whether a server is a server or a toy.
About 1 min of reading
Find the ceiling
Write the pool size
Before you run it, say out loud what it will print. Then open the answer.
Send more than the pool
Before you run it, say out loud what it will print. Then open the answer.
Stamp the upstream timeout
Before you run it, say out loud what it will print. Then open the answer.
The ceiling only shows when the count of requests exceeds it. A measurement below the ceiling measures nothing.
The upstream timeout. A server that closes before the client retries is a server that loses requests on time.
Beyond the pool, requests wait. The wait is the queue, and the queue is the latency.
Reusing a connection saves a handshake and holds a socket. The socket held is the cost of the saving.
An idle connection is closed after a time. The time is chosen against the client, not against the server.
Each one is defined here on this site. Open the entry for the full definition and the other words it points at.