Have you found yourself waiting in line at a supermarket and the guy in front decides to pay by check? One person can slow everyone down. We had a similar problem at Twitter.

Every server has a fixed number of workers (cashiers) that handle incoming requests. During peak hours, we may get more simultaneous requests than available workers. We respond by putting those requests in a queue.Read more…