Motivated by applications to wireless networks, cloud computing, data centers, etc., stochastic processing networks have been studied in the literature under various asymptotic regimes. In the heavy traffic regime, the steady-state mean queue length is proved to be $\Theta({1}/{\epsilon})$ , where $\epsilon$ is the heavy traffic parameter (which goes to zero in the limit). The focus of this paper is on obtaining queue length bounds on pre-limit systems, thus establishing the rate of convergence to heavy traffic. For the generalized switch, operating under the MaxWeight algorithm, we show that the mean queue length is within $\textrm{O}({\log}({1}/{\epsilon}))$ of its heavy traffic limit. This result holds regardless of the complete resource pooling (CRP) condition being satisfied. Furthermore, when the CRP condition is satisfied, we show that the mean queue length under the MaxWeight algorithm is within $\textrm{O}({\log}({1}/{\epsilon}))$ of the optimal scheduling policy. Finally, we obtain similar results for the rate of convergence to heavy traffic of the total queue length in load balancing systems operating under the ‘join the shortest queue’ routeing algorithm.

We study a class of load-balancing algorithms for many-server systems (N servers). Each server has a buffer of size $b-1$ with $b=O(\sqrt{\log N})$, i.e. a server can have at most one job in service and $b-1$ jobs queued. We focus on the steady-state performance of load-balancing algorithms in the heavy traffic regime such that the load of the system is $\lambda = 1 - \gamma N^{-\alpha}$ for $0<\alpha<0.5$ and $\gamma > 0,$ which we call the sub-Halfin–Whitt regime ($\alpha=0.5$ is the so-called Halfin–Whitt regime). We establish a sufficient condition under which the probability that an incoming job is routed to an idle server is 1 asymptotically (as $N \to \infty$) at steady state. The class of load-balancing algorithms that satisfy the condition includes join-the-shortest-queue, idle-one-first, join-the-idle-queue, and power-of-d-choices with $d\geq \frac{r}{\gamma}N^\alpha\log N$ (r a positive integer). The proof of the main result is based on the framework of Stein’s method. A key contribution is to use a simple generator approximation based on state space collapse.

A parallel server system with n identical servers is considered. The service time distribution has a finite mean 1 / μ, but otherwise is arbitrary. Arriving customers are routed to one of the servers immediately upon arrival. The join-idle-queue routeing algorithm is studied, under which an arriving customer is sent to an idle server, if such is available, and to a randomly uniformly chosen server, otherwise. We consider the asymptotic regime where n → ∞ and the customer input flow rate is λn. Under the condition λ / μ < ½, we prove that, as n → ∞, the sequence of (appropriately scaled) stationary distributions concentrates at the natural equilibrium point, with the fraction of occupied servers being constant at λ / μ. In particular, this implies that the steady-state probability of an arriving customer waiting for service vanishes.

We consider a system of N parallel queues with identical exponential service rates and a single dispatcher where tasks arrive as a Poisson process. When a task arrives, the dispatcher always assigns it to an idle server, if there is any, and to a server with the shortest queue among d randomly selected servers otherwise (1≤d≤N). This load balancing scheme subsumes the so-called join-the-idle queue policy (d=1) and the celebrated join-the-shortest queue policy (d=N) as two crucial special cases. We develop a stochastic coupling construction to obtain the diffusion limit of the queue process in the Halfin‒Whitt heavy-traffic regime, and establish that it does not depend on the value of d, implying that assigning tasks to idle servers is sufficient for diffusion level optimality.

In this paper we give an analytic solution for graphs with n nodes and E = cn log n edges for which the probability of obtaining a given graph G is µn (G) = exp (- β ∑i=1ndi2), where di is the degree of node i. We describe how this model appears in the context of load balancing in communication networks, namely peer-to-peer overlays. We then analyse the degree distribution of such graphs and show that the degrees are concentrated around their mean value. Finally, we derive asymptotic results for the number of edges crossing a graph cut and use these results (i) to compute the graph expansion and conductance, and (ii) to analyse the graph resilience to random failures.

We study the problem of dynamically allocating flexible workers to stations in tandem or serial manufacturing systems. Workers are trained to do a subset of consecutive tasks. We show that the optimal policy is often LBFS (last buffer first-served) or FBFS (first buffer first-served). These results generalize earlier results on the optimality of the pick-and-run, expedite, and bucket brigade-type policies. We also show that, for exponential processing times and general manufacturing networks, the optimal policy will tend to have several workers assigned to the same station.

In this paper, we develop a unified approach for stochastic load balancing on various multiserver systems. We expand the four partial orderings defined in Marshall and Olkin, by defining a new ordering based on the set of functions that are symmetric, L-subadditive and convex in each variable. This new partial ordering is shown to be equivalent to the previous four orderings for comparing deterministic vectors but differs for random vectors. Sample-path criteria and a probability enumeration method for the new stochastic ordering are established and the ordering is applied to various fork-join queues, routing and scheduling problems. Our results generalize previous work and can be extended to multivariate stochastic majorization which includes tandem queues and queues with finite buffers.

We prove a theorem which can be used to show that the expectation of a non-negative function of the state of a time-homogeneous Markov process is uniformly bounded in time. This is reminiscent of the classical theory of non-negative supermartingales, except that our analog of the supermartingale inequality need not hold almost surely. Consequently, the theorem is suitable for establishing the stability of systems that evolve in a stabilizing mode in most states, though from certain states they may jump to a less stable state. We use this theorem to show that ‘joining the shortest queue' can bound the expected sum of the squares of the differences between all pairs among N queues, even under arbitrarily heavy traffic.