Dynamic Batching of Online Arrivals to Leverage Economies of Scale

Many settings, such as matching riders to drivers in ride-hailing platforms or in-stream video advertising, require handling arrivals over time. In such applications, it is often beneficial to group the arriving orders or requests into batches and process the larger batches rather than individual arrivals. However, waiting too long to create larger batches incurs a waiting cost for past arrivals. On the other hand, processing the arrivals too soon leads to higher processing costs by missing the economies of scale of grouping larger numbers of arrivals into larger batches. Moreover, the timing of the next arrival is often unknown, meaning fixed-size batches or fixed waiting times tend to be poor choices. In this work, we consider the problem of finding the optimal batching schedule to minimize the sum of waiting time and processing cost under both offline and online settings. In the offline problem in which all arrival times are known a priori, we show that the optimal batching schedule can be found in polynomial time by reducing it to a shortest path problem on a weighted acyclic graph. For the online problem with unknown arrival times, we develop algorithms that are provably competitive for a broad range of processing-cost functions. We also provide a lower bound on the competitive ratio that no online algorithm can beat. Finally, we run numerical experiments on simulated and real data to demonstrate the effectiveness of our algorithms against the offline benchmark.