Error exponents of quantum state discrimination with composite correlated hypotheses

We study the error exponents in quantum hypothesis testing between two sets of quantum states, extending the analysis beyond the independent and identically distributed case to encompass composite correlated hypotheses. In particular, we introduce and compare two natural extensions of the quantum Hoeffding divergence and anti-divergence to sets of quantum states, establishing their equivalence or quantitative relations. In the error exponent regime, we generalize the quantum Hoeffding bound to stable sequences of convex, compact sets of quantum states, demonstrating that the optimal Type-I error exponent, under an exponential constraint on the Type-II error, is precisely characterized by the regularized quantum Hoeffding divergence between the sets. In the strong converse exponent regime, we provide a general lower bound on the exponent in terms of the regularized quantum Hoeffding anti-divergence and a matching upper bound when the null hypothesis is a singleton. The generality of these results enables applications in various contexts, including (i) refining the generalized quantum Stein's lemma by [Fang, Fawzi & Fawzi, 2024]; (ii) exhibiting counterexamples to the continuity of the regularized Petz Rényi divergence and Hoeffding divergence; (iii) obtaining error exponents for adversarial channel discrimination and resource detection problems.