Optimal Fidelity Selection for Human-Supervised Search

We study optimal fidelity selection in human-supervised underwater visual search, where operator performance is affected by cognitive factors like workload and fatigue. In our experiments, participants perform two simultaneous tasks: detecting underwater mines in videos (primary) and responding to a visual cue to estimate workload (secondary). Videos arrive as a Poisson process and queue for review, with the operator choosing between normal fidelity (faster playback) and high fidelity. Rewards are based on detection accuracy, while penalties depend on queue length. Workload is modeled as a hidden state using an Input-Output Hidden Markov Model, and fidelity selection is optimized via a Partially Observable Markov Decision Process. We evaluate two setups: fidelity-only selection and a version allowing task delegation to automation to maintain queue stability. Our approach improves performance by 26.5% without delegation and 50.3% with delegation, compared to a baseline where humans manually choose their fidelity levels.