PLanet: Formalizing Assignment Procedures in the Design of Experiments

Carefully constructed experimental designs are essential for drawing valid, generalizable conclusions from scientific experiments. Unfortunately, experimental designs can be difficult to specify, communicate clearly, and relate to alternatives. In response, we introduce a grammar of composable operators for constructing experimental assignment procedures (e.g., Latin square). The PLanet DSL implements this grammar. Researchers specify assignment requirements. PLanet compiles these into a constraint satisfaction problem over matrices that determines viable experimental plans. In an expressivity evaluation, we find that PLanet is the most expressive compared to two existing experimental design libraries. Its composability enables expression of both canonical and customized designs in HCI experiments. Case studies with three researchers reveal how PLanet helps them make complex design choices explicit, explore alternatives, and develop a deeper understanding of experimental design.