Key Takeway

A-B testing tools on online ad platforms do not randomize ad exposure in ways that allow causal inferences about user response because targeting algorithms serve each ad to different mixes of users optimized for each ad. This paper offers prescriptive guidance to experimenters based on their decision-making goals for A-B ad tests.