Online marketplaces use ranking algorithms to determine the rank-ordering of items sold on their websites. Standard practice is to determine the optimal algorithm using A/B tests, but this approach may be misleading if outcomes of one treatment depend on treatments to the rest of the population, leading to incorrect inference and sub-optimal decisions. In this webinar, Professor Ali Goli will present a framework to characterize the Total Average Treatment Effect (TATE) of a ranking algorithm in an A/B test and demonstrate the presence of interference bias. His research also proposes a novel solution that can recover the true TATE of a ranking algorithm based on past A/B tests, even if those tests suffer from interference issues. This approach can be used with data from standard A/B tests readily available to many firms – the existing data from previous tests – to de-bias TATE estimates and serve as the basis for decisions going forward. Attendees will come away with an understanding of the interference issues present in commonly used rankings tests, and a strategy for de-biasing future tests.