Key Takeway

A/B tests to optimize ranking algorithms used by website marketplaces may be misleading if outcomes observed under one treatment depend on treatments assigned to the rest of the population. A novel approach can recover the true Total Average Treatment Effects (TATEs) of ranking algorithms based on past A/B tests, even where they suffer from such interference issues.