Your A/B Testing Isn't Working Nearly as Well as You Think

... tests per day and have hundreds of thousands of hits, only a few offers can be tested at one time in cases like direct mail. The variance that these tests reveal is often so low that any meaningful statistical analysis is impossible. John Senior and ...

http://www.wired.com/opinion/2013/11/what-to-do-when-ab-testing-doesnt-allow-us-to-c-enough/