Taguchi Sucks for Landing Page Testing
In the world of landing page testing there are two common mathematical approaches: A-B Split testing, and parametric Multivariate testing. A subset of Multivariate testing is known as “Design of Experiments” (DoE) and is also called “fractional factorial”. A common fractional factorial approach is called the “Taguchi Method”.
Some online marketers consider A-B Split testing to be kind of wimpy, and endow fractional factorial methods with an almost mythical quality.
I spend way too much of my time explaining to people that at least when applied to landing page optimization fractional factorial methods are a really bad idea. Despite this, the illusion persists that this kind of testing is somehow state-of-the-art, when in fact, nothing could be further from the truth.
For a lot more detail (30 pages worth), download the whitepaper - “The Truth About Taguchi”.
Testing is composed of two important activities:
- Deciding what to test and coming up with good ideas
- Finding the best solution among your tested alternatives
People claim to get really good results with fractional factorial multivariate testing, and they credit this to the method that they use to analyze the data.
In reality, the improved conversion rates are the results of the great ideas for new landing page elements that go into the test. If all of your alternative landing pages designs are better then the original, it does not really matter what method you use to confirm that. Fractional factorial approaches may actually miss the best version of the landing page in your test and often lead […] Read more »
Technorati Tags: blog