Creative
A/B Test
A controlled experiment comparing two ad variations (A vs B) with equal traffic split to determine which performs better on a specific metric. Requires statistical significance before declaring a winner, which typically means enough conversions to be confident the difference isn't random. Simple and reliable, but slow. At typical ecommerce conversion rates, an A/B test needs 1,000-5,000 impressions per variant to reach significance. For brands testing 20+ creatives monthly, sequential A/B testing is too slow.
Why it matters
It is the only method that isolates one variable, which makes it the only one that produces a transferable lesson rather than a local result.
In practice
Change one thing and size the test before running it. A test that cannot detect the effect you care about will return no significant difference regardless of the truth.
Related terms