Back to Glossary

Creative

A/B Test

A controlled experiment comparing two ad variations (A vs B) with equal traffic split to determine which performs better on a specific metric. Requires statistical significance before declaring a winner, which typically means enough conversions to be confident the difference isn't random. Simple and reliable, but slow. How long a test needs depends on conversions per variant, not impressions, and at typical ecommerce conversion rates that takes a while. For brands testing 20+ creatives monthly, sequential A/B testing is too slow.

Why it matters

It is the only method that isolates one variable, which makes it the only one that produces a transferable lesson rather than a local result.

In practice

Change one thing and size the test before running it. A test that cannot detect the effect you care about will return no significant difference regardless of the truth.

Related terms