A/B Testing 101 for Facebook Ad Newbies

A/B Testing 101 for Facebook Ad Newbies

You hear about Facebook Ads all around the industry on how it’s the hottest platform to run ads for any type of business. So you want to try them as soon as possible, but of course, with extreme caution. 

This is where A/B testing comes in. You can try new things with at least 90% confidence, literally. 

What is an A/B split test? 

An A/B test, also known as a split test, is an experiment between two variables to find out which one performs better. Each variation is shown at random to get the most unbiased results. Typically, statistical analysis is used to determine the success of each A/B test where we aim for at least a 90% confidence in the data. 90% confidence is the threshold where we’re “confident” that the results will repeat itself if the variation was shown all the time.

Take A/B testing as a strategy to prove success before fully committing to an idea. Because we all know, money can go from $0 to $100 real quick with any advertising platform.

Popular Beginner Mistakes That You Can Avoid

  • Testing too many variables at once: When people first hear of A/B test, they usually get so excited that they want to test everything at once. But with so many variables at play, how do you know which one’s really leading the pack? And not to mention, you’ll end up with 500 combinations you’ll have to sift through in the end, which we all know we can spend our time better elsewhere. 
  • Not giving the test enough time: This is a tricky one because Facebook Ads doesn’t have a built-in “confidence level” metric in its platform. It just tells you, “Oh, this one’s the winner.” But most of the time, Facebook Ads jumps the gun and you’ll notice there’s not even much reach or even conversions to make a confident decision. So you’ll need to consider more variables before ending a test by yourself. But don’t worry, I’ll show you how to tell if it’s “enough.”
  • Focusing on the wrong metrics: I must say, this is usually the most overlooked. Because we expect Facebook Ads to track everything for us. The truth is you have to do some homework on your end to add what means the most to your business. 

How to Run an A/B Test in Facebook Ads the Right Way

#1. Identify what you want to test the most. 

In today’s age, we want to test everything, so we can get all the answers right away. But you would end up with more Ad Sets than you could even manage. 

Let’s filter it down to the most impactful variables that determine a Facebook ad’s success. We’re not trying to just change 2-3 words in the ad copy where Facebook users would normally skim anyways. We’re thinking about the top ad creation options that Facebook offers that can really move the needle. 

Impactful Facebook Ad Variables

  • Target Audience Makeup
  • Copy Message for Persona Styles
  • Creative Style for Brand Vision
  • Creative Format (Video, Image, Carousel, etc)
  • Facebook Placements
  • Ad Headline
  • Call-to-Action
  • Landing Page


This is the most basic list of variables you can start with when you’re trying to figure out if Facebook Ads is for you. How you create them are all based on your business’ brand and goals. As you figure out each element, move on to the next. 

Remember to only test one type of variation at a time. Each of these variables can have other external influencers that you may not account for, so it just makes identifying them much easier in the long run. 

#2. Determine your success metric beforehand.

Facebook Ads starts you off with all the pre-click to click behavior to understand how engaging your ad is. Does it attract people to start clicking the ad? 

But it stops there. 

What will even tell you if your A/B test was a success? Obviously, anything that is your business’ ultimate end goal of the campaign. So you’ll need to complete extra steps to integrate conversions - your bottom line - into the Facebook Ads interface.

With the main Facebook Pixel, you’ll be able to create a conversion based on your target website URL. If you want to get even more specific with a purchase or a lead, you’ll need to look into Custom Conversions. If you’re already using Google Tag Manager, this will be a bit easier for you. 

Refer to this article to get the skinny on URL based conversions. 

For other action-based or other conversions, read all about it here.

Once you create a custom conversion, you’ll be able to review the success of each test all within the Facebook Ads platform. Then, Facebook automatically optimizes ads towards getting you more conversions and calculates the cost per conversion for you. 

Also very important to note: Cost per Conversion should be the #1 easiest metric that you focus on because of two reasons. It relates conversions to money. And it shows you how much it’ll cost you to reach your goals. You’ll know when you’re just spending too much.


#3 Apply your A/B test variations to the Facebook Ads structure.

Now the critical thinking piece, how do you set up the A/B test manually in Facebook ads? 

But wait, I bet you’re asking, why aren’t we using the Split Test feature that’s already built-in? 

Well, you won’t have full control or flexibility in creating your ads. And the tests would end once it reached a “conclusion” and who knows how they’re evaluating the success. Don’t let Facebook take all the control of your money! 

So the manual way, it is. 

Facebook structures ad creation from top-down: campaign, ad sets, to ads. You can structure an A/B test using these three tiers any way you want. It depends on how complex your A/B test is.

But the most valuable A/B variables to test are usually in the Ad Set and Ad creation pieces. Keep these in mind as you build your variations. Remember, you don’t want to have too many combinations. 

Think about this example: 5 different ad sets with 5 different ad creatives. That’s already 25 variations to compare against one another. Just keep it simple.

You can either do a single ad set with multiple ads or multiple ad sets with the same ad. This will help you to stay focused on the goal without the mess. 

Aside from campaign and ad A/B variables which are self-explanatory, ad sets are based on these three valuable A/B variables. 


  • Audience Targeting: The ads associated with each ad set will only show to this unique group. Be careful here in creating audiences that overlap, you may be paying extra as it’s skewing “competition” for that group. But audience targeting is the best way to filter down to your desired customer for a certain campaign. You’ll find out who likes your campaign the most. 
  • Placement: Facebook has a wide range of placement options from different areas in the News Feed, in Instagram Stories or Feed, as well as their 3rd party Audience Platform vendors. Testing the placement will let you know where your ad works best in getting your target audience’s attention. 
  • Target Conversions: Here, we’re testing around with how Facebook optimizes your campaigns. When Facebook Ads prompts you in your Conversion goal, they’ll aim to get you more conversions for less money. This means they’re predicting what type of audience would get you results, so play around with this. Maybe telling Facebook to target something completely opposite actually gets you the conversions you need. 


Get to testing these three separately and eventually find the best combination for your campaigns. Audience Targeting can get complex, so I say you drill those down first. And move on to the next. 


#4 Calculate the Right Budget

Ok, the big ol’ question. How much should you start spending on Facebook Ads to really get the answers you need? Well most articles out there will say to find out how much it cost per conversion for your previous Facebook ad campaigns. Problem is, you haven’t even started yet. 

So ask yourself these questions to help you figure it out.   

How big is your audience? 

When you set up the audience you want to target for each ad, Facebook Ads will tell you the potential amount of people you can reach. Use that as a reference point of how much it might cost to reach everyone. 

So it costs about 25 cents per 1000 impressions. Let’s say your audience is 1 million people. It would cost at least $250 to reach everybody over time. But keep in mind Facebook gradually shows the ad multiple times to each person, so the total cost will definitely cost more. 

How many sales can you potentially get from Facebook? 

It’s all in the numbers, right? The more people I can show my ad to, the more people will buy. Figure out your average conversion rate from your main website and use it as a benchmark for Facebook Ads. Then, extrapolate the conversion rate to the amount of people you need to reach on Facebook each day. 

Let’s say you want 10 sales a day from Facebook and you have an average of 2% conversion rate. You’ll need to reach 500 people a day at least to get 10 sales. 

Again, these are questions when you’re just starting out where you have no benchmarks at all. Who knows, you might even find a better conversion rate on Facebook and this will adjust your strategy moving forward. Cross your fingers!


#5 Calculate the Confidence Level to Help Guide You

Lastly, when do you know if your A/B test is ready to end? It’s not the amount of time you let it run or the amount of money you put in. And I’m sure you want to turn it off once it hits your budget limit. But you’ll need to wait and make sure you’ve collected enough data with results above the 90% confidence level, at least. 

A confidence level above 90% is a statistical threshold where you can be confident that the results will repeat itself another time. I’d push for 95%, but we don’t need a super confirmation for these A/B tests. Just enough for us to know that 9 out of 10 times, we’ll see similar results in production as the A/B test.

Click here for a fun little calculator you can use to help you out.