Roundup of Top 5 strategies (Beyond A/B testing) and Misconceptions to Avoid in CRO

Many digital marketers approach A/B Testing and Conversion Rate Optimization (CRO) as synonymous. They often use the two terms interchangeably.

Conversion rate optimization has a simple goal: get more visitors to become customers. To achieve this, you must build a CRO practice that goes beyond A/B testing. In this article, we will share 5 strategies beyond A/B testing (as well as some other common misconceptions about CRO), so that you can achieve growth and scale your digital business.

5 CRO Strategies Beyond A/B Testing

1. Customer service reports

You should start by reviewing the things your customers have already told you. Your customer service team is on the front lines and can provide you with detailed examples of where customers have been frustrated or confused. These details are a critical part of understanding what the most urgent problems are in your user experience.

You can also review digital interactions, such as your chatbot messages and email so that you can get a deeper understanding of what challenges your high intent visitors have in becoming customers.

2. Talk to customers directly

You should also talk to your customer directly through proactive qualitative research. Based on what you want to learn, you should choose a research method:

  • Interviews
  • Intercept surveys
  • Informal guerilla research in stores

You should ask your current customers questions like

  • How well is it working for you?
  • How well is it not working for you?
  • How do we compare to other competitors?

By engaging with customers and prospects, you will uncover areas your team will not have considered because you’re too close to the user experience and product or service offering.

3. Observe your customers interacting with your digital experience

You need to understand how your customers and prospects interact with the digital experience, so you can uncover and solve their friction points keeping them from becoming customers. This is done through usability studies.

You’ll go through the following steps:

  • Set up the participant with a scenario to represent a typical use case of reaching your site (e.g., you’re looking for a new pair of athletic shoes)
  • Ask the participant to go through specific tasks on the site and to think aloud as they go
  • Observe their gestures, thoughts, and emotions to understand what challenges they are experiencing.

For usability studies, you can use remote research tools such as, TryMyUI, and If you have a smaller budget but still want to observe how people use your website, you could use click tracking analysis tools like Hotjar, CrazyEgg, and Microsoft Clarity to get a sense of how people move through the site.

4. Talk to your competitor’s clients

Most businesses rarely think to talk to their competitors’ clients or visitors. There are various ways to get access to folks who buy from your competitors and to learn what they’re doing well and what they’re doing poorly.

You should run a usability study on your competitor sites to see what their strengths are. You can then have the participant compare and contrast against your website experience.

5. Leverage your sales team for insights into your customers

You should also talk to your frontline sales team to understand your customers and what they need. Ask them what information they provide during the first sales conversations and what information they provide to close the deal. This information should become your landing page copy

Misconceptions to Avoid in CRO

1. CRO Programs = A/B Testing

CRO Programs are more than the specific solutions you will A/B test to improve conversion. You need to start with the problems, not the solutions. A/B tests will not tell you what problems or challenges your visitors are having on your website; it will only tell you if your solution solved a problem or not. It’s really important to understand your visitors and their friction points before you create ideas for the A/B test. A CRO Program based on ideas alone will not move the needle for your business.

2. A “fully” optimized site

A lot of people ask, “how long does it take to optimize my site?” There’s no such thing as a fully optimized experience.

Customer experiences and expectations change over time, and you may have aspects of your business offering and the competitive landscape that change as well. All of these factors mean that your site can never be fully optimized. Instead, your site can outperform other experiences by understanding what visitors need and what challenges they have, and ultimately addressing those problems with meaningful solutions.

“If you think about the web in the 90s, versus the web in the 2010s, versus now, we’ve all evolved and what we expect from a website. We no longer tolerate flashing banners or sites with complicated checkouts” - AJ Davis, CEO of Experiment Zone

If you optimized your site 20 years ago, it’s certainly not optimized today. It’s an ever-evolving landscape that needs to be addressed with curiosity in order to stay ahead of the game.

3. Test everything and anything

The philosophy of “testing everything” is a common misconception when it comes to experimentation programs. Testing everything can give you lots of learning, but it can also be time-consuming, and you will not have the ROI to test each and everything.

For example, if you decide to change your footer, is that going to make a difference to your customers? Will it impact whether they convert (or not)? Or should you focus your testing resources on higher impact testing - such as product images and value propositions?

If you’re Amazon with tons of traffic and seemingly unlimited testing resources, then go ahead and test everything. If you’re not Amazon, you need to prioritize carefully and test what matters.

Because if your customers are struggling to figure out basic things like finding information, navigating through the site, etc, and you aren’t achieving those basic usability needs, then you shouldn’t be testing anything else until you’ve solved those problems.

4. Changing button colors

This is by far the most popular misconception in CRO: that everyone should test whether their buttons should be blue or red.

Many technology companies enabling A/B testing use this as an example of demos because it is easy and quick to set up.

If that’s the A/B test you’re running, you don’t know what problem you’re trying to solve and you’re not likely to impact customer behavior on your website.

Content originally featured on our partner site


In the evolving and growing CRO industry, you can learn from common misconceptions so your optimization program won’t fail. You can also improve the success of your optimization program by working with leading industry experts, such as partnering with seasoned industry experts.

Experiment Zone is a leading optimization agency based in Austin, Texas. We help companies identify opportunities and evaluate ideas using data so that they can improve customer experiences and increase sales. Contact us today to get started with optimization.

Need help getting your website to be high converting?