Jeremy Smith Feb 19, 2019 11:33:15 AM 26 min read

Should You Attempt Conversion Optimization If Your Data Sucks?


That’s the answer to the question posed in the headline.

Let's go through it again: Should you attempt CRO if your data sucks?


Don’t do it.

If you try to optimize your site with sucky data, you won’t be able to determine whether your changes are making a difference.

That’s huge!

Or, even worse: you’ll be given the wrong information and make bad decisions.

That’s terrible.

Your data is the raw, cold, hard, unbiased, numerical information that tells you whether your optimization campaign is on the right track. It’s an essential part of any successful CRO effort.

Trying to optimize a site without good data is like trying to build an outhouse without wood, nails, or a hammer.

Wooden squat-style outhouse on lawn with door open

(Image source)

You’ll make something, sure, but you might end up with crap.

That’s why you need to get your analytics house in order before you go CRO-magnon.

So, here you are, whimpering because your data sucks. You can’t do conversion optimization.

So what can you do?

You can get your data in order.

Here are a few steps to help you do just that.

1. Start with Google Analytics.

There are plenty of analytics tools on the market.

Some cost money.

Some are free.

Some are amazing.

Some are crap.

Some are really complicated.

Some are super simple.

Fortunately, there’s one tool that you’re probably already using that doesn’t cost any money: Google Analytics.

Screenshot of Google Analytics dashboard

(Image source)

Google Analytics isn’t just free; it’s also feature-rich. It has exactly the kinds of analytics you’re looking for to get you started with tracking customer data.

If you haven’t already done so, set up your website with Google Analytics. Then, wait a few days and browse through the analytics reports so you can see what the tool offers.

Keep in mind, though, that in its default configuration Google Analytics probably doesn’t give you everything you’re looking for.

Especially if you want to enter the ranks of conversion professionals.

For example, if you’re running an ecommerce website, you probably want to track conversions for specific products. For that, you’ll have to set up the tool for ecommerce tracking.

Screenshot of Google Analytics Ecommerce Report

(Image source)

That’s just one feature you might need that requires additional configuration. If you want to track goals or link your AdWords account to Google Analytics, you’ll need to take more extra steps.

Screenshot of site link for AdWords account

(Image source)

Bottom line: make sure that you not only have Google Analytics running on your site, but that you also have it configured properly for every metric that you want to track.

2. Add Custom Variables If Necessary

You might be at the point now where you have Google Analytics all set up, you’ve configured it properly, and you see that the tool is properly tracking your metrics.

So now you’re ready to begin CRO, right?

Nope. Sorry.

Google Analytics is very comprehensive, but it still might not report all the data you’re looking for.

This is where you take a step back and tell yourself the following: “I will not let the tool tell me what I need to track.”

Say it several times (click your heels if necessary) until it finally sinks in.

Google shouldn’t dictate the metrics you want to track. You should dictate the metrics you want to track.

Now, get away from Google Analytics, meet with your team, and brainstorm about which metrics you want to track, so that you can properly optimize your site for conversions.

Let your imagination go wild. Don’t let anything hold you back.

Are you done with that step? Good. Now it’s time to go back to Google Analytics.

When you get to GA, you’ll probably notice that there’s no way to track any of the great metrics you just brainstormed up.

Maybe Google Analytics isn’t so great after all and you need to fork over a bunch of cash for another solution.

But probably not.

Why? Because Google Analytics lets you use custom variables.

Custom variables are variously referred to (correctly or incorrectly) as custom attributes, custom dimensions, custom implementations, and custom metrics.

Basically, if you hear the word “custom,” you’re in the ballpark.

Custom variables are the GA wild card. You can make the program do what you want it to do. You can set as many of these custom variables as you please.

screenshot of 10 Unique Custom Variables set up in Google Analytics

Think of custom variables as “the metrics you want to track that Google Analytics doesn’t already track.”

Screenshot of Google Analytics navigation menu with Custom Variables highlighted

It’s quite possible that you’ll need to set up a few custom variables before you can properly optimize your site.

Which ones? That depends on the industry you’re in, the type of website you’re running, the nature of your target market, your business model, and the results of that brainstorming session you just emerged from.

Here are a few suggestions, though:

  • Customer ID – While Google Analytics tracks demographics, it doesn’t track personally identifiable information (PII). Fortunately, you can do that with a custom variable. If you’re running a site that allows people to login, track them so you can determine who your most loyal customers are. Then, come up with an optimization campaign specifically designed for them.
  • Checkout Account Type – If you’re running an ecommerce site that allows people who aren’t logged in to check out as a guest, you might want to measure how many people check out as a guest versus how many log in before checking out. Then, you can design an optimization campaign that will encourage people to log in.
  • RSS Subscribers – Congratulations! Your content marketing efforts are so effective that some people have added your blog’s RSS feed to their readers. Wouldn’t it be great to know about RSS subscribers? With a custom variable, you can track that information.
  • Author Metrics – Your blog might have several authors. Wouldn’t it be awesome to know which one has the lowest bounce rate? Which one has the most popular articles? Good news: you can track that information with a custom variable.
  • Sales Region – You might be running a fancy startup that’s already established sales regions (Southeast, Midwest, etc.). How would you like to optimize your site for a specific region? Add a custom variable to make it happen.

You’ll likely come up with a whole different set when you determine what’s best for your site.

In my view, one of the biggest advantages of custom metrics is their ability to help you with segmentation.

Daniel Waisberg, a Google Analytics Advocate, wrote this:

"Segmentation is the mother of all website optimization. That's an extreme statement, but I believe anyone in the field would think twice before disagreeing with it. Segmentation enables us to understand who are our customers and how different groups of people behave on the website; this information is extremely valuable in order to provide a richer and more relevant experience to customers. The result is a happier customer, a more profitable website, and more bonuses to Web Analysts!"

That first sentence is clutch: segmentation.

Avinash Kaushik, another Google Analytics demigod, has a similar take on the issue:

"There is a reason your analytics data does not make any sense. There is a reason you are unable to find an iota of insight that you can action. There is a reason you feel data is your enemy. You see, you are not following one of the holiest of holy covenants when it comes to data analysis: Segmentation."

Beyond a shadow of a doubt, the most powerful segmentation feature on Google Analytics is the Custom Variable.

The reason why these guys use terms like “holiest,” “holy,” “covenant” and “mother of all” is because segmentation is really that important.

And you don’t get sacred data like that without the custom variable.

That’s why I’m very comfortable with my own extreme statement, that “segmentation is the secret to any A/B testing strategy.”

It goes like this:

  1. You need data for CRO.
  2. The data you need comes from custom variables.
  3. Custom variables require segmentation.
  4. Segmentation is the key to CRO.

One leads to the other.

Custom variables take a little bit of learning, but it’s a learning experience that is well worth the effort.

For a hard-hitting and deep learning experience on custom variables, check out Waisberg’s great post on the topic.

3. Run the Right Reports

Now that you’ve established your custom variables in Google Analytics, it’s important to know how to run the right reports.

For that, you’ll probably need to familiarize yourself with this phrase: “secondary dimension.”


Screenshot of Googe Anaytics Secondary Dimensions tab annotated with arrow and box

(Image source)

That’s Google’s not-so-cool way of describing additional data that you want to see in your reports.

Take a look at a standard report in Google Analytics. In the example below, we’re looking at the Acquisition report for all channels.

Screenshot of Google Analytics Default Channel Grouping report

Do you see that “Secondary Dimension” drop-down at the top? That’s what you’ll use to add more data to the report you’re looking at.

For example, you can see from the above graph that most of the traffic is coming from social media. But that really doesn’t tell you everything, does it?

How much of the traffic is coming in from Facebook? How much from Twitter? How much from MySpace?

Okay, probably none from MySpace, but the other two questions are valid.

If you want to see the answer to those other questions in the report, just click on “Secondary Dimension,” then select “Acquisition” from the drop-down that appears. Underneath that option, select “Source/Medium.”

You’ll get a report that looks like this:

Screenshot of Google Analytics Default Channel Groupings report with "Source / Medium" secondary dimension added

In this case, you can see that traffic from Facebook and Twitter is just about equal and accounts for about 70 percent of all traffic.

Why is this important? Because even if you have Google Analytics configured properly and you’re tracking custom variables, it doesn’t matter if you don’t include those variables in your reports.

Fortunately, the “Secondary Dimension” drop-down allows you to also select custom variables. That’s how you’ll include that information in your reporting.

Even better: Google Analytics gives you the option to create your own custom report. You can set up the report to include any metrics you want, including custom variables.

Screenshot of Google Analytics "New Custom Report" tab annotated with arrow

(Image source)

Again: don’t let Google constrain you. It’s up to you to determine what you need in terms of reporting and then create the reports that are required for your optimization campaign.

4. Come up with a hypothesis.

Now that you’ve set up Google Analytics to include your custom variables and custom reports, it’s time to start an optimization campaign, right?


You’re still missing something.

A hypothesis.

That makes sense, doesn’t it? You must have something to test before you can run a test.

So begin with a hypothesis. It should be something that is quantifiable and testable.

For example: “If we change the color of our CTA button to blue, conversions will increase 20% because blue symbolizes trust.”

First of all: LOL, but let’s just roll with it for sake of an example, mkay?

When your test is complete, you’ll use the data from your analytics to determine whether your hypothesis was valid.

Infographic explaining what makes a great hypothosis: It's testable, addresses barriers, and it's aimed at gaining insight

(Image source)

5. Why A/A Testing Is Important

Now that you have a hypothesis, it’s time to start A/B testing, right?


Before you start A/B testing, you need to start A/A testing.

Ha, ha. Gotcha.

What’s A/A testing? It’s basically a “test” in which you compare the existing color of the button against itself.

Now, why would you do that?

To make sure that there’s nothing wrong with your data.

Remember: It’s all about having non-sucky data.

It is very, very important that you conduct an A/A test and look at your data to make sure everything is being reported correctly. Otherwise, the results of your A/B test could be off.

If you’re noticing that the A/A test isn’t breaking down evenly for conversions, then there might be something wrong with your data.

And that’s what this article is all about, isn’t it? You want to make sure that you have solid data before you begin CRO.

This is what good A/A testing should look like:

Graphical representation of very little variance that should be found in A/A testing

(Image source)

Once you’ve established that your data is accurate based on the results of A/A testing, then you can initiate the A/B test.


Good data is essential to optimizing your conversion campaigns. Without it, you have no rational basis for validating (or invalidating) a hypothesis. That’s why you should make sure that your analytics are properly established before you begin CRO.

CRO begins and ends with data — good data.

As passionate as I am about conversion rate optimization, its glories, triumphs, and awesomenesses, I’m first passionate about data.

Great data is the key that will open the treasures of CRO.


Jeremy Smith

Digital marketer with a penchant for dance; helping clients see the light through the jungle of tweets since before Twitter was cool.