How to measure the success of your marketing automations—and then improve them

Once you have your marketing automations up and running, they should pretty much work on their own.

But that doesn’t mean you and your marketing team can forget about them forever.

“I don’t know if we’ve ever reached a place where we just stop and say, ‘OK that’s it,’” says Sean Donahue, director of email marketing at nationally recognized digital marketing agency Power Digital.

“We’re of course being mindful as we set it up, but then we also run optimizations to make sure that the engagement we’re getting says this is the right thing to do,” Donahue explains. “And then we act accordingly. If we see things aren’t working, we tweak that in any number of ways and continue to drive in that direction.”

How, exactly? Here are 4 ways to test and optimize your marketing automation efforts in a marketing automation platform like Klaviyo.

1. Determine your top key performance indicators (KPIs)

In order to understand whether your automations are doing what you need them to do, you first need to know what you’re looking for.

“We review the data—the results and the benchmarks we have available within Klaviyo—and the data speaks to us,” Donahue says. “We’re then able to use that data to figure out where to pivot.”

What constitutes the “right KPIs” “depends on the business, the flow, and the goal of the flow,” says Chris Gordon, head of client success at leading ecommerce and marketing agency Noticed.

Overall, though, experts recommend marketers pay attention to the following important metrics when evaluating whether their marketing automations are working:

• Open rate
• Click-through rate
• Conversion rate
• ROI
• Deliverability
• Bounce rate
• Unsubscribes
• Customer lifetime value (CLV)

While “conversions are always top of mind,” Gordon says, he actually pays more attention to clicks.

“In the school of thought I come from, we tend to think that the job of the email is to get the potential customer onto the site, ready to buy, and then it’s the site’s job to convert them,” he explains. “So the email should be trying to drive just as much website traffic as humanly possible, and then hopefully the site converts them from there.”

If your automations are earning big-time clicks and a high number of visitors but your subscribers are not converting, for example, “maybe you need to look at your homepage or the landing page you’re driving traffic to,” Gordon suggests. “Look at core KPIs across your overarching data, even outside of email.”

Visit our help center to learn how to dig deeper into your marketing automation analytics in Klaviyo.

2. A/B test your automations

By using your core KPIs as a roadmap, you’ll be able to “figure out what’s working, what’s not, and then do some testing around that,” Donahue says. “There are all these different ways we can pivot the messages.”

In Klaviyo, you can set up both automatic and manual A/B testing to compare variables like:

• Message frequency
• Send time
• Time delays between messages
• Subject lines
• Copy
• Images
• …and more

“We always want to increase, or at least maintain, our engaged customers, so A/B testing is a non-intrusive way to see if something is working or if we need to make adjustments,” explains Heather Browne, director of performance marketing at leading digital customer experience company Blue Acorn iCi.

“We look to gain statistical significance in these tests in order to determine if we change things up now, or keep testing,” Browne adds.

“After we get everything launched, it’s an iterative process where we’re continuing to review how things are performing and then taking an educated guess and saying, ‘OK, we think if we tweak this specific thing, that it’s going to move this specific needle,’” Donahue agrees. “And then we make that change, see what happens, and just continue that process over and over again.”

How often should marketers modify their existing automations? “It depends on volume and cadence,” Donahue explains. “We’re going to get much better and quicker response data with flows that are going out on a more consistent and regular basis.”

It’s “much easier for us to make data decisions within a month” on an automation that sends multiple times a week, for example, than on a welcome series where sends are 14 days apart, Donahue points out. “The data coming in is going to be much slower, in that case, so it’s also going to be much slower for us to figure out what changes we need to make.”

Visit our help center to learn how to A/B test a flow in Klaviyo.

3. Analyze individual steps in an automation, not just the automation as a whole

Overall flow performance is, of course, important to track. “You want to understand what’s working for you,” Gordon says. “If your welcome series is a big part of what drives revenue for you, you’ll want to prioritize that. If your abandoned cart is a big revenue driver for you, prioritize that.”

On the other hand, if your welcome series is underperforming, for example, “maybe you’re not explaining your brand enough, or maybe you’re not providing enough social proof to show the customers who have tried this product really love it,” Gordon suggests. “Look at your data, figure out what it’s telling you from a very high level, and then, from there, build your roadmap.”

But getting a bit more granular with your data can reveal equally important insights, points out Lindsey Murray, VP of performance marketing at Blue Acorn iCi—and also help you make sure you’re respecting the inboxes of both paying customers and new customers.

“Because people are typically more engaged early on in a flow, we have checks built into our lengthier flows. If someone has not opened or clicked within the last 3 messages, for example, we might drop them out of the flow,” Murray explains. “We don’t want to continue sending someone emails if they’re clearly uninterested.”

“We like to look at the overall flow and just see how it’s performing, but it doesn’t really tell us the whole story,” Donahue agrees. “You could have a first message that’s just crushing it, driving all of the revenue, but then everything else just drifts off. It’s important to look at every single one of those steps and figure out what’s going on.”

4. Make sure you don’t “creep anybody out”

The biggest challenge with marketing automation, experts agree, is finding the right balance—in other words, “boundaries,” Murray says.

“We know how beneficial all these automations are, but you don’t want to overwhelm your customer, either,” Browne points out. “This comes back to the fact that you need to know your customer—whether they’re email or SMS people, how often they like to be talked to, what’s too much, etc.”

What’s too much—and also what’s alarming. As Donahue points out, we’ve all had that moment after having a casual conversation when we open our phones, start scrolling, and immediately see an advertisement for the exact thing we were just talking about.

“We don’t want to reflect that in email marketing,” Donahue emphasizes. “We don’t want to creep anybody out. So how do we do all of this, and make it a really personalized experience, without coming across as Big Brother?”

As with so many things in marketing automation, the answer lies in the data, both real-time and historical. “If we see an increase in unsubscribe rates, or a decrease in engagement overall, it would be time to reassess the overall customer journey strategy and the quantity of marketing campaigns and flows,” Browne explains.

“Engagement tells us everything we need to know,” Donahue agrees. “If we’re getting too close to the sun with personalization, that is absolutely going to reflect in the data and we’ll know to back off.”

If you’re struggling to strike the right balance, remember that the ultimate goal of marketing automation, Donahue says, is to “give the customer value.”

“Be helpful,” Donahue says. “Be there for the customer when they need you, and not when they don’t. Marketing automation is about finding those opportunities—making sure you’re in the right place at the right time, and not where you shouldn’t be.”

Want to go beyond marketing automation KPIs? We cover everything you need to know about building and executing a smart, effective marketing automation strategy in this series. Check out:

• Why marketing automation that prioritizes personalization is non-negotiable
• How to set up a strong marketing automation strategy
• Marketing automation examples that go beyond the basics

Marketing automation KPIs FAQs

What are the most important marketing automation metrics?

The most important marketing metrics for automation depend on the business, but experts recommend marketers pay attention to engagement metrics like open rate, click rate, conversion rate, ROI, deliverability, and unsubscribes when evaluating whether their marketing automations are working.

How to test the success of the marketing automation process?

In a platform like Klaviyo, you can set up both automatic and manual A/B testing to compare automation variables like message frequency, send time, time delays between messages, subject lines, copy, images, and more. Look to gain statistical significance in these tests in order to determine if you should change things up now, or keep testing and optimizing.

What are the best tools for measuring the success of marketing automation?

The best tool for measuring the success of marketing automation is one that provides automated A/B testing as well as marketer-friendly custom reporting and benchmarks that allow you to compare your performance to others in your industry or vertical. A solution like Klaviyo also empowers you to analyze individual steps in an automation, not just the automation as a whole, so that you can get more granular with your optimizations.