How customer insights saved CXL Institute from near-disaster

Table of contents

• What are customer insights and why are they important?

• Failure to launch: the CXL Institute backstory

• A real-life example of how customer insights saved the day

• Creating solutions based on customer insights

• Key takeaways

What are customer insights and why are they important?

Customer insights are the result of gathering qualitative and quantitative data about customers from a variety of sources. They can include customer interviews, surveys, in-product behavioral observation, and user feedback, among others.

The goal of customer insight analysis is to unearth what customers deeply hope for and why, what they experience, and what drives them to act. When analyzed carefully, customer insights can tell you where you’re going wrong and why—and how to make the best possible choices for your customers and your business.

Most importantly, they fill in the knowledge gaps created by assuming you know what your customers need and want, as Peep Laja discovered when he launched the CXL Institute.

Failure to launch: the CXL Institute backstory

Peep Laja is the founder of ConversionXL (now CXL) and CXL Institute. What began in 2011 as a marketing agency focusing on data-driven conversion optimization quickly expanded into one of the most popular marketing blogs online.

CONVERSIONXL’S WEBSITE CA. LATE 2011

The success spurred Peep to survey his audience of 50,000 subscribers and ask: “What else would you wanna buy if we were smart enough to build it?”

The answer that surfaced: training courses.

More specifically, training courses for marketers to learn how to leverage data for higher conversions. It seemed like a no-brainer—just train people to do what CXL already did so well.

So in January of 2016, he hired a team of product developers and researchers to work on a spin-off business. Five months later, CXL Institute launched as a subscription product.

CXL INSTITUTE’S CURRENT HOMEPAGE

For $65 a month, CXL Institute students had access to a conversion optimization course and weekly original user experience studies.

Peep had a product, a plan, and a massive email list to launch to. His team crafted an email campaign and blog announcements. His costs were rising, but he assumed he’d cover them easily once the sales cart opened.

But that’s not at all what happened after the launch:

“We only made about $25,000 in the first month, which was not even enough to cover our costs. But June will be better! In June we made even less money. In July, even less. In August, even less. In September, we made just $10,000, and we had depleted the cash reserves.”

The venture lost money for six months straight. Peep was ready to pull the plug and tell his team “Sorry, I have to let you go because I’m out of money.”

A real-life example of how customer insights saved the day

The first clue that something was amiss with the launch was the low open rate for the email campaign—around 25% of their 50,000-strong list. Click-through rates were much worse.

Peep’s team could see their ship sinking and tried to diagnose what was going wrong.

Here are the seven exact steps they took:

Step #1: talking to customers one-on-one (to figure out why they signed up)

“We were frantic about customer development,” says Peep. For every new sign-up, the onboarding process included a welcome email saying:

“Thank you for joining. By the way, would you have time to hop on a 10-minute phone call with us?”

Two team members were assigned to call 5-15 people a day as part of their daily routine, armed with a set of interview questions to find out who they were, what made them sign up, which problems they were trying to solve, and how they planned to use what they learned. And, since the conversation was part of customer onboarding, the call included a short walkthrough of the product to introduce the new customer to features they might need.

“Basically, we were really trying to understand what they were trying to get out of the course.”

Every customer interview was documented in a Google spreadsheet, which the team read in their weekly meetings. It didn’t take long for the team to notice that “the things people said started to repeat. A lot. We immediately started to see patterns,” says Peep.

The patterns they saw in the qualitative data were supported by their findings on the quantitative side—with in-product analytics.

Step #2: using in-product analytics to gather quantitative data

Talking to customers is a can’t-miss step in customer development, but asking people what they intend to do is not enough. Humans are notorious for being really bad at predicting their own behavior. Peep’s team also tracked what customers actually didonce inside the product.

From their in-product analytics, they found that most users weren’t reading their weekly original user experience studies at all. That finding was supported by the qualitative feedback they gathered: Customers never cited the UX studies as a reason for signing up.

“So we dropped them," says Peep. "And it helped us cut costs to stop producing content nobody wanted to pay for.”

Not ones to let good research go to waste, they published the studies on the CXL blog, which helped the company gain a wider readership and fill the top of their sales funnel.

“I like to publish studies because nobody else does original content. It’s great for attracting backlinks.”

But, mostly, he says, stopping the studies—which took time and resources to produce—allowed his team to focus on what really mattered: the courses.

Step #3: using a retention survey to uncover why people leave

Peep’s team now knew why people were coming, and they knew what they were doing when they used the product—but they didn’t know why customers were leaving.

The first question that came to mind: what was CXL Institute missing that would retain customers for the long-term?

The first step in this learning process was to send emails to every person who canceled their subscription, asking for another 10-minute phone call.

They didn’t get many takers. Even fewer responded than for the onboarding calls (which weren’t very popular either). So they sent email surveys asking: Why did you cancel? What did you not like? How can we improve?

But the response rates were low. So low, the team decided to change tactics.

“We flipped it around so when people canceled, the couldn’t just hit cancel, they had to choose from a list of reasons.”

Using their qualitative feedback, Peep’s team came up with a ‘top five’ list of reasons customers canceled and required customers to choose one in a multiple-choice survey or write in their own.

“That was a really good move because, finally, for every cancellation, we were seeing why people were leaving.”

The number one reason customers left: “I don’t have time.”

For Peep, this was understandable. Very few people plan ‘time to learn’ into their already over-packed daily calendars.

The second most common reason: “it’s too expensive. We can’t justify it.”

It didn’t take long for Peep and his team to solve the first objection—and find the reason underlying the second. But before they did that, they needed to learn even more about who these customers were.

Step #4: finding the answers through customer segmentation

Peep and his team had been collecting qualitative and quantitative user feedback on every single customer since their launch, and they noticed one curious discrepancy: some customers left really fast, after just a month. Others left after three months.

What was the difference between the two groups?

To get to the root of these all-too-common patterns of customer behavior, Peep’s team exported each customer’s subscription data into an Excel spreadsheet. Then, they split the subscription listing into short-term subscribers and customers who had a longer retention period (four months or more).

They filtered out free email domains like Gmail and Yahoo to concentrate on corporate clients only. This step allowed them to do SEO analysis for the corporate domain names, looking at their domain authorities and whether the sites were large or small.

Then, using Clearbit, they collected more data on the company’s revenue and size, as well as details like employment seniority level on the individual members.

“Clearbit will give you like 50 data points. We didn’t know what we were looking for, but quite soon we started to see that company size was a common factor. So we looked at annual revenue and Alexa rank and job titles. The revenue combined with domain authority made it pretty clear that more established, bigger companies were the ones we retained longer.”

This discovery flew in the face of Peep’s original expectation that smaller companies would be more interested in the training.

Step #5: understanding the why behind “it’s too expensive”

It turned out that the single biggest difference between customers who churned quickly and those who stuck around was company size. Larger companies stayed on as subscribers; smaller companies churned.

“We thought we were targeting small businesses; people who can’t afford our agency. But actually, the target audience turned out to be the same audience that uses our agency services. Our ideal customer is somebody who makes at least $10 million a year online, and agencies. Agencies are the exception of small businesses that think long-term, invest in education, and are used to paying for it.”

Smaller companies have less money to spend on education. And they can’t afford to plan for the long-term since they’re busy trying to survive in the short-term. Peep’s qualitative interviews supported this theory:

“In our interviews with small companies, we learned that they are very tactics-oriented. They wanted something they could implement and make money with right now, whereas bigger companies are thinking long-term strategy. Because they have the luxury to think long-term.”

Based on this information, the CXL Institute team decided to change their target client to those larger companies and long-term thinking agencies. The product wasn’t too expensive—it was just too expensive for smaller businesses to justify.

Step #6: addressing the “I don't have time” issue

As Peep observed, very few people put ‘self-education’ on their daily to-do lists. It’s just not something we usually prioritize unless we have to.

The qualitative survey research uncovered a low course completion rate. And, those who did complete the course churned out, figuring, as Peep says, “I finished it, so why keep on paying?” They weren’t seeing continued value.

Peep says: “They had a point.”

In the quantitative data—the user analytics—the team noticed that many churning users weren’t even consuming the content.

“Most bought the course, logged in, and never logged back in again. Or maybe they logged in twice.”

This is actually very common in the online education industry.

If you have all the time in the world to finish a course, odds are, you never will.

“It’s the same problem for Udemy, Udacity, Coursera, edX - everybody. I don’t think any of these providers are getting their money back, because the completion rates are horrible. It’s like 10%, maybe 12% of people who enroll in a course finish it.”

But those numbers weren’t good enough for Peep.

So how did he change a prevailing mindset around online learning? Especially one that’s so common and ingrained?

Step #7: changing the offering to change customer behavior

Peep had an idea.

He’d been running his own conversion coaching programs for years at high ticket prices. And he’d always had very high engagement rates.

The only difference between those events and the courses he was offering through CXL Institute was that his classes were live.

Peep realized live classes are ones you put on your calendar as can’t-miss events.

But he also realized, “I don’t scale.”

He couldn’t teach every class as the Institute grew, so he recruited a handful of experts to teach the courses for him.

Now they had live training with third-party trainers, allowing CXL to produce two new live courses every month—giving users a reason to maintain their subscriptions and keep learning more.

“The format works. People learn better. We make money. And the completion rates for these live courses is way, way higher.”

Live classes were the key to improving engagement and commitment levels, as well as delivering ongoing value. They force users to make time to learn.

Creating solutions based on customer insights

Successful products and services all start with understanding the customer, of course. But the real key to Peep’s eventual success with CXL Institute was how he included gathering and using customer insights from the very first email, asking “what would you like us to build?” to the customer interviews conducted every day to find out what was going wrong.

Customer data-gathering was baked into his onboarding and offboarding processes, too. These insights were how he knew customers wanted training in the first place. Tracking behavior was how he knew he hadn’t delivered exactly what they needed. And interviewing and surveying customers was how he learned what prevented his customers from achieving their goals.

Peep’s solutions, however, didn’t come from the customers.

And that’s where many people go wrong with customer insights. Many founders, CEOs, and developers think that by asking customers what they want, the customers will tell them how to solve their problems.

It’s our job to come up with creative solutions to our customers’ problems. Customer insights just tell us what those problems are.

Key takeaways

When your product launch goes wrong and customers start churning, here is what you can do to help your product succeed:  

1. Gather information about your customers as part of your launch, onboarding, and offboarding routines. Every touch point is an opportunity to learn more about what your customers want, hope for, and actually do. Consider making it a requirement for people to tell you why they leave as part of the offboarding process.

2. Use surveys and one-on-one phone calls. You can get deeper insights in a phone call than most people will write down, but you’ll get more responses via a survey. Use both.

3. Cross-check your quantitative user data with your qualitative data. The first tells you what users are doing; the second tells you why they are doing it.

4. When users churn (or stay), use segmentation to take a good hard look at who they are and what they have in common. Based on the data you get, you may need to re-think your target customers.