A Complete Guide to Self-Serve SaaS Metrics


Let me tell you a little bit about myself first to give you a little bit of a background of where I'm coming from. And then I hope to geek out with you on some data. So I run consumer product and growth teams at Malwarebytes. What that really means is it's performance marketing, it's design teams, it's product teams, it's growth teams, and it's all of the CRM teams as well. So everything that in anything has to do with consumer product, for Malwarebytes, which is a cybersecurity company is on my plate. And then I have a counterpart on the corporate side, which is a platform sales and enterprise and corporate marketing. So prior to this -- I've been at Malwarebytes just around two years -- prior to this, I was at Survey Monkey. I was at Survey Monkey for almost eight years.

I started at a very early on when there was less than 10 people in our office. And then I've exited two years ago and they went public six months ago, which was very exciting. I started my career in analytics. I graduated from Berkeley with a stats degree. Data was my right and left leg. It was a fluke that I got into tech. And I'm so thankful that I did. I also got into tech with amazing mentors, such as Selina Tobaccowala and Dave Goldberg, who was Sheryl Sandberg's husband. And he was our CEO at Survey Monkey up until 2015. And that's the time that I really saw how powerful data can be, even early on, for you to build a sustainable business, predictable business, and for you to actually understand how to orient your business on the outcomes and not the processes. So I'm going to talk to you about how I think about analytics.

Analytics is still the backbone of a lot of my decision-making, even though I run product and marketing. I did run product marketing at Survey Monkey. I also ran performance marketing at Survey Monkey. I also did my own product management. So to me, it's all kind of a blur. It's a customer experience. And how do we use data to inform and our decisions on how to improve it and drive business value out of it. So I'm going to get started. I, and really what I'm hoping to do is to give you an understanding of how to start very early on, on your self-serve KPIs, how to think about your metrics and how to empower yourself and your team to use the data as your ally, as opposed to being confused about the data or not knowing how to do the data, or really not even knowing how to tie your day-to-day actions to what the outcome of the data should be.

Let's first get a couple of things straight. Revenue is something all of us want. We get the funding, we need to grow the revenue, but revenue is an outcome. It gets very dangerous. If you start tracking revenue on daily or weekly or monthly basis as a team, leave revenue for your CEO. If you are one, obviously look at it. And for your finance team, your teams, your product marketing and the engineering teams should never look at revenue as something that they're trying to change, because that's an outcome. If you actually use revenue as means to getting to understand how to flex it, you're way too late. You're way too late. Something already happened. And revenue is now getting impacted your weeks past the point of inflection. So how do you not look at the revenue? Well, the first step is to forecast on levers and they'll talk with those levers are, but you don't want to actually just say, okay, I want to grow 10% year over year, or say month over month on my revenue.

What does that mean? What are the actual numbers and drivers or levers that go into it? What is your forecast built out of that outputs in revenue? So we'll talk about that in a second, but then most importantly teams start talking about KPIs and those are the key performance indicators that impact the levers that result in revenue. And now you actually brought it all the way down. You got to them to the KPIs where they actually can drive an impact and grow as opposed to looking at the top total dollar number and not knowing what to do with it. So it's about to get really heavy here, but I think it's very important for us to understand how it actually all pulls in. So financial levers of how to build a subscription model, a fairly simple, if you take number of new paid subscribers that you get, you cost some churn subscribers that are happening every single month as well.

And you have your total paid subscriber base in the ARPU that will get you your revenue. So new paid subs, ARPU and churn sobs become your levers. You want to increase number of new paid users. You want to increase ARPU that you get or average revenue per user, that you get out of your subscriber base and you want to reduce churn. Those are your financial forecast, levers, that output revenue, these is what you focus your activities around, but even the is a somewhat of an outcome. How many new paid subscribers can I drive? If you give it that as a goal to anybody increase it, they will have to do more fundamental, smaller activities to actually get to new paid subs. So how do we do it? We go all the way down to KPIs and KPIs as something tangible, something that the team can actually grab onto something that they can understand an impact and drive so heavy charged, fully appreciated.

But this is the way I think about my world. When I'm thinking about driving consumer business forward, I have my billings and revenue at the top. That's what my CEO cares about. That's incentivized by it, obviously as well as driving consumer units. But then I have three levers that I'm trying to move new, paid ARPU in churn. And then I go deep. And this is based on the team structure and what I believe can actually impact those levers. So I'll pick cybersecurity product. This is Malwarebytes. Levers has both mobile and desktop. We treat them very differently because the actual offerings that we have differ so much based on the operating system that you are because we're a downloadable product. So we go into a stock about desktop. You have paid acquisition, how much acquisition will I get from paid channel? How much will come from SEO down to unique visitors that are actually coming to our site?

How many downloads does that? Does that result into how many activations will I get out of those downloads? So how many people will hit that aha moment. Those are tangible metrics that I can give to a person and say, go drive it. Now I believe that all of these have a causation to my lever. So if I improve number of downloads, I think it will improve number of new paid subscribers. Now it's not a one-to-one causation. Some of them are more causative than others. We test into it and we take things out that are correlations and not causations. And we put things in that are causation and correlation at the same time. But you don't know until you try to flex the KPI to know how much it actually flexes the lever, or does it actually put just more pressure on another KPI pricing funnel is a perfect one pricing funnel.

Obviously it will affect you number of paid subscribers. So how many trials starts do I have Malwarebytes product as a freemium? You can come in. If you have an issue we'll always scan and we'll fix your problem. Absolutely free, no commitment. If you want to drive protection. So you want time protection from getting infected again, that's the paid offering and that's a subscription offering, but how many people try it to me is a KPI. So how many trials stars can I drive? How many pricing entries do I get? Just because they started a trial doesn't mean they've seen our pricing, how many pricing to order conversion rates do I get? So how many of them that visit a pricing will result in the order? What's my monthly versus annual mix, because that will impact that moves into a little bit ARPA-E, but that's still important to me in terms of how many new paid subscribers that I can get.

So I believe fundamentally that I give out these KPIs to my teams and we focus on them and we go into the war room and we figure out the actions or results that we're going to do, order to drive this lever and the goals all across the spectrum. So let me walk you through a little bit on the new page side. So I'm acquisition. My acquisition is site traffic downloads and installs. So I monitor site traffic very carefully, but traffic can be a vanity metric. So what I draw a bunch of traffic that is very low qualified. That doesn't really mean anything. So traffic, as you saw was never a KPI in my dashboard. And actually nobody in my team is incentivized to drive traffic download is a KPI. Absolutely, but I need to drive traffic to drive, download, download. You actually downloaded our product. And then you move into the install and I need you to finish an installation. I need to make sure that the download to install ratios in the high nineties in order for this to work. But that's my acquisition. I have an acquisition product manager that all he worries about is how many installations our product will get.

And that becomes pretty powerful because then he's aligned to an outcome, not necessarily process activation. So innovation in our case, like I said, is to run a scan. You come to us, usually with the problem, your computer is slow. It may be it's taking way too long to boot up. You might be infected. You might have some trackers that are on your computer that are actually stealing you information. So we want you to get you into a product, and we want you to run a scan for you to hit that aha moment, because the moment you see scan results and 60% center of our scans result in the affection that we found a new machine, just because even as drop our trackers on your machine, that actually grabbed you information to steal your browser history, that we can show it to you. And that's your aha. I understand the value of the product. So my main focus is to land people here and for them to do.

But what about conversion? So Malwarebytes is very interesting product. It's been around for a really long time. Same as Survey Monkey. It's known for its free user base. It's known for its freemium. It actually took really long time to grow and to start monetizing users. We still, I would say struggling with monetization, why our CEO actually started the company when he was 14 years old, when he pirated a game off the internet and he had an AB running on his machine and he got infected and he'd start getting all of these pop-ups blowing up. He was terrified to tell his parents that he infected the machine, but he had Norton running on it. So that obviously did not help anything. So he wanted forums and he found that what he calls heroes to help him create, basically come up with this 20 page document, the how to remove the smell were off your machine, which those heroes helped him. And then he decided with one of them to automate the process. And he created this lightweight agent that basically would do this 20th step process, 20 page process in an automated way. Then he put it up online and then he went to high school

And then he went to college, but he graduated college.

And then he decided he actually wants to make a business out of it. That was six years ago. So the product was fundamentally free for the first eight years of its existence. It sold some lifetime licenses to basically get unlimited access to it, but it was very small. It's not up until at least actually four years ago that we started a subscription offering. So we actually constantly battled with this behavior of what is actually free because everybody just sees us as free. And now we need to move them into perception of paid. So we are premium and free features are our paid conversion triggers and free feature for us as the protection layer. But nobody actually understands the value of it until we do the trial. So we have a mix of both trial and F and freemium model. But the trial for us is time-based that's 14 days, but it's really it's for you to hit that aha moment on the real-time protection event.

That's why we do the trial because otherwise you don't see necessarily the value in paid as opposed to giving a trial for you to do additional functionality. Because for us activity comes to us just as long as you browsing online, but we don't have to drive you to do anything. You just have to do you, and then you will see the value of our product. So those are the con conversion moments, but we're always focused on that aha moment, that time you hit that blocked page. When we blocked you from going somewhere, we try to iterate over and over again, this is the paid feature. This is what you're going to be upgrading for. And then there is a bunch of additional trial ending notifications that are starting to come up at the end. The main thing that I see a lot of companies miss out as that, the value actually does not equal monetization.

The more and more you get a hot moments that doesn't have one-to-one relationship with your conversion. And that's a problem. You have to understand the value metric of your customer. So you can monetize on that value metric and the more and more your iterate of understanding what that value metric that becomes your unit of monetization. But it's sometimes hard because the functionality or the effort required because behind any given feature might be disproportional to value that it's actually providing to the customer. If it's some customer advantage, that's great. But if it's in your advantage, so it took you way more to do that feature. Then the customer values it that's too bad and you cannot make that human unit and monetization for your business. But too often, we try to monetize based on how we build and how hard it was versus how valuable it is to customers and what they want us to monetize on.

But at the end of the day, all we're trying to do is to create this called bookings, layered cake, uh, Evernote did for the first time that I saw it online, I do it for every single business that I engage with advising or full-time at this point, because to me, it's absolutely fascinating to watch how the layered cake of the businesses are built. So the way that you to read this chart, I is, so let's say in 2014, you get your first cohort of paid subscribers and then it diminishes and 15, it diminishes further in 16 and then 17. Then in 15, 2015, you get another new pace number of new paid subscribers. And then it starts to go down. This has put in normal, this is called self-serve churn. You cannot expect to continuously monetize more and more people. And only best businesses in class.

Don't have this type of curve. They have smile curves. The longer time goes by the more you actually monetize out of that user base, but let's look at it from another perspective. Why do, why do most people want to do subscription businesses? Because let's look at 2017 here and that light blue light blue area, which has new bookings that came in from new subscribers in 2017. The beauty in this is that your revenue is not that dependent on acquisition anymore. And every single year, you are less and less dependent on acquisition. That's amazing because the by law of large numbers, the more and more you penetrate out there in the world, the harder it will for you to get acquisition through the door and share you get more and more layers of renewables that are contributing to your revenue, as opposed to you having to drive purchases from new customers or returning units every single year.

And this is beautiful because you, you, you, you diversifying your ability to grow revenue. You can grow it by reducing churn, or you can grow it by acquiring more customers. It's a lot more sustainable. And that's why everybody wants that recurring revenue. It provides you the bility predictability of your business, because guess what, as you can see from this darker blue line after first year, your curves will flatten out. You will lose 25% of your annual subs in the first year. It's okay. In the second year, you will lose less than 10% in a third year. You will lose less than 3%. And then I actually hit reverse churn where more people reenter

Than losing.

And that becomes powerful because then you only really worry about the first or second year churn in order to maintain your users. The rest comes because they see the value there with you monkey, which actually had over 11 years of subscription business, not only over 85% of bookings came from renewals, which is awesome on the couple of hundred million dollar business. But more importantly, those bookings were so stable and predictable. We could forecast on our, pull it down to a hundred dollars because we know I knew how much we could do an acquisition. And more importantly, when you down to a cent of what we will get out of our renewal bookings and predictability is important because then, you know, actually what you need to do, but smile curves are also great. And really this is best in class. You want to be in smile, curve land.

So how do you make your smile curves? You're still gonna lose customers. You can't, you can't, you can just drive and, and not turn any customers and self-serve business. Whereas the product led growth, you really have to do a lot of amazing experiences for your customer in providing value. Some of them will not see, or there will be price sensitive and they will turn, but more importantly, out of those people that do stay with you for three years, how do you increase their ARPU? How do you sell them more? How do you make them engage with your business further helmet? How many, how can you buy more? How can you have them buy more of your product, more of your teams, more users, whichever value metric that you actually do. How do you, when you release new products and get them engaged on that product and have them that byproduct, because they become your evangelists because this cohort out of 2014, right now at Malwarebytes drive the most acquisition for us even. So forget the smile curves. We have a referral viral loop. Um, main persona is what we call a home CTO. That home CTO is that technical guide that you have in your life. That if your computer has any problems, you call them up and you say, what's up.

And they refer Malwarebytes and you

Download Malwarebytes and you go activation conversion, retention process, and then we can convert you, or we can at least engage you. But that cohort of loyal customers, we keep them happy. We do anything and everything to make sure that that cohort is happy because if they have not turned out and if they've been with our product for so long and their main persona, they drive more acquisition for us than anything we do in marketing. They are our brand ambassadors and they are a marketing agents out there at Survey Monkey was very similar because certain monkey had a lot more condensed content loop, create a survey. I send it out to you. Uh, you will now all know about Survey Monkey because it was in the link. It was in the footer. It was on the final end page and actually free users of Survey Monkey had the most acquisition compared to anything that we were doing too, because those surveys were fully branded with Survey Monkey material.

So those users not only very important for you to continue monetize them, but view them as your marketers, keep them happy because they will drive acquisitions to your product and they will refer. You left and right, if they're a consumer, they will refer you to their friends. If you were in the self-serve B2B business, they will take you to the next place of work with them. And I'm sure you've done it yourself. You know, the tools that you like, the moment you switch job, you engage with it immediately. Those are those people and build sustainable, predictable growth businesses. So in terms of how I set up my teams and to be data-driven to make sure that we get to those smile curves, to make sure that we get to those levers and get outcome of the revenue. We set up a goals. We don't set up activities to say, we're going to launch this.

We set up goals and what we actually want to do from our KPIs. So as an example, completely random. So we have 25% year over year growth in downloads. We want to get to a 5% free to pay conversion rate and 75% annual retention rate with NPS of 60. So how do we [inaudible] get there? Well, I'm going to pick on the churn one specifically, because I went through a little bit of acquisition and seventy-five percent, 12 month retention rate for annual subs is fairly best in class in the subscription businesses. So if you only have annual offering, you basically expect it to retain 25%. Sorry. You're not going to lose more than 25% of annual subs. So now let's break it down. How do we actually enable the team to do it? Well, first you need to understand why, why am I customers leaving? You can send surveys.

You can analyze data. It all boils down into two things. Involuntary or voluntary churn in voluntary churn is the most, I would say potentially it's easy to fix because that's just the renewal payment failure. I didn't mean to churn, but my credit card fail. Guess what? You will have close to 40% of your annual subscription payment in the world that we live in. We get credit cards are issued all the time because somebody that there's yet another data breach that happened. So you lose 44% of your users. So it's all about how do you get them back? Because you don't know you're going to lose them until the churned event actually happened. And that deploys very different techniques versus voluntary churn. I want

To leave you. That's

A very different intent from the customer. So you cannot attack both of the use cases the same way. So why do I want to leave you now? There's a lot of qualitative analysis that you need to do, but there's only very, so much mechanical things that you can do to actually leave a company. You can turn your auto renewal, you can get a refund or you can go straight to the bank and issue a back charge back. So now we're going to identify the actions that actually keep you with the company. So in voluntary, involuntary is a very reactive way. The turn already happened. You have some grace let's call, how do we actually engage you? So you obviously, you need to make sure that you got connected with a good, great payment processor that does auto reach retry attempts. Otter. We try attempts should be able to save you 30 to 50% of your payment failure churn. Make sure that you go best in class. There needs to be multiple retry attempts, different days, different, different hours of the day. It will get you 30 to 50% of

The failure back.

Now, not obviously not all of it is due to retry. Retry probably makes up probably 20% of it. The rest of it will come through notifications that you have to do with communication with the customer, to tell them that your payment has failed. A lot of times, we assume that they know they have no idea. So you have to email them. You have to be in product notifications. You have to be in your face to say, you're about to lose your access, but you have to talk to them. And payment processes. Selection is also very, very important. Obviously in the early stage, you partner with somebody that does the full e-commerce for you. Then you take start taking pieces in house, but more importantly, partner with somebody that is industry leading because you actually get good auto retry attempts at Malwarebytes was switched from one e-commerce provider to another. That was a lot more robust in subscription auto retry. And we've improved our retention by 15%,

Which is huge, just from a different

Payment processor. Don't let the mechanical fails actually drive your retention, but on the voluntary proactive efforts, that's where it gets really a lot more interesting. So obviously anytime anybody cancels with you, please ask them, why is the price too high? Are you going to come back survey at the time of the cancellation? Don't start them afterwards in product. If they're canceling auto renewal survey them, you will get plethora of information for you to act on. Most importantly, communicate with the customer. If you can at least track, what's actually your activity metric for you to predict if the trend event is actually going to happen, does engagement cause retention hopefully, but not always. So at Survey Monkey engagement meant creating a survey or receiving a response on the long running serving. We basically knew that if you either continue receiving responses or you create another survey that has the highest causation to improve retention, but it was very difficult to do because to create another survey you had to have intent.

And a lot of times we had to create that intent for you create an intent in the product for a customer is very difficult. What if you boss just asked you to run a survey, how do we tell you to do it again and take your time and do it again? But then you would see so much more value out of the product. You would see your results over time, and then you'd be basically guaranteed to stay with us, but it was very difficult intent creating problem. Malwarebytes has been absolutely fascinating. When I started two years ago, we had our churn and I'm like, what, what can we do to change it? What can we do to change it? And we looked at anything and everything possibly try to correlate to churn from the product actions. Did you run a scan? Did you hit a real-time protection event?

Did we find anything in your skin? Would, how many times did you scan nothing correlated to retention? It was mine boggling. We couldn't figure out how can nothing that we do. How can no interaction with the product actually cause retention. It took us a couple of months until it hit us. The head we're installation based product can buy online, but you have to actually activate it on the device and to activate it on a device, you have to punch in the license scheme. The only predictor for retention for us has been how many people punch in the key versus not how many buy and never utilize their key versus how many people buy in utilize. And that utilization metric, wasn't the only thing that mattered for retention and that masked all other engagement metrics within the product. So we became, it was mission critical utilization.

How do we drive utilization of our product? How do we auto activate? If you make it in purchase, purchase inside the product, if you make it on the web, how do we communicate to you? Activate your key, activate your key, activate your key. And that started driving retention. So it's not always as obvious you have to do an action in the product sometimes depending on how your product is set up, it actually something much bigger can be masking any possible action that can happen in the product. So to me, it was fascinating because I was so God, I was so sure that there has to be something that you do in the product to do it, but we were literally, we have no idea yet. So in our first cohort of users and our first year, right now, we're just improving utilization, which we have.

We moved it from being in the mid sixties to 90 in just couple of months. And now we're gathering data to understand the next level of what now will impact retention. So take it in the baby steps, but at least it was a data-driven decision and we're able to make an impact and then step four, make a hypothesis. So after you understand actual mechanics of what that lever, that KPI, how do you actually figure out what to do about it? It's difficult. Everybody has a great idea. How do you prioritize it? You obviously need to listen to your customers. I mean, especially coming from Survey Monkey, I run surveys all the time. It's important. You need to understand what they are saying about your product. You need to scope the landscape. I'm subscribed to so many products that I receive all of their communications, all of the notifications. I want to know what everybody else is doing.

Pick the best idea. Only you will know your customer in your business to figure out what is the actual best idea, evaluate its difficulty. Sometimes your infrastructure will just not allow you to, to execute on that. And then you'll actually go out and do it. Now, how do you make sure that actually this process works because moment you start breaking it down into actionable pieces. You again, when the revert to this process oriented results, I will complete this action. I will launch this email. I will launch this campaign. I will release this feature. I will improve this communication. And then I'm done, as opposed to saying, this is still our goal. And we have to make sure that launch is just one of the pieces of the puzzle. You go from the development, you go to actually launching it and then you drive to an outcome.

So don't stop at the release. Don't stop at the launch of activity, stop and celebrate. Once you did right, did got to the outcome. And for that, you need to make sure that you talk about that outcome all the time. So talk about your KPIs that you want to impact. Talk about your learnings from it, prioritize it, try to test as much as you can because not AB testing. This is my favorite quote is on my LinkedIn profile, but not AB testing is testing on the a hundred percent of users just without any quantifiable learnings.

If you have traffic, why not test it,

See what it does sometimes you don't have ability to do so. That's okay. Take a guess, take a leap of faith.

But if you do what's the downside

Side is that you actually know what you did. The other downside is that you can fail

Celebrate failures.

Now I don't mean that to say, okay, I failed. I'm going to, um, um, I'm going to really launch a big party about this, but look a failures there's learning. So you actually had a hypothesis that something's going to happen. And they didn't. That's the most valuable value metric internally that you should have you've assumed wrong, which means that you will learn something new about your customers. And that will launch additional analysis and understanding of how you should form your hypothesis. But oftentimes we're ready to fail by like humans. We want to succeed. We want to show winnings to our bosses. So as much as you can drive the failure as a learning, learning is a very valuable experience for us. And now we can do better. As a result is important on my growth team, on my experimentation team, I have it set as their KPI. They have to fail 50% of the time


Because if they don't, all they do is either change colors on the buttons, which are safe bets. And I don't want them to do it, or they don't think big enough because they're afraid to fail. So to say, you actually have to reach and you have to guess, and it's okay to fail. That's actually amazing because we need to talk about it afterwards and figure out why to me is the most important thing that you can do. That's all I have. Thank you very much.