The best to track the effectiveness of product changes is to measure their impact on revenue. We measure the impact of onboarding changes, for example, by calculating the effect they have on the first revenue event, signup to paid conversion.
Growth teams pay a fair amount of attention to signup-to-paid conversion. But it's equally important to measure how many people make it to each revenue event beyond the first. Look at a weekly cohort of users (the number of people who signed up for your product during a given week); its crucially important to measure the number of them that make it to rev one, but it's as important to measure how many of them stick around for the second payment event, and the third and so on.
Tracking decay curves (how many users you lose per revenue event per cohort) is important for two big reasons:
- You have a better sense of your cohort-to-cohort profitability: If you're tracking all the event to your CAC payback period, you know how many users per weekly cohort are sticking around long enough to pay back the amount that you've invested in bringing them to the product. If you use cohorts, you can tell whether the percentage of the signups that you're getting in one given week is going up or down as you continue week over week. This gives you more granular and recent cashflow information than a metric like MRR.
- You can turn revenue into a leading metric: Say a 1000 people signed up for Plan A at $10 in the week of September the 6th. Say 600 of them made it to Rev 1 on October the 1st. You can look at past cohort patterns and estimate the drop-off you expect on November the 1st. Working backward from the expected drop-off, you can estimate the amount of revenue you expect to receive from the cohort on November the 1st (Rev 2). Say you expect 400 users from thee cohort to make it to Rev 2, equalling $4000 in revenue. If you can make product or path changes that result in the actual number beating the expected number, you have a clear and reliable indicator that the changes are working for the better.
From: VPP Episode 2 (30.19)