How We Analyze DAU at Alarmy

If you want to increase active users, check this first!

How is your DAU chart doing?

Many services define Active Users as their primary metric and work hard to increase it. Cumulative metrics that naturally build up over time, like total installs or total registered users, are often categorized as vanity metrics. However, user activity metrics like DAU and MAU are recognized as solid indicators of a service's growth and are heavily considered in IR and PR.

Most services monitor their Active Users using a chart like the one below. You put the date on the x-axis and the number of active users on the y-axis. They track DAU and MAU, and depending on the company, might add an engagement metric calculated via DAU/MAU.

Monitoring the daily active user count is great, but just looking at this chart makes it hard to answer questions like "What should we do to increase Active Users?" or "Are we doing a good job with our activities to boost Active Users?" There are hardly any decisions you can make just by looking at this chart. The number of Active Users itself is a classic lagging metric.

Because of this, many services also track leading metrics that seem likely to impact Active Users. The most common ones would be number of sign-ups (or installs) and retention (or repeat rate). If you have a lot of sign-ups + a high percentage of users using the app consistently so that fewer users churn, your Active Users will naturally increase. So, looking at acquisition and retention metrics as leading indicators seems completely rational. But is this enough?

I once read an article titled Repeat rate is a vanity metric. I don't agree with everything in the article, but it pointed out some points I hadn't thought about before, making it an incredibly fascinating read. I'll briefly introduce the points I resonated with (along with my own thoughts added in).

• The original article used "repeat rate" closer to the concept of a repeat purchase rate. Since we're talking about active users here, I'll explain it in terms of retention rate.

• Most importantly, the conclusion I arrived at is slightly different from the original article's conclusion. ^^;;

Is a 75% monthly retention rate a positive number?

Let's assume there is a service like the following.

A service where 75% of users return in M+1—isn't that incredible? (It depends on the service category, but generally, even a D+7 retention of 30% after install is considered pretty decent;;;) On top of that, if new users steadily flow in every month, you could probably consider this an excellent service. What would the monthly active user graph for this service look like?

This is the Active User chart for Service A, which satisfies the assumptions above. The X-axis represents the period (months), and the Y-axis represents the number of Active Users. Cohorts with the same install timing are marked in the same color. For example, users who first installed the app in January are shown in light blue, and 75% of these users show up as having returned in February. Similarly, 75% of the orange users who first came in February stick around and are active in March. You can call this a service where, as assumed above, 75% of visiting users return the following month + new users steadily flow in every month.

Looking at the graph, a surprising(!) fact emerges. It's a service with a whopping 75% retention rate, but after about a year, Active User growth pretty much stalls. If growth stops in just one year even though new users are steadily increasing and the retention rate is 75%, then exactly how high does the retention rate need to be for Active Users to keep growing continuously?

Let's look at another one, Service B. The beginning is modest, but you can see Active Users increasing along a J-curve at an incredibly fast pace. It looks like a completely different graph from Service A above. In stark contrast to the stagnant Service A, Service B looks like it's going to grow even faster in the future. So, what is the monthly retention rate for Service B?

Surprisingly, Service B's monthly retention rate is 75% (the same as Service A). Similarly, if we look at the monthly cohorts separated by the same colors, we can see that 75% of the active users from the previous month are returning the next month. If both Service A and Service B have an identical 75% retention rate, what makes their Active User graphs so different? The difference lies in the number of new users coming in each month. Service A has a constant number of new users flowing in every month. On the other hand, the number of new users joining Service B steadily increases by 20% each month. (In the original text, this is expressed with the term CCGR = Cohort/Cohort Growth Rate).

If there's such a dramatic difference depending on the number of new users even for services with the exact same retention rate, does it make sense to focus the whole company entirely on acquisition rather than retention? Should the marketing team just drop everything else and grind away at acquisition marketing as the best way to increase Active Users? (Nope...)

The original post wrapped up with the conclusion that the retention rate metric shouldn't be used in isolation, but rather as a function of CCGR and Churn. While I partially agree with this, I think we need to pay attention to another point entirely.

There is one thing these graphs are missing. Hidden in these graphs is the assumption that "a user who churns once never comes back." In other words, no matter which cohort you look at, Active Users linearly decrease at a constant rate, and churned users do not return. But realistically, user activity isn't expressed in such a simple form.

In the DAU or Retention charts we typically look at, user behavior is displayed very flatly.

• Logged in / Did not log in

However, the user activity patterns we actually encounter in a service are incredibly diverse and multidimensional.

• Signed up

• Churned right after signing up

• Uses it consistently

• Can be considered a core user

• Is on the verge of churning

• Used it well for a certain period, then churned

• Churned, but came back

• …

Duolingo, well-known for its language learning service, has previously shared that they have a Growth Model that breaks down the top-line DAU metric into several meaningful user segments for analysis. When you actually break down activity metrics and look at them this way, you can gain insights on a completely different level compared to simply monitoring the Active User count.

Alarmy User Analysis Framework

Alarmy is analyzing Active Users in a similar way, dividing the Active User metric into various Segments and breaking down the transition trends between each Segment over specific periods. Due to the nature of an 'alarm' app, which is used frequently over short cycles, it's crucial to quickly catch continuous usage or early churn and respond appropriately. To easily check the level of user activity by Segment—something that isn't revealed by simple DAU metrics—and to analyze Segment Change paths, we created and utilize the following user analysis framework.

Alarmy's user analysis framework is divided into User Segments representing Stock and User Segment Changes representing Flow. Since it's confidential, I can't share the specific criteria or metrics, but our data pipeline and dashboards are set up so we can easily look up detailed metrics in the following formats.

• By separating User Segment (Stock) and User Segment Change (Flow), we can see how related metrics move on a daily basis.

• We can grasp the flow of metrics by dividing them into broad categories: 1) Acquisition, 2) Activity, and 3) Churn.

• By tracking the changes in User Segment metrics over N days, we can figure out the cumulative amount of change over a period.

Being able to break down activity metrics into various forms like this has allowed us to check data at a much higher resolution when running experiments to increase Active Users. Rather than simply sending an app push and watching the DAU go up, we can see if enhancing service features so users check in more often (e.g., wake-up challenges) actually has the effect of converting Light Users into Heavy Users. Also, being able to see segment changes for New Users or Heavy Users after N days gave us hints on whether the onboarding journey for new users is structured well, and how to design the churn prevention process for core users. Based on this data, we are internally preparing to execute much more granular user communication and CRM actions, and I'm really looking forward!!! to writing a new post in the near future featuring steeply rising DAU metrics. I hope it works out well. Haha.