Choosing Usability Over a 2x Revenue Boost
Should we add interstitial ads or not?
In late 2019, the AdCarry squad, which is tasked with ad monetization, started talking about interstitial ads.
At the time, many major alarm apps were showing interstitial ads right after users dismissed their alarms, and these ads had significantly higher ad rates compared to other banner ads. We ran a quick internal simulation and found it could drive at least a 1.5x increase in ad revenue.
Honestly, from a usability perspective, interstitial ads hadn't even crossed our minds. But the discussion eventually shifted to, "Let's at least figure out what the business upside is and what the usability downsides might be."
Ultimately, we decided to test interstitial banner ads and use this opportunity to think deeply about the balance between ad revenue and usability, so we launched an experiment.
Experiment Results
We ran the experiment on 150,000 users in the US for two weeks. As our key metrics, we tracked ARPU (Average Revenue Per User) to measure the revenue boost, D1–D7 retention, and negative user feedback regarding ads.
ARPU jumped about 1.6x. We projected that with our standard internal ad optimization settings, we'd easily see over a 2x increase in revenue. (For the sake of a fast experiment, we ran this without doing ad optimizations like mediation setups.)
Retention, surprisingly, didn't drop. We checked D1 through D7, but there was no difference between the control and experiment groups.
In terms of user feedback, we received an average of two ad-related complaint tickets per day.
The conclusions we drew from the experiment results were as follows:
There are users who feel inconvenienced in terms of usability (to the point of actually expressing frustration through feedback). However, it's not a severe enough inconvenience to drop retention; it's something they can tolerate while using the app.
The Start of Thinking About "Ad Principles"!
We had discovered an opportunity to double our ad revenue with no clear drop in retention, so in theory, it seemed like a no-brainer to implement it. Since this was before we launched our premium subscription model, it literally meant "2x ad revenue = 2x company revenue."
That's when the deep pondering began. "Does that mean it's right to keep adding ads for monetization right up to the point where retention drops?", "Is retention a metric that captures everything about usability?" and so on.
No matter which point we chose between usability and revenue, there was no single "right" answer. We realized what mattered most was the direction we wanted to pursue. Ultimately, after deep discussion and deliberation, we came to the following conclusion:
Monetization through ads is a means to build a great product, not the end goal.
As a result, we established sharper ad principles for when we incorporate ads into Alarmy. During this process, we set up several quantitative & qualitative principles. Here are a few representative examples:
Quantitative Ad Principle: Retention must not drop.
(We generally look at D1 to D7, but depending on the case, we check up to D14.)Qualitative Ad Principle: The user shouldn't feel like there are too many ads.
(Qualitative principles can be subjective, but having a baseline made alignment on our thoughts much easier.)
In a way, you might misunderstand and think, "Aren't you just chasing usability while missing out on business opportunities and neglecting revenue?" But it's not like Alarmy blindly sacrifices ad revenue for usability. In fact, if you look at our annual ad growth rate:
Compared to 2020, our 2021 ad revenue grew more than 2x, from 3.3 billion KRW to 8.8 billion KRW. What’s even more encouraging is that this growth came not from adding more ad placements, but from improving ad efficiency.
In other words, rather than simply showing a ton of ads to create business upside, we increase ROI on the same ad placements through ad efficiency optimization, essentially safeguarding usability in the process.
What We Gained
We validated the possibility of doubling our ad revenue. (+ We also gained the confidence of knowing, "We have a seed here that can double our ad revenue whenever we want.")
Although it wasn't implemented into the product, the interstitial banner ad testing process helped us establish Alarmy's very own ad principles, and we still use these principles to make decisions about ads today.
Through our discussions, we were able to find a balance point and achieve alignment on what everyone thought about "product monetization" <> "product usability."
👉 Check out our Instagram for a sneak peek into daily life at DelightRoom