The Preset Experiment That Boosted Revenue by 30% in Just 30 Minutes of Work

The hidden impact of presets lying right under our noses

0. Presets?

Also known as "default values," presets refer to the initial settings or configurations within an application. Whenever new features are launched, there's always a debate on how to set these presets. We usually rely on user interviews, research data, or our own "gut feeling" to assign the setting we think users will want the most. After launch, however, we often get so used to that preset that we leave it there untouched, treating it like a permanently fixed product spec.

But we shouldn't forget the powerful impact of presets, as they essentially dictate the user's very first action. Depending on what you set as the preset, your target metric can fluctuate wildly. Specifically, if you feel like a newly released feature isn't gaining traction, it's worth reviewing the preset. I’d like to share a recent case study from Alarmy regarding this.

1. Launching Annual Subscriptions and Growth

Alarmy's premium subscription products come in two types: monthly and annual subscriptions. We launched the monthly subscription first, followed by the annual subscription as part of our subscription growth strategy. We then ran experiments with various price sets as test groups and rolled out the winning price set across the board. Through these successive growth experiments, we thought our subscription revenue would steadily climb. Indeed, looking at each experiment individually, the test groups won by significant margins, proving it was a clear case of growth.

You can read more about how we achieved this growth in Jason's previous posts:

Why Annual Mobile Subscriptions Are Better Than Monthly
Subscription Price Testing (1)
Subscription Price Testing (2)

2. Re-evaluating the Monthly Subscription

Early in 2022, we revamped the purchase screen. We wanted to address an issue where users were unaware of the monthly subscription because we had overemphasized the annual one. Since we had high confidence in the annual subscription and were still monitoring its mid-to-long-term metrics, we debated whether it was even necessary to make the monthly option more prominent. However, we felt the existing screen used a "dark pattern" against the monthly option, so we deployed the update directly without an A/B test. Despite just comparing before and after, we saw an astonishing change in metrics.

Monthly subscriptions jumped significantly without causing a major drop in annual ones. In other words, we didn't just cannibalize users who would have subscribed annually anyway by showing them the monthly option; instead, we managed to capture users who would have otherwise dropped off without subscribing at all. This made us re-evaluate the power of the monthly subscription. Had we subconsciously been pushing the annual option too hard?

3. The 1-Year Mark for Annual Subscriptions

You can't judge the effectiveness of launching an annual subscription in the short term. Since the subscription lifecycle spans a full year, we need to observe it for at least that long to properly evaluate its impact. Depending on its actual performance, we might need to adjust the absolute price of the annual subscription or tweak its relative balance against the monthly one.

Alarmy's annual subscription is 30% cheaper than 12 months of the monthly plan. So, looking purely at the price, an annual subscription carries a Lifetime Value (LTV) equivalent to 8.4 months of a monthly subscription. But since some annual subscribers request refunds, factoring in the refund rate pushes the LTV slightly lower. Also, because an annual subscription requires a larger upfront payment, its trial conversion rate (Trial CVR) and trial-to-paid conversion rate (T2P CVR) are lower than the monthly option. So, how much higher is the annual subscription's LTV (+), and how much lower is its conversion rate (-)? Ultimately, are we generating more revenue?

At the 1-year mark, we calculated and compared both the LTV and the subscription conversion rate based on historical data rather than estimates. This gave us some incredibly valuable data unique to Alarmy.

• The annual subscription showed an LTV that was N times higher than the monthly one, but a conversion rate that was M% lower.

• The value of N came out very close to our initial estimate. The annual LTV is definitely excellent.

• The value of M was lower than expected. Users find the annual commitment more burdensome than we thought.

Suddenly, a thought crossed my mind: our purchase screen's preset was set to the annual subscription.

4–1. Spotlighting the Preset

I started doubting the "preset = annual" configuration that had always been hiding right under our noses. When we launched the annual subscription, we set it as the preset without a second thought, and in all subsequent price tests, every price set kept the annual option as the default.

What if the preset had been the monthly subscription? Post-payment retention is a lagging metric, so it likely wouldn't be heavily affected by the preset. In other words, the LTV would probably remain at the N multiplier regardless of what the preset was. However, I had a strong hunch that the Trial CVR would be massively influenced by it. After all, hadn't we just seen the conversion power of the monthly subscription during our purchase screen revamp earlier in the year?

Setting the burdensome annual option as the preset versus setting the less intimidating monthly option as the preset—which one would ultimately generate more revenue? I was curious. It was obvious that the sheer number of subscriptions would be higher in the test group where monthly was the preset, but factoring in the annual option's N-times-higher LTV, it wasn't easy to predict which would bring in more cash.

4–2. Let's Just Book It

At the time, our squad was fully absorbed in developing "Multiple Missions," a highly requested new feature. Even though we were fiercely guarding against context switching, we decided to run this experiment first thing in the quarter. We believed the impact and our confidence in it were high, but the biggest factor in our decision was that it was a simple experiment taking less than 30 minutes in total from product spec to design and engineering. And just like that, swoosh—it was deployed.

5. Picking a Winner

Both the control and test groups generated annual and monthly subscriptions. Influenced by the presets, the control group (where annual was the preset) saw more annual subscriptions than monthly ones, while the test group (where monthly was the preset) saw more monthly subscriptions than annual ones.

For the monthly subscription counts, we simply multiplied them by the monthly price. For the annual subscription counts, we multiplied them by the annual price, divided by 12 to convert it to monthly revenue, and then multiplied by N, since annual subscriptions are retained N times longer than monthly ones. By doing this math for both groups, we finally arrived at the total revenue for the control and test groups.

6. A 30% Increase in the Test Group

The test group, where the preset was the monthly subscription, clearly showed an improvement of around 30%. Android, where we felt a deeper lingering unease about the original annual preset, saw a larger jump, and the improvement varied slightly by country. The influence of the preset might differ by OS and region, or perhaps the perceived burden of an annual commitment varies among them.

The thrill of victory was brief, quickly followed by a dizzying realization. With presets wielding this much power, were there other screens we were completely missing? I started questioning whether the presets we had historically chosen for specific reasons—like the "mission" to dismiss an alarm, the alarm "ringtone," or the "volume level"—were actually optimal today. It felt like a few simple experiments could massively boost overall retention.

The preset experiment was rewarding not just for the meaningful revenue bump, but for the many lessons it taught us. It prompted us to look closer at other things hiding right under our noses, and it solidified my conviction as a PO to never ignore that "lingering unease" but to tackle it head-on.

Now, it’s time to loop back to price testing. The winner of our annual vs. monthly price test a year ago came from an experiment where both the control and test groups had the annual subscription as the preset. If we can find the optimal price set in our current state—with the monthly option as the preset—it might just drive another 30%+ revenue improvement.

Here's hoping my next post will be titled "The Price Test That Boosted Revenue by 60% in Just 60 Minutes of Work"...

I hope this encourages all product managers reading this to go review their existing presets. :D

👉 Check out our Instagram for a peek into daily life at DelightRoom