How to Analyze a Drop in Revenue
Breaking Down and Structuring Ad Metrics
Why did revenue drop?
There are times when we need specific answers to why data behaves the way it does. Especially when it comes to business revenue, members are highly sensitive to it, so we naturally treat it with high importance and tend to actively dig for the root cause. If there is a sudden drop in revenue, we want to quickly find out why and respond. Conversely, if revenue spikes, we need to figure out what drove it so we can use that as a basis for our next moves.
The problem is that a metric like revenue is a lagging indicator—the result of various ongoing efforts—so the causes behind its fluctuations are incredibly diverse. There are simply too many factors influencing a top-level metric like revenue. Even if there is a correlation with other metrics, if the direct link isn't obvious, it’s hard to intuitively grasp what exactly caused the impact. While something like a "decrease in revenue per user" is directly linked to "revenue" making it easy to infer causality, something like a "decrease in product keyword search ratio" might seem less directly related at first glance, making it trickier to nail down the relationship.
Ad Revenue
We often run into similar situations from an ad monetization perspective. Changes in the "ad revenue" business metric are something our members take very seriously, making it crucial to pinpoint the exact cause of any shift. However, it's often difficult for the following reasons:
Limited information due to platform dependency
To monetize efficiently and effectively, many publishers rely on platforms (e.g., mediation, SSPs, etc.) for ad monetization. While using these platforms brings many benefits, the downside is that because we rely on them, it's almost always difficult to check data at the resolution we'd like. Also, when using multiple platforms, the information and definitions they provide differ slightly, making it challenging to treat all that data under a single standard.
2. Unpredictable metric fluctuations
The ad domain is full of various players. With publishers trying to monetize through ad inventory, advertisers wanting to place ads in that inventory, and various intermediary platforms tangled up like a spiderweb trying to achieve each side's goals, the volatility is immensely high. Since the revenue metric that publishers want to maximize is determined by a combination of changes in various factors beyond just the publisher's inputs, it's extremely hard to predict and equally hard to judge the pure impact of any single change.
Building a Metric Hierarchy
That's why it's so important to properly establish the hierarchy and relationships of the metrics being managed within your product's ecosystem. A Metric Hierarchy is a way to manage metrics so that their relationships are visually or intuitively easy to understand, taking into account their leading/lagging relationships or relative importance. By mapping out the relationships between various metrics well, you can pinpoint the causes behind changes in top-level metrics or sharply define the actions you want to attempt from a strategic business perspective.
The metric "Profit" consists of the detailed metrics "Revenue" and "Cost" (Profit = Revenue — Cost). Changes in profit can be explained by changes in revenue and cost. If you repeatedly map out these direct relationships from the very top-level metric down to the lower-level metrics, you'll eventually draw out a relationship map for the entire product. Initially, just knowing that things are related or affect each other is a good enough start, but it becomes even more meaningful if you can figure out the exact exchange rate of how a change in a lower-level metric impacts the top-level metric.
Even with limited information, you can elevate your understanding of the ecosystem inductively by accumulating trial and error and gathering lessons. At first, even if you can't logically understand why a change in A leads to a change in B, after experiencing a few similar cases, you can form hypotheses and build up your own learnings one by one. To do this, structuring the "hierarchy" of your metrics within the possible scope is absolutely essential.
Drawing the Ad Metric Hierarchy
Let's draw a simple relationship map for ad data. It's best to start with the top-level metric, ad revenue, and break it down into its individual components. Revenue is made up of a combination of the following components:
Just by adding this single layer, you gain valuable insights. A situation where revenue increases by 10% compared to the previous month could be the result of any of the cases below. By breaking down the simple fact of a "revenue increase" into changes in its components, our understanding of the phenomenon becomes much more transparent. And our response to it becomes much sharper as well.
10% increase in impression
10% increase in eCPM
5% increase in impression, 5% increase in eCPM
20% increase in impression <> 10% decrease in eCPM
20% decrease in impression <> 30% increase in eCPM
Just as we split revenue into impression and eCPM, we can similarly represent impression and eCPM as combinations of other, deeper components. Let's expand on what factors influenced the upper metrics. This time, let's break Impression down into a combination of components like this:
If we assume the cause of a 10% increase in revenue is a change in impressions, the specific changes in underlying factors could be as varied as follows:
10% increase in ad request
10% increase in fill rate
10% increase in view rate
Once we pinpoint exactly which factor caused the change in impressions, the root cause of the revenue change consequently becomes more specific. And the more specific the cause, the sharper our response can be, all while improving our understanding of the domain.
Let's explain the change in ad revenue using the components of Impression. Expressing it as a combination of four factors looks like this: (Ad request, Fill rate, View rate, eCPM)
As a result, our resolution for understanding a 10% change in revenue has drastically improved. The following are examples of how detailed factors might have changed to produce that exact same result:
Example 1) Ad Request +5%, Fill Rate -10%, eCPM +10%, View Rate +5%
Example 2) eCPM +10%
Example 3) Fill Rate +10%
....
Similarly, we can continue expanding Ad Request, Fill Rate, and View Rate as well. I believe that drawing out the entire relationship map of the metrics you manage in this way will vastly improve your overall understanding.
Wrapping Up
The entire process of finding insights through data and trying out new things is incredibly important. But the fundamentals must start with properly checking and tracking the state changes in our product. We need to look back and ask ourselves if we're just repeating new attempts without even knowing the current state of our product's metrics. Throughout this process, I think it's worth rethinking how to structurally manage metrics so we can quickly grasp metric changes and accurately identify their root causes.