Does User Feedback Actually Help Improve the Product?
Customer Service: Do we really have to do it?
I'm sure a lot of startups wrestle with how to handle Customer Service (CS).
Because the importance of user feedback is constantly preached everywhere, many companies give it a shot at first. But I've seen plenty of them push it down the priority list because the payoff just isn't clear. Like anything else, for the importance of this work to truly hit home, you need visible results. In other words, if CS doesn't help improve the product or drive growth in our target metrics, it's inevitably going to be back-burnered.
(DelightRoom is also constantly racking our brains to ensure that CS doesn't just become a mindless routine, but actually helps grow or improve the product.)
In the case of Alarmy, the alarm app DelightRoom runs, we grew by leveraging user feedback so well that you could say being "Customer-driven" was our core strategy. All our members still pay close attention to user opinions, but the environment has changed too much to handle it the old way. As Alarmy has grown, the daily influx of user feedback has skyrocketed, making it impossible for engineers to read every single comment and make decisions based on gut feeling and intuition like they used to.
For this reason, DelightRoom formed a dedicated CS team, and we urgently needed to establish a process for the work.
What the DelightRoom CS Team Does
In the early days of taking on this role, I couldn't even get through the daily flood of customer feedback (a.k.a. tickets) if I spent my entire day on them. Before I knew it, I was blindly churning through tickets, unable to even answer fundamental questions like, "Why are we doing CS? How does this activity actually help the team?" After naively wondering if slapping some trendy machine learning on it would save time, I finally decided to start from scratch: "Set a clear goal and find the most optimized way to achieve it."
DelightRoom's CS work can largely be divided into two activities.
The first is directly responding to customer feedback with comments and replies, and the second is extracting meaningful information based on what we gather while processing tickets. Both of these activities must directly or indirectly influence our product improvement direction or growth metrics.
Goals of the DelightRoom CS Team
1) Maintain high app ratings to contribute to UA (User Acquisition)
2) Derive insights for product improvement
3) Quickly detect issues and minimize damage
Every day, once ticket processing is done, we share a one-page "Daily Report" summarizing the day's results with the team. Out of this report, we compile insights for product improvement and consistently share and debate them with teammates during our weekly team meeting (DelightRoom holds a weekly meeting covering the overall status of the company), filtering out what needs to be moved to the planning backlog.
Of the three goals mentioned above, I want to talk a bit more in-depth about 'Deriving insights for product improvement.'
Finding the Insights Hidden in User Feedback
User feedback is highly engaging data that customers spend their own time and effort to create. And usually, the core message they want to convey and the reasons behind it are laid bare. Because of this, user feedback gives us a fantastic opportunity to look at our service through the user's eyes. Trying to empathize from the user's perspective is incredibly helpful for finding valuable information when it comes to product improvement. (On the flip side, we always have to be on guard against hasty generalizations—asking ourselves exactly where on the bell curve a single person's feedback might fall!)
User feedback requires a collection pipeline and a trigger to prompt user action. In other words, the context in which feedback is collected is already largely predetermined, and we can use this contextual information to understand the user's perspective much more deeply.
No matter how much we put ourselves in the "customer's shoes" and uncover penetrating insights for the product through "empathy," it only holds true value if it's shared with the team in an appropriate format. Only by going through this process can we reach the ultimate goal of improving the product through CS.
| How Alarmy Used Customer Feedback for Product Improvement
Alarmy user feedback pours in by the hundreds every day. Data accumulates from about six different feedback collection channels based on their respective contexts. Fortunately, with the help of our god-tier engineers, the collection and processing workflow has gotten much easier, and we're now able to gather various supplementary data alongside it.
While processing the daily quota of tickets, my brain goes into overdrive trying to 'find insights.' (This part inevitably relies on the ticket handler's subjectivity. However, this is backed by the belief that the person handling them is more intimate with the product than anyone else, and therefore has a finely tuned radar for it.)
When you look through thousands of tickets on the same topic while genuinely trying to empathize with users, a moment eventually comes when the familiar suddenly feels completely foreign. That exact moment is when all your hard work sprouts into an 'Insight.' And these little sprouts are later stored in the planning backlog for product improvement, going through an incubation period until they finally bloom.
For example, Alarmy's 'Emergency Dismiss' was a feature that always divided opinions among teammates. (Emergency Dismiss is a feature provided for emergency situations where a user can't dismiss a mission alarm for whatever reason). The trade-offs are incredibly stark depending on the policies decided during planning, and this divide showed up just as clearly in customer feedback. The effort to empathize with opinions about the 'Emergency Dismiss' from the user's perspective gave us an out-of-the-box viewpoint and handed us the clues to solve a problem we had been struggling with. Currently, that specific feature is cooking in the planning backlog, waiting to be reborn as an improved 'New Emergency Dismiss.'
One thing to note is that with qualitative analysis alone, no matter how loud the person in charge yells, 'This is what users think!', it can end up falling on deaf ears. So, we're always trying to back up our claims with hard numbers. In particular, for product-related issues that we manage closely, we track them by turning things like the number of tickets related to a specific issue against the total ticket volume into a metric.
Wrapping Up
Personally, I'm quite interested in anthropological approaches to user research for product improvement. I'm especially keen on finding ways to utilize Digital Ethnography (or Netnography), which merges these approaches with user data collected over the internet. From this perspective, I think there is still so much that CS can do. To achieve DelightRoom's ultimate mission to "Help people wake up, Effectively," we're mulling over various ideas to build a CS team that is more proactive and experimental, while also being a huge catalyst for product growth. I hope this post opens up opportunities to share diverse opinions with anyone interested.
We’re hiring!
If you're curious about what roles are open, please click the link below :)