Junior Devin and Claude
This year, both the breadth and depth of features we need to look after in our product have grown significantly. Plus, given the nature of an acquired product, there were a lot more under-the-hood behaviors we didn't know about yet. While trying to act as the PO for two squads at the same time, I found myself in a situation where it was physically beyond what one person could handle. That's when two junior AIs stepped in to solidly fill the gap.
Devin is a junior Dev who knows the codebase inside out
Claude is a junior PO who runs alongside me from building product roadmaps to post-release planning after shipping the backlog
Devin: The junior Dev who knows the code and database inside out
I assign Devin four main roles.
[Reader] If I ask, "How does this feature work right now?", he reads the code, outlines the flow, and even pulls up the relevant numbers. Since he reads and structures complex code for me, I can save a lot of resources during the planning stage. This is especially useful in an environment where we have an acquired product with many behaviors we don't fully understand yet.
[Debugger] When a problem occurs, we dig into the cause together. He reviews various variables in the code that could be affecting things, and since he knows the past commit history, he can trace "how it used to work" as well.
[Developer] He can also handle simple code modifications directly. Of course, he could probably do more advanced engineering, but we're not quite at the stage where I can fully trust those results yet. (Since he can't take responsibility for them...) He also plays a role in resolving operational inconveniences with a single snippet of code. Honestly, I should be utilizing this role more, but I didn't use it enough this quarter. I'll aim to do better next quarter. He'll definitely shine more when the backlog is piled high.
[DA] He also performs expected impact calculations, cohort analysis, and cross-tabulation. For revenue data alone, the sources to reference vary depending on whether it's a web or app payment, and user activity values also require cross-referencing server and client values carefully. Devin navigates across multiple data sources like AMP, Stripe, and Qonversion to do this, and since he already understands our taxonomy, I don't have to explain it every time. Also, there are times when I want to run granular data analysis that might seem pointless (since occasionally we do find good seeds in there), and even then, Devin silently and swiftly brings back the results, giving me a great sense of relief.
Claude: The junior PO who partners on everything from product strategy to planning
Claude's main roles are threefold.
[Discussion Partner] This is the role I use most often. When the problem definition goes off track, when there's a better alternative, when there's a hole in the logic, or when a risk is missed — we catch it through conversation. In fact, I utilized this role in various situations, such as reviewing backlogs, organizing key issues, defining problems and planning solutions after onboarding, organizing squad retrospectives, and restructuring the onboarding flow.
[Document Editor] I tend to just throw raw data at him entirely to provide as much context as possible. He organizes raw data like my rambling train of thought, original user interview transcripts, and survey responses to fit the purpose of the document. Most of our major documents, like PRDs and analysis docs, are written by Claude. Especially this quarter, we conducted user interviews in English-speaking countries, so there was a lot of jumping between English and Korean in our documents, but he handled it so neatly on his own that I couldn't have asked for more reliable support.
[DA] Based on the provided data, planning intentions, and context, he validates hypotheses and extracts follow-up action items. While Devin has the capability to navigate the actual codebase and database, Claude has a sharper brain for reading and interpreting that data. Together, they produce meaningful analysis results.
There are also some minor uses. As a [Visualizer], he quickly whips up things like graphs, diagrams, and presentation decks, and as a [Copywriter], while not giving perfect copy, he helps brainstorm seeds. On the other hand, I rarely use him as a [Researcher]. Usually, when doing reference research, you really need to get your hands on the actual apps and check the screens and flows to truly understand them, which makes it hard for AI to replace.
Like managing juniors
Reading up to this point, it might sound like a single "click" gets everything done.
But at the end of the day, these two are juniors. And juniors with rather different personalities, at that. You need to lower your expectations a bit and delegate appropriately.
Devin is diligent but a bit clumsy.
He often mistakes the codebase or data source he should be referencing, so you absolutely have to cross-check his output. Also, if you don't clearly narrow down the area to investigate, he'll wander off pretty far into the wrong places before coming back. It's important to set a clear scope for him.
Claude is smarter, but requires a different kind of attention.
Because he can't grasp the entire context, he sometimes gets priorities wrong on his own, or adds too much unnecessary content, which actually hurts readability. It's better to boldly cut out anything that doesn't affect the overall understanding. In estimating impact, he sometimes applies assumptions to various variables, and the PO's intuition must validate whether those assumptions are appropriate. Direct review of the organized output is essential.
The same principle applies to both in the end.
They can't possibly know the context better than I do. I have to act as the highest-level brain.
However, treating them like juniors doesn't just mean recognizing their limits.
If you hand over the context well, define their roles clearly, and give them feedback, they increasingly get into sync with you, just like smart juniors do. They are the kind of teammates who do better the more you teach them.
AI technology is advancing so rapidly...
I'm hoping I might be able to give these two a promotion next quarter...
Oh, and I have a feeling things would be even smoother if I let them communicate directly with each other, but I'll save that for next quarter -