What happens when an engineer focuses on efficiency at a startup?

Streamlining collaboration at a startup

At a startup, time equals survival. Every startup gets the same amount of time, and while many companies and people work hard within it, how that time is spent ultimately dictates the company's growth and revenue.

DelightRoom operates in 2-week sprints. Assuming no holidays or PTO, a 2-week sprint gives us 80 hours of available time (8 hours x 10 days). If we subtract meetings and buffer time (we set aside 25% of our available time as buffer), we're left with an average of about 40 hours of actual working time.

How should we use those 40 hours?

Because our 2-week sprints offer a short window for development, everyone needs to work at peak efficiency to deliver peak performance.
In other words, our performance is determined by how efficiently we use our given time.

Not just using time efficiently, but
making it easier to use time efficiently

Collaboration efficiency is much more important than individual efficiency.

I'm an iOS engineer in the Subs squad, which handles subscription-related work at DelightRoom.
To figure out exactly when collaboration happens the most from an engineer's perspective, I mapped out our sprint workflow in order.

01. MVP or A/B test experiment tasks are assigned
02. Development begins based on priorities
03. QA once development is complete
04. Inclusion in the release build after QA

Of the steps above, collaboration is absolutely essential in step 1 (task assignment) and step 3 (verifying that the completed work functions correctly).
Between those two, the stage that involves the most collaboration and poses the most challenges is step 3: requesting QA upon development completion. (This can vary depending on the MVP, but since our team deals with a lot of data, it's usually step 3.)
This stage requires heavy collaboration with the teammate doing QA, and often involves verifying the same task multiple times. Because of this, it's a frequent source of inefficient time use.

The frustration born of inefficiency

Since we're a growth team, we value data immensely.
Because of this, we analyze data and run countless A/B tests, which means we have to design events and implement A/B test experiments.
We execute both of these every sprint, and several bottlenecks popped up whenever we ran data-related QA for them.
(Maybe our data analytics tools just aren't very friendly to us...)

An engineer can easily check things via console logs, but that's not the case for the teammate doing QA. (You can trust computers, but the person writing the code should always doubt themselves. That's why verification during the QA phase is so important.)

If the engineer found it difficult to QA their own work during development, I imagine it's even harder for the PO or designer trying to verify it.

Turning inefficient time into efficient time during data QA

During the aforementioned data QA phase, our squad checks the following two things every single time:

• Checking if events are sent correctly at specific moments

• Checking if the control/variant groups are assigned properly

If a bottleneck happens here, the time used inefficiently stacks up with every sprint. I realized that cutting down on this wasted time would boost the team's overall performance.

I want to share what those bottlenecks were at these two points and what we did to improve them.

Checking if events are sent correctly at specific moments
What was the bottleneck here?

When we send an event based on a user action, we verify it through an analytics tool, and this is where the bottleneck occurs.

• Wait time until the client's event shows up in the analytics tool.
  Some tools support real-time data streams, but others don't, and they are heavily affected by network conditions.

If the wait time takes about 5 to 10 minutes, considering we tackle multiple tasks per sprint, we end up wasting an average of 20 minutes of wait time per person inefficiently.

With 2 sprints a month and 24 sprints a year, the snowball effect of the inefficiency caused by this wait time would be massive.

Reducing the wait time

To cut down on the wait time, I developed a way to check events directly within the app. (I had been wanting to try out Swift AsyncStream, and this was the perfect feature for it.)

The event console features the following:

• Show/hide the event console log using the left arrow

• Real-time event verification

• Select a specific event to check its detailed information

• Check A/B test control/variant group assignments

Using this feature, we were able to reduce the wait time not only for engineers but for all teammates who needed to verify events.

Checking if the control/variant groups are assigned properly
What was the bottleneck here?

To verify the behavior for the control group and various variant groups, you often have to reinstall the app multiple times, and the bottleneck occurs during this reinstallation for experiment assignment.

• App installation -> Check control/variant assignment
  If you need the variant but get assigned to the control, you end up having to reinstall the app.
  Or, you face the hassle of having to manually change the allocation to 100% control or 100% variant in the analytics tool.

Reducing reinstallation time

I developed a feature that allows you to change the control/variant group directly within the app.

enum ABTest: String {

    case control_none = "대조군 키값"
    case variant = "실험군 키값"
}

extension ABTest: ABTestable {

    var key: String { "abtest" /* 실험 식별자 */ }
    var id: String { rawValue /* 대조군 혹은 실험군의 키값 */}
}

• A struct or Enum type containing the experiment identifier and the key values for the control/variant groups adopts the ABTestable protocol.

• Using Sourcery, we create an auto-generated array object for currently running experiments. (I developed this so that the engineer only needs to create the experiment design, and Sourcery automatically generates it into an object that can be modified in debug mode.)

• In debug mode, you can view the list of currently running experiments and change the control/variant assignments for them.

Through this feature, we saved time for both engineers and teammates who needed to verify if experiments were functioning properly.

A good team is one that thinks about boosting team efficiency through process improvements, in addition to feature development.

'Our' time over 'my' time

The time I invested in building these two features to reduce wait times and reinstallation times was less than 3 hours.

As 'our' time passes, those 3 hours spent by an engineer will snowball, enabling us to efficiently use 10, 20, or even 100 hours.

Because time equals survival at a startup, small features built to use it efficiently can trigger a butterfly effect that drives the company's growth.

Want to join teammates who make things happen? 🙋‍♂