In Product Engineering, one of the major charters is to evolve the product by building new features. It’s often the hope that a new feature will change usage patterns to unlock step function growth.

Building something that unlocks step function growth is extremely challenging, especially with respect to mature products, and only occurs at the intersection of strong execution and feature/market fit. The most recent example that comes to mind is Instagram Stories, which not only curbed competitor Snapchat’s growth, but accelerated Instagram’s YOY growth from 25% to ~40%…

Pinterest Product Engineering managers recently did a look back on a variety of projects we worked on over the past several years. While the Product Engineering team launched a number of succesful products like Lens, Sections, and Skin Tone Filters, there were a fair share of features built that never got traction and were ultimately shutdown.

During the lookback we evaluated each of the features we had invested in over the past several years, and discussed whether we felt the projects were successful. After discussing each project, we determined that we evaluated the success of a project not by whether it shipped but on its return on investment for the product and the company.

For instance, there was a project we built around ARKit in late-2016 with two engineers working a few hours a week. The engineers built a prototype of viewing objects within pins in augmented reality, but in the process determined that the existing corpus of AR content was both scarce and low quality. The recommendation at the end of the project was that the company should hold off on AR until there was a critical mass of high quality AR content.

While the project ultimately did not ship, it took less than 20 hours of engineering time to deliver this insight and it ultimately helped to inform strategic direction of the company. As a result, we considered the project a success.

The projects that were the least successful were the ones where we really overinvested on a single execution, which neither delivered the impact we expected nor resulted in learnings that were as valuable as the time invested. Even with the benefit of hindsight we felt that each of these projects was worth an exploration, and may have considered them successful had we delivered the same learnings with far less investment.

It’s clear to me that ultimately this type of failure is a result of the team structure, and as such lies with management. In each case, when staffing speculative features with the hope deliver step function growth, it would’ve benefited the product and engineering managers to determine the minimum viable team required to deliver the initial project.

The benefits of staffing a minimum viable team on a risky project go beyond helping to control the costs associated with delivering the results (whether that be in the form or impact of learning):

  • Provides constraints to help focus the team.
  • Reduces the thrash involved with changing direction.
  • Avoids sunk cost fallacies associated with a single execution.

In contrast, the most successful projects were the ones started by a small group of engineers (1–3), and only expanded the size of the team and investment in the project after determining the promise or success of the initial execution. These projects also had the benefit of shipping faster because they had been significantly derisked by the time a larger team started working on it.

While there were projects we overinvested in, fortunately they were few and far between. I have no doubt we will continue to fund projects that are speculative and risky, and it’ll be on us as managers to ensure that we’re staffing these projects in accordance with their minimum viable team.