Product Operations is an emerging discipline that seeks to turn Product and Tech teams into well oiled machines, equipping them with processes to achieve operational excellence. While very few people were talking about Prod Ops even 5 years ago, the function has boomed across companies and industries in the years following, with a whopping 32% of companies estimated to have had a Product Operations person or team staffed in 2022 (“2022 Annual Product Report (opens in new tab)”, Product Plan).
Product Operations was established as a centralized function at Oscar in 2021 by Senior Director Clare Hawthorne (opens in new tab), with a north star vision to “unlock Oscar’s ability to ship more product, better, and faster.” We’ve been growing quickly ever since! Each of our 13 Product Operations team members works closely with two or three cross-functional “pods,” collaborating closely with Product Managers, Engineers, and Product Designers. We’ve found that this model and scope allow Product Operations members to build deep subject matter expertise within their domains and partner most effectively with their pod-mates. To assess the efficacy of our partnership, we send out a satisfaction survey to stakeholders two times per year, and to-date, this Prod Ops team structure has yielded scores that consistently fall into the “excellent” rating category.
The original business case for Product Operations at Oscar was to enable Product Managers to spend more time on “Discovery and Planning,” which includes activities within strategy development, goal definition, scoping, and design. Over the years, we’ve sought to find a way to quantify our progress against this goal.
In 2021, we introduced a survey to our Product Managers and Product Operations team members to learn more about how they were dividing their time across the full Product Development Life Cycle (PDLC). The PDLC is the succession of stages required to get a product or feature from ideation to refinement. Our goal is to empower all parts of Oscar’s Tech org to focus on the work they do best, and this survey provides us with transparency into the areas in which time expenditure can be shifted to enable each team to deliver maximum value to the business.
When we sent our first survey, the majority of Product Engineering pods were not yet working with a member of the Prod Ops team, so we were able to gather valuable baseline data on the time that each team was spending in the categories of “Discovery & Planning” vs “Delivery & Runwork”.
Together, the “Discovery & Planning’’ and “Delivery & Runwork’’ buckets encompass the full Product Development Life Cycle (PDLC). “Discovery & Planning” captures activities within the first half of the PDLC, and “Delivery & Runwork” captures activities in the second half of the PDLC. You can see how Prod Ops has defined these buckets in the diagram below. We define “Discovery and Planning” as the category of work that allows for brainstorming, innovation, and definition of a path forward to meet a business need — it encompasses activities within strategy development, goal definition, scoping, and design. “Delivery and Runwork,” on the other hand, has a heavier tactical focus on driving and maintaining results, encompassing activities within building, launching, running, and refining a product.
As mentioned above, the original business case for Product Operations at Oscar was to enable PMs to spend more time on “Discovery and Planning” — the reason we chose to hone in here is that the PM skillset is uniquely tailored towards these activities. We believe that aligning roles (eg. Product Management and Product Operations) with their strengths will allow each team to deliver maximum value and increase the probability that we’ll achieve our organizational goals.
What have our results looked like?
Now, fast-forward two years — Product Operations sent out our third semi-annual “PDLC Time Expenditure Survey” (H2 ’22) to Product Managers and embedded Product Operations members.
We completed an analysis of our H2 ’22 survey results and are excited to share how we quantified the impact that Prod Ops has had on Oscar’s Product org. In addition, our H1 ’23 survey was sent out in August and the results are currently pending.
Some of our most exciting and interesting findings from H2 ’22 can be found below:
Since our very first survey in H2 ’21, we have introduced embedded Product Operations members to 26 additional pods and have seen overall Product Manager time spent in “Discovery & Planning” go up by 7%, with time spent in “Delivery & Runwork” down by 14%.
In H2 ’22, Product Managers whose domain areas had an embedded Prod Ops team member spent an average of 8.2% more time in “Discovery & Planning” and 13.6% less time in “Delivery & Runwork” when compared to Product Managers whose domain areas didn’t have an embedded Prod Ops team member.
The “Delivery & Runwork” gap between pods with and without embedded Product Operations resourcing also grew significantly from H1 to H2 ’22. We may be seeing seasonality play a role here: Oscar’s annual Open Enrollment and 1/1 readiness activities require that many teams spend less time in “Discovery & Planning.” “Open Enrollment” refers to the time period at the end of the year during which members can enroll in, re-enroll in, or change health insurance plans for the next plan year through the Health Insurance Marketplace. Oscar does plenty of prep work in the months leading up to Open Enrollment and 1/1 to ensure that we operate smoothly and efficiently during this critical time period. This requires the business to dedicate more attention to tactical YoY activities. However, we saw less of a rise in the tactical “Delivery & Runwork” bucket in the pods that had Product Operations resourcing than in the pods that did not. Approaching Open Enrollment, PM time spent in this tactical bucket only increased by 3.7% in pods that had Prod Ops resourcing, while it spiked by 13% in pods that did not. This strongly suggests that Prod Ops’ work to take on tactical Open Enrollment and 1/1 activities for Plan Year 2023 absorbed a significant portion of the seasonal effect!
While enabling PMs to focus more time on discovery and planning is no doubt exciting and inspiring to our Product Operations team, we also know there is always room for improvement. Every round of this survey helps the Prod Ops team fine tune its role, evaluate new potential responsibilities, and reassess the most valuable ways to collaborate cross-functionally. For instance, the H2’22 survey flagged new opportunities within areas such as bug intake and triage, QA and testing, and documentation and enablement. We are even piloting a new User Experience Research (UXR) partnership with our Product Design & Research team this quarter. Our goal is to enable more efficient and frequent user research by introducing process improvements and iterating upon the operational components of recruiting, scheduling, and managing studies. We’re looking forward to continuing to track these metrics over time and identify new ways to work towards our north star vision of unlocking Oscar’s ability to ship more product, better, and faster!
