In an AI-first world, what does it mean to be a great product manager?
In a recent interview with Reid Hoffman’s Masters of Scale, Marissa Mayer shared what led her to create Google’s programme for hiring and training product managers (PMs). According to her, as Google grew increasingly complex, they needed more of people with minds nimble enough to cover any and every aspect of the company’s rapidly-increasing range of products. Hiring qualified product managers, however, turned out to be more difficult than Mayer anticipated. She thought: “I can hire new people right out of school and train them to be great product managers at Google faster than you can hire the people you prefer, who are more experienced and senior”. Google’s APM (Associate Product Management) programme is just one example among many that establishes the key role PMs play within internet companies and digital businesses.
“A great VP of product is someone who has the brain of an engineer, the heart of a designer and the tongue of a diplomat”, Deep Nishar
Product managers play a significant role, if not the most significant role, in a product’s success. They make engineering, design, strategy, sales, marketing, operations, and other specialties work together with minimum friction. They own (or at least, hugely influence) the decisions about what gets built and how it gets built. Ben Horowitz’s famous article summarises this the best: “Good product managers know the market, the product … and the competition extremely well and operate from a strong basis of knowledge and confidence … a good product manager knows the context going in (the company, competition, ….), and they take responsibility for devising and executing a winning plan (no excuses).”
On the other hand, the world of software has seen ongoing shifts and transformations. From the web and mobile, to the most recent and important one: AI (or to be more specific, Machine Learning). Additionally, it has entered industries that were never thought likely to be “eaten by software”. In medicine and finance, for instance, digital transformation (including, but not limited to, the introduction of AI) is happening at a great speed and scale. New unicorns are emerging, such as Oscar, Babylon Health, Robinhood, and Revolut, just to name a few. Large incumbents are hiring digital and AI leaders to fuel their ambitious transformation initiatives. Such constant change in the scopes and domains of software products, paired with the importance of AI in modern software, has meant being a good product manager is a fast-moving target. What used to be considered a great PM a few years ago in one industry, might not even qualify for an interview in another — even the same industry — today.
“Software is eating the world, but AI is going to eat software”, Jensen Huang (CEO, Nvidia)
Assuming that both incumbents that embrace digital transformation and startups who come to the scene with new digital offerings see AI-first design as the key to making their products and services future-proof, I think that there will be a need for hiring qualified AI-first PMs or good product managers for AI-first products. As a result, such verticals are likely to face challenges that are similar to the one that Google faced: Scarcity of qualified PM talent for an emerging class of products and problems, and potentially the need for a program like APM to mould the talent they need. I’m excited to see the evolution of the product manager in light of all these. Below I’ve outlined the key skillsets and practical talent strategy needed to be a good product manager in an AI world.
The Basics: Skills a General PM Must Have
The past decades of internet software taught us a lot about the effective ways to build and ship stable software, fast decision cycles, and the emphasis of diverse talent classes in modern software teams. And more importantly, from a strategic angle, we now know that the main reason products fail is because they don’t meet customer needs in a way that is better than other alternatives: the lack of product-market fit. The most basic skills of product managers should enable them to establish product-market fit and solve for their users’ problems.
Dan Olsen’s Lean Product Playbook is a must-read if you’re working on shipping great products. This book provides an overview of skills, such as:
- Understanding the customer
- Problem space vs solution space thinking
- Deciding on MVP, iterations and pivot
- Analytics for optimising the product
These are just some of the skills a PM needs to navigate the lean product process and continuously ensure product-market fit and commercial success. Someone who lived this book (or experienced it on the job, as a PM), will surely have excellent general PM skills (and what I’ll refer to in this article as a General PM).
The Power of Context and Market
Understanding the market is critical in establishing product-market fit. In many consumer apps, such knowledge can be (at least, partially) trivial through personal experience, friends and family, quick online tests, etc. But in more complex industries such as commercial insurance, medicine, and asset management, this will need a level of understanding that usually comes from working in the industry for a reasonable period of time and meaningful interactions with domain experts. Additionally, in many industry verticals going through digital transformation, the PM’s role is to rethink and reimagine the business — a big task that requires a close-to-complete view of the value chain and first principles of the corresponding businesses.
For example, in insurance, this requires a certain level of knowledge about underwriting, pricing, claims, risk management, asset management, and various other inner workings of an insurance company. Plus, there is a huge ecosystem of partnerships and competitions (of reinsurers, insurers, brokers, regulators, and more) that adds to the complexity of decision-making. In healthcare, there is a huge variability from country-to-country in the ecosystem of health: different forms of relationships among payers, providers, and regulators; many diseases, treatments, and technical terms; medical ethics and sensitive personal data; a-fast developing scientific literature — the list goes on.
Deep knowledge of the industry / market can help a PM go beyond basic user requests and introduce outside the box features.
Deep market knowledge and industry expertise can help build a PM’s credibility when interfacing with various stakeholders and users. Also, it can help a PM map the conventional wisdom coming from their general PM skills, to the present situation and market. For instance, imagine the digital / product unit of a big asset manager that is building a product to flag credit risk events through news-driven signals for a portfolio of $100 billion fixed-income assets. The number of users and key decision makers here will be very small (maybe less than 5 and 50, respectively). Here, embarrassing MVPs — perhaps due to an avoidable AI error in finding the right signals and making accurate predictions on credit spread — can lead to huge negative financial impact on the portfolio and a potential loss of credibility for the product team.
This dynamic goes against the conventional wisdom in the consumer internet world (sometimes with billions of addressable users) that if you are not embarrassed by the first version of your product, you’ve launched too late. In this case, the users are more like venture capitalists whose feedback can lead to go / no-go decisions about the product’s funding. Plus, there are not enough of them that the product team can count on learning the basics through continuous testing. Balancing the conventional wisdom from the internet businesses vs realities of a given industry is a delicate art that PMs with contextual knowledge can do with lower risk of failing the whole product.
“If you’re not embarrassed by the first version of your product, you’ve launched too late”, Reid Hoffman
A deep knowledge of the market can help a PM go beyond basic user requests / feedback and introduce outside-the-box features. In 2006, Daniel Ek of the music streaming service Spotify had to compete against music piracy — almost like a free streaming service competing against a paid streaming service. His deep understanding of the market made it obvious that speed of streaming (through his knowledge of the neuroscience of hearing) and breadth of music library (through his knowledge of the market and users) were the key to win potential users. Daniel said: “I read in this book that the human brain takes about 200 milliseconds to perceive anything. I said to the engineering team, we gotta get this down to 200 milliseconds”. He wanted the gap between a user hitting play and the music hitting the user’s eardrums to be imperceptible, based on a deep understanding of how the product interacts with its users. He also wanted the users to feel like they had all the world’s music on their hard drive. He knew that through creating that feeling, Spotify would build something much better than piracy. Two crazy ideas at the time that led to Spotify’s success today, both arising from deep market and contextual knowledge.
Machine Learning Know-how
In the past decade, we witnessed the mass adoption of software in every aspect our lives: from search, shopping and travel to health, finance and beyond. The positive feedback loop of more software >> more convenience + more data >> better AI >> more mass adoption / funding, and repeat seems to have enough fuel to help AI-first software reach almost every angle of our lives. That’s why it’s inevitable for a good PM to understand how the goodness of the algorithm that drives or is inside the software, can impact the success of the user experience.
Imagine if you called Alexa and it only worked 60% of the time. Would you be pleased with the overall product and continue using it? How accurate should a conversational AI agent be in order for the customers to see the experience as something worth coming back to? How will this answer change if we go from banking to healthcare, for instance? Should a hospital employ an emergency-readmission prediction model that is 90% accurate for prioritising people for early check-ups to mitigate their risk of readmission? And how should accuracy be measured? Is it area under the ROC curve, or precision / recall, or something else? These are just a few examples of questions that should occur to a PM when defining the product roadmap and assessing the goodness of a model for a user problem. Of course, it does not mean that AI-first PMs must have a degree in machine learning; it refers to their ability to map the UX KPIs to machine learning KPIs.
To take the story further, when building AI-first products, AI is the starting point of the design process. Rather than adding AI features to a pre-AI product (which is sometimes referred to as an “AI inside” product), it is about creating an entirely new AI-centric experience that just would not make sense without AI. The emergence of such features will have implications for the use of agile framework, which most PMs are familiar with and advocate for the delivery of the product (as per conventional skills that a general PM has). Unlike the pre-AI products of recent decades — which had many small features that could be delivered over a series of multi-week sprints — AI-first products usually have a small number of deep features, each needing months, sometimes years, of R&D. Additionally, success in the implementation of such features usually has uncertainty associated with it. For example, will computers ever be able to predict the time to Alzheimer’s Disease with 99% C-statistic? It requires a different approach in defining the roadmap and feature milestones.
How to develop the ideal Product Management Programme
When hiring AI-first PMs, companies outside the technology sector face three key challenges:
- Scarcity of great AI-first PMs in the market, as described above
- Lack of appropriate in-house training and development programs for PMs
- Poorly defined PM career trajectories
Overall, when hiring at small scale, such companies’ needs for formal PM-specific talent programmes will be a lot less if they devise an approach that only hires either AI-first PMs or general PMs; for the latter, of course, they must have strong AI and market experts working closely with the PMs to help them develop into great AI-first PMs.
“When you can’t find the employees you need, you have to make them”, Marissa Mayer
Due to the scarcity of both AI-first and general PMs (the two hiring scenarios mentioned above) while easier to deal with after hiring, are extremely difficult to succeed. That’s why when going big on AI-first digital transformation and product development, companies must be open to hiring a broader range of profiles to fill the need for PM talent faster. Given that great AI-first PMs are likely to be fairly senior individuals, the more inclusive approach in hiring PMs will create an organic path for junior PM talents of various core skillsets to grow into great AI-first PM profiles. And that can only be a good thing.
Regardless of the level of seniority at which the hiring companies start to onboard the product managers at, product management primarily requires learning on the job. This is why there is a need for such organisations to put in place mechanisms to support such learnings. Through formal and informal training, frequent transitions between products and businesses, and exposure to new talents and trends that are shaping the industry, great PMs will rise. A well-designed product management program will be the key to hire, develop and retain great PMs and offer them a fulfilling professional experience.
