In this conversation, Schwartz talks about what inclusivity actually means for a product team — not a single interface that tries to serve everyone the same way, but a product that lets a first-time player and a 25-year veteran each find their own moment of joy. She discusses building teams that include both the expert and the beginner in the room, translating jargon like reverb and distortion into something a new player can feel rather than just read, and why she pitches inclusivity to leadership not as a values statement, but as a growth number.

Redefining who counts as a musician

Who has the musical instrument industry traditionally been designed and built for?

It depends. Inclusivity can relate to any category. Even though I happen to be in the music industry, it comes down to who is your audience, who is the audience you want to grow into, and how can you include them? It can be anything or anyone.

There are some companies who target a user in a particular part of their journey. I’m going to use some of the more school-focused, education-based products that are targeting that younger demographic, newer to their journey. They’re specifically targeting that person, but they can grow as that person learns. They can continue to grow with that company.

But what I’m talking about in my experience is more the popular music, the recording-based music, or fretted instruments, as they say. Who is that customer? Who do they design for? I think largely it becomes and has become the player. They want to talk to, speak to, and engage with those who use their products. Usually, once you start playing — let’s use guitar for example — you start understanding those things within a year or so, and as you grow in your journey, you can continue that for a lifetime. You are a player, so therefore you fit into their category.

Who’s making music now that wouldn’t have called themselves a musician 10 years ago, and how does that redefine the audience?

You nailed it. That’s exactly why this becomes an interesting conversation now. Over the course of 10 years, we could boil it to five years, even two years, and the definition of who is a musician, who can create music, has dramatically changed. Going back to pre-AI, pre- a lot of the technology that has grown to support new players, that target audience for MI companies 10 years ago would have had to invest a good amount of time and money to be comfortable enough to be one of their engaged users.

But that changed—needle scratch—to a point where you no longer need years of formal musical training to begin creating music. All you need is to be able to access your keyboard and an app, and you are making music to a degree that you can be composing orchestras. You could be making beats, designing and writing rock bands, without ever having to know one note or be able to sing one note. That’s how it changed so dramatically and why it’s so interesting.

Designing across the beginner-to-pro spectrum

How do you build a single product experience for someone playing their first chord and someone who’s been playing for 25 years?

That’s where it gets a little complicated because, from my perspective, trying to make sure we are inclusive to all doesn’t mean you have to have a one-size-fits-all solution — one onboarding journey, one UI that has to satisfy somebody who just picked up an instrument for the first time versus someone who’s been playing for 25 years. It doesn’t have to be the same. You just need to make sure that each of those categories, in their extreme, can do what they want to do, can engage with the product, can find that moment of joy as easily as the other one. That’s all you have to do.

I’ve seen established music companies say, “We don’t want to look stupid, or talking down or condescending to a player who knows exactly what they’re doing. We don’t want to ask, ‘do you know how to play a chord?’” Of course you don’t, but two things: First, you need to support that user at every step of the journey. And you would be surprised how open more established players are to making sure other people can learn — they’re not going to turn their nose at why you’re offering this kind of information. And, quite frankly, we have sophisticated MarTech tools right now to make sure you don’t even have to see that information if it doesn’t pertain to you.

If you don’t want to insult that power user, where’s the line between making something effortless and making the skill behind it feel unnecessary?

That’s exactly the fine line that one has to traverse: to make sure you don’t condescend to a pro user, but that you’re also inclusive. That’s sort of where the experience starts, and it’s where, from a product perspective, I get really interested: How do you do that? What language do you use? What does it look like, and what does it feel like? We need to maintain the brand integrity — we don’t want to look like we’re pandering or going beyond who we are as a product. That’s the real fine line, but it’s really important.

Building inclusive teams to build inclusive products

Are there any techniques or frameworks you use — questions you constantly ask, or mantras you have?

I do it rather intentionally, and how I do it is by creating the team itself to develop a feature. Say we’re going to develop a feature, an onboarding, or roll out a new product. I like everything to be collaborative, so it’s not just one vision or one focus. It’s been crucial to me to build my teams to make sure we have somebody to represent each side inherently. We have a player who really does know, speaks the language, and knows what they want and don’t want. Conversely, we need the person who is just beginning their journey. That’s the most important thing, having the two together. As we’re ideating, as we’re doing wireframes and user flows, both sides of that user are in mind, always, in the room at every step of the journey.

You’re designing something, and someone will use a word or a phrase that a beginner wouldn’t understand, and the beginner says, “Wait, what is that?” “Oh, yeah, you don’t know that yet” — so how do we support you at that moment? That’s how it’s building that team. It’s not even just about product people — it’s about your content providers, your designers, your UI, UX. Everything should in some way have that breadth and depth of experience in the room for collaboration.

How does who’s actually in the room designing a product change what the product ends up saying to the people using it?

Just data. We start with our existing data — what do we know about this person or this audience from a data perspective, and how can we apply that as sort of a shot in the dark, an assumption theory? Then we plot that out as quickly as possible, get to some MVP, or minimum viable product, and get that out there, and start testing and iterating: Were we right? Are we wrong? Should we adjust? Should we pull back? It really just starts with that best guess.

Speaking to different fluency levels

When a technical term means everything to a veteran and nothing to a beginner, how do you decide whether to teach the word or design around it?

That’s something I’m still experimenting with. That question is so relevant when you’re talking about music — how do you describe sound? How do you explain distortion or delay or reverb to somebody who has no idea? They’re words, and you could read the words, but until you actually hear it, you go, “Oh, that.” That’s a big challenge. But as we move toward these more experiential moments, where you’re able to, say, hear what reverb sounds like, I’ve been able to design experiences where you can show it dry, meaning without that particular effect. Let the user hear it for themselves, let them understand it. And, ideally, let them play around with it, take a knob or a dial to pour more in or take more off, to really start to feel it.

That’s just one example of how jargon will impact your experience, because you want to get a sound — you know the sound, you’ve heard it, you know an artist who plays in that style or sings in that particular way. How do you get there with words? It’s impossible.

How do you know whether a design choice is working?

As easy as a heat map, or using triggered events. When they came to this part of their journey — “Do you know what reverb sounds like? Here, press this” — how many people pressed that? That’ll tell you right there. And then, following that down, what are the other KPIs you have for that feature? Whether it’s converting, buying something, or taking on a subscription, you start to see that if you get people to understand and feel and really get what they’re doing, your KPIs will increase across the board. If they know what they’re doing, they’re going to enjoy it and do more of it.

Making the case: Inclusivity is growth

When inclusivity doesn’t show up as a metric the way conversion or retention numbers do, how do you justify keeping it on the roadmap?

I justify it by not having it on the roadmap — by not saying, this is going to be an exclusive experience, because that doesn’t really resonate in a boardroom. I don’t mean it literally, but when resources are limited and you’re trying to hit numbers, that’s something that’s not really valued.

So what is inclusivity? If you take a step back, it’s really about growth. That’s it. If I can say, I will increase our user base by X percent, they’re like, great, go do that. That’s inclusivity in practice — reaching people who may not have previously felt these products were for them and helping them achieve what they came here to do. It’s making sure we’re speaking their language and applying learning techniques in a way that’s meaningful to them. Growth follows when you create that value.

At several companies I’ve worked with, that is a metric we track. We call it the moment of joy, that aha moment. Anyone who’s trying to make music can hear it. They did it, they get that feeling of joy, and you can really capture that and go, ding, that moment. From a product perspective, we want to get you there. We back into the moment you first enter our experience to that moment of joy: How do we get you there, how long will it take, and what are those inflection points we can look at from a data perspective that say, if we follow these steps, then you’re 7X more likely to have that moment. And we’ve got you for life. We’ll be that specific and meticulous about every hour in that journey to get you to that moment.

How do we know that? Anyone who’s a player who’s been playing for 25 years, we’ve asked this one question: What was the first song you learned to play? They all know it. They all talk about how they learned it. They all talk about that moment when I figured out that last chord that George was playing — “Aha, that was it, I was in.” Or that moment I sang and heard my voice sound like so. Every musician has that moment, so how can we help you get there quickly?

Leading inclusively, from culture to the next generation of creators

As a leader, what are your golden rules for building a culture of inclusivity?

Being absolutely inclusive. How can I expect product leaders or designers to be inclusive in all their work if I’m not inclusive with a team? How can I build a team that has a variety of voices, opinions, backgrounds, and facts?

Diversity isn’t just gender-based or background-based — it’s also about what kind of people they are. Are they super type-A, the type who always achieves their goals and likes to make a checklist? If I’m designing a learning product and I’ve got all those types of people, it’s kind of easy. I also need to hear the voice of the person who really wants to do it, but they’re too busy — how can I support them?

What about someone on my team who’s a busy parent with kids to work around, but who really wants to satisfy their dream of being able to play an instrument? I want to hear from them too. How do you find time? How can you get 20 minutes out of your day? It’s about bringing people with different backgrounds, perspectives, experiences, and ways of approaching the world — and creating space for those perspectives to shape the work.

As the definition of a musician keeps stretching, what’s the next kind of creator most companies in this space haven’t noticed yet?

I don’t know about “haven’t noticed.” There’s a tremendous amount of attention right now on people who may not identify as traditional musicians but are creating music. If you’d asked me that question a year ago, I would’ve answered the same way. But as of now, the spotlight is on that cohort, on those potential users, with such a flurry of excitement. And I love it — I’ve always believed everybody should be able to make music, even though a lot of people in the industry are resistant to that. This very underserved, huge user base is now getting the love and attention it deserves.

What was your aha moment when you fell in love with music?

I get the chills even thinking about it. I am what I like to call a perpetual beginner — I’ve never really learned how to play, but I do remember being a teenager and wanting to be in a band because I loved music so much. My aha moment was that time when I was in a garage with my friends and I became the lead singer because I didn’t have any instruments, and it was easy. I could do it. But having that moment where you’re singing and the drummer’s playing the drum part and the bass is playing, the guitar is going, and suddenly — we’re a band! It was so exciting. There’s nothing like that feeling of playing together and making a noise that sounds at least somewhat similar to the sound you’re trying to make. It’s amazing.

It’s so much fun. It’s frustrating and hard, but that’s why I feel like I’m in a good place to support that user — I’ve made a career and a life of music without really being able to play. So you can do it. Anybody could do it. It’s true, especially now.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In our conversation, Swapna talks about designing AI around the work employees are already doing rather than asking them to adopt new habits. She discusses building trust in nondeterministic systems, the new evaluation practices AI products require, and why the future of enterprise AI lies in agents that can carry work across systems.

Making enterprise software disappear

You’ve built products across payroll, HR, recruiting, and workforce technology. What’s the biggest lesson you’ve learned about what employees actually need from the software that they use every day?

Employees just want to get the job done. They need the software itself to disappear. Nobody wakes up wanting to use an HR system — they just want to get paid correctly, get answers to questions they might have about policies, get a laptop replaced, that type of thing. Their problems have essentially remained the same despite changes in technology, and I’ve been through those technology transformations. People want to use the software to do what they need to do and then be on their way.

With Kore.ai’s AI for Work platform, what problem were you trying to solve beyond simply adding another AI assistant to the workplace?

This is true for any enterprise: the biggest problem employees have is needing to use 10 different systems to get their jobs done. Ideally, they should be able to go to one place to ask their questions and get the help they need.

The first question is, “Where do I need to go to do that stuff?” As everybody starts to add AI, the problem compounds. Even if I need to go to HR, do I need to go to a specific HR system for policy questions? And another for benefits? As an employee, that’s not a good process.

The problem that I was trying to solve is how to make employees’ lives easier using AI. We need to make it simple for people to do their work without having to go to 10 different systems. This starts with meeting them where they already work. That could be in Teams or Slack, or a web UI. We don’t need to make people develop a new habit.

Building trust into enterprise AI

Enterprise AI operates in regulatory environments where mistakes can have real business consequences. How does that change how you approach the product design, testing, and rollout?

Even in enterprise software, compliance has to come first. I’ve built those products from the ground up. Mistakes in payroll, compliance, and HR have huge consequences. It’s the same in AI, and that risk compounds even further.

You need to make sure answers are thorough and that humans are in the loop in sensitive areas. You need to be very careful about preventing hallucinations, providing accurate answers, escalating to humans when needed, and accounting for compliance-level rules. The overall compliance landscape should be thought through before you ever push something live. Building specific guardrails — like masking sensitive data, for example — is not an afterthought. It needs to be included in the design itself.

When you’re doing regression testing, you should have golden datasets and rules that you’ve already thought through. What types of policies and procedures will inevitably come up? These are called evals in AI. When you’re rolling the product out, you have to start with context grounded in the organization’s data so that the agent doesn’t hallucinate. If it can’t answer, there need to be correct fallbacks so that it doesn’t make things up. It should either escalate to a human or say, “I don’t know the answer to this question, but here’s a document for you to review or here’s someone you can ask.”

How do you mitigate the hallucination risk of LLMs and get the end user to trust an AI agent with sensitive data?

You have to build trust from the moment a product is rolled out. As a user, the first time I interact with a system, I expect it to do the job I want it to do — almost as well as a human does. If it doesn’t know the answer, it should send me to a human rather than make one up.

A company’s policy document repository is very important here. For example, I’ve worked with customers who have laid out “moments that matter.” In an HR scenario like a bereavement leave case, LLMs are very good at handling that type of sensitive information. But as soon as the situation involves sensitive data or something like an employment violation, those are the “moments that matter.” Once an AI detects that type of intent, it should give a very empathetic answer and pass the situation along to a human because mistakes in those contexts are very costly.

When you’re designing AI workflows, you need to understand those workflows end-to-end: what are the connection points between different systems and different users, and how do I now redesign and rethink this entire workflow while keeping humans in the loop at specific moments?

What new practices have become essential for evaluating an AI product before it ships?

One of the biggest things is evaluations — making sure the product is working correctly. There are two types: one happens during the build itself, and the other happens during internal rollouts or pilots. During the build, you anticipate what people might ask, but when you actually roll the product out and people start asking questions, you may realize that you haven’t thought of every scenario.

What’s different here is that you’re not evaluating a UI; you’re evaluating conversations. There’s no UI to speak of, so the design practices become different. Are you empathetic? It’s a whole new way of thinking. There also isn’t the single point of failure that you used to have in traditional software development. It’s generative so that the answer can be different every time. Pinpointing where something is going wrong requires investigating a whole range of possibilities: “Is it in retrieval? Where did it actually go wrong? How do I even fix this?” There’s not a straight answer.

It’s very different from traditional UI development because now you are dealing with conversations or voice. Traditional development practices are completely thrown out of the loop. Still, spec-driven and test-driven development concepts, which have existed in the software world, are now reflected in practices like evals and spec-driven development. They’re new practices that you need to adopt.

Rethinking product development for agents

Having worked in both traditional software and this non-deterministic new world, do you have a preference?

Currently, I do not know how I would go back to the old way. Now, I’m thinking in terms of agents and how to develop them. When I’m given a problem, I try to map the workflow, and my mind immediately goes to, “What agent would solve this workflow? What systems are involved?” My thinking has changed dramatically. I’m also thinking in terms of conversations. To paraphrase Satya Nadella’s term, AI has become the new UI. I find it difficult to go back to traditional software development at all.

Your team rolled out the platform internally first. What did that reveal that you couldn’t have learned through customer interviews or usability testing alone?

Initially, I was rolling it out to different users. We spoke to different user groups, really trying to chart out their days: “What’s your day-in-the-life like?” These are UX research practices that have existed for a long time in software development. Those practices are still important. Nothing substitutes for good UX and UX design, though the traditional terms have changed a bit. Talking to users, talking to customers, and finding the right problem to solve have not changed at all.

I spoke to our HR, IT, and sales departments to learn what their jobs-to-be-done are. Once we had a good set of interviews and a grasp of our internal systems and workflows, we evaluated whether the use cases we built were still applicable. What tweaks would we actually need to adopt it within the enterprise? We had the golden datasets, and we’d thought through the test sets, but when it came to rolling it out for real, there were surprises. People use every system differently, so we didn’t anticipate certain questions.

Like any software rollout, we learned as we went, adapted, and evaluated. It’s true for every single enterprise: Who are the users? What problems are they trying to solve? What systems are they touching? What workflows are they using? Then, after rollout, does it work? Have people actually adopted it? Are they coming back? Is it successful?

Can you share an example of something that surprised you in the rollout?

Initially, when we were working with policy documents, we had to configure the system so that the right countries’ policies were displayed to the right people. We only realized that during testing. It all came down to how you organize your documents and make sure answers come from the right documents in the right format.

Designing for adoption, not disruption

AI is changing how people work almost in real time. How do you design a platform that can evolve just as quickly without forcing employees to reinvent their workflows constantly?

You need to meet employees where they work — whether that’s in a chat or a web UI, you shouldn’t change that. The user’s habit actually lives in those systems. In most cases, it is in Slack or Teams, and anytime you make changes, the experience should survive every model upgrade, every capability change, and every new architecture change behind it.

That’s why the platform-first approach works. Models can be swapped, but you need to be able to do new regression testing. The workflows that people have already gotten used to should not break.

When you’re building an employee-experience platform, what signals tell you that the product is actually improving how people work?

This has always stayed the same in enterprise software — it’s about retention more than adoption. The first metric is whether or not people have taken to it. Then, the real test is whether they are coming back to use it.

For example, with HR software, you are often required to use it — there’s no other way for you to submit a time-off request. But if it is optional, and now you can submit a time-off request using AI, are people actually using it? Have they come back because they liked it and it made their work simpler? Further, are they now starting to use it for complex tasks? As we move to agentic AI tasks, a good example is a user telling the AI, “Plan my leave. Check my vacation, look at my policies, and tell me the best leave that I can take that optimizes my time off with my vacation days.” That’s a complex-level task that an agent can actually do.

In IT, for example, the AI can go in, check when your laptop lease is going to expire, and automatically send emails to an administrator that the laptop needs replacing. That’s a level of autonomy, but of course, I always stress having a human in the loop.

When people use it as their first stop because it’s that useful, that’s when you know it is successful. There are some easy metrics to track, such as lower ticket volume, because you can conclude that people are getting their questions answered by the AI. HR can then do higher-level tasks and focus on strategy rather than answering questions.

From conversations to end-to-end work

Looking ahead, what do you think the best enterprise AI platforms will do three years from now that today’s products aren’t yet capable of delivering?

They’ll be able to carry work across systems instead of just answering questions. You’ll hear about orchestration in the AI world — a typical enterprise will have 50-plus systems. There’s an end-to-end workflow that can execute across those systems rather than relying on these single points of contact with AI tools.

Further, today’s products are mostly conversational. They’ll summarize documents and answer questions, but the opportunity is to cut across workflows. For example, “I can manage your end-to-end day from morning to evening. In the morning when you come in, I can tell you what meetings you have. Here’s your dossier for them.” Work should be a lot simpler, and it still isn’t. These systems need to be proactive, anticipating what you need and taking that busy work away. This will make work more enjoyable because AI systems can do a lot of the grunt work.

As expectations for output accelerate alongside AI, how can AI help employees keep up?

It’s the cause of these increased expectations, but it can also help you. Nowadays, there’s no other way — to do your best work, people are being asked to do more work with less. It’s not stopping. So AI is an essential component.

We all talk about using humans in combination with AI. I truly believe in that and think AI should be seen as an asset rather than a replacement. If you view it this way, it can augment people to the point of driving higher satisfaction. That’s a win-win for both the organization and the employees who work there.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

Her perspective is grounded in a central conviction: as AI lowers the cost of building software, features alone will become increasingly difficult to defend, and durable differentiation will come from the context, semantic infrastructure, workflow intelligence, and trust systems that let products understand a business and evolve with it. In this conversation, Soni discusses what it means to make an enterprise legible to AI, why products must earn the right to become invisible, how personalization can remain transparent and controllable, and why the products that endure will be the ones that continuously increase their value to customers.

Beyond features: The new durable moat

With tools like Claude Code dramatically lowering the cost of implementation, many products are starting to look remarkably similar. What still creates durable differentiation?

Durable differentiation is moving away from the feature itself toward more infrastructure, context, and the semantic layer that sits underneath it. Features are already copyable. I like to think of them more like menu cards. One vendor introduces a feature capability, the other can copy it fairly quickly, and it’s another version of it that is sitting with a different prompt, different model, maybe slightly different implementation approach. Now, as the cost of building falls, the shelf life of the standalone product differentiation is reducing significantly.

What’s becoming really hard to replicate is the system that enables the AI to understand the business environment, like the relationships between the data, the meaning of the workflow, the users’ context, what decisions are being made. What are the decisions that the customer cares about? What are the outcomes that they’re driving for? The semantic and the contextual layer is what makes the entire enterprise legible to the AI.

Once you have that, you can enable the model to make smarter decisions, orchestrate across workflows, and deliver an experience that is relevant to the end user, rather than simply generating a generic response to them or building a generic feature. And the second part of the differentiation is the infrastructure itself. The ability to connect systems, the right systems, preserve the context over a multithreaded set of information, govern the actions, and create the feedback loops, so the product can learn from the usage and the outcomes itself.

So I believe that the products that will win will be self-evolving systems, honestly. They will not simply ship a fixed set of features. They will continuously become more intelligent and valuable as they understand the customer’s environment better. Otherwise, we’ll be just building point solutions with increasingly similar capability, and that’s really not differentiation. So as the features start to become a commodity, the advantage is the context graph, the orchestration and the infrastructure underneath, and a self-healing, self-evolving system for the product to evolve with the customer’s needs.

If feature parity is becoming inevitable, where should teams invest to create experiences that are difficult for competitors to replicate?

Teams should invest in understanding the customer’s environment deeply enough that the product becomes connective tissue within it. That starts with listening carefully to the customer — not just to the feature being requested, but to the outcome they’re trying to achieve, the systems they already use, the workflows they depend on, and the friction preventing them from realizing value.

From there, the opportunity is to build the semantic layer and contextual intelligence that let the product fit naturally into that environment, rather than sitting beside the customer’s infrastructure as another isolated tool. Teams also need to invest in curation and adoption: providing a capability isn’t enough. You have to configure it for the customer’s context, help embed it into the way they work, and make sure they’re realizing the intended outcome.

A competitor may be able to replicate the visible feature. It’s much harder to replicate the accumulated understanding of the customer’s environment, the integrations, the adoption model, and the trust that develops over time. That’s when a product becomes more than software — it becomes part of the customer’s operating infrastructure, and therefore much harder to replace.

The paradox of invisible design

Sometimes the best products are those that become “invisible.” What does invisibility actually mean from a product design perspective?

I think it’s such a powerful moment when a user lets you put your product that is almost invisible to them on their mobile phone. A product earns that right to become invisible when the user trusts it deeply enough to depend on it without having to constantly manage it. My perspective is drawn from my earlier work in security. The best security systems are often the ones that recede to the background, that do the work for you but you don’t need to worry about them, because they only surface when there’s a threat or an anomaly or an insight that really requires attention.

I think the same principle applies more broadly today beyond the security industry. A product becomes invisible when the interface is no longer the center of the experience, but the solution is, the outcome is. The user continues to receive the outcome that they care about, while the system blends naturally into their workflow or the work routine.

The customer should be able to define the problem, or you sit down and understand the outcome clearly that the customer’s driving for. The product should almost become so frictionless, the way we deliver it, that it becomes a part of the routine, whether they need an interface, or they need a recommendation, or they need an automated action or no visible interaction at all. What do they care about, and how can we deliver that in a very orchestrated manner, in a more trustworthy manner? That’s where a product can really be invisible, but I do not mean invisible to mean opaque. The user must still understand what the system is doing, retain control over the important and consequential decisions, trust the product, and understand the business context in which it is operating.

Ultimately, invisibility is really not the absence of the product, but it’s an evolution of the experience that you bring to the user and the presence of value, without the burden of operating the product or the product requiring the user’s constant attention. In short, a product becomes invisible when trust is high, friction is low, and the outcome stays visible even when the interface doesn’t.

Personalization without complexity

As products become more personalized to each user’s context, how do you keep that from becoming overly complicated or error-prone?

Personalization, again, has to start with a deep understanding of the problem the user is trying to solve. Personalization should not require users to reorganize their work around the product. The product should adapt to their context and working model. This was valuable in the pre-AI era as well.

The first principle for a product to get personalized or to be adopted in a personalized manner is to show the work, especially early in the relationship. The AI should make reasoning, assumptions and proposed actions, visible enough such that the user can understand how it reached the conclusion. Over time, as the system improves itself, the user may choose to delegate more authority, but that trust at the beginning has to be earned. That’s where human-in-the-loop comes in.

The second principle in personalization is more about inspectability. Let’s say the product goes more invisible, but the user should always be able to go back, query what happened, why a certain decision was taken, and understand if or why the AI may have taken a wrong turn. It should be reversible, because that’s what is sometimes needed. That makes the system more stable, because when it’s imperfect, it’s not inscrutable.

The third principle is flexibility, like how I was talking about being able to revert back, but it’s the flexibility in how the outcome is delivered. Personalization should not be limited to the way that the interface has to change. Different users may need the same outcome, same capability, but delivered to different experiences. It could be a mobile experience, it could be a web interface, it could be an automated workflow, it could be an MCP-enabled interaction. It could be as simple as, “Pull it into a Google Sheet for me.” That’s the flexibility of personalization. That model hasn’t changed. So it’s not about personalizing the interface, it’s about personalizing the experience by leveraging the workflow that works for a user’s mental model, and it’s deeply rooted in the user’s problem. Again, transparent enough to be able to earn the trust, flexible enough to fit the workflow, and controllable enough that the user never feels that they are not in control or the product is unstable. Ultimately, the AI has to earn the right to make decisions on the user’s behalf.

Products people don’t want to leave

Switching costs are often framed around data lock-in or integrations, but there are also emotional and behavioral switching costs. How do you design products a user doesn’t want to leave?

This is actually my favorite question, because the premise hasn’t changed between pre-AI and post-AI. That has always been the core challenge: how do you build something that people just love to stay in, day in and day out? I think, again, it goes back to understanding their mental model, how they work, how they make decisions, what they trust, and where friction appears for them. The product has to fit naturally in their mental model, rather than asking them to get out of it and continuously adapt again and again to the product that you have built for them, and that’s challenging. That’s complete friction right there.

The second part is trust. Customers stay when the product is reliable, it’s understandable, and it’s consistently helping them achieve the outcomes that they care about. Then the third, which is increasingly important — this is the part that gets me really excited now that it’s possible — is evolution. The context today is already evolving. The data is always fresh. It’s real-time. The priorities are always getting updated. Workflows are getting really flexible and real-time adaptable as well. The insights available to them are getting generated. It’s learning and it’s feeding you information. What if the product evolved with the context? The product should not remain static.

I think the strongest products now will need to become increasingly self-healing and self-evolving. So they will detect where something has changed, adapt the experience or the workflow, automate what can be automated, request permission when appropriate, continuously improve, without requiring the user to go elsewhere to fill minor feature gaps. Because you’ve got the right context which is constantly updating. You’ve built it over time. You’ve built a connective tissue where it works with your infrastructure. All you need is the product to evolve, and you’ve already set the experience in the way that you want to deliver it. That creates a much stronger form of retention for the customer and customers want to stay. This way the product understands their environment, reduces their effort, and becomes more valuable over time.

I think the goal is not to create a dependency through lock-in. The goal is to create confidence that it’s continuously relevant and it’s working for you, not the other way around.

Enterprise UX in an AI-native world

With AI, has the definition of “consumer-grade experience for enterprise” changed?

I think if you root the definition of a good enterprise experience in deeply understanding the problem, then you’re fine either way, with AI and pre-AI. What has really changed is the evolution of what experience means. Good enterprise UX was once largely defined by making complex software easier to use: clearer navigation, more coherent information architecture, fewer steps, and less training. Those qualities still matter, but they are now table stakes.

What has changed is the unit of experience. Enterprise UX is no longer confined to a screen or device. It now spans workflows, roles, data, integrations, automated decisions, and interactions across multiple systems.

A beautifully designed interface inside a fragmented workflow is not a good experience. If users must reconstruct context, repeat information, move between disconnected systems, or manually coordinate decisions, the underlying problem remains unresolved.

AI expands the remit of design further. We are now designing how systems interpret intent, communicate uncertainty, request approval, take action, and learn from outcomes.

Good enterprise UX is therefore no longer simply about making complex software usable. It is about designing coherent, intelligent, and trustworthy business systems. Screens still matter, but they’re no longer the primary measure of the experience.

If voice and natural language become the primary frontend interface, how does that change the role of product design?

The experience and the way you deliver it is definitely changing. Previously, you had to go gather the intel. You would go to solutions engineers, you’d go to customers, you’d set up labs, you’d look at competitors’ information, you’d look at documentation. You were gathering context. Now that context is automated for you, so how do you automate the ingestion of context? I like to call it the ExperienceOS. So you’re now building an ExperienceOS first, the ingestion of that customer feedback, the competitive intel, how products are being used, and behavioral data together, creating an AI-ready to read an automated .md file with full context of the product experience.

Once you have that, leverage a design system that is AI-ready that can read natural language and an .md file for AI-dev creation, and now you’ve got end-to-end. You’ve got context, a prompt, and you’ve got your natural language input that pulls it all together. Your design is now fed by voice, natural language, and with the context inputs. Again, the work needs to be done on the context layer, semantic layer, pulling information from so many different sources and making sure there is no hallucination. That’s one place where you put a human in the loop or a learning system there.

The second is if it’s spitting out automated experience, whether it’s screens, whether it’s MCP, whatever that might be, how is that being checked? There needs to be a person who’s reviewing that, or an agent that has learned what it needs to deliver. So I think that’s how product design is evolving, from the overall ingestion of an evolving contextual layer of multiple sources, rooting it into the business outcome, rooting it into the experience that you want to deliver holistically as a company, as a vendor, and then using natural language prompts. A lot of work needs to go on the prompting itself. You do that work, and then it leads to a much more streamlined model of delivering product experience.

This can make it feel a little like design is not important, but I think a lot of work now goes on building the right systems design, building the right libraries, rooting it into the right decisions such that a single command, even from a designer, can fix the system completely. So a lot of focus goes on the platform layer, context layer, and it’s not about tweaking one screen at a time. It’s tweaking the system, making it more intelligent at the same time. The visible frontend may increasingly become language, but the real design material is the behavior of the system.

What survives the commodity trap

If every company can generate code, ship features quickly, and access increasingly similar AI capabilities, what will separate the products that endure from the ones that become commodities?

I think products that will endure are those that are rooted in business outcomes — products that customers can trust, that they can almost have a relationship with. That’s because the product is compounding in value for them; self-healing is one example of that. Then why would I buy from you versus somebody else? Because you have an infrastructure that it’s rooted in, that you understand my business, and that together both of them become a connective tissue. I think that’s the one that endures.

Products that become commodities will just provide feature-level capabilities, point solutions. Enduring products will help lead organizations through their own transformations — continuously providing that transformation to customers is what endures. Commodity products provide capability; enduring products continually expand what the customer is capable of achieving.

Many valuable product metrics today weren’t measurable just a few years ago. How is AI changing what product leaders should be measuring?

AI is changing product measurement in two ways. First, it allows us to measure signals that previously existed only in unstructured conversations. Second, it allows us to connect those signals to product behavior and business outcomes. Product leaders no longer have to rely exclusively on surveys, interviews, support tickets, and manually synthesized field feedback. AI can analyze customer conversations, implementation notes, sales commitments, product usage, support interactions, and behavioral data together. That creates entirely new measures. We can identify which promises were made during the sales cycle, whether the product delivered on them, where adoption stalled, which customer problems are becoming systemic, and what interventions are most likely to improve the outcome.

These capabilities also make measurement more dynamic. Once an organization identifies and resolves one source of friction, the next constraint emerges. The KPI itself may evolve as the business becomes capable of seeing and solving new problems. That is particularly important for CIOs and transformation leaders. Many know they need to lead change but are looking for partners who can help determine what to measure, interpret what the signals mean, and identify the next action.

The most valuable product organizations will not simply report what happened. They will continuously identify what is changing, explain why it matters, and recommend what the business should do next.

As AI becomes more embedded in how products work, how do you see the role of product and design leadership evolving?

The role of product and design leadership is expanding. For years, we largely designed software that waited for users to operate it. We are now designing systems that interpret, recommend, act, learn, and increasingly participate in the work itself.

That requires a broader form of leadership. Product strategy, design, AI, customer experience, go-to-market execution, infrastructure, and organizational transformation can no longer be treated as separate disciplines.

The opportunity is not simply to ship more capability. It is to design the complete system through which a business becomes more intelligent, more adaptive, and more capable. The leaders who shape the next generation of products will be those who can make complexity coherent, make intelligence trustworthy, and make transformation adoptable.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In our conversation, Will discusses how experimentation evolves as organizations scale and why leadership mindset is the foundation of a successful experimentation culture. He talks about how GoPro reduced time-to-market while creating more capacity for experimentation, and shares his learnings from unexpected outcomes.

Building an experimentation culture

You’ve had experience in experimentation programs from several vantage points — as a consultant, as a transformation leader at Gap, and now leading GoPro’s Digital Product team for the ecommerce site. From your perspective, what’s changed about what makes experimentation successful?

The first thing that comes to mind is my early days as a consultant. I was fortunate to work at a boutique agency that specialized entirely in experimentation and personalization. We worked with organizations like Barnes & Noble, Ralph Lauren, Under Armor, and Fidelity.

At first, it was hard to understand what test to run and where to start. It was very confusing until I realized it really comes down to the customer. Every organization has unique customers who expect different things and want different experiences. The thread that’s remained the same throughout my career is understanding the customer and meeting their needs.

One thing that’s changed is my perspective on scaling. As a consultant, I used to think, “Why don’t these companies just double their experimentation output? We could go from four tests a month to eight tests a month tomorrow.” Then I worked at Gap, with four global brands, and now at GoPro. You realize the complexity of the organization itself often becomes the blocker to increasing experimentation velocity. As a consultant, I had the luxury of focusing on one objective. Within a large organization, you appreciate just how much coordination is required to scale experimentation.

When you’re driving experimentation, at what point does it become part of the culture?

I think a really important factor is leadership’s disposition toward data and experimentation. I’ve worked with leaders who know experimentation very well and are really gung-ho about it from the beginning. I’ve also worked with leaders who don’t have much experience but quickly understand the logic behind it. In both types of organizations, experimentation thrives.

Where it gets difficult is with leaders who have been in the business for a long time and think they already know how it works. They have a roadmap from past experience and come in ready to check the boxes because they believe they already know the outcome. In that kind of environment, there’s less room for experimentation.

Experience can absolutely inform decisions, but there’s got to be a real humility around the uniqueness of each organization. What worked somewhere else — or even what worked before in the same org — doesn’t necessarily translate to the current situation. That’s where experimentation becomes much harder to gain traction.

Many organizations say they want to be data-driven, but still make roadmap decisions based on the loudest voice in the room. What separates companies that run experiments from those that make decisions through experimentation?

I think the difference is what I call a culture of experimentation. That means making experimentation visible to everyone. Everybody can submit ideas and guess which experiment will win. Everybody can see the results. Sometimes that’s through emails or polls, and sometimes through more formal meetings or forums. Not everyone participates, but that openness and transparency are really important. In organizations that embrace that mindset, experimentation starts to replace the loudest voice in the room.

Learning from unexpected results

Throughout your career, you and your teams have driven significant improvements in conversions, engagement, and customer satisfaction. Can you share an experiment or product decision that surprised you?

Sure, there was one experiment that almost didn’t happen because so many people thought it wasn’t worth testing. The idea was to add a recommendation carousel to the order confirmation page. People said, “They’ve already made a purchase. Why would they buy again?”

What got people on board was the strategy behind it. There is no point during a visit when you know more about a customer than after they’ve completed a purchase. You know what they browsed, what they bought, how much they spent, and whether they’ve been there before. This was for Barnes & Noble, where the average order value is relatively low. We ran the test, and it produced a statistically significant increase in revenue. As far as I know, it’s probably still on their site today.

Again, it comes back to understanding the audience. These are readers — they’re much more likely to add another book to their order than someone who buys a product only once.

On the product side, one of the biggest changes at GoPro was the introduction of an NPS score for our ecommerce site. We had already measured NPS for our customer service call center, but we hadn’t measured it for the website experience itself. We implemented a quarterly NPS survey, and that gave us a consistent benchmark for the site’s experience. When the score improves or declines, we can pair it with behavioral analytics to understand why.

Your team has also significantly reduced time-to-market. How did you accomplish that?

It was a very focused effort, and I’d like to give credit to our principal product manager, Brandon Watts, who led much of this work. We started with a survey asking teams which parts of their work took the longest and how much time product launches required across UX, Merchandising, Engineering, and Studio.

The responses quickly pointed to one bottleneck in Merchandising. Creating product detail pages inside our CMS required navigating six or seven layers just to add images or content. We focused our effort there. By simplifying that workflow, we reduced the time required to create those pages by more than 50 percent, which was a huge win for the team. The Merchandising team now has more time to focus on higher-value work, including experimentation.

Scaling experimentation across organizations

At Gap, you had four distinct brands under the company umbrella, including Banana Republic, Athleta, and Old Navy. How does experimentation change when you’re optimizing for a diverse portfolio of customers vs. a single, deeply engaged community like at GoPro?

GoPro has one experimentation program with a tight group of stakeholders across Product, Merchandising, Studio, and Engineering. Because we all understand the same customer, company strategy, and KPIs, we can move very quickly.

Gap was very different. I led the personalization pillar across Gap, Banana Republic, Athleta, and Old Navy. Those customer personas are completely different. A win for Old Navy — something like promotional badging or urgency messaging — fits naturally with that brand. The same experience could feel completely out of place at Banana Republic and even weaken the brand.

That’s one of the biggest differences when you’re scaling experimentation across multiple brands. It’s not just about finding wins. It’s about getting stakeholder alignment while respecting each brand’s identity.

Many experimentation programs optimize conversion or click-through rate. How do you make sure teams are improving the overall experience?

I’m still surprised when I see roles focused exclusively on conversion rate optimization. Conversion isn’t the end-all metric for retail organizations. It’s actually fairly easy to increase conversion if you’re willing to sacrifice revenue. A retailer could push more people to buy less expensive accessories rather than higher priced flagship products. Conversion might improve, but the business will lose overall. The right KPI drives the right improvements. One metric I really like is revenue per visitor because it combines revenue and traffic into a single measure. It accounts for changes in average order value and conversion together, rather than optimizing one at the expense of the other.

I’ve also become a big believer in measuring NPS directly on the website. Customer lifetime value can also be valuable, but it’s less clear cut. Organizations calculate it differently. The important thing is making sure the calculation actually reflects the holistic value of your customers to your business.

AI, strategy, and scaling responsibly

As AI makes experimentation faster and cheaper, how do you distinguish productive experimentation from experimentation theater?

It comes back to strategy. Whether a test wins, loses, or ends up flat, you should learn something from it. If you’re not generating insights regardless of the outcome, you’re probably not spending your time wisely.

The second part is ensuring you can sustain the winning experience. Organizations often realize they can personalize experiences for 15 different customer segments. That’s exciting until you ask what it means for UX, Merchandising, Localization, Engineering, and Studio.

Before we run a test, we ask whether the winning experience is something we can realistically maintain over time.

Lastly, what’s the biggest mistake leaders make when trying to scale experimentation?

The biggest mistake is shortchanging strategy or analytics. AI can generate fifty creative variations in seconds. The bottleneck isn’t producing assets anymore, but if your tests aren’t strategic — if they aren’t designed to teach you something regardless of the outcome — you’re moving too fast.

The same applies to analytics. If your analysis starts to degrade because you’re running too many experiments, or you’re no longer using those insights to shape future tests, you’ve scaled beyond what the organization can support. The moment strategy or analytics begin to decline, I’d pause any further scaling.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In this conversation, Lisi talks about what it really takes for a smaller company to compete against dominant players — not by copying them, but by staying closer to customers, moving faster, and knowing when to say no to a feature request. She discusses the discipline behind Singular’s product decisions, the self-serve platform that helped drive 70% year-over-year growth in the mid-market, and how the company decided to invest ahead of Apple’s IDFA changes. Along the way, she reflects on what separates a good product manager from a mediocre one.

The David and Goliath advantage

What kinds of advantages can a smaller company realistically create through product strategy, rather than trying to out-build larger competitors?

This is a really fun question because it’s something we internally debate all the time. If you take a business class, they’ll tell you that if there’s a dominant player in the industry that owns anywhere between 60 to 80% of the market, that’s a total blocker — don’t go into the market. But you see throughout history over and over again examples where these disruptors come in and go from being the small player to being the dominant player. I think there’s a bit of a middle path where you can leverage your strengths to become a significant enough player within that market space. What’s really important is being very creative about your strategy and having a clear understanding of what your strengths are.

You can’t tackle things effectively by being a copycat company. If you’re going to continuously chase after that larger company, you will not succeed. You have to figure out what niche you’re going for and hack your way into that. But you have to have that disruptive mentality and really understand what the added value is, and what your unique take on the problem space is. If not, you’re always just going to be just chasing after the larger competitor and you’ll never be able to catch up because you won’t have the same resources that the larger competitor has.

Do you have explicit decision criteria that help keep roadmap decisions from becoming overly competitor-driven?

There’s theory and then there’s reality. It’s always funny because I hear all these product interviews and they’re like, “Yes, we have this framework and this is how we work.” But, to really be strategic about the product choices you make, you have to be able to pivot, which is the worst thing any product manager ever wants to hear because we do like to have that stability. The biggest strength you have as the smaller incumbent is that you can pivot and be more agile. So you do need to have a balance. If a new customer comes in and they’re willing to pay for whatever it is that you’ve figured out, you build that.

I know a lot of product managers will tell you that’s the wrong thing to do, that you don’t want to constantly be swapping out tasks and initiatives and having distractions as you go along. But I think it can also work to your advantage, because if you stick too much to a framework, it makes you too static. It doesn’t give you the flexibility you need to move fast. Especially in the marketing world, where the industry is changing a lot. Being agile is one of the really great things we’ve been able to leverage — working with our partners to understand what’s coming and being proactive about the changes we need to make.

So it’s not necessarily that we have an explicit framework, but it’s more about making sure we’re on top of all the changes coming through so we have a good position in the marketplace and we’re ready for it.

If a competitor makes a change, how do you decide whether it deserves a response or is just a distraction?

This is really what comes to the core of our product theory. Our biggest focus as a product team at Singular right now is that no matter what, we’re always obsessed with our customers. That’s originally an Amazon quote, and I really love it, because it means we are constantly prioritizing customer calls, joining whatever QBRs or sales calls we need to go to, and really listening to our customers.

This does two things. One, it’s understanding when there’s a feature out there that we need to determine if we also need to have it versus it’s just noise. Is it actually being adopted? Is it actually providing value to customers? You will not know until you talk to a thousand of those customers to understand if there’s really value there.

Having a competitor build something out first also gives you the ability to really analyze what they did well and what they didn’t do well, and then you can iterate and come out with a better solution. It’s not that you’re just being a copycat for whatever’s coming out — you understand the problem that was solved and then figure out how to do it better. Sometimes it’s better to be second to market and have a better product for it.

Saying no to the loudest requests

Can you give an example of a time you intentionally chose not to build a feature customers were requesting, because it would have diluted your product’s positioning?

This happens a lot with features that are buzzwords. Specifically in the marketing industry, we’ve been hearing the word “incrementality” for several years now. How do you measure incrementality? What’s your incrementality report? But when you start having conversations with customers to really try to tackle it and break it down — “What does incrementality mean to you?” — you realize it’s not one thing. It’s the same with AI: what does AI mean, or AI analysis? We can say it because it’s part of the hype, but if you’re not really breaking it down to the core of what you’re trying to solve, then it’s just a buzzword.

It’s more than just listening to what people are talking about — it’s also being able to really synthesize the information you’re hearing from all these calls with different customers to come up with a product that’s going to serve them best. A lot of times, the issue with customer calls is you can’t take the feedback at face value. You have to understand what’s underneath it. That’s really the difference between a good product manager and a mediocre product manager — especially in this AI world. OK, you take amazing notes and really listen to what they have to say, but what are they actually saying?

How do you validate whether something is worth building — probing questions, or watching users in action?

Ideally, it would be great to validate that by putting out a prototype or some sort of beta and having customers actually use it. That’s the real key: you know a product works if people come back and use it over and over again. But a lot of times, you may not have enough time to validate things as much as you’d like. That’s where I’m a big believer in being able to break things fast — launching with smaller MVPs and seeing if that small version is good enough, then iterating from there, because it doesn’t make sense to work on something huge and then find out nobody wants it.

Is moving fast ever a detriment?

For sure. Internally, sometimes we become sort of myopic. I hear this feedback a lot: “We’re putting out a lot of betas but then we don’t finish fully going to market with a complete feature.” I don’t actually see that as necessarily a bad thing, because you’re trying things out and seeing what works, and that’s what it means to be agile.

The downside is making sure that, internally, you’re keeping the right motivation, because working like this can sometimes feel chaotic. You need to be clear about what the objectives are, what you’re moving toward, and what the logic is behind some of the decisions you’re making. Sometimes you pivot in a way that isn’t so classic and it feels a bit random, but I think that really marks the strength of being able to think outside the box.

Building a culture that can pivot

Is it easier to push new tools through in an earlier-stage company?

It’s definitely that startup mentality where it’s an organized chaos. You give everybody the freedom to do whatever it is they need to do to get it done — we don’t necessarily have an organized process. We have certain values we know are important to us as an organization, but you have to give people the freedom to explore, try things out, and get things done on their own, as long as they’re accountable for the outcomes of their actions. And it works.

A lot of what I’m saying might be going against the classic way of doing product, but I feel like it’s really critical to be able to see the long-term — to understand the problem you’re trying to solve and not focus so much on the day-to-day task, the feature you’re working on at hand. To do that, you need to be a little bit more disruptive in how you think things through. Sometimes it’s having uncomfortable conversations, because people get very attached, in a good way, to what they’re building, and it hurts to have to pivot to something completely different.

Is it harder for product people with a background at a more traditional, enterprise company to move into an experimental environment, or the other way around?

Yes, definitely. I suffered that at first when I started working at Singular. I came from a much larger, enterprise-style company and I didn’t understand the chaos. Now I’ve fully embraced the chaos.

Sometimes we get too caught up in the process and it’s not an efficient use of our time. Sometimes it is, and you have to be able to make a decision and understand: is this providing value or not? But if you can’t back it up, you can’t articulate the value of it, and nobody else agrees with you, then you should toss it out.

Winning the mid-market

You launched a self-serve platform that helped drive roughly 70% year-over-year growth in the mid-market. How different is the product strategy when you’re trying to win customers who could very easily choose a much larger vendor?

It’s two completely different personas. Especially in the B2B space, if you’re working with very large enterprises, the decision-making they do — “I need to find a new vendor; how do I choose a new vendor?” — they’re never going to get criticized for choosing the industry standard, the go-to for everyone. But people from smaller-scale, more mid-market companies that are just starting out, especially in the mobile world, are looking for value for free or value for very little money. They don’t want the same features, they don’t want the same pricing, they don’t want the same plans. You have to understand what they want to be able to serve them differently.

Did that experience change how you think about different customer segments?

Yeah, definitely. This is a common B2B dilemma: What do you focus on to win an account over? If someone’s waving money in your face, you’re like, “Yes, let’s do it.” But if it’s a feature request where only one enterprise customer is going to be using it, it doesn’t make so much sense to build it out. It really depends — you have to weigh the decision. But, generally speaking, if we get a product feature request from one customer, we do our best to validate it with other customers. If it’s useful for them as well, we try to determine if we can build something that serves more than just one customer. That really helped us change our mindset, because we realized we needed to stop overly customizing the product specifically for enterprise and build something that has more value for different segments.

When you focus on it as segments, you’re not customizing to fit one customer — you’re building out a product that serves more than just one.

Reading industry shifts

Attribution, privacy regulations, AI, and platform policies are constantly reshaping the mobile marketing landscape. How do you distinguish between changes that warrant a strategic pivot and short-term disruptions that simply create noise?

Sometimes it’s really hard to tell. That’s where being small also helps because the communication isn’t as diluted in a small company. If there’s going to be a big change in the industry, we try to figure out how it’s going to affect partners and potential customers. We can go to the different teams and get feedback right away.

For example, when Apple initially released the changes where they removed the IDFA, the device model ID used for attribution, everyone was panicking. We knew it was coming and we invested quite a bit. A year or two after that, things changed again, but I don’t think it was a waste of our time, because it put us in a position where we were considered an industry leader, because we were ahead of the curve.

Sometimes you invest in things that don’t catch on, but having the right marketing around it still puts you in a good position in the market — it’s kind of like making lemonade out of your lemons. Sometimes you take a gamble and it doesn’t necessarily work out, but if you can spin it and use it to your advantage, show the market that you’re prepared, that still wins you a lot more credit than you realize.

Are there specific signals that give you confidence that an emerging trend is worth paying attention to, versus just buzz or hype?

We do regular check-ins both with our customers and with our partners, including an annual NPS. That’s been really important for us, because a lot of times you get feedback from customers, but you can’t put a number to it. Doing an NPS has been incredibly valuable for us to put numbers around things and make decisions based on what’s most important to our customers.

In the marketing space, you have very dominant partners — if Facebook, Google, TikTok, Apple care about something, then it’s important to us. Sometimes they make mistakes and gamble on things that don’t work out, but it warrants paying attention if it’s something they’re validating as well.

Finding white space

Product leaders often talk about finding white space, but in mature software categories those empty markets are rare. What techniques have you found most effective for identifying opportunities that larger competitors overlook?

It’s going back to being obsessed with your customers. A lot of times the big aha ideas come from these conversations you’ve been having over and over and you hadn’t realized it. That’s really hard to do — you need to be a little bit obsessive about certain subjects. For us, our ideal persona is either a VP of marketing or the head of BI, so I put myself in their shoes and actually go through the exercise of running the reports they need, looking for the data they need. That’s when I feel like I get a better understanding of that white space.

I also think there’s an idealized notion of what white space is. The marketing industry is changing all the time, so you have to constantly be reflecting and digesting all the changes that are happening for new opportunities to come up. That’s really what’s interesting and keeps us relevant — being able to stay on trend more than necessarily a magical white space. It’s about figuring out how the industry has changed and finding an opportunity that didn’t exist before.

Do your biggest wins come from solving entirely new problems, or from looking at a familiar problem through a different lens?

I think it’s from looking at the existing problem through a different lens.

Sometimes our CEO will get on a call with the randomest BI person from one of our customers, just because he liked the guy when he met him at a trade show or something. Then they start talking, and he’ll come up with this genius idea for something, a new way of tackling the same problem. And we take it and run with it.

There’s so much value in that, even though you’re rehashing and having the same conversation — and sometimes it’s very frustrating. But doing your best to have these conversations in a healthy way, one-on-one with people, getting their opinions on things and breaking thor input down over and over again, sometimes leads to really great ideas. You need to be patient.

When we hire for product, we look for people who have that mentality of how to tackle a problem, and then how to have a conversation around it, and how do you explain themself around it. That’s the skillset of a really good product manager — being able to not only define the problem, but also to have a dialogue and debate around it.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In our conversation, Zach talks about identifying careers that will remain resilient as AI reshapes work and why Penn Foster focuses on preparing learners for tomorrow’s version of a job. He also discusses using AI to individualize education at scale and how Penn Foster is designing AI tutors that encourage productive struggle and simulation-based learning. Zach also shares why adaptability and lifelong learning are becoming increasingly important.

Viewing careers in terms of AI-resiliency

As AI changes the nature of work, how do you determine which careers are worth building education and training products around?

Some people are far smarter than I am who spend all day thinking about how AI is going to reshape work, so I don’t pretend to know where all this is headed. My opinion on AI is fairly nuanced, whereas others tend to think in extremes, from “AI will replace everyone” to “AI changes nothing.”

Our job at Penn Foster is not to predict the future perfectly, but to build education that can adapt as that future becomes clearer. People often start with the question, “Which jobs will AI replace?” We actually start with, “How is this specific career changing? And then what will employers expect someone to be able to do in three or five years from now?”

We spend a lot of time talking directly with employers, looking at labor market trends, trying to understand hiring needs and how those are changing, and identifying the careers that continue to offer real opportunity and show signs of strong demand in the labor market. Though we’re still preparing people for jobs, which is core to who we are as a business, we’re preparing them more for tomorrow’s version of that job.

What do the roles you see as most resilient to AI have in common?

I like the framing around AI resilience — it’s a more useful way to think about careers. I don’t know that anything is truly AI proof anymore. That said, the careers that seem more resilient combine technical expertise with a lot of human judgment and interaction. They involve working with people, whether they’re coworkers, customers, or patients; making decisions in uncertain situations where there is no clear right answer; communicating effectively; and interacting with the physical world in some shape or fashion.

One example that is near and dear to our hearts at Penn Foster is a veterinary technician. For pet owners, this is the person who assists the veterinarian, performs many of the tests, and handles animals at the clinic. For them, I think AI will help interpret information and streamline documentation. But when somebody has to calm a nervous pet owner, notice subtle changes in an animal’s behavior, or work with the rest of the clinical team, that is still going to take a human being who’s trained in that profession.

AI is becoming incredibly good at generating answers. Humans are still going to need to decide which answers matter in the moment. For that reason, I believe many of these professions will continue to show demand and growth.

Keeping education aligned with a changing workforce

You build a product that helps with training in entry-level healthcare professions, such as medical assistants, dental assistants, pharmacy technicians, and more. As job requirements change, how do you make sure a training program evolves quickly enough to keep pace?

Anyone who’s worked in education knows that, historically, academic timelines don’t necessarily match the real world. I’ve seen areas where it can take years for a program to move from the conceptual phase to full deployment, and then years again for changes to be made once the curriculum is live with a set of learners. The world of work has always changed much more rapidly than that, and now those timelines are accelerating.

We’ve known at Penn Foster that we needed to change our operating model to keep up. One of the things that I’m most excited about is the work we’re doing on cohort-based learning experiences. This is different from our historical model, which is more self-paced. Instead of treating a program as something you update every few years, we’re creating environments where we learn alongside the students we’re serving.

If we see friction in one week of a cohort, or learners consistently struggling with one particular concept, we can make improvements while that cohort is still progressing, rather than months or years later. That’s a completely different operating model than the one we’ve deployed in the past. From one cohort to the next, we can make a curriculum change where we see skills starting to shift in the profession.

As product leaders, it’s exciting because it starts to look more like continuous product development than traditional curriculum development. We’re learning, iterating, and improving results in something a lot closer to real-time than has ever been possible before.

As AI changes what employers expect people to know, how do you identify foundational skills that need to be added before the market explicitly demands them?

There’s a real balance here, because “before the market demands them” is a risk. We don’t want to wait until every employer explicitly asks for something, but we also don’t want to get too far out in front of the market; otherwise we’ll end up teaching things that employers don’t yet actually value.

We source expertise from anywhere we can get it: employers that are putting people through our programs; industry and job-specific advisory boards where we invite experts in from the industry; labor market data; certifying bodies in these various fields; and our own learners and graduates. With all of that put together, we can paint a picture of how industries and careers are evolving.

A good example of not getting ahead of the market is our medical billing and coding program. Since 2023, we’ve been hearing from pundits that this job was going to be more or less wiped out by AI. While it’s true that AI is changing the workflow for people in these positions, it’s not reducing the need for people this many years later.

Medical coders increasingly need to validate and work alongside those kinds of intelligent systems, rather than simply producing every code manually as they used to. Still, the demand for the core knowledge hasn’t disappeared. If anything, we’ve actually seen it grow over that time. For us, that means the difference between adapting an existing program versus looking for the next best thing and maybe retiring an old program.

Across industries and roles, we are looking at how we can teach AI literacy, not just the tools themselves, because the tools are going to change. The same AI platforms that you and I are using today are going to be different in five years. But understanding how to evaluate the output AI is giving you, recognizing when it’s wrong, and using it responsibly — that’s one of the new, durable skills we need to incorporate in all of our programs.

Individualizing learning at scale

When you design an AI tutor, how do you build an experience that knows when to answer, when to ask a question, and when to guide the learner toward an answer themselves?

This gets at one of the stickiest problems in AI and education so far. If you ask someone what makes a great tutor, they’ll usually say someone who knows the subject matter better than anybody else. Frankly, that’s wrong. Great practitioners do not always make great teachers because teaching is a skill in its own right.

A great tutor is somebody who knows you, the student, better than anyone else, because they can understand how you learn. They can understand what motivates you, recognize where you get stuck, and understand when you’re frustrated or ready for a new challenge. We’re not completely there yet as an industry, but we’re getting remarkably close to being able to do what a great tutor does with AI. The tooling allows us to design around the individual in a way that’s never really been possible before.

The reason it’s taking more time than people may have expected with a tutor, compared to something like a general customer service bot, is important to call out because the goal here isn’t simply answering questions faster. A really good tutor knows how to probe your thinking in different ways, when to ask follow-up questions, where to provide hints, and when to let you struggle on your own to answer the question, because learning is hard. There has to be some struggle involved for real learning to occur.

I’ve seen a lot of examples so far of chatbots that just give students the answer. That might help someone complete an assignment or pass a test, but it’s a real disservice to actual learning. I’ve definitely seen pieces of this AI tutor done well. I have yet to see anyone put it all together, but I do think we are very close.

How is AI changing the learning experience itself?

Historically, Penn Foster achieved scale through standardization. We are serving hundreds of thousands of learners a year, and the only way to effectively do that was to make operating those programs efficient. Think about things like standard courses that we can use across multiple programs, or a single support model that every learner engages with.

In the last few years, we’ve been able to turn that concept on its head. Technology now allows us to deliver individualized learning experiences at scale. We’re adapting program pacing so some people can move faster and some can move slower. We can give examples to different kinds of learners depending on their experience, offer more practice or remediation where it’s most needed at an individual level, and recognize when someone is ready to move on or move ahead. For the first time, personalization or individualization and scale don’t have to be competing priorities for us.

For product leaders, anytime you’re in an industry where that kind of paradigm shift is happening, speed matters, but I would say what matters even more than speed is your willingness to question old assumptions. I’ve seen a lot of companies get stuck and get in their own way because they don’t realize how quickly this is changing. They assume what’s worked for them in the past will always work in the future, and it’s not always true. If you can’t spot those trends and shift accordingly, then somebody else is going to do it before you do.

Practicing the human side of work

When you’re building interactive experiences for learners, how do you decide which parts of a job need to be practiced rather than simply explained?

Practice is so important, no matter what we’re talking about. This is classic learning science — reading about something and becoming fluent in it are two completely different things. We know that people learn through a series of practice, feedback, reflection, and repetition. That’s the model of a good learning experience. That’s especially true for career education, where you’re trying to master new skills, not just remember new facts.

If your future job involves interacting with patients or customers, or working with equipment, machinery, or software — which, let’s face it, describes most jobs — you have to practice those situations before you encounter them on the job. It’s really a prerequisite to success.

Technology is making this much easier to do at scale. At Penn Foster, we began investing in real simulations and interactive experiences a few years ago, and we can build them even faster today with new tools that our product team is building. We want to present learners with a real situation, the kind that they might encounter on the job, and then present them with decisions they need to make and give them feedback in real-time as they make those decisions.

Then we want them trying again and again, because that’s what’s going to build confidence before they’re ever in the real workplace. We can couple online learning experiences with simulations or scenario-based learning.

We still take on-the-job learning experiences very seriously as well. Nothing really beats getting into a clinical setting, having some supervision, and actually getting a chance to use your hands and do the job.

Can you share an interactive experience your product team has built?

One of the best examples I can give is in our HVAC technician program. This is somebody who’s learning how to respond to a service call and fix an air conditioner that’s gone down. We can give them an interactive simulation where they work with the equipment, choose the right tools for the job, and identify different parts of the unit and how to take them apart and inspect them. They do this with a 3D model they engage with on the screen.

Each time they’re asked to make a selection or perform an action, they’re getting feedback. If they got it wrong, we prompt them to try again and give them a little bit more information. If they got it right, we reinforce the knowledge involved and help them understand where that would come into play in an actual service call environment.

Students don’t even realize their progress because they’re used to learning by reading static text or watching a video. When they’re actually doing, they’re learning better than in any of those other environments, but it feels more like play. It feels more like getting a chance to practice, which creates more engagement. Our student satisfaction scores have gone up once we’ve started introducing more of this. It’s a win-win because it gives students what they want and also helps them learn the material better.

Which interpersonal skills will help one worker win out over another as AI takes on more technical work?

Skills like effective communication, curiosity, empathy, and human judgment are becoming increasingly valuable and skill-defining. I want to be careful not to create a false choice between the interpersonal and the technical — many of the careers that we serve still require a person to pass a certification exam, and those certification exams measure deep technical knowledge associated with the field. Technical mastery still matters, but the opportunity is to layer increasingly authentic practice experiences on top of that knowledge. That way, when someone shows up to work on day one, whether it’s in an office setting or a clinical setting, they feel more like a seasoned professional.

That’s where simulations become incredibly powerful. We have scenarios where you’re practicing difficult customer conversations or explaining a diagnosis to a pet owner, going back to the vet tech example. We can use those simulations, technology, and real-time feedback to get people comfortable with more of that interpersonal element of a career.

It’s one of the few ways to safely practice human interaction at scale. You can do it online in ways you didn’t use to be able to. As AI handles more of the routine, technical work, those interpersonal moments become even more important. If you can show a potential employer that you’re competent and confident on the job, and have an ability to work with a team and work with customers effectively, that’s what they’re going to look for and how they’re going to make their hiring decisions.

Building a career that evolves with AI

What does it mean for a career to be AI resilient now, and how do you expect that definition to change over the next decade?

My answer to this has changed in the last couple of years. If you asked me that a year or two ago, I probably would have talked more about choosing the “right professions.” Today, I think it’s a little bit more about people’s mindsets. Even working with the product managers on my team, the people who will thrive aren’t necessarily the ones who know the most today. They’re the people who stay open and willing to learn and adapt as technology changes the roles that we’re all in.

You can’t assume that your job, or even your career in the field that you’re in, will look the same in five or 10 years the way that you used to. That’s scary, but the one thing I know for sure is that change is inevitable. Resisting change is not going to get you anywhere. It’s about acknowledging that and getting comfortable evolving alongside it. As new tools come along, work with them, practice them, get to know them, and make your own judgments about how valuable they are in your day-to-day.

I also think that changes how we think about education. Education can’t be something that you’re ever really finished with. It’s not the thing that comes before the career phase of your life. It has to evolve with you and your career over time. That means it’s a lifelong endeavor, a lifelong process.

At Penn Foster, we hope to become a lifelong partner for workers in these fields and employers in these industries — a trusted source of knowledge, information, and skills that they can continue to come back to as this technology changes and as these roles change over time.

Can you share any advice you’d offer to an 18-year-old high school graduate, and would that differ for someone who’s mid-career?

A big part of our work at Penn Foster is our large online high school program. We are working with 15-, 16-, and 17-year-olds every day, and they’re working toward their high school diploma. As you would expect, a lot of them want us to help them make those decisions about what comes next.

For the last 30 years or so, as a society, we’ve been sending one message: college for all. The only surefire way to get a middle-class lifestyle is to get your four-year degree, and that probably has always been wrong. It’s more wrong today than ever because a lot of these professions that can lead to a stable career don’t require a college degree.

None of the careers that we’re preparing people for, outside of a few associate degrees, require college. They require a set of skills, and they often require a certification to get started, but then they can lead to a successful career.

My biggest piece of advice is to ignore the people who tell you that there’s only one way. Broaden your exposure to different careers and different fields, and don’t be afraid to try something. It’s not the biggest risk in the world to do something for a couple of years and then decide that you want to go a different way.

People get hung up on: I have to make this choice at 18, and it’s the only time I get to make this choice. The stakes are super high. I think the stakes are lower than a lot of people realize. As education becomes something that you come back to again and again throughout your life, it’s also that recognition that you can change your mind and try different things.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In this conversation, Aaron makes the case for why every PM should build a personal agent, not just a work one. He talks about how doing so has changed the way he uses the internet, what it teaches him about the agent-using customers now showing up to every product, how to pick a first project, what he’s learned from his own agent failures, and what it actually costs to get started. He also makes the case that, underneath all the practical upside, it’s just fun.

Why build a personal agent?

You’ve argued that every PM should build their own personal agent. What changes in the way someone thinks about products after they’ve actually built one for themselves?

I think it will blow their mind. It’s just such a different way of using a product. As a PM, you’re sometimes detached from the outcome. You build the thing, you wait to see users use it, you might watch user tests, you go to learn and you observe. The thing with personal agents is you’re a feedback loop of one — if you build the thing and it breaks, you’re in trouble immediately. In some ways that reminded me of originally playing with Tamagotchis, except the stakes are way higher. So it’s a very different way of building, but it also gives you such a perspective into the things that engineers go through and what your teammates go through.

You now have, not accountability, but you’ve got to take care of everything. The agent is the thing you’re interacting with, but you have to think a lot more about the prompts and the logic and the context. That’s why I use the Tamagotchi analogy — you used to have to water it, play with the thing. Where if you build an agent and give it no context and no tools, it’s like, “Well, it’s just a chatbot.” There is no difference. It’s kind of stupid.

But if you build yourself an agent and go through that exercise — think about what context you can give it, what you can teach it so it could be more useful for you, what tools you can give it so it can enrich itself further — that’s when your mind is blown. It becomes really, really useful, and it’d be hard to go back. I don’t want to do this manual thing over and over anymore.

How has it changed the way that you use the internet?

Obviously, the introduction of LLMs changed people from using search behavior to using chatbots. That was step zero. A lot of folks have already shifted to using ChatGPT and Anthropic to do search queries with a chatbot. The agent way of changing the way I use the internet is going beyond just search.

Here’s such a stupid example, but it’s fun: I was with my wife and we were recently looking at toys from the 1980s, just pulling things out of a pile. Before you might do a Google image search and then try to figure out how much a thing costs, but I already have these agents built. So instead we’re snapping pictures, sending them to the agents and just saying, “Go figure out how much this is. Go find the eBay listings. Go find the conditions,” and we’re sitting there feeding this to the agent — we’re just talking. I’m not even on the internet. I’m just snapping a photo, sending it off to the agent, and using voice to chat. And every few seconds we’d get back a response like, “That toy’s worth five bucks. That toy’s worth three bucks.”

It was so silly, but instead of me having to sit there and Google search and reverse image and then pull all this information together, you can delegate this little minion to go get the information and bring it back. It didn’t interrupt the flow of the conversation, and I think that’s the ideal. The goal is this should supplant and amplify the thing I’m currently doing without being disruptive.

What building an agent teaches you about your users

Many PMs are experimenting with AI through prompts and chatbots. What do they learn by building an agent that they wouldn’t learn just from using AI tools?

I think it’s incredible that PMs are experimenting, period. I think curiosity shouldn’t stop at just the chatbot. The thing they’re interacting with is an end product. There’s a lot to be learned by building an agent because it makes you understand what goes into it. It goes back to giving it context, and being responsible for it staying alive, for lack of a better term. How does an agent fail? And once you understand that it failed, you start to build things differently, because you need something that fails out loud. It can’t fail silently.

A person doesn’t expect to keep learning UIs. We’re seeing that change already. If you only interact with the chatbot, this little box is where the work occurs. The thing with agents is you’re bringing the agent to wherever you’re doing the work, which is very different. A chatbot lives in the chatbot’s window. So if you’re downloading Claude Desktop, you’re working in Claude. When you start to build agents, the agent goes with you where you need it to be. If you’re working in a terminal and you’re looking at financial data privately on your computer, you didn’t have to upload that to Claude and put it in the cloud. You can do that on your machine and your agent can be with you on that journey.

Even in the way I use products — if you have an agent, one of the first things you think about is, well, how do I get my agent to help me explore this product? People are less like, “Go build me a B2B product that I need to learn how to use.” Instead they’re saying, “I want to use your product. Can I bring this to this other thing I’m doing where my agent already exists?” An agent becomes a first-class product need. And if you’ve never built an agent, how the hell do you build for that? You don’t understand the user you’re building for if you’ve never built one. It’s like you’re blind.

Finding your first project

How do you identify the right first project? What makes a problem a good candidate for an agent?

What annoys you? I think that’s always where I start. The project most people recommend is to build a daily morning briefing; even I built tutorials telling people to do that as a starting point. But maybe that doesn’t annoy you, and so maybe you won’t do it. I think it’s great to build an agent for something that you find deeply annoying or deeply repetitive, and you’re like, “I don’t want to do that anymore.” That’ll motivate you to get through the learning to get to building the context so you don’t have to do it again.

How many agents do you have? What’s a good number?

I think one is a great starting point. Building one that is on and persistent and gets to know you is what I advocate for. One held for a really long time.

My Discord channels setup — being able to bifurcate an agent’s access to different channels and loading contacts in different channels — one agent can handle that. We just remind it what memory it should load when it’s in that channel. For instance, if I’m interacting with an agent in the inbox alerts channel, it’s flagged that we’re talking about email. I’ve given it a memory. It’s the same agent, but it’s saying which set of memories apply.

But I do have two agents now, and there was a reason for the second agent. The first agent is on a Raspberry Pi, and the RAM and the storage on that device is much less than a computer. I’m very interested in using more local models, and I’d like to give it more advanced use cases, like analyzing my finances, and I’d like to be able to give a more powerful machine with more storage so I can keep the thing inside my house, versus pushing this information to someplace that I have no control over and no visibility. So the second agent I’ve added — which, again, is just the name of the computer, it’s an M1 MacBook — is called Agent M1.

It has different tools. The first agent has general access, less tools. It can answer questions, it could work across channels, but it can’t cause too much damage. This second agent I built has more tools. M1 can self-administer my GlutenOrNot application. It can access the logs, it can file error reports, it can push code to fix it without me overseeing it. I’ve given that machine more power, piece by piece, as I trust it more and more.

Learning from loud failure

Can you talk about some of your agent failures that have taught you the most?

All the time, nonstop, 24/7. Using OpenClaw when it first came out, I probably spent hundreds of hours tinkering, and it was fail after fail after fail. But the ones I can pull better patterns from — I think letting agents fail silently is a big mistake, and I’ve learned a lot from it.

A great example: if I’m using an agent powered by Claude or Anthropic and the agent gets logged out, or the API key expires, it’s really hard, if you let that fail silently, to understand what went wrong. You could spend a lot of time chasing what’s actually a really basic error, like you’re logged out — that’s the error. But if you build the webhook or a notification to say, “Agent X has been logged out,” you save yourself from wasting a ton of time trying to debug.

A big mental shift for me was making sure agents fail loudly, making sure how they think is visible. So then if I send something to them and there’s an issue happening, we have some sort of log of the thinking and the dialogue and the interaction.

Another pretty big failure point for me: at first, I spent a lot of time trying to approve every single thing. Because I didn’t know what I was doing at all, I thought, “Oh, if I read all the interactions, I’ll figure it out.” I think that was not the best use of time. I learned more from letting an agent try a thing and screw it up than from trying to read the code it was writing.

The better approach was to understand: success for this thing is being able to classify emails, read emails and triage them. I decided I’d do an experiment where I give it 5% of my email for a week. Then, if it stinks, we fix it. If it doesn’t do a good job, we’re just going to work on it. It’s iterative versus, “It’s got to be perfect.” I would’ve gotten much faster learnings earlier on in working with agents if I was just willing to let it fail in a contained environment.

What it costs to get started

What should a PM who wants to build their own agent budget for?

Personally, I currently am using a Claude Max subscription, the $100-a-month one, and that is ample to power the two agents, because I’m using Claude Channels as a setup. Claude Channels is the thing that allows a persistent session to stay alive, and I can access it via Discord. You don’t necessarily need that. I know plenty of folks that are using OpenRouter and using different models and different setups, and they’re spending less than $50 a month. So I would say budget $100 to get started, get up and running, and then pretty quickly, once you figure things out, you could swap out anything you want with different models and setups.

I didn’t mention hardware, though. You can use old computers, that’s worth saying. I’m fortunate I had an M1 MacBook I was not using for this new agent, but originally I did buy a Raspberry Pi. You don’t need the newest one — you can get a Raspberry Pi 4 or 5 for, I think, between $100 and $200. I always recommend Raspberry Pis. They’re super cool personal computers that do all sorts of fun stuff besides building agents. It’s really small, barely consumes power and it can do everything a computer can do. But it’s slow.

Why it’s fun to build agents

If a PM spent a weekend building a personal agent, what would you hope they walk away understanding about the future of software that they didn’t understand on Friday?

I’m hoping by Saturday they get the agent alive, Saturday and Sunday they spend some time figuring out what tasks they’re going to give it, then Monday comes and maybe they’re reflecting. What I would hope for them is they enter Monday understanding that the internet and the way we’re using it is changing. No matter how you feel about it, I don’t think agents are going away. They’re so convenient, and as soon as people get convenience, they’re very unlikely to put it back in the box. So a transformative outcome for a PM would be just feeling that in some capacity.

That’s why I always advocate for finding something that annoys you, because the minute you offload one thing that annoys you to an agent, you’re like, “Oh, what else annoys me?”

Apart from the learning aspect, is it just fun to do something that’s not work-related?

One-hundred percent. That’s why I advocate for personal agents — I am not here to offload building an agent for work. Software tools are going to do that for you. Every freaking company’s building an agent at this point. But building a personal agent — who’s doing that for you? The fun here is you are building something for yourself that can benefit you in your life. It’s a piece of technology, so as excited as I get, it doesn’t replace the human side. I like building things. That’s why I got into product management. I just knew I like to build stuff, and I’m not a real engineer, and building agents scratches that itch of building something.

I am the user of the thing I am building, and I’m continually surprised with the things I can do with an agent. At first, I was having an agent watch my email. Sure, that was cool. Then I was figuring out how to build another agent that can watch Twitter using a Twitter API to give me information I want. You start to stack these things up, and piece by piece you’re like, “Oh wow, each of these little functions in my life that required some form of overhead or mental capacity, I can now offload back onto an agent.” It’s a bit freeing.

So, it’s fun to build, and then it’s fun to get time back, and have more time for the stuff you want to do. I needed to figure out a part for a cabinet in my house that broke, and I did not have the time to fix it. Instead of having to go log into Claude and sending a photo and doing all that stuff, it was just a conversation. I flipped it to my agent, came back, and had three Amazon links with the dimensions, assessed off the photo. That is a thing that would’ve gone on my to-do list and sat there for two weeks. Instead, in conversation, in flight, handed off, got back a result, able to order the part, not a beat missed.

My to-do list that used to be 20 items long, maybe there’s still three items on the list, but they’re really the ones I should do. I’m throwing the rest to the agents — let them get on those things that would’ve just sat there.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In our conversation, Ephie discusses the differences between leading UX in enterprise and nonprofit environments. He shares how AI is changing the role of designers rather than replacing them, as well as how creativity and storytelling shape better product experiences. Ephie also talks about building and designing trustworthy technology in the age of AI.

Building products around real user needs

You lead UX across large enterprise platforms while also leading product strategy for a much smaller nonprofit. How has moving between those two environments changed your perspective on what great product and UX leadership actually looks like?

There are major distinctions between the two. In a large enterprise company, things take longer to build because there are so many moving pieces, including the number of customers with active accounts. You’re working with so many different people and creating new technology, so you can’t just do things on a whim. You have to plan ahead, be strategic, and ensure what you’ve built is well-vetted and tested before you ship anything.

It’s refreshing to work in a small nonprofit space where we can ship things quickly and experiment. There’s a lot of fun about that, but at the end of the day, it’s the same principle — we’re working to solve real user needs and deliver a product that works for them.

In either space, it’s easy to fall into the trap of thinking that everybody will want a specific feature or product. You can go ahead and build it, only to realize that nobody actually wanted it in the first place. That’s where strong UX leadership comes into play. You have to ask, “Who’s asking for this and why? What’s going to make our users’ lives better?” It’s important to think about the whole workflow.

This is also how I approach AI — you don’t want to build an AI product or feature just because it’s trendy and cool. You need to think about the biggest pain points. Whether it’s a massive company processing millions of dollars per day or a small nonprofit using your software sparsely, what tasks are tedious for them? This is true across all industries, and this is where AI can come in.

Do you have any learnings from working in a smaller space that you leverage in an enterprise role?

One thing I oversee in both spaces is leveraging AI for auto-filling forms. You may think that’s a no-brainer, but it isn’t. AI has to understand the various ways people answer the same question across different forms, so you have to train it for your industry. Working in the nonprofit space gave me an appreciation for how difficult seemingly simple problems can be. We had people handwriting forms, writing in the margins, and drawing little sketches of what they meant. It made me appreciate the broader AI challenges we’re solving in the enterprise, and it gave me better language to work with our AI developers.

Do you feel that running product has made you a better UX leader, or has leading UX made you a better product leader?

I’d say it’s a blend of both. My background is primarily in UX, and bringing that experience into product leadership has been incredibly useful.

People will say, “We’re facing this huge product problem we’ve got to solve.” I can put on my UX hat and think, “We could solve this very easily.” The user might be asking for some elaborate feature that will involve months of work, but maybe we only need to move one interaction or change one button. Suddenly, we’ve gone from months of development to a couple of days.

It’s been fun bringing that mindset into product leadership. You don’t just want to build something viable — you want it to be usable and simple. That’s my UX mantra with everyone I manage: “Keep it simple.” The elegant solutions that look obvious are often the hardest to arrive at.

Enterprise software especially gets complicated over time because every customer requests something new. The customers who pay the most are often the ones who want something built specifically for them. That’s when I wear my product hat, even though my official role is UX. For example, early in my time at Applied Systems, we had a customer with a long list of requested features. Instead of trying to solve all 30 requests, I asked, “What have they consistently told us are their biggest problems?” We reframed the conversation around outcomes instead of outputs. Rather than talking feature by feature, we focused on what would actually make their team’s life better. That one larger outcome was going to move the needle far more than checking off dozens of individual requests, and the shift changed our relationship with the customer.

Why AI makes UX more valuable

Are there UX practices that have become less valuable because of AI? Which have become more important?

The days of what we used to call “pixel pushing” are coming to an end. Think about spending endless hours making a prototype look perfect — AI can do a lot of that now. That begs the question: what should the UX designer be doing?

I’d argue the role of UX is more important than ever because AI can generate so much so quickly. We had a product manager take something that had been on a six-month roadmap and come up with a working solution in just two days using vibe coding. The developers spent another two weeks validating and refining the product before it shipped.

Where did UX come into play? We stepped back and asked, “What is the user trying to accomplish? What are their biggest pain points? How can we smooth those over?” That’s where UX adds value.

Right now, we’re seeing roles start to blur. Even though a product manager may wear a UX hat and a developer hat, that doesn’t mean those responsibilities become their job. We still need specialists, but we also need to be willing to move outside our lanes.

I like the idea of T-shaped professionals. You go deep in your own discipline, but you broaden what you’re capable of across others. I’m seeing more of that at my company, and I’m doing it myself. Recently, I’ve been pushing code for the first time in my life. For example, one of my hobbies is doing blind wine tastings with friends. I wished there was an app for it, so I vibe-coded one. It’s improving so quickly that I’m now planning to release it.

The most important thing UX professionals can do today is stay connected to real users. Talk to real humans; don’t just rely on AI. AI-generated work might earn a passing grade, but it’s rarely A-level work. For example, my wine app looked great at first, but once I actually started using it, I realized the database wasn’t very good, and parts of the experience didn’t work well. After iterating, I got it much closer, but I still had to test it with real people.

One friend I tested with is in his 60s. He immediately needed to zoom the screen, and that completely broke the experience. I never would have discovered that sitting at my computer. You don’t know what you’ve missed until you watch someone use your product. The important lesson here is that AI gets you started, but it doesn’t get you finished.

How do you quantify whether design is creating business value?

Measuring UX value is difficult because it’s closely tied to product success. One of the biggest successes I’ve seen was when a designer did user research on a roadmap initiative that already had designs, planning, and development scheduled. The research showed users simply didn’t need it, so we decided not to build it. To me, that’s an enormous UX success. We saved weeks of engineering time, product time, UX time, release effort, adoption work, and, ultimately, a tremendous amount of money.

Instead, the research uncovered something users actually wanted — and it was much simpler. Sometimes the greatest value UX delivers isn’t launching a feature. It’s preventing the wrong one from being built.

Creativity as a competitive advantage

There’s a growing narrative that AI is commoditizing design. Do you agree? Are there parts of the discipline becoming even more valuable?

Absolutely. AI is creating many solutions, and if you aren’t pushing beyond those initial ideas, everyone will produce the same things. There’s something positive about that because users benefit from familiar patterns. If somebody has seen a particular interaction before, they’ll probably understand yours immediately.

What worries me is that it can stifle the human creativity that’s required to solve problems in better ways. Sometimes the right answer isn’t the big, flashy solution — it’s the simple change that removes friction. Those are the kinds of solutions that still require human creativity.

My brother recently opened a restaurant and was frustrated with the software he was using. My first instinct was to imagine building an entirely new inventory and recipe management system. Then I actually talked with him, and found that his software worked fine. All he wanted was for his recipes to print differently.

My brain had immediately jumped to a huge, exciting project when all he really needed was one small improvement. That’s the opportunity — talk to the user before you start building.

How do you stay creative as AI becomes more capable, and how does that creativity influence your work as a product and UX leader?

I’ve recently caught myself thinking, “I’ll just ask AI,” only to realize AI doesn’t actually know the answer — I just need to think. We have to be intentional about protecting our creativity.

For me, that happens outside of work. I’m an author, and I love building worlds and characters. That creativity carries back into my product work.

Storytelling helps in two ways. One is communicating vision and helping people understand a product direction in a relatable way. The other is thinking about the user’s story. What’s their journey today? Where are the pain points? What’s the ideal journey? What’s the gap between those two? I think about that the same way I think about writing novels. There’s a beginning, some conflict in the middle, and a successful ending.

I’m also a big fan of Jeff Patton’s user story mapping. Just like a movie storyboard lays out every scene, you can map a user’s journey with sticky notes. As you walk through it, you start asking, “Do we really need these steps?” If someone can accomplish the same goal with four steps instead of 20, everybody wins. Less is more.

That way of thinking is influencing the work my teams are doing around agentic AI. We’ve already built several AI capabilities into our software, like autofill, but that’s only one step. Our vision is for AI agents to assist throughout the workflow while keeping a human in control.

Imagine an email arrives. An AI agent reads it, moves it into the system, identifies which form is needed, autofills it, identifies what’s missing, drafts a response requesting the remaining information, and then pauses for human review. Instead of a user completing six different steps, they review one workflow.

Further, I try to disconnect every day. I’ll take a walk at lunch, get away from the screen, look into the distance, maybe walk with my wife. When I come back, I usually have better ideas. You have to give your brain space to do its work.

Designing AI people can trust

That’s good advice. What are the biggest mistakes you’re seeing product or UX teams make as they integrate AI into their work?

The biggest mistake is trusting AI the first time through. People generate a summary, a prototype, or some research synthesis and immediately share it because it looks impressive. Someone else will read it closely and discover that the conclusions are wrong or that the sources were misunderstood. The first draft is rarely A-level work.

The other mistake is the opposite: not using AI enough. People get overwhelmed because it’s capable of so much. I’m experimenting with using AI to analyze financial markets. It can process more information than I ever could, but many people still use it only as a chatbot.

Start connecting it to your analytics platform. Connect it to your research repository. Ask it questions about user behavior. Those are the kinds of workflows where AI becomes incredibly powerful.

How do you build trust into AI experiences?

Trust is the biggest challenge. Everyone we’ve shown our concepts to says the same thing: “I don’t trust AI.” Honestly, I don’t blame them — we’ve all seen AI make mistakes. That’s why we’re focused on principles like transparency, keeping a human in the loop, making actions explainable, and allowing people to undo or edit what AI has done. Those things have to be designed intentionally.

For large enterprise software, that’s a significant effort, and a large portion of our UX team is focused on AI experiences because getting trust right takes a lot of work.

What advice would you give UX designers who are early in their careers on the skills they should focus on building?

First, learn how to do real user testing. Talking to real people is becoming the most important part of the job. Second, don’t be afraid of AI — but don’t trust it blindly either. I’ve seen AI produce impressive-looking work that completely fell apart once I verified it. I even tried using AI to organize my taxes. It made enough mistakes that I eventually started over, but I still used AI where it was appropriate. I used Expensify to scan receipts and compile the information, for example, and it worked flawlessly because it was designed for that purpose.

Use the tools and be smart about them. Don’t accept AI slop. I’ve told my whole team that if they don’t embrace these new tools, they’ll get left behind. The people who are learning how to work with AI are already doing things they couldn’t have done just a few months ago. We’re living through a true revolution in what people are capable of, and it’s a very exciting time.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In this conversation, Hoffmann talks about the global shift from barcodes to QR codes under the Digital Link standard, a transition he expected to take a decade that’s now happening in a fraction of that time. He explains why packaging is becoming the start of the customer experience rather than the end of it, walks through real examples of brands using dynamic QR redirects for everything from sustainability messaging to Super Bowl content, and makes the case for why product managers need a more commercial, customer-facing skill set to keep up.

The accelerating shift from barcodes to Digital Link

Walk us through what’s going to happen in the coming years as we transition from barcodes to QR codes.

The barcode is a very established code, but it’s more than 50 years old. There’s a global initiative going on that describes the transition away from a barcode to QR code. The reason why this happens is you can store so much more information behind a QR code rather than behind a barcode. More and more brands and retailers are adapting the new QR code standard.

When we started working in that space, if you would ask me about one and a half years ago, I would’ve probably said I’m expecting a transition phase from a barcode to the QR code experience over the next, I don’t know, decade or something. It appears to happen much faster. We do have a bunch of retailers in the world that completely skipped the transition period for some of their products (not the entire shelf) — they’re already done with moving from barcodes to QR codes. Some of them are still transitioning, so everyone’s going at their own speed, but it’s definitely happening. This opens up completely new product experiences, completely new brand experiences, and there’s a huge potential for us as product managers behind that.

Packaging becomes the starting point, not the end

Product teams have traditionally treated packaging as the end of the customer experience. How do QR codes change where the product experience begins and ends?

That’s going to shift the order a little bit. Packaging will not be at the end of the customer experience anymore because it’s tremendously important to start strategizing around that new opportunity. When we talk to our customers, the companies that have started strategizing around that — what content they want to leverage, what customer experiences they want to build — will have a huge jumpstart against competitors who just wanted to execute or who go directly to the execution. Packaging creation has to be an essential part of the customer experience right from the start. First of all, you need to put a QR code on the package. Then it goes to the question, where are you putting a QR code on the package? It requires a little bit of space. Space on packages is limited, so you have to start thinking about how your package design would fit into that.

You also have to think about what will be behind the QR code. We recommend to our customers to really start thinking about that at the earliest possible stage. It can just start with creating a strategy — what brand and product experience you want to leverage. That’s always the first point.

What does a QR code do that a barcode can’t do?

First I’ll talk about what the barcode does. The barcode that we all know literally stores a number, which is the GTIN, the Global Trade Item Number, and that identifies a product. The QR does something similar — it also hosts a GTIN — but a QR that you scan with your mobile phone directs you to a link.

That link follows a taxonomy — that’s a global standard. Every link that fulfills this Digital Link standard, when you scan a QR code, has to follow a specific taxonomy, and that includes the GTIN that still identifies a product. But the difference between a QR code and a barcode is that the link can redirect you dynamically. You can scan a QR code while you’re at a Target in, let’s say, Chicago, and I can scan the same barcode in a Walgreens in New York City. We’re scanning the same barcode, so we’re opening the same link, but we’ll get different redirects. For example, you could get a promotion from Target — buy this product, buy two, get one free. I could be getting sustainability information. It can be completely different information. It opens up a new product and brand experience, because the content you put behind the QR code is completely up to you, and that’s where the magic happens.

New responsibilities for product managers

If every product becomes a live digital touchpoint with a QR code, what new responsibilities does that create for a PM that they didn’t have in the barcode era?

It’s another customer touchpoint and another channel to deliver unfiltered brand content directly to the consumer. Typically, if you try to create some brand content in any other channel, it would go through a retailer filter, where the retailer decides what they put on the shelf. This is one of the very few opportunities to deliver content you want directly to a consumer without getting filtered.

Product managers should be part of that strategy process. What content makes sense? What content is relevant for my target audience? Typically, looking at the role of product management from a traditional perspective, they would know their customer segments, and they would know their personas. This can be very valuable information in the process of creating the content that’s behind the QR code.

You’ve talked about creating different experiences for different customers using the same QR code. How do you decide which use case is actually worth building first?

That’s one of the topics that really excites me the most. The amount of use cases that you can create or leverage are endless. Whenever a customer comes up with a new use case or idea, in almost every case, the answer is, “Yes, that’s possible!”

We’re talking to customers every day about possible use cases. Typically the conversation with the customer goes to a little bit of brainstorming around their specific needs and ideas. You can do so much. It starts with a very simple landing page about sustainability information or brand information. Think about gamification — you can do a little micro game. You can do a raffle. If you’re sponsoring an event, you can display the content that’s relevant for your customers.

One of our customers sponsored the Super Bowl, and they displayed specific Super Bowl content before and during the game. After the game was done, the packages were still out in the market in stores, and they just changed the redirect — they said, “Congratulations, team X, Y, Z. Really great game.” They can change it right on the spot. It takes 30 seconds. That could have been done with changing the content on the landing page, too, but maybe you want to apply more changes than just text.

A successful Digital Link experience involves content, commerce, engineering, and customer experience. How should a PM bring all those disciplines together?

The role of a product manager is trending more and more into a commercial role. When dealing with topics like the QR code transition, you really have to know your customers. And you can only achieve that by working directly with customers. You can’t just hide behind your sales forces or your customer success management teams — that’s not an option anymore. A product manager needs to be customer-facing and hear directly from customers about what they’re trying to achieve and what their goals are. The role of a product manager is to think about how we can transition those goals and rough ideas into customer experiences.

When I started working in product management, the role of a product manager was very much tied to the tech teams, and that’s still valid, but the customer perspective comes on top of it. It’s tremendously important that you know your customer, your customer segments, and your personas, because then you can try to adapt to that to determine the best way forward.

Let’s talk about the tech side. What technical decisions need to be considered with the Digital Link?

First of all, there has to be a decision to transition — not only from a technical perspective, but companywide. Then you would need to have something that we call a resolver. The resolver looks up the link that sits behind the QR code and redirects the scan dynamically to a target URL. Either you build this capability on your own — I’m not really recommending this because there’s a global standard, but there might be changes in the future so you have to be on top of it — or you work with a company like ours that offers a resolver. The promise that we make to our customers is if you use our software, you don’t have to read the document about the global standard. You just use the software, and then you can be 100% sure that you comply with these global standards.

Why a background in PR, marketing, and sales matters now

You worked in PR, marketing, and sales before product management. What skills outside of traditional product management will become important with the Digital Link transition?

When I started working in the PR space and in the marketing space, no matter what company I worked for, I always wanted to be as close as I could be to our customers. That’s why I stepped away from PR and marketing — for me, that wasn’t close enough to customers. I moved into sales, and then transitioned into product management about a decade ago. I think this path shaped my view on being customer-facing. People often criticize — and they might not be wrong — that I’m too close to sales. But I actually think that’s a good thing, because product management is a commercial role.

Being a product manager is 80% communication. Having worked in marketing, sales, and PR builds your communication skills. It’s important when you’re working with customers and doing stakeholder management, but communication also determines whether you’re successful in building products. The same is true when you’re creating use cases for QR code experiences.

In PR and marketing, content creation is the focus. How does that carry over to the Digital Link?

The tech part becomes less important because it’s pretty straightforward. It’s more about the content, and having worked in PR and marketing brings you as close to content as you can be. In those fields, you are thinking about what content is relevant for your customers. You think about how you write articles or how you position your products or your brands. It’s the same with the QR code experience — it’s literally brand and product positioning. I think my background was really helpful.

Lessons learned and advice for PMs today

Can you walk us through a customer implementation that changed your thinking?

A customer implementation typically starts with something very straightforward, because you have to convince multiple functions within an organization, like marketing, sales, logistics, and the production teams. So we always recommend starting with something easy, like a sustainability page. Everyone speaks about sustainability, so if you’re working in certain industries, that can be very relevant to you and your customers. We always recommend setting the entry barrier as low as you can, and then doing the analytics — check if the content is relevant for your customers, and then adapt.

What surprises me during an implementation process is how quickly companies learn. If they start with a sustainability landing page, they learn quickly — we thought this content is relevant but, effectively, it’s not; we see in the analytics that no one’s interested, or no one scrolls. So we change it. We do a little bit of research, we experiment, we do A/B testing. The pace at which brands learn about what’s relevant has been kind of surprising to me. They learn much more about their customers this way, on top of whatever methodologies they already have to learn about their customers.

Did you have any early fails that you learned from?

A bunch of them — I think every product manager does. One of the big fails is misaligned go-to-market and missing product-market fit. If there’s a misalignment, it doesn’t show up on a board. You can work with JIRA, you can work with any roadmap tool, but if you haven’t thought about how to market the product, or whether there’s a real product-market fit, it’s hard to fix that at a later stage, because you’ve already put in the effort. We started developing our own software, and we thought it’s great, everyone wants to have it. No one actually wanted to have it. That’s changed significantly over the last two years, but we might’ve been a little bit ahead. If I could turn back time, I would love to have spent more effort and time on product-market fit and thought about go-to-market a little bit harder, because I think that would’ve saved us a bunch of internal conversations.

They’re important and hard to surface. JIRA tells you a lot about velocity, or if you’re delayed in a sprint, for example, but it doesn’t say anything about whether your go-to-market strategy is nuts. You have to determine that.

As brands prepare for the transition to Digital Link over the next few years, what should PMs be doing today to make sure they’re ready?

This is a new topic, and I think this relates to every new movement in an industry: PMs should really know what they’re talking about. That sounds simple, but in reality it’s not. Customers rely on what PMs are saying because it’s completely new to them too. They want to learn more about what’s happening in the industry. PMs build a relationship with the customers, and one prerequisite of building a relationship is building trust. Trust builds if you rely on each other, if you provide value.

Also work actively with customers. Don’t sit there until a customer sends an email, but working with them if possible on a daily basis, learning from them, being open to new use cases, to new opportunities a customer might bring up that the PM didn’t see coming. And work with sales. That’s tremendously important, because in times of AI, code is commoditized. Running a business is not, and PMs are part of that running-a-business process. They should know their customers, they should work with sales, they should know about the business cases behind it.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In our conversation, JP discusses why encryption should be viewed as a product experience rather than just a security feature. He explains how AI is reshaping data governance and how, with AI technology, product teams need to think differently about access and permissions. JP also talks about the role of transparency and user control as it relates to customer trust.

When encryption works best for users

Traditionally, encryption has been treated like an IT or security concern rather than a product experience. Why should product leaders outside of security care about encryption right now?

The ideal outcome is that encryption never becomes a product concern. The best encryption is completely invisible, so you can do the things you’re supposed to do while your information remains protected behind the scenes.

The real question product leaders should be asking is what happens to the information people share with us. There are a lot of free services where you’re not paying with money — you pay with your information. That’s a perfectly reasonable trade if users understand it. The question becomes, “Can I still control my information after I’ve shared it?”

Right now, by default, once data is shared, it’s gone. The problem we’re trying to solve is how to give people continued control over their information, even after they’ve shared it.

If the best encryption is invisible, where does it actually show up in the user journey?

A lot of encryption already exists — you just don’t notice it. For example, when you’re on a banking website, your connection is encrypted. Nobody asks whether you’d like the encrypted version of your online banking experience; it’s simply there as the default. When you’re on a video call, you’ll often see an icon showing the session is encrypted. Many laptops encrypt everything stored on the drive, so if someone removes the hard drive, they still can’t read the contents.

The bigger question is who holds the keys. In most enterprise software, the vendor controls the encryption keys, but in many consumer products, the platform does. That’s why end-to-end encryption has become such an important conversation. Ideally, the people communicating — not the platform — control access to the information.

There’s also a usability problem. Say I go to a doctor’s office, and they want to send me medical records. Instead of emailing them securely, they’ll likely ask me to create an account, since that’s how they’ve chosen to manage encrypted information. I already have thousands of accounts, and I don’t want another one. However, the purpose of creating another account is simply to verify that I am who I say I am.

We should be able to accomplish that identity verification without adding friction for users. That’s what Virtru is working to solve — our products ensure users can send encrypted emails in Gmail or Outlook without creating a new account. We have many other products coming soon, but that’s the core of what we do.

How does encryption come into play with chatbots and AI assistants?

Encryption itself is just a tool, but the true end-goal is protecting information. Information is only valuable if it’s shared. Say I write my autobiography and want an AI assistant to help summarize it. The AI needs access to that information, but if I give it unrestricted access, it will also process information I might not want included.

AI has dramatically increased our ability to process information. It can analyze enormous amounts of data in minutes, which means organizations need much better control over what those systems can actually see. Today, the only practical way to do that is to separate the information manually. We’re working toward allowing people to label or tag portions of information so only the appropriate content is accessible to AI, while protected information remains encrypted and invisible. Ultimately, it’s about giving people control over what AI is allowed to access.

How product teams are thinking about data

What mistakes do you see product teams make when incorporating encryption into their products?

In my view, the ideal scenario for encryption and decryption is that it’s completely invisible. The biggest mistake is making security so restrictive that information can’t be shared, because Information that’s never shared isn’t useful. I’ve heard this repeatedly from customers in the intelligence and defense communities.

One example people often reference is that multiple organizations each had pieces of intelligence before the September 11 attacks. Because the information wasn’t shared effectively, no one assembled the complete picture.

Organizations already classify information as public, internal, or confidential. The next step is enforcing those classifications. Today, someone can accidentally send a highly confidential document outside the company. Product experiences should help prevent those mistakes by enforcing the policies that organizations already have.

AI systems are fundamentally data-hungry, but organizations are becoming more cautious about where sensitive data flows. How does that change product design?

AI changes one of security’s long-standing assumptions. Historically, teams could grant broad access and revoke it later if something went wrong. That approach doesn’t work with AI. Once an AI system has processed information, you can’t undo that. There’s no way to put the toothpaste back into the tube. Instead, product teams need to start with the minimum access possible and expand permissions only when necessary. That requires much more deliberate security thinking early in the design process.

Building customer trust through transparency

Are there certain industries where this becomes even more important, such as healthcare or finance?

Yes — there’s a push-and-pull model. Imagine a large pool of information that I’m dumping all my data into. It’s still my pool, but I’m letting an AI model that’s been trained by someone else, such as Anthropic, OpenAI, Google, or another company, come along, swim in it, and walk away with all the information inside it.

More and more, the approach is shifting away from giving AI all of your information. Instead, it’s, “Come and swim in my pool, but here’s the lane you can swim in. Everything else is off limits.” That’s where I believe things are headed. Rather than simply giving AI models access to our data, we’re inviting them in, creating structures, and setting boundaries that define what they can and can’t access. There are already companies specializing in creating those boundaries. From our perspective, if you’ve already segmented your information, you can build on that and use those segments to define exactly what the AI model is allowed to use.

Are there specific product decisions that can either erode or strengthen customer trust regarding data security and privacy?

One of the biggest trust-builders is transparency. Show people exactly what information is being collected, why you need it, and how it’s being used. Regulations like GDPR and state privacy laws increasingly require companies to disclose those practices, but organizations that go beyond compliance and are upfront about data use build much stronger customer trust.

Another important factor is relying on open standards and open source software. With closed-source systems, you’re effectively asking customers to trust that you’ve implemented security correctly. For example, Anthem was hacked in 2015, although it was only disclosed a few years ago. It was one of the largest data breaches ever seen, with nearly 80 million people’s health records exposed. When incidents like this happen, the question becomes, “Well, did you not know that there was someone in your system snooping around? Is your software safe enough?”

When you use open source software, you’re basically saying, “Here is the source code that we’re running, check it out.” You can actually go in and look at it. When software is open, people can inspect it, identify issues, and improve it.

Our own software is based on open standards, and we run bounty programs that reward people for finding security issues. That openness creates confidence because security isn’t based on “trust me” — it’s validated by the community.

I also think AI is changing how companies think about competitive advantage. Writing code is no longer the differentiator it once was. AI can generate code extremely well, so I foresee a shift where the real value comes from designing great user experiences, providing excellent support, solving meaningful customer problems, and building products people trust.

For organizations evaluating an encryption platform, what questions should they ask?

One thing people forget is that companies don’t last forever. When you’re choosing an encryption or data security partner, you should ask what happens if that company disappears, gets acquired, or you simply decide to stop using the product. Can you still access your data?

There are the three questions I recommend organizations or individuals always ask:

• Who controls the keys?

• Is the encryption format open and documented?

• What happens to my data if I stop using your product?

The first question about who controls the encryption keys is related to access. If the vendor controls the keys, they ultimately control access. In our case, we have solutions that encrypt information, but we can’t decrypt it ourselves. Customers retain that control.

The second question is whether the encryption format is documented and based on open standards. If it’s proprietary, you’re dependent on that company forever. If it’s open, you have options even if you eventually move away from the platform.

The third question is about offboarding. If you leave the product, what’s the process for getting your data back? Can you still decrypt it? Can you still use it?

Ultimately, our mission isn’t solely about encryption. It’s about helping create a world where data remains under your control wherever it goes. Encryption is simply the tool we’ve chosen to achieve that goal. The objective isn’t to make security something users constantly think about. It should work in the background, protecting information without getting in the way of people doing their jobs.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In our conversation, Adi explains why AI is a force multiplier for high-judgment product managers rather than a replacement for them, and why outcomes, rather than output, still decide whether a team is winning. He shares what separates high-craft product organizations from everyone else, and talks about how marketplaces successfully scale, including how to layer advertising onto a marketplace without cannibalizing the core business.

How AI is changing the PM craft

There’s a lot of talk that AI has reduced the cost of execution and normalized the PM function, and that the more traditional execution skills are less valued now. Do you agree? And are there PM capabilities that are more valuable now?

Product management is a vector, not a scalar. Direction matters more than speed. AI has definitely accelerated many parts of the product development cycle, from discovery to prototyping to building, testing, and launch. Teams are now able to do more with smaller pods — with fewer engineers and analysts. And judgment and direction are even more important and elevated in the craft of product management than they were before. Great product management has always been about driving outcomes for the customer, which are closely tied to the business, versus shipping more features.

From that perspective, AI is a force multiplier for the high-taste, high-judgment product manager who knows how to prioritize the right problems for the target customer. It gives them speed, and it also gives them greater conviction because many AI tools help de-risk and accelerate solutions. But it has also diminished the value of the orchestrator product manager (the facilitator archetype) and the feature-shipping product manager (the output or feature-delivery archetype). That’s a shift in the skills that will be increasingly valued as AI usage continues to grow.

With AI making the build process easier, you can ship nearly any feature you can think of. So, how do you ensure you’re creating customer value rather than just shipping features?

This problem — the tendency to be output instead of outcome-oriented — existed before AI, and it will likely be exacerbated post-AI as it becomes ever easier to ship features. The key is to go back to the fundamental principles of staying outcome-oriented for your customer and the business.

Let’s say you’re a feature PM or you own an experience on a marketplace or a social networking app. First, you need clear measures of customer value metrics and qualitative feedback to ensure you are improving customer experience. For example, if you’re tweaking the algorithm on an app like a rideshare, you’ll want to ensure you’re matching the driver and rider more efficiently. If you change the user experience in your product, you’ll want to ensure you’re improving customer satisfaction and engagement.

The key is not just launching something for the sake of launching it, but making sure the measure of success (qualitative feedback or a quantitative metric) that you’ve carefully chosen has improved the product experience — this could be personalizing your feed for shopping, improving search relevance, or introducing an AI-enabled experience The measures of success need to improve whether you ship one feature using AI or a series of 10. It’s about advancing the outcome you’ve chosen as a beacon of success, versus the volume of features that were shipped because the new tool in the shed allowed you to do this faster. AI can accelerate the work, but the PM remains accountable for whether the work creates real value.

In complex ecosystems like marketplaces and social networks, lots of teams are working on different parts of the same system — buyers and sellers, consumers and creators. AI may help us ship faster, but we still have to make sure those features create real value for the whole ecosystem. A win in one area isn’t really a win if it comes at the expense of another.

In complex products, improving one metric isn’t enough — the change should make the overall system better. For example, sending more push notifications may drive more conversions, but it’s not a real win if each notification becomes less effective or hurts another part of the experience.

As AI helps teams ship faster, strong measurement becomes even more important. We need to know that a feature improves the broader business, doesn’t hurt partner-team metrics, and creates a true net gain. Metrics tell us what happened, but not always why. That’s why we also need to talk to customers and use research to understand the full picture.

Finally, from a culture standpoint, we want teams to embrace AI as a tool but incentivize outcomes instead of usage. Recognition, promotion, and performance management systems should be attuned to outcomes, because then you create the right incentives for using AI in organizations.

Is it difficult for early-stage PMs to gain the experience to be more strategic, when their role is more operational by nature?

Sure. Early on, product managers tend to be more execution focused, but execution still means driving impact for customers and the business. Even though early-career product managers seldom pick ambiguous problem spaces, they should be cognizant of whether their work is succeeding or failing, and as a consequence, learning whether their solutions are solving customer problems via the measures we discussed earlier.

Being strategic means developing judgment to prioritize the right customer segments and their most pressing problems for building products and features that move the needle for the business. More than ever before, AI-enabled customer research tools like VoicePanel help us better understand customer needs. Such tools provide insights at scale, quickly and cost-effectively, to better inform PMs — early or seasoned to sharpen their understanding of customer problems and the opportunity at hand.

Product management taste develops over time. The key to good taste and judgment is having multiple reps at shipping products from discovery to impact. Another complementary method is to observe patterns and information architecture in the most popular apps — the small details like how they get to know you when you first sign up, or how they reach out when and how often they want you to come back. The best apps do this almost invisibly well so that you are entirely focused on the content versus trying to access the content. The best PMs (early or seasoned) should connect the dots and bring them to the context of your customer and your product, to see what’s missing or what can be elevated.

You’ve built teams at startups and at much larger companies like Facebook, TikTok, and Poshmark. Are there differences in how you build high-performing product orgs depending on the size and stage of the company?

Oh, significant differences. The stark difference is in product craft. For me, product craft is about the motions of how product gets built — the people building the product, the tools and methods they use, and the fundamental philosophies and beliefs embodied while building products. There are certain behaviors typical of high-product-craft organizations, typically found in larger companies — the behemoths like the Facebooks and the Amazons of the world.

First is customer obsession. Most of these teams deeply understand people’s problems and their conscious and unconscious needs. They get to the root of what motivates people to use products. Typically, it’s not a feature, but a people problem. For example, people use TikTok for entertainment, not because they’re looking for short-form videos. Having a well-developed user research function that can understand customer motivations in a very non-leading, non-biased way is super important. Empathy and understanding are number one — especially understanding the deep psychological motivations that customers themselves aren’t able to articulate!

Two is having a product culture that rewards bold initiatives and celebrates failure. You can’t do anything big unless the culture normalizes failing fast and learning quickly. This encourages product managers to make bigger, broader bets that can really move the needle for the business.

Three is a very high sense of accountability and ownership. There are always mandates that come from the top, but typically these organizations hire and/or coach product managers into having a high sense of accountability and ownership around their product area. This translates to deeply understanding the customer problems in the space, prioritizing them, building solutions, and measuring impact in terms of: did my feature improve the day in the life of the customer?

Four is the measurement piece — being really thoughtful around metrics or measures of value in general, both qualitative and quantitative. Ask: How do I know that the customer is better off after shipping this product or feature? If you are in the lead-generating business — say you’re an auto dealer or Tesla — all you can collect is a lead, so the core measure of value for the digital/online team is likely lead volume (weekly or monthly).

But teams can be poorly incentivized if their work is goaled on leads. A good and responsible product team sets goals for weekly (or monthly) leads generated, but also guards themselves against low-quality leads by monitoring the conversion rate (this becomes the “guardrail metric”). The message is to choose your incentives wisely because the organization responds accordingly!

Say your leads generated grow 30 percent quarter over quarter, but your conversion rate (cars sold divided by leads) drops by 40 percent, then the team basically just gamed your metric incentive. Being careful with your selection of metrics and making sure you’re not gaming the system is critical.

The tooling for experimentation, customer insight gathering, and team rhythms around planning and alignment are highly evolved at high-product-craft companies. At smaller companies, startups, or emerging small-cap companies, these are typically developed by bringing in someone who has spent significant time at a high-craft product organization. That’s the biggest difference, and it’s usually a journey of transformation that needs to take place.

A significant portion of your career has been spent in marketplaces. Is there an aspect of marketplace product management that you feel may be misunderstood?

Typically, marketplaces are bootstrapped by aggregating supply — whether it’s cars on eBay, collectibles, or commerce on Amazon — you build supply and then generate demand. Once you have a sufficient selection of clothes and shoes, or labor or rides, and have figured out your demand-gen engine, you work on improving the matching liquidity and efficiency of connecting demand and supply. Some marketplaces are nationwide or global, like eBay, Etsy, and Poshmark. Some are highly local, like Thumbtack, Facebook Marketplace, ride-sharing, or dating sites.

What is often misunderstood is that the effect of demand, supply, and matching on overall marketplace growth and health varies significantly based on the different segments or dynamics within each marketplace. For example, take Facebook Marketplace. It’s a classified marketplace, which could be demand- or supply-constrained. It’s a global marketplace, so if a product management team’s mandate is to grow in a specific country, they’ll first need to understand whether that market is demand-constrained or supply-constrained, because usually it is one or the other.

Good marketplace product managers work with analytics partners to quickly identify the highest-leverage opportunity within the marketplace. Facebook Marketplace created artificial suppression tests on both the supply and demand sides. If there’s a hypothesis that a specific country is more supply-constrained (there are more shoppers than sellers and listings), the marketplace would artificially suppress certain listings from showing up in searches or browse for a finite amount of time. This enables the team to detect whether overall transactions decrease more acutely than the suppressed listings on the supply side.

If transactions drop more acutely, you know the market is supply-constrained. Similarly, if the hypothesis is that your marketplace is demand-constrained — that the bigger opportunity is bringing in more shoppers — and transactions fall more acutely, you know the market is demand-constrained.

Understanding which side the market is constrained on, then going one level down to understand exactly which category — automobiles, rentals, electronics, or clothing — gives product managers the right unlock and where they need to invest, whether to increase the shopper funnel or increase listings. It’s a de-averaging on either side, and it will provide the highest ROI investment opportunity to grow the ecosystem.

Marketplace health and the advertising layer

How does having an ads layer on top change the way you think about the health of the marketplace, the liquidity, and long-term value creation?

Most marketplaces, like Amazon, Walmart, eBay, Etsy, and even Poshmark, have two main revenue lines: transactional revenue, which is a commission on the commerce they facilitate, and advertising revenue. Advertising is a monetization lever, but it’s also a marketing tool for sellers. Many businesses on marketplaces are willing to spend money to increase distribution and sell-through of their inventory. So it’s a marketing service — but there’s a tension between introducing advertising and preserving the shopping experience so it doesn’t feel irrelevant.

Marketplace health matters because having sellers turn over inventory more rapidly via faster sell-through retains your best sellers and keeps them listing more. On the shopping side, it’s all about showing the right ad to the right shopper at the right moment, based on their sensitivity to advertising and the experience they’re in. If a shopper is brand new to the marketplace or has a really specific query, they’re very high intent, so you probably do not want to show them an ad. But if a shopper is just browsing through what’s popular, their intent is lower at that moment; there is an opportunity to insert relevant advertising responsibly.

It is very easy to become greedy and irresponsible when you run an advertising business, because the revenue realization is almost immediate. Most marketplace advertising models are click-based, so every click generates revenue. The responsible product organization wants to ensure that, as you diligently insert advertising, it’s accretive rather than cannibalizing the transactional business. That comes with introducing ads in the right volume (or ad load) at the right stage in the shopping journey for the right user.

We typically ensure this by leveraging a concept called a no-ads holdout — a population of shoppers that sees no ads for 6–12 months, so we can understand the long-term longitudinal effects of advertising. Even though we see an increase in revenue in the short term, you want the long-term health of the marketplace to be unchanged; we don’t want to see a drop in visitors or shoppers in the long term. If the holdout is significantly healthier than the population that sees ads, we know we’re showing too many ads and need to dial down in certain areas. The best teams do this very responsibly.

How do you decide when it’s the right time to introduce advertising in a new marketplace? Is it based on the size of your shopper population or their behavior?

It’s both, actually. At Facebook, they always said monetization is step 99 in the product development process. There are two types of customers on marketplaces: demand and supply, and buyer and seller. You want product-market fit on both sides, so you have healthy cohorts that retain over the long term — buyers who keep coming back, say 30, 60, 90 days out, and the same for your sellers. The key when you’re launching a marketplace is to first make sure you have product-market fit and have maintained healthy retention for both demand and supply audiences.

You also figure out channel-market fit — your economics of acquiring users via owned or paid channels are sustainable, so you know your long-term value-to-customer-acquisition-cost ratios and your payback periods. That’s the first necessary condition. The second is scale — significant scale and size, where you feel good about volume and growth rate.

Amazon introduced sponsored advertising for sellers 10 years after launching, and eBay introduced promoted listings 20 years after launching. These large-scale marketplaces think of advertising as a service that benefits sellers — a business expansion lever, a monetization engine — but never at the expense of the core business. Unless you have a healthy ecosystem of demand and supply being orchestrated at scale, you can’t introduce advertising. It’s at least a decade after inception — that’s what we’ve seen historically, and that’s a responsible way to do it.

As product discovery evolves, what do you anticipate when it increasingly starts from sources like ChatGPT, Perplexity, or AI agents rather than in the marketplace itself?

There are a couple of implications. It’s very hard for consumer businesses to battle intent and customer journeys from emerging media such as ChatGPT and others. If consumer behavior has started shifting toward a new source for discovering, say, clothing for their upcoming trip to Venice, you can’t really change that.

The implication is ensuring you show up well in those experiences. If you’re a marketplace — whether labor or commerce or travel — how does information on your site, app, or feeds need to evolve so you’re more discoverable in these moments, for users looking for a recipe or an outfit for their next vacation or party? SEO is what companies did for search, so how do we evolve for agentic to make platforms more discoverable?

Google announced a new protocol that encourages commerce vendors to send more attributes for greater discoverability. So the question becomes: if consumers have embraced this new modality for discovery, how do we make sure we show up well? Also, if consumers get accustomed to conversational interactions just as they did with using the search bar after Google introduced Search in the early 2000s, then how should these modalities be introduced within apps and websites, and how do they coexist with traditional modalities like search and browse?

How do we do it in a way that accommodates emerging behaviors, or at least cohorts of users that prefer this emerging modality, without throwing out the baby with the bathwater? How do we introduce this gently in a way that accommodates the early adopters, but also the more traditional users of apps? Google is testing that and trying to figure it out.

And eventually, as newer behaviors become more the default, we’ll need to ensure our experiences also include this modality as an option, or eventually the default.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In this conversation, James talks about how his unlikely path from psychology coursework and Wall Street trading floors to ad tech shapes the way he thinks about risk, value, and machine learning at scale. He walks through the three-way tension every marketplace has to resolve between suppliers, the business, and buyers, how his team organizes itself around that tension, and what separates an ML model that scores well in a lab from one that survives real production traffic. He also gets into how his team’s architecture mirrors Conway’s law, and how he expects product judgment to change as AI compresses the cost of generating new ideas.

From Wall Street to ad tech: An unorthodox path into product management

You took an unusual path into product management. How has it shaped your approach?

My career trajectory, both into product management and machine learning product management, is definitely a little bit unorthodox. But I think it’s helped a lot and brought a lot of creativity and decision-making, as well as frameworks, to what I do.

It started when I was in college — I was a psychology major at Penn. There were many clinical classes, but I was always mostly interested in the behavioral economics and game theory components of it, which is where I did a lot of my coursework. My first job out of college was on Wall Street in the sales and trading division of a large, global investment bank. While that’s very different from what I do today, a lot of things have carried over.

That was my first exposure into what a marketplace is, because in sales and trading, you are a marketplace. Back when I started, it was when things like high-frequency trading were starting to pick up a lot of steam. So I was able to learn a lot about how a marketplace works, especially as tech gets more involved. But I knew that the trading culture wasn’t for me, and I had this kind of entrepreneurial bug. I wanted to be closer to technology. I wanted to build stuff.

I ended up finding this small company called Kargo back in 2016. I joined their finance team and learned a lot about how the ad tech industry worked. When I joined, we were building this new marketplace technology called an SSP, a supply-side platform, and I was able to transition from the finance team into the product management team. That’s where I’ve been ever since. Over the last decade or so, I’ve been focused on building marketplaces at scale, specifically with a machine learning backbone.

A lot of what we do today, when we connect buyers and sellers, is driven by some sort of machine learning optimization or algorithm. There’s always a financial analysis involved in how you scale a marketplace — we’re very P&L driven — but the fundamentals of a marketplace always stand true regardless of what industry or sector you’re in.

Did that finance background teach you when to hedge your bets, or when to cut your losses?

It taught me to be a bit less conservative when it comes to failure, and to embrace failure. My background in finance is helpful, because at this scale — in just assets under management, we’re talking about billions — a little mistake can cost a lot. And in ad tech, the scale of the data is actually in that same magnitude, when we’re talking about tens of billions of ad requests happening at any given day.

But the difference is that I was able to see that that scale should encourage your experimentation philosophy, because there is capability to have so much data that even the smallest test can have a really big impact. It can be quite frightening at times to see how big things get. That was a learning lesson that took some time: If there’s a lot to work with, figure out how to work within those guardrails.

Defining value in a two-sided marketplace

How do you define customer value when each side of a two-sided marketplace wants something different?

It’s constantly evolving. If you go back to the marketplace model, as far back as ancient times, the game theory behind it is always the same. In order to be a successful marketplace, you have to accomplish three things: you have to provide revenue to your suppliers, you have to provide profit to yourself, and you have to provide value to your buyers.

What’s really interesting is that profit and revenue to your suppliers are quantitative financial metrics that you can calculate very quickly. Value, on the other hand, is constantly changing, and it is up to you — via research, experimentation, and the occasional bet — to really decide and define what that value is. The equilibrium of achieving those three things, which are technically always pulling at each other, starts with defining what that value is. If you do that first and drive value to your customer, then the rest will follow.

Is defining that value where teams struggle most?

Absolutely. It changes, and you end up in a lot of places where the customer is not really going to tell you what that value is. In our industry, for example, even though we might sell to a customer, we’re not actually selling to the customer directly. Our machine learning model is interacting with their machine learning model. So even though you’ve done what the customer has asked, or established the marketplace fundamentals to drive those three things, you might end up in a world where the models don’t agree with each other on their own optimization goals.

What ends up happening is that you have to react quickly, but you also have to understand that there is technical complexity to how you set up what you’re offering. The contextual understanding of the business model, and what that buyer is trying to achieve via your marketplace, should always come at the forefront of whatever decision you’re making.

Organizing teams around a marketplace’s north star

How do you translate marketplace complexity into product decisions the whole team can align around?

That’s kind of the secret sauce of the job sometimes. Something that pushes too much in one direction can make it very difficult for the team that represents the other side of the marketplace. So first and foremost, establish a north star goal: to always try to hit that marketplace equilibrium.

There’s going to be some push and pull, but as long as you’re driving that value, the rest will follow. There’s a cultural component to it — this is the direction we’re going in order to drive value and scale our marketplace. But it ultimately comes down to how the team is organized and how you manage communication within that team.

For example , my team is called the Outcomes and Auction Pod. We’re a group of product managers, data scientists, machine learning engineers, and software engineers, and our tentacles spread and interact with a lot of data engineers and analytics engineers. We’re super close to product marketing as well, and sales, of course.

We accomplish building the system and the proper infrastructure, while also having business context at scale, by verticalizing every stakeholder into various squads. We organize around who each squad is ultimately serving — some squads are aligned to a specific customer relationship, and we also have a squad focused on our own internal marketplace needs. It’s about making sure everyone has a north star in terms of who they’re trying to serve, and then there’s an orchestration layer to make sure we’re all on the same page.

One of the things I talk about a lot is the concept of failing. Every two weeks our team meets, we do a retro — what went well, what didn’t, where are the blockers. But we recently decided to add “Where did you fail this week?” Because we’re really of the belief that if you’re not failing, you’re not trying something new. Why did you fail? Was it a disconnect with the north star fundamentals, something technical, or contextual information that was missing? Even though we’re organized in a squad format, the knowledge sharing has to stay collaborative, because the best ideas can come from different types of people.

What happens when different squads have conflicting priorities?

I know this sounds weird, but those are some of my favorite problems to solve, because it means we’re doing something right. Imagine there’s a yield issue that could impact a customer outcome, but that yield happens to be with a very important supplier, or vice versa: you might have a customer that’s super important who needs a certain access to supply.

Those problems are fine — that’s when you get into the heart of what we’re trying to do. The solution is usually machine learning-driven to some degree because we need a prediction and optimization layer that can help balance the tugging forces between those two squads. But everything starts with the conversation of: What problem are we trying to solve? As long as we’re focused on the problem space, building that ML algo is going to be much more successful than if we hadn’t.

From clicks to real-world outcomes

How do you build products that connect digital marketplace dynamics to offline behavior?

This is something that anyone who’s ever tried to run their own advertising campaign has felt — am I getting the return on my ad spend and driving those sales? You have to consider your goal: do you want foot traffic, or do you just want to make people more aware of your brand? There’s the measurement component, which is a really important factor in ad tech in general: make sure you are measuring the true impact of your ad strategy.

The second factor comes down to how we position ourselves as a marketplace: the differentiation of supply and the differentiation of data. Value is constantly changing for the advertiser, and so is their user journey. Just a few years ago, the way we researched what product would best fit our needs was very different. ChatGPT, or any other AI chat service, has really compressed that user journey. Kargo is one of the first ad tech companies to partner with ChatGPT to get access to its supply for advertising services. It’s really about understanding your measurement capabilities, and whether you’re meeting the customer at the correct point of the user journey with the appropriate messaging.

The next component is the differentiated data. Commoditization is a risk a marketplace always faces. Machine learning capabilities are a layer that add value to the customer, but much of that information is open source, so the true differentiation is the data you have in your training set. Having access to that data enables you to reach the user at the right point in the journey with the right messaging. This is what ultimately leads to things like brand lift, foot traffic, and eventually sales.

Building ML that survives production scale

What separates a machine learning model that works well in experimentation from one that survives at production scale?

Assuming the technical implementation is the same in both scenarios, I think there are two answers. The first is solid ML engineering and infrastructure — you can run your machine learning model in a training environment and get specific scores that show you how strong it is. But if the infrastructure isn’t there to serve it at a very high scale — one of our models is actually called 500,000 times a second — a training environment doesn’t usually go to that level of scale. So that’s the first thing: solid ML infrastructure engineering.

There’s also the contextual business understanding of the problem that model is trying to solve. One of the things that’s beneficial for ad tech is that we’ve been using machine learning for the last 15–20 years. The more specific the problem you’re trying to solve, the better the output of that model will be. If you try to use ML as a magic bullet without the contextual understanding of how market forces react when you introduce these changes, or without the specific value proposition you’re trying to solve for, that becomes the difference between getting good scores in training and deploying a model in production that doesn’t do what you expect.

Where do ML-driven marketplace products usually break first?

That’s part of the investigation. There will always be technical issues, maybe a pipeline issue, maybe a scale issue. But the most common reason for something not going right is that you’re trying something new, and you can’t define the value upfront, because that’s the point of the experiment.

The contextual understanding is usually where we start: we deployed this model in production, we’re not seeing the results we expected — why? We start following the flow, and if everything is technically set up correctly, then it’s a market force we can’t predict that has to be studied.

This ties back to what I said earlier: a two-sided marketplace has three parties, and all three are tugging at each other to get what they want. If the equilibrium isn’t met, we go back to those three pillars: was revenue lost, was profit lost, or was value not delivered to the customer? That’s usually where we start our investigation.

How do you design a system to change as fast as customer value does?

From a technical perspective, it always comes down to trying to avoid monoliths and breaking things down into microservices. The more specific you are with a problem you’re trying to solve, the more modular your system becomes, which means the more adaptable you become to reacting to those market forces without building up tech debt — which is, of course, inevitable in every system.

It comes back to solid infrastructure and systems engineering that allows you to be malleable in the future. But it also comes down to how you communicate as an org and get everyone on the same page. As long as you’re aligned in the mission, the people making those changes can become more adaptable too.

When something stops working, most people ask, “What do we do? How do we fix it?” If you take a step back and ask instead, “How has the value changed to the advertiser?” — that already gives you a framework for how to solve the problem.

Product judgment for an AI-accelerated era

How does organizational design show up in marketplace product architecture?

As Conway’s law suggests, organizations and systems are built following the same communication model that’s used within the company. This is something that goes back to the 1960s, and the genius of that statement cannot be overstated. It’s even more powerful today than ever before.

The way we organize ourselves, and the way our products fall under that umbrella, is that we have the supply team, the demand team, and the internal marketplace team, and underneath that a horizontal team that serves those internal customers.

Whiteboarding is one of my favorite activities to do, both with new hires and people who’ve been at the company a long time, because even if everyone is a subject-matter expert, as you start drawing things out and you start to uncover logic in a very complex system that you didn’t realize was connected to something else. We’re the marketplace, we’re connecting buyers and sellers. Through a whiteboarding exercise, we may realize we have a piece of code on the supply side that’s actually impacting a decision happening on the demand side. That’s normal as marketplaces scale.

How you pitch the value to your buyer is how you define it internally, and that’s how your org is going to be structured. Your tech follows that same shape, and your logic starts to work under those same rules. It’s almost like a game of Operation at times, because while you’re trying to maintain your current line of business, you’re also moving as fast as you can just to stay still. It’s that kind of Red Queen hypothesis thing.

How should PMs evaluate ideas now that AI makes it inexpensive to produce more of them?

One of the things that’s really exciting about the advent of chat-based tech, or LLMs in general, is that it has lowered the barrier for accessibility to code contributions and understanding. As a research-driven product manager, what that does is open the floodgates to new ideas. People who were otherwise unable to understand the code or how our systems worked can now ask a chatbot questions and get those answers — which means ideas can come from other places.

I’m a big believer that in any organization you’re a person first — you’re not an engineer first, or a product manager, or a data scientist, or a seller. The best ideas can come from outside sources. This is an amazing time for a PM, because now we have even more access to new ideas at our fingertips. But with every great power, you have the responsibility of filtering those ideas, and that’s where it becomes very difficult. The fallback has to become: How does it fit within that framework of delivering value? How specific is the problem you’re trying to solve?

There are two things that matter most. The first is the infrastructure it’s built on. In our industry, our auctions run within a second, and we’re talking about events that happen within milliseconds. LLMs cannot operate at that speed just yet, but LLMs can be orchestrators — they can run their own experiments and help us come up with ideas, but only if your infrastructure is in a strong place. So having new ideas means you have to invest much more in infrastructure to capitalize on them.

The second thing is that it becomes even more important to define the level of effort, the business criticality, and the risk of not doing something, and to prioritize initiatives based on those definitions — always keeping in mind what value it brings to the marketplace.

How will things change once LLMs can operate at latency speeds fast enough for real-time auctions?

When they do, I think there’s going to be more emphasis on things getting better — not just bigger. Quality will become even more important. Right now, it’s much easier to write documents or create new videos, but it’s not the quantity of the documents or the video — it’s their quality that’s most important. We’ll see how things change from an organizational perspective, or what it means for product managers to focus on better, not bigger. You can only do that if the fundamentals of your understanding of the problems within the marketplace stay true.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In our conversation, Nilesh talks about how developing product sense is not only about technical expertise but also requires skills like judgment, systems thinking, AI adoptability, and more. He discusses why AI is raising the bar for PMs rather than replacing them, as well as how organizations can adopt AI responsibly. Nilesh also shares his perspective on balancing speed with governance.

Bringing technical thinking to product management

You’ve spent most of your career in deeply technical roles in product management. How do PMs with engineering routes tend to approach the role differently?

The real advantage isn’t what most people assume. It’s not that technical PMs are better at building — they’re better at understanding that a solution will take three sprints, not three days. That’s the asymmetry that matters.

When you’ve written enough code — as I did early in my career at Microsoft — you develop an intuition for how systems age. An API contract written for the feature in front of you rather than the feature you’ll need 18 months from now doesn’t look like a problem on day one. It becomes a problem later, when another team is trying to move fast, and the architecture is working against them. Without that lived experience, you understand the risk intellectually, but you don’t feel it. Product decisions get made at the level you feel, not at the level you understand.

I watched this happen at AWS. I saw a connection management architecture that was sound at 20,000 nodes hit a ceiling as the fleet scaled — all because API contracts had been designed for the problem in front of the team rather than the one three years out. By the time it surfaced, it was baked into a system that tens of thousands of businesses depended on. The technical fix was possible, but the cost was measured in engineer-years, not sprint points. A PM asking what this would look like at 10x might have changed the design decision. Nobody did.

At Unicity, we manage backend systems across more than 50 markets, each with different regulatory requirements and currency environments. Design decisions made today compound across every team that touches the system.

A technical background doesn’t make you a better builder in this role. It makes you much more expensive when someone brings you a feature idea that hasn’t been thought through past the demo. Since you understand how much effort it takes to implement a feature, you can push back much earlier. You understand the backend, the technical implications, and can explain why something isn’t feasible within the expected timeline.

With the AI tools your teams are using today, how has the PM playbook changed over the last year, the last few months, and the last few weeks?

It’s not just the playbook that has changed, but the expectation baseline. We used to evaluate product managers on their ability to write BRDs and PRDs, launch products, understand product-market fit, and build go-to-market strategies. Today, AI can produce a solid BRD in 45 minutes. That doesn’t mean you’ve saved a day and a half — it means the baseline for “ready to share” has moved.

Stakeholders have calibrated to that, whether they realize it or not. They think, “AI can write your documentation. What are you busy with?” The compression is real, but it’s unevenly distributed. Discovery has also become much faster. AI is excellent at synthesizing research, generating first-cut frameworks, and helping structure a problem space.

With that said, stakeholder alignment hasn’t gotten faster. The conversations where you’re figuring out whether legal approves, finance supports it, or engineering agrees on the trade-offs still take just as long. If anything, faster documentation has exposed how much alignment was happening during the process of writing the document together rather than in the document itself.

Today, I think PMs need two things. First, they need curiosity. Ask questions, even if they feel naïve. AI isn’t judging you. You can ask as many questions as you want. And second, they need to implement. We used to say ideas are cheap and implementation is hard. Now you can take an idea, use AI to prototype it, bring it to customers, demo it, gather feedback, and iterate much faster than before. Your ideas no longer have to die in a boardroom.

We recently migrated hundreds of endpoints from a legacy framework to a modern architecture. The work required translating enormous amounts of existing behavior into structured, testable contracts. That translation was necessary, but not intellectually demanding, and AI handled it well. We completed the migration in two months. Without AI, it would have taken years.

Building AI into product organizations

What are you most excited about with the way that AI is transforming the PM role, and what are your biggest concerns?

What excites me most is the collapse in the cost of being wrong early. I’ve spent years working in domains like order management, healthcare, and backend infrastructure, where validating a hypothesis required a meaningful engineering investment before you had anything to show a customer. We’d spend two weeks on a spike before discovering we’d misunderstood the problem. That cost shaped everything: which hypotheses we tested, how many we could test in parallel, and how reluctant teams were to abandon something after investing so much to build it.

What concerns me is a specific failure mode that I’m already seeing. A PM generates a spec with AI. An engineer implements it with an AI assistant. A reviewer scans it, it looks polished, the structure is right, and the edge cases appear covered, so we ship it. Every individual acted reasonably, but the system failed.

An underexamined decision travels much farther and much faster because the friction of manual production, which used to create natural review checkpoints, is gone. In complex backend systems, by the time you catch the mistake, it’s already in production, and the blast radius is real. The answer isn’t to slow everything down. It means creating explicit review gates instead of relying on “this looks right” as a proxy for “this is right.”

At Unicity, we’ve embedded automated compliance checks into our CI/CD pipeline. We operate in more than 50 countries with different currencies, GDPR requirements, PCI compliance, and other regulations. Before a human reviewer even sees the output, automation validates those requirements. The automation doesn’t replace human judgment. It creates a workflow where human judgment can actually be trusted. When everyone is moving ten times faster than before, “this looks right to me” is no longer a reliable standard.

How are developers responding as PMs gain more direct access to production? What’s the path to making that relationship work?

We’ve definitely had pushback from developers. At Unicity, we held a hackathon around this idea. The message wasn’t, “PMs can write code now.” It was, “We want to help the engineering team increase its velocity.” I used an analogy from healthcare. In the US, there’s roughly one provider for every 857 patients. Nurses have taken on responsibilities that used to belong exclusively to doctors because physicians should be spending their time on the work that requires their expertise. The same principle applies here.

We want developers focused on architecture and the most complex engineering problems. If PMs can safely handle configuration changes or other low-risk tasks with AI, developers have more time for the work only they can do. We’re not trying to replace engineering — we’re trying to become better partners.

One concern developers raised was legitimate. They said, “If you start creating pull requests with hundreds of lines of AI-generated code, why should we be responsible for reviewing code that you didn’t actually write?” That’s fair, but my view is that this has to happen gradually. Start with configuration changes. Build trust. Show that you can safely make small improvements. Then maybe you move to a single line of code, then small functions, and gradually expand responsibility as confidence grows.

The goal isn’t for PMs to own engineering work. It’s to ask, “How can I help?” If I can take low-level work off an engineer’s plate, they can spend more time acting as architects — deciding how AI should be implemented, what standards we should adopt, what security requirements matter, and how the system should evolve. That’s where developers create the most value.

You come from a background in healthcare payments, cloud infrastructure, domains where cutting corners is not an option. How do you think about that tension of moving fast with AI and maintaining the standards that those environments demand?

I would actually push back on the framing that this is a tension between speed and quality. The real tension is between knowing your risk surface and acting as if you do when you don’t. At Unicity, we tier our backend systems by consequence rather than technical complexity, and that distinction matters:

• Tier 1 — Where an error creates an embarrassing result, such as a display issue or a minor data inconsistency. We move fast, and AI tools operate freely

• Tier 2 — Where an error is expensive, such as a failed payment, a duplicate transaction, or an issue that requires someone to wake up at 2 a.m. for manual intervention. Those systems move with structured reviews and specific guardrails

• Tier 3 — Where an error affects customer access, creates regulatory exposure, or introduces legal risk. Every change receives deliberate human review before it reaches production

We operate in 56 countries, so systems move into Tier 3 very quickly. An architectural pattern that seems harmless in one market can become a regulatory issue in another. We experimented with AI-assisted pull request reviews, but we found they weren’t sufficient. Every pull request is still reviewed by at least two engineers before it’s released.

The important thing is that the risk model reflects business and regulatory consequences, not just technical complexity. That’s how we balance using AI while maintaining the standards our environments demand.

Helping teams become AI-native

Not every PM adopts AI at the same pace. What’s your strategy for closing the gap?

The approach that consistently doesn’t work is leading with the tool. If you tell someone, “Try this AI tool, it’ll save you hours,” the response is usually, “I don’t have hours to learn something new.” That’s not resistance; it’s an accurate prioritization call. What works instead is making the output visible first.

I’ll have someone who’s already comfortable with AI produce a first draft of a specification or analysis, then hand it to someone who hasn’t adopted AI yet with one simple ask, like, “Find what’s wrong with this.” This is an invitation to do something they excel at, and, in doing so, they engage directly with both the AI product and where it falls short. The moment someone catches a meaningful mistake in AI-generated work, they stop thinking about being replaced. They become the editor of a very fast first draft, and that’s an identity that fits naturally.

At Unicity, we’ve also created a secure AI sandbox for product managers. They have access to approved enterprise AI tools, demo data, and a controlled environment where they can build prototypes without depending on engineering or production systems. The goal is to let PMs validate ideas with customers before asking engineering to invest in a full implementation. Once we’ve demonstrated product-market fit, the engineering team takes over and builds the production version.

That has dramatically increased adoption. One of our PMs came from a design background and had never written production software. Using the sandbox, he’s built multiple applications that customers are actively using and talking about. Seeing that success has encouraged other PMs to experiment as well. We also run an AI Guild where PMs, TPMs, and engineers share what they’ve built, whether it’s a customer-facing product or an internal workflow improvement. We even reward impactful AI projects because we want experimentation to become part of the culture.

At the end of the day, I come back to the same two principles: be curious, and act. Ideas are valuable only when you put them into practice.

There’s a version of the future where the PM becomes a generalist who does everything (discovery, design, code), and a version where specialization wins. Where do you foresee the profession going?

My prediction — and I’m willing to be wrong about this — is that within three or four years, a full-stack PM who can run discovery through deployment will become the baseline. The interesting question won’t be whether you’re a generalist or a specialist, but what you know that nobody can replicate simply by prompting AI.

The backend systems we manage, like order management, payments, distributor compensation, authentication, and regulatory workflows, take years to understand. Good documentation helps you understand how a system works, but it doesn’t teach you how to make judgment calls. That comes from watching things break, analyzing why they broke and understanding the downstream consequences two quarters later. AI can’t tell you why an architectural decision was made, what alternatives were considered, or which pieces of tech debt are actually carrying the system. Those things live in experience.

I’ve seen both sides of this. When you’re the person who made the original architectural decision, you understand the constraints that shaped it and the alternatives that were rejected. Someone equally intelligent but new to the domain simply doesn’t have that context. Not because they’re less capable, but because the relevant context doesn’t live in documentation. It lives in institutional memory.

When those people leave, the context leaves with them. The PM who captures that institutional memory is creating something that compounds over time. I think the combination that becomes genuinely valuable is broad operational capability as a PM, including discovery, design, code, and data, combined with deep expertise in a specific domain. Generalist skills become the entry requirement, and domain depth becomes what you compete on.

If you’re early in your career, build that broad range now, but also choose a domain and go deep. By the time you realize that expertise matters, you’ll already be behind the people who started earlier.

What advice would you give a PM to stay current in the AI era?

There are three things I’d focus on. The first is organizational memory. Why did we make this architectural decision? What did we try first? What happened? Which stakeholder objections are real blockers, and which are signals that something else is going on? AI doesn’t have access to that context unless someone builds a system to preserve it, and most organizations haven’t.

In cloud infrastructure, I’ve seen decisions made in one year continue shaping roadmaps three years later. By then, the people who made those decisions had moved on, and the reasoning had left with them. Engineers were forced to reconstruct that history from code that couldn’t explain itself. The PM who preserves that context becomes incredibly valuable.

The second is understanding the political economy of the roadmap. Every roadmap has a visible version and an actual version. The visible version is what the roadmap says. The actual version is who agreed to it, under what conditions, what trade-offs were made, and what happens to stakeholder relationships if priorities change. AI doesn’t have relationships with your VP or your CTO. It doesn’t know that the same concern means something very different depending on who’s raising it. Understanding those people and those relationships takes years, and better tools don’t compress that experience.

The third, and probably the hardest, is developing product judgment. It’s the ability to look at a specification, a prototype, or a feature proposal and know that something is wrong before you can explain why. That intuition comes from years of watching products succeed and fail, developing a feel for the gap between what looks good and what actually works. That’s the hardest skill to build, and the hardest one to fake.

PMs who invest in that judgment now, while everyone else is trying to outsource it to AI, will have something genuinely scarce five years from now.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In this conversation, Mike talks about building an AI-native product inside a 75-year-old brick-and-mortar retailer and why he doesn’t think AI is married to any one business problem. He walks through the graph database that lets Pearl Planner understand a bride’s personal, interrelated preferences, his guiding principle for when to surface AI and when to let it work invisibly in the background, and the 20-plus-agent architecture running behind the scenes. He also shares the story of a critical API dependency that got deprecated days before launch and what the team did about it.

Betting on AI at a legacy company

Why make such a significant bet on AI, and what business problem were you actually trying to solve?

I think as a technology, AI is not married to one business problem or one customer problem. AI is an opportunity. Having been close to engineering and close to design, but never been the person who’s as good or able to actually do that work myself, AI is the thing that really takes down the barriers so you don’t have blockers.

So if you’re a resource constraint, say you have a tight budget, you have something you’ve committed to for the year, you can’t pivot as quickly as maybe a new startup would with funding sitting in the bank and three people, you can use AI to give the org, the team, or the company, more agility in terms of what can be turned around and who could do what.

The other piece of it comes from a UX perspective. Having built software for years, I couldn’t tell you how many times we came up with an ideal experience and then heard “That’s too complicated or too expensive to build,” or, “We’re not going to be able to do that right now. It’s not possible.” AI shortcuts most of that in a lot of ways. The standard in the industry for a long time has been that if you want information, you put a form in front of the user and try to get them to fill it out.

When we talk to brides, they tell us that one of the hardest things about planning the wedding is getting other people to understand their vision because it’s hard to put into words. And so a really good opportunity we saw early on was like, “What if we didn’t make them tell us? What if they just showed us?” So instead of filling out a form or going through multiple steps, the bride literally just shows us pictures of things they like, and then we leverage our expertise as a business that’s been in the wedding industry for 75 years to identify the things that actually matter to the decision-making process. We make a bunch of decisions on the backend that brides don’t have to worry about, regarding what we surface or how we personalize the experience. Little things like that just make it much more streamlined to what they need, especially in a space where they’ve been underserved.

Are there changes in the market that influenced that decision, like the growth of digital wedding sites like The Knot?

When you have a chance to start fresh, you don’t have legacy tech debt or a platform that you’re already invested in. What we saw early on was that nobody had actually done planning from day one through the day of the event. It’s usually just a checklist with the big milestones, but most people can think of those milestones off the top of their head, so that’s not very helpful.

It’s also skewed toward where those companies monetize, and brides see that instantly. When our platform launched, brides were like, “Hey, why are there so many dresses in here? That feels too pushy. It feels like you’re just trying to sell dresses.” It was only like 5 tasks out of 300, but they noticed it right away. So we took a different approach. When you start fresh, you don’t have existing infrastructure, legacy revenue, and restrictions, so you can just focus on the human, which is nice.

Building a data foundation for something deeply personal

In adding new technology to an existing business with legacy systems, were there any gaps that you found in data that you had to try to address?

I’ve never worked with a company that hasn’t had data gaps as an ongoing thing. There’s always improvements to make. But what I brought was the experience in the new tech that says, “Based on where things are going, the standard database is not how we want to solve these very personal interrelated things for storing and recalling and doing all this.” So we went with graph storage so that we could emphasize the relationship between — “My wedding color is red. I don’t know which shade of red it is, but this one is too dark, this one is too light, and my shade is somewhere in the middle.” Or, “I like this color because it suits my theme and I have a personal connection to it.” All those things make that experience much better when they talk to the AI or when we surface a recommendation or even when we default to certain settings to just shortcut a process so that a bride doesn’t have to make as many decisions.

We had to work out things like whether it’s the same user across touchpoints, or whether a wedding date means the same thing in every context — that’s the semantic layer we built out. Luckily, David’s has invested over the years in a strong data foundation. We have Snowflake, so all of our data ends up in one big place that we can pull forward. We have our own middle layer that actually transacts data from one point to another so that if we change our platforms over time, we still have the relationships and ways to pass things back and forth. That was one thing I was really impressed by from the tech side of David’s. Maybe it could have had better documentation, maybe it could have been updated a bit more, but overall the architecture was pretty solid on the data side, which I was happy with.

Deciding where AI shows up — and where it hides

As you rebuilt parts of that digital experience, were there certain principles that guided you in terms of where to bring in AI automation and where that human touch is still needed?

As we build the platform, the principle has been: if the user can do it, the AI can do it. So we have a budget feature: the user can enter an expense, or they can just upload an image, and the platform will scan it and put the expense in for them. Similarly, they can just type a chat to Pearl like, “Hey, I spent $150 on supplies for my centerpieces,” and she’ll add that expense to their budget. They don’t have to click to a tab, open the form, and manually enter the expense.

But over time, as they start pushing Pearl — “Actually I don’t like that you grouped these tasks in this milestone. Can you move four of them to a new one that’s called something else and push it back a month?” — Pearl can just do that. They might not be using Pearl that way yet because we have to do some education, but we’re building a future state where more people are assuming it can do that.

The other piece is that we don’t always have to tell the user we’re leveraging AI. If we have a color feature and we know the bride has picked out certain colors, we build on that without labeling it. When we generate palettes like, “Hey, how could this look with different palettes and different themes?” we don’t communicate that it’s an AI feature. We’re just like, “Hey, let’s see what it looks like with different combinations.” And then when the bride picks one, we preview what their vision board can look like with those colors applied. We don’t say that’s an AI feature, it’s just part of the experience.

We’re working on a new version of the vision board right now, and we want it to feel like magic. The only case where I tend to say, “Let’s explain a little bit,” is when the AI is doing a lot more behind the scenes. So if it’s like, “I checked your wedding day, I looked at information from your venue, I checked other comparisons, I read reviews, and here’s my recommendation.” I think it builds trust. There are certain scenarios where you should surface that you’re using AI, because otherwise it looks like you pulled something out of thin air and that could erode user trust.

For many things, it’s not necessary to communicate that we’re using AI, but if we’re sharing critical information or advice that’s needed to make a decision, then let’s at least verify we did the right things to get the information.

Taking anxiety out of wedding planning

You mentioned that one of those core challenges is the bride’s dress. Are there other examples of things you’ve done to reduce that anxiety level?

The budget tool is a good example. The perception is that a lot of brides already know everything about their wedding and have been planning it since they were little, but that’s really not the market. A lot of people we talk to are like, “Yeah, I figured I would get married, and every now and then think about it, but I don’t have a vision board. I don’t know anything about this.”

We take a more opinionated approach. The app says, “I already know my budget,” or, “I need help.” And if you need help, OK, where are you getting married? How many people do you want to have there? Elina, our president, had this plan a long time ago, and I thought it was one of her best ideas: What if we just asked them what things were most important to them and we did the math? So instead of like, “How much do you want to allocate toward apparel?” it’s like, “Do you want to splurge, or do you want to save here?”

And if it doesn’t check out, if they are going to have 300 people and they want to splurge on everything and their budget is small, we’re going to say, “Hey, here are the gaps. Let’s brainstorm where we can save.” They can lean on Pearl to help with some of those decisions. We don’t make them math at all. We can do the math. We just give them the results, let them think in human terms of what’s most important, what’s less important, and what they need. That gives them a starting point, and they can go on from there.

When your API dependency disappears days before launch

You mentioned that a few days before launch, there was a critical API dependency that was becoming deprecated and you had to pivot. Could you say more about that?

The initial idea, in line with everything we’d talked about, was: maybe brides have already collected a set of images in a popular social media tool, and it would be better if, instead of making them look through more images, they just showed us what they had. So we built it out that way. The fundamental principle, the shortest path: just share a link, we’ll go find the pictures, and you don’t have to do anything else. That was the ideal. Then we found out right before launch that the API was going away. So our backup plan became the primary plan, and we had to curate more and build a different loop for that content.

The fundamental principle is still there. The entire pipeline — look at these images, identify traits from different categories, create a file, send it downstream to be embedded — was all still there and usable. But the user experience side of it was a bit of a bummer. Since we’re attached to the appointment booking piece too, it didn’t hit as hard as I thought it would. We still saw good throughput for everybody who got that far — I think we had a 60-70% completion rate. And some people really loved it — instead of five pictures, they picked 20 or 30. That’s great data for us to have.

Building lean with AI-native tools

Operationally, you’ve described a relatively lean team that’s moving much faster than the traditional product organizations. Since you started incorporating more AI tools into the workflow, are there certain buckets of work that have disappeared or that you’ve gained very large efficiencies with?

We started out playing with Replit to prototype and get some initial code. Then we thought, “You know what? No. Cursor’s out. We can actually keep our engineering workflow.” My team did try Copilot for a little bit; but I was not a fan, and they ended up not being fans either. Cursor’s 10 times better.

It’s crazy how much product management has changed in the last year. Agent skills roll out, and I share the insights. And this is across David’s, to all company leaders. That’s kind of how we’re handling rollout — give them access, tell them what’s possible, let them apply it to their specific area.

One of our engineers, Valerie, did a workshop with the rest of the engineering team and IT, showing that, for each repo that she works in, she has her own set of skills for the agent, different workflows and changes in her system prompts so it always checks the design system, or it always goes to Figma and pulls in the code. She’s a 10x engineer on her own — not because she’s working ridiculous hours, but because of the way she thinks about getting the work done and the way she thinks about building the systems to do the work.

More broadly, how do you think about architecting AI adoption across the organization?

At David’s, we’re leaning into Claude pretty heavily. I would say Claude is a harness, and it already has the agentic under the hood, so it’ll spin out sub-agents and things like that. So what I’m doing from an architecture standpoint is trying to find the leaders and the use cases and then build up a range of impact as they get more comfortable with it. Start with, what can you do with chat? What can you do with Cowork and MCPs integrations? And then how can you reshape that to be something you do consistently or offload with skills?

Generally speaking, most companies don’t need something complicated to maintain, like a custom agentic workflow. If you’re building that on the product in the backend, great. There’s a lot of benefits to that. Pearl Planner uses agentic on the backend, and there are 20-plus agents in there that handle everything, from updating budget, to updating colors, to gathering context from what’s on the screen at the time, to updating tasks, or researching different things from the web, or pulling content from our library for specific moments. We have our own orchestration layer with all those specialized agents within it.

Where do you see this going next?

We’re hoping someday it’ll branch out beyond weddings, and people can just use it to plan anything. I did not have wedding retail on my bingo card, but it’s a very unique strategy, unique opportunity, a lot of the dimensions that I’ve never worked in before, so I’ve had a lot of fun.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In our conversation, Josh explains why sustainable subscription growth depends on balancing acquisition with engagement, retention, and long-term customer value. He explains why identity, monetization, and discovery are the building blocks of products customers continue to value, and why the strongest subscription brands will extend beyond digital experiences to build lasting customer communities.

Building subscription growth through customer value

You’ve spent much of your career building consumer products across different industries. Thinking back on those experiences, what have you found to be the biggest misconceptions product teams have about sustainable subscription growth?

A big misconception is that sustainable growth primarily comes from acquiring new customers, and to be honest, I’ve been guilty of that myself. Subscriber growth is obviously visible and incredibly important, so it’s easy for teams to focus heavily on that.

Over time, I’ve realized that sustainable growth comes from consistently delivering enough value to keep your customers coming back. Your existing customers are incredibly valuable, not just because they’re already paying, but because they validate your product and often become your strongest advocates. They might go out through word of mouth, social media, or elsewhere and explain why someone else should become a customer.

It’s important to balance acquisition with engagement, retention, and habit formation. Otherwise, it’s like a leaky bucket — you’re bringing people into something that’s not sustainable. The goal shouldn’t just be to grow subscriptions, but to build an experience that people continue to value and ultimately want to recommend to others.

When you’re thinking about the whole customer lifecycle — acquisition, onboarding, engagement, retention, and monetization — are there certain things a team can’t overinvest in?

Yes. Some things are so critical that you don’t want to underinvest in — mainly identity and monetization. Certain parts of a product feel more visible, such as the video experience, content merchandising, or, in ecommerce, product detail pages and recommendations. Those things matter, but they’re only effective if you understand the customer: who they are and how they move through your business.

You need to know where they sign up, how they pay, whether they use promotions, which platforms they use, when they engage, when they leave, and how you can win them back.

One team that needs a seat at the table in virtually every discussion is the monetization and identity team. Without that foundation, you can’t personalize experiences, build a lifecycle marketing strategy, or make smart product decisions. It may not have the sizzle, but it’s definitely the steak of the operation. It’s the critical piece that allows everything else to run effectively.

Are there areas that consistently provide a higher return on investment, or does it depend on the business?

I think it depends. In a pure ecommerce business where you’re buying products, some parts of the experience might be more critical than others; similarly, in entertainment and streaming. One piece that’s equally critical in both is discovery, especially for new customers.

In ecommerce, someone comes to your site, and maybe they don’t land on the right product page, or they don’t like what they see. You want to make sure they can discover something else. If they’re looking for something specific, your merchandising and search need to help them find it.

The same applies in streaming. You want customers to find what they’re looking for and get value quickly. Even though subscribers are already paying, you’re still trying to earn that subscriber beyond the current billing period. If they can’t find something they want, you’ve lost them. Discovery is equally critical in both businesses because it helps customers realize value immediately.

Turning engagement into long-term retention

Most streaming services have enormous content libraries. Is there a customer signal that indicates long-term engagement?

A couple of signals stand out to me. One is the number of different series someone watches. There will always be people who subscribe to binge-watch one show and then leave. The real opportunity is getting someone to discover something else they’ll enjoy, even if they weren’t looking for it.

That’s where recommendations and personalization become so important. If that second show is something they didn’t even know about, you’re creating additional value beyond what they originally came for. This does two things. It creates more value in the moment and, over time, builds trust in your recommendation system. If customers consistently enjoy what you recommend, they’re more likely to trust future recommendations. Ideally, that helps retain them longer.

Another important signal is engagement within a household. If multiple people are using the service, especially with different profiles, watchlists, and favorites, the subscription becomes more valuable. It’s less likely they’ll churn because more than one person has a stake in keeping the service.

The broader context of consumption — what people watch, when they watch, and where they watch — helps you personalize the experience and continue to help customers find value.

You’ve worked in both ecommerce and subscription streaming. Do customer expectations differ when people are buying products vs. entertainment?

At a high level, they’re similar. You want someone to arrive, find something relevant as quickly as possible, and take the next step, whether that’s making a purchase or starting a stream.

The difference is that ecommerce extends well beyond the digital experience. There’s packaging, shipping, delivery communication, returns, customer service, and product reviews. Those things determine whether someone buys and whether they come back. Shopping is also very emotional. The experience needs to create excitement during the purchase and reinforce it when the package arrives. It’s one cohesive experience online and offline.

With streaming, it’s between you and the service. The customer is already paying, so the challenge is making sure it’s easy to find something that fits their mood and interests and continues to feel worth the price. Ecommerce has to deliver both a great digital experience and a great physical one. Streaming has to keep earning the subscription every time someone opens the app.

Customers don’t differentiate between the website and the box that arrives at their house. They don’t think, “The site experience was great, so I’m happy,” if everything else disappoints them. They think about the entire brand experience from beginning to end. That’s something I’ve learned over time, and it’s why organizations need to consider every customer touchpoint.

AI, prioritization, and outcome-driven product management

AI has made it much faster to synthesize data, identify patterns, and generate insights. Has that changed the way your teams build or prioritize what to build?

At a high level, AI enables product teams to get to the starting line much faster. It can quickly synthesize customer feedback, identify patterns in product and behavioral data, test early hypotheses, and prototype ideas. Instead of debating an abstract concept, teams can sit in a room and look at something tangible. It’s imperfect, but it makes the discovery process more efficient and helps teams make stronger prioritization decisions with fewer cycles of analysis and discussion.

Ultimately, AI provides another layer of context. It’s not the decision-maker. It can surface useful insights, but it doesn’t understand the full context around the customer, the business, technical resources, or how the organization works. That’s why you still need human judgment to decide what matters and what to build.

Is there an example where you’ve seen AI change the customer experience rather than just improve internal efficiency?

Personally, I’ve been building with AI on the side, and one of the biggest opportunities I’ve found is how it can make personalization feel more useful and contextual. Instead of simply presenting recommendations, AI can explain why something is being recommended based on onboarding signals, how someone is using the product, or other context behind the scenes.

That additional layer helps people understand why something is relevant to them rather than just seeing “This is recommended for you.” It creates a more responsive, personalized experience.

We’re just scratching the surface. The more human context a product can provide, the better AI can make recommendations that feel relevant rather than generic.

You’ve talked about moving organizations from idea-driven roadmaps to outcome-driven execution. As AI shortens planning and delivery cycles, does that change how you prioritize?

I think it has to. As planning and delivery cycles shorten, teams receive customer signals and results faster. That makes it even more important to prioritize outcomes over features. That doesn’t mean I don’t believe in a long-term roadmap — you absolutely need one. But the roadmap needs to be more flexible than ever, and teams need to be willing to change course when the data shows there’s a different problem or another opportunity that matters more.

The starting point should always be the customer problem, not a solution. AI can help identify problems, test ideas, and accelerate learning, but product teams need to become more comfortable being uncomfortable. The discipline is staying anchored to the outcome while being flexible about the path.

Are there decisions that become harder as execution becomes easier?

One of the biggest things product managers need to become comfortable with is saying no. There will never be a shortage of stakeholders bringing ideas to the product team. One of the biggest muscles you have to develop is knowing when to say, “Let’s hold on this. We want to understand it better.”

As AI accelerates development, that becomes even more important. People will naturally ask, “If it’s so fast to build, what’s the harm? We can ship something and learn.” The ability to pause, be thoughtful, and know when not to build something — even when speed is there — will become increasingly valuable.

Building brands customers want to belong to

Looking ahead, what competitive advantage will help subscription businesses continue to grow?

I think the biggest advantage is still delivering an exceptional customer experience. A lot of that will happen within the digital product itself — creating an intuitive experience that’s personalized and consistently delivers value. But I also think there’s a big opportunity to go beyond the product and create real-world experiences.

You don’t want customers to feel connected only to a show or a product. You want them to feel emotionally connected to the brand and what it represents in their lives. Real-world experiences bring affinity groups together. They create shared memories, build stronger communities, and give customers another reason to feel invested in the brand.

Fanatics Fest is a great example. Fanatics could have remained a traditional ecommerce business, but instead it expanded into a much broader fan experience. It brings together athletes, collectors, and communities, elevating the brand beyond a point of purchase into an experience of its own.

The companies that win will be the ones that create an exceptional product experience and then find meaningful ways to build connection around it, both online and offline.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In this conversation, Renee talks about what it really takes to bring users along through major product transformation, from winning organizational buy-in before a redesign begins, to migrating tens of thousands of users off legacy platforms without losing their trust. She discusses how to decide which friction to remove and which to preserve, what she learned consolidating products after an organizational merger, and how AI is changing (and not changing) the fundamentals of good UX.

Aligning the organization around users

Bringing the organization along is often one of the hardest parts of transformation. How have you approached getting organizational alignment?

As technology advances, transformation is constant, particularly right now with AI. But I’ve also been through transformation caused by organizational changes, not just technology changes.

In terms of getting organizational alignment, taking time for discovery and input from users, in my experience, can be one of the most challenging things to convince internal stakeholders to accept. They generally agree in principle. It’s not that they ever say, “We don’t want to hear from our users.” They always say, “We want to hear from our users.” But then in practice, time schedules, wanting to ship faster, wanting to meet deadlines, revenue concerns, and that sort of thing will cause some people to bypass what we’re actually hearing from users.

I’ve found success by involving stakeholders from all different areas of the organization, because it doesn’t work if only product is on board with how we’re going to make this big change. Marketing also has to be on board. The engineering team has to be on board. You need to include stakeholders and make them aware of what users need in that transformation. Sometimes that means having them actually sit in on user interviews. And also share data, because hard evidence is difficult to ignore.

It’s challenging, though, because between the time schedules and the resource considerations, there are a lot of different factors going into how things need to happen. But it’s not going to be successful if you’re transforming into something that users aren’t going to want.

Discovery is a big piece of it, and I’ve found that’s where corners often get cut.

Migrating 80,000 users without losing them

You recently helped consolidate multiple legacy platforms into a single experience and migrate more than 80,000 users. What did that teach you about the difference between building a better product and getting people to adopt it?

Talking to users constantly throughout the discovery and the development process helps to identify and anticipate areas of potential friction. Even if the new product provides easier, faster ways for users to meet their goals, people have established habits in using digital products, especially things they have to use for their jobs.

We were working with professionals who were using our software to identify potential funders. Once they’ve established those habits, they’re not really keen to learn new methods. Even if the new methods make it easier, it’s like, “I go through these steps. That’s how I do my work.” So it’s really important to anticipate the friction points, eliminate them where possible, but also provide guidance where necessary.

We used tools like Appcues, a customer engagement tool that provides a variety of methods for customer guidance. For different features in the product, we would use different methodologies. Appcues would help us build multi-forked tours of the product. It could also provide really quick guidance around a particular feature, if we felt like that feature was valuable, but we need to give them a little bit of a handhold to get used to it. Like features that use AI — I had this experience with LinkedIn. They just deactivated their classic search and replaced it with an AI search that’s supposedly better, but all my microfilters are gone.

We experienced a lot of things that were similar to that. In addition to providing guidance within the product that’s right in front of the user, we created short videos around certain features: “Did you do this in the other product? Here’s how you do it in this product. Here’s how you achieve that thing.” They weren’t polished videos, they weren’t something that we would put on our website. But, when customers were reaching out to our customer support team, they could share the videos with them. It was a very quick and visual way for people to understand what the transformation was.

We had a variety of users, from the once-a-year user to the daily or weekly user. It was really those daily and weekly users we were most concerned about — a smaller group of the paid subscribers and the people who were consistent users of the product. When I did a revamp of another product, we offered customers a choice: Check out the new thing, but you can still go back to the old thing. At one point, we gave them the option to commit. By the time we reached the end of that transition period, we only had a small group of diehard stalwarts that we had to forcibly move over.

It’s a matter of really listening to the feedback — as people are moving over and telling you, “I can’t do this,” or “I’m having a problem here,” sometimes you have to go back and redevelop some of the pieces of it.

Deciding what friction to remove and what to preserve

During a major migration, some amount of friction can help users understand what’s changing and build confidence in the transition. How do you decide what friction to remove and what to preserve?

Friction can be caused by the removal or change of a feature that users deem essential. I find that the best way to identify those points and try to resolve them is by observing users in both the old product and the new product.

We would do a lot of sessions where we would give the users prompts and ask them to walk us through. They would be speaking aloud and talking us through. Then you would see immediately where they would get stuck and just freeze up and you would say, “Well, what are you thinking?” You don’t lead them to the next step; you just observe them using the product, because it could be that they’re just stopping to think. We had a data product, so it was a lot of information to process, but it could be that they got lost and the friction was inspectable.

Looking across the modernization initiatives you’ve led, what decisions had the biggest downstream impact on customer experience?

Staying on top of the changing levels of technological literacy of our customers. As technology advances, users have expectations and an understanding of products that function well and that give them confidence in the product. For example, e-commerce flows have evolved really significantly over time, and a poor e-commerce flow can be a dealstopper. It could work fine, but if it doesn’t feel polished and elegant and inspire confidence in the user, nobody wants to put their credit card into something that looks janky.

But the biggest decisions were those where user feedback and data were ignored. Those have the biggest downstream impact. I went through some of that — where UX recommendations got overridden or overruled.

Oftentimes it’s done, almost always, in the interest of time savings or rush to market. And it ends up being costly: I’ve experienced situations where parts of a product had to be rebuilt. That’s why having top leadership agree to take in all that user feedback is really critical. There are just times where everyone feels like they’re making the right decision, but forcing through expedient solutions that end up being problematic has a huge impact on customer experience.

Building feedback loops that keep transformation honest

You’ve led designers, researchers, engineers, and product teams. What operating models have you found most effective for turning customer insights into product decisions?

Face-to-face contact with users, observing how they use the product and understanding their relationship to technology — not just how they used your piece of technology, but technology in general.

In terms of development, there are a lot of different models. Over my past career, I’ve worked with everything from waterfall to agile to scaled agile to product operating model. From my perspective, a true agile development process is the best for continually integrating user feedback throughout the development cycle because there are multiple points where you can inject the user feedback into revisions of the product or additions to features.

With other product development models, it’s very hard. We followed scaled agile for a while, which is really closer to waterfall, and it was really hard because once you got the feedback, it would take months to get the report as opposed to weeks. Being able to have that cycle continually, that continuous development and user feedback going through, helps to make sure that’s taken into consideration.

Is post-launch feedback the same face-to-face approach, or does it look different?

Yes, but a lot of really critical initial feedback comes from the customer support team. If it’s a situation where people are reaching out to support, we need to tackle it and address it urgently.

You can’t just put something out in the world and then leave it alone. We continually observe user patterns using automated tools that do recordings of usage and show where their cursor is moving on the screen. When new features are released, I like to do spot mini surveys around a new feature. Just one or two questions, like: “Is this working for you? Did you have trouble with it?” We’d get a decent amount of feedback from NPS, which we’d use to gauge the overall tenor of customer satisfaction.

Merging two products into one

Following the merger of two established organizations, you helped create a unified product experience. How did you decide what to keep, what to retire, and what to reinvent?

We identified our critical customer personas and their needs. We looked at the usage data of features in the legacy products to determine which features were absolutely essential to replicate. We spoke to users about what could be improved from the old products.

You have to take all of that information in — What are they using? How are they using it? What are the problems that they have to solve? But, you still have to go at it as a completely new product that you’re building from the ground up, because it’s too easy to get mired in the way things were previously coded. Especially with technology moving as fast as it is, you need to be freed up to take all of that information, decide what you’re going to build, but don’t try to replicate. Build it in the most modern way possible.

We did make a list, kind of like a big chart: “Here are all the critical features from this product, here are all the critical features from this product, and here’s how you do it in the new product.” That was mainly for educating our own teams.

Did you end up compromising on some features?

We did, certainly. Trying to replicate some of what was there before slowed us down. I feel like some of the compromises didn’t work out. Anytime you build something new and the customers, as they’re getting used to it, have an opportunity to try it out, they’ll tell you, “I need to do this and I can’t do it.” Sometimes you have to say, “OK, we actually have to move fast and build a way for them to do this.”

There were many, many compromises. The compromises came from any number of corners of the organization — some from users, some from customer support, some from marketing, some from engineering.

What AI changes — and what leadership must get right

AI has made it dramatically easier for people outside of design to generate interfaces, content, and workflows. Are the core principles of UX changing?

I don’t think the core principles are changing. With AI, some people might choose to ignore them, but that can result in some sloppy stuff. But, AI makes it a lot easier to do rapid prototyping and put solutions in front of users faster and more frequently.

Working in nonprofit technology where resources are limited, prototyping was always a challenge before AI because oftentimes it required engineering resources. I’m a big fan of paper prototyping, and everybody makes it sound really easy, and it’s not. The fact that you can use AI and build a prototype that feels like people could get their hands on it — that’s going to really help, if people use it to actually get more user feedback.

Are there parts of the design process that need to remain with humans?

I think so. My designers loved using AI. But I don’t think it’s so different from using AI to write a first draft, using AI as almost like a scratch pad to play around with a lot of ideas quickly as opposed to building them yourself. A human needs to apply their aesthetic — AI doesn’t have an aesthetic, and that is subjective. There are some people who are very talented in that area and people who aren’t, and we still need those very talented people who see something that nobody else sees.

I think it can be a launchpad. If somebody needs a simple website and AI can build it for them and they don’t need to worry about brand or any of that, have at it. But there are a lot of small things that only a human can really pay attention to in terms of detail. And those details, I strongly believe, impact usability.

If you joined a product organization tomorrow to start a new modernization effort, what’s the first thing you’d assess?

The organization’s top leadership has to have high interest in truly understanding the problems that they’re trying to solve for their customers. User-centered product design and development has to be supported from the very top of the organization to be successful. If I were joining a new organization, I would want to know that human-centered design was a priority for the leadership all the way up to the CEO.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In our conversation, Idan discusses how AI is reshaping product development by giving individuals more agency and changing the way teams collaborate. He shares why prototypes are becoming the new language of product development, where the next bottlenecks will emerge as execution accelerates, and how leaders can create the guardrails needed to maintain alignment without slowing innovation.

AI’s expansion of individual agency across teams

Over the past year, AI has expanded individual agency faster than organizational capacity. What kind of work have you seen collapse into a single role faster than you expected?

There have been a lot of experiments — people doing things that wouldn’t traditionally be part of their job description. I wouldn’t say there’s been a full collapse or merging of roles, but boundaries are absolutely becoming blurrier. With more agency, individuals are taking on areas that would have previously been unexpected.

From where I sit leading design, I see PMs prototyping their ideas very early on. That shift from a document and a hypothesis to tangible output, helps us raise questions, create alignment, and understand edge cases that otherwise would have required much more iteration. I find myself reviewing work with PMs and having design discussions that I previously would have had only with designers. PMs are using design agents and making decisions that enable designers to build on top of that work instead of starting from scratch.

Today, our product and design teams are often prototyping directly in our product repositories, and we are working to expand this approach — bringing product and design closer to the production code that ultimately ships to users. This brings us much closer to engineering and creates questions about how we reduce the translation between PMs, designers, and engineers, so that we’re continuously building on each other’s work, rather than starting over every time.

I wouldn’t say the roles are collapsing, but actually expanding. The boundaries that used to be much clearer are no longer distinct. I think that’s a good sign.

How does engineering feel about this shift? Do they find that when a design gets to them it’s better vetted, or do they feel isolated from the beginning of the process?

Our engineers are actually getting involved earlier because ideas are being shaped into tangible output much sooner. Once I have a prototype and a direction, I want engineering involved immediately to validate that we can build it.

While this approach has allowed engineering to begin work earlier, particularly on backend and architecture, it has also created confusion when there is no clear source of truth across multiple artifacts being developed simultaneously — for example, a product prototype, a design prototype, and Figma. Some designers, including myself, have submitted pull requests into production, but it’s very contained. These types of changes can be risky when they happen at scale in a mature company with millions of users, that’s why it’s important to have strong guardrails.

This could look very different in a startup environment where founders wear multiple hats and the priority is shipping quickly. The boundaries are probably even blurrier there. At Via, due to our scale, engineers now see expected outcomes in much greater detail much earlier. For example, most of our product surfaces are maps. Translating a static Figma file into expected map behavior is incredibly difficult. Think about Google Maps and the transitions between zoom levels, when points of interest appear and disappear. Those interactions are hard to communicate.

When you can prototype that behavior instead of describing it, it creates alignment, improves quality, and increases velocity. The next question becomes how to bring that work closer to production instead of rebuilding it from scratch.

AI tools sometimes introduce friction or disruption. What have you experienced? And are there forms of disruption that are actually healthy signs of adaptation?

Absolutely. I’ve spent a lot of time over the past year thinking about how we adopt AI and change our processes. I can share a good example from one of our mobile teams. They created an incredible prototype of a very complex feature that showed how the experience would work with all the interactions. It looked amazing. They started with just a few designs in Figma to create the foundation, and then built the rest in Cursor.

When we reviewed the concept, most of the conversation centered around the prototype. It created much better alignment between leadership, PMs, designers, and engineers. It also helped us discuss scope because we could clearly see what could be phased. Then the engineers asked, “Where’s the Figma? Where are the specs?” We only had a few screens in Figma. Everything else existed in the prototype.

This was a healthy problem to have because the prototype created so much value. It improved velocity, reduced manual work, helped leadership discuss scope, gave engineers feasibility input, and communicated interactions that would’ve been difficult to explain otherwise. Our challenge now is figuring out the best way to translate those prototypes into production code.

Where I do see negative friction, though, is when PMs begin prototyping. The goal should be to define the direction, not create the final design. But, once a prototype is shared with leadership and other stakeholders, it can begin to establish expectations for what the final product should look like. That puts designers in a difficult position because once a design direction has been established, it’s much harder to challenge or rethink it. We need to be very mindful of that dynamic.

AI is changing the shape of product development constraints, but it hasn’t eliminated them. As teams become capable of doing more with the same resources, where do you see the bottlenecks shifting?

In the short term, I still think the bottleneck is translation. Going back to the prototype example, if you have an advanced prototype that represents the ideal outcome, how do you reduce the amount of engineering work required to build it? We’re experimenting with spec Markdown files, building directly on our repos, and improving our design system so it’s more effective for agents, not just humans. That’s where we can unlock even more velocity.

The value already exists, but AI helps us reach alignment faster and build conviction earlier. The remaining bottleneck is how we hand that work off. Longer term, I think the challenge shifts toward distribution and communication. If you look at companies like Cursor, Claude, or ChatGPT, they’re shipping updates almost daily. Every time I open Claude, there’s another version. The question becomes: how do users absorb that much change?

This creates a different bottleneck around distribution, marketing, and communication. It’s a healthy problem because historically, engineering velocity was the bottleneck. Now, the bottleneck may shift to helping users understand and adopt what’s changing. So, in the short term, it’s about removing the translation layer, but in the longer term it’s about managing distribution and adoption effectively.

Why leadership needs stronger guardrails

As AI gives individuals more agency, are there responsibilities that should remain firmly with a specific function instead of becoming more fluid?

I think in the short term, yes. In the long term, it’s harder to say because the models are progressing so quickly. But, the trajectory is very clear — the AI models continue giving more agency to each individual. Right now, I think there’s a big difference in the level of risk that different stage companies are willing to take.

At Via’s scale, a big change will happen when everyone adopts more of a builder mindset. This will enable team members to cover more ground, while different functions shift from being executors to contributors earlier in the process.

For example, a PM can prototype without producing a polished design. Designers can help with system thinking, identify risks in the proposed solution, and continue shaping the prototype before taking ownership later in the process. Instead of a clear handoff, designers fade in as the direction becomes more mature.

With this model, I think each team member will be able to move farther ahead while other functions act as consultants until it’s time to take ownership. The important part is making sure no one gets too far ahead before designers or engineers have had the opportunity to provide input. That’s why the guardrails matter.

With development accelerating so quickly, what areas have become more prone to breaking down?

I think the biggest risk is around the fundamentals. A couple of years ago, we spent much more time defining the problem. Why are we solving this? Why is it a priority? How do we know this is the right opportunity? How do we build conviction that we’re solving something valuable? Only after we answered these questions, would we spend time designing the solution.

AI lets you move much faster into advanced stages of building, but that creates the risk of losing focus on what you’re actually trying to solve and why it matters. Ideation, experimentation, and building have become much cheaper, so it’s tempting to explore more and more solutions instead of staying disciplined about the problem.

I’ve seen teams make progress and then come back a week later with a prototype that’s completely different. My first question is always, “What happened? How did we get from here to there?” That’s the risk. When building becomes cheaper and faster, it’s easy to keep testing new ideas without staying anchored to the original problem.

Is there anything you’ve changed your mind about over the past year as you’ve watched your teams adopt AI?

Definitely. And I’m sure there will be more changes over the next few months because everything is evolving so quickly. A few months ago, I imagined a process that would give every function much more agency. There are already many examples of companies where PMs can go end to end — from defining the solution to merging small changes into production. I used to think that designers would move further into frontend development while engineers would spend more time on backend work, architecture, and other complex engineering challenges.

Today, I look at that differently. The question I keep asking myself is: what value are we actually creating? I’m no longer convinced that value comes from PMs producing designs that are nearly as polished as a designer’s work. The value is already enormous when PMs can build conviction, clarify requirements, and define the product direction through prototypes. That’s where designers can step in and build on that foundation.

Similarly, I don’t think designers necessarily need to make pull requests. Personally, I don’t want bugs assigned to me that I don’t know how to fix. So I’ve become a little more conservative. That might change again in six months, but today I think the biggest opportunity is for PMs to create strong prototypes that establish direction, and for designers to create polished prototypes that help engineers clearly understand what should be built. I’m focused on finding ways to help product, design, and engineering extend and build on one another’s work, rather than rebuilding at every handoff, and leveraging that to accelerate the path from idea to execution.

Preparing organizations for AI-native work

Are the behaviors of your team’s top performers different compared to a year or two ago?

Sure. Today, they’re using AI to articulate and visualize their thinking. When a PM is working through complex logic, it can be difficult to absorb that by reading a document. But when they build a quick prototype, it immediately becomes much easier to understand how different states and interactions work. The same is true for engineers. Instead of explaining backend architecture or APIs through documents or spreadsheets, they can build lightweight prototypes that communicate those ideas much more clearly.

The strongest performers understand the complexity they’re dealing with and use AI to visualize it in ways that create alignment and accelerate decision-making. The output isn’t always faster code or faster design. Often it’s those side projects — small demos and prototypes — that make the main work move much faster. From a leadership perspective, that’s incredibly valuable because it creates alignment early and reduces the need to revisit decisions later.

The second thing I see is constant learning. Our top performers are always testing new tools, those problems much cheaper to solve. Instead of leaving something in the backlog indefinitely, they’re asking, “Can I solve this now because AI makes it practical?” They’re using AI to cover more ground and create more impact.

Many established companies are layering AI onto structures designed for a different era. What organizational patterns do you think will feel outdated in a year or two?

The scale and complexity of work are going to increase dramatically. If every team member is running multiple agents, the challenge becomes understanding what’s happening across all of them. That will require rethinking how we work. I suspect that some of our existing rituals, like standups, will evolve. Those checkpoints exist to create alignment, but when agents are constantly producing information, we’ll need smarter systems that decide what to surface and determine when humans actually need to get involved.

This will likely lead to smaller teams. Startups are already showing us what’s possible in this regard. Companies that once required 50 people are now operating with five or 10 people supported by AI. For larger organizations like Via, that transition will take longer because the risks are much higher, but I do think we’ll eventually see smaller teams supported by intelligent agents that help coordinate work and highlight where human judgment is needed. Communication will become one of the biggest challenges, and it will force organizational change.

What advice would you give leaders who find their orgs are moving faster but are becoming harder to align?

To be honest, I’m still figuring this out myself. It helps to establish very clear guardrails around when different functions should become involved. As individuals gain more agency, they can do more work independently, but designers still need to be involved early enough to provide their perspective, identify risks, and shape the direction before it becomes fixed.

The same is true for engineering. Once there’s a clear direction, bring engineers in early to validate the approach and identify constraints that others may not see. Being explicit about when different functions should be involved — or at least informed — is critical during this transition. Until we reach a more agentic way of working, clarity around those collaboration points is one of the most important things leaders can provide.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In our conversation, Scott shares why empathy is the foundation of great product management and how the simple mantra, “I am not my customer,” has shaped his leadership philosophy throughout his career. He discusses balancing user research with data, preserving empathy while building products at YouVersion’s global scale, and where AI can — and shouldn’t — play a role in product development.

Understanding people before building products

You’ve led products that range from booking travel to designing hypercar experiences to helping people build spiritual habits. What have those very different products taught you about human behavior that you think every product manager should understand?

The products and experiences I’ve been involved in are wide-ranging, but I think there’s a common thread among them. In the end, people come to a product or service because they have a need they can’t meet themselves, or they’re looking for expertise to meet that need.

In travel, it’s much less about the destination and more about the experience. In hypercars, it was about the quintessential thing that could not be had that’s customized or unique. In my current role, people come to Scripture or faith with a particular desire, history, or path, and they’re looking for something. Everyone’s on a journey, and whether you make a physical product or a service, the idea is that someone invites a product or service into their life because they’re looking for something from it.

In the age of AI, as automation gets good enough to deliver a flawless product experience, what human function can still not be automated away?

This is such a big topic right now — not a day goes by without some kind of conversation about what the future of product management looks like. While I believe AI experiences and automations will improve, I still haven’t seen where taste meets function. It’s very hard to understand good taste from bad taste, or appropriate responses, questions, or ways to deal with a situation.

If you’re coming to something and you say, “I have anxiety, and I need help,” there’s a different response than, “I want the fastest hypercar in existence.” Being able to contextualize your responses and build products with those human needs or desires in mind is something I don’t know will ever be automated.

The fun part is that you can now try more options or have more experiments. We don’t necessarily have to get it right the first time. We can try three, four, or five times for a much lower price or with much less energy spent. I look forward to bringing the right tone, the right experience, good taste, and simplicity, while also being able to go up to bat multiple times at a lower cost.

When you’re trying to understand what customers actually need, how do you separate what they’re asking for from the underlying problem they’re trying to solve?

It’s nuanced because sometimes what people say in a user interview doesn’t match the behavior you see in the data. Someone might say, “I want X, Y, and Z,” and then, when you observe their actual behavior, they want something different.

At every organization I’ve worked at, we’ve spent a lot of energy on user research, putting that alongside the data, then measuring and analyzing them together. If you bring only one or the other, you don’t have the whole story.

I’ve also learned not to ask direct questions about particular products. It’s tempting to ask, “Do you like the thing that we made?” But it’s better to ask, “How did you get here? What made you step in this door?” Learn the why before the product. I tend to go back one or two steps, and that’s usually where the real nugget of information is.

Understanding what customers need from a product requires empathy. What does empathy mean to you before it becomes anything on a product roadmap?

To me, empathy means being able to think about someone else — or their experience — in your day-to-day life. For example, how does what I’m doing impact others? What are others around me thinking, feeling, or responding to as I work or live?

‘I am not my customer’

Did you have a moment in your career that taught you the importance of user empathy?

I definitely experienced this during my time at Expedia. One of Expedia’s policies was that you go to the call center and listen to people who had called in — because that meant that something had gone wrong with their vacation, like a delayed flight or extra, unnecessary charges. If you ever want to feel terrible, listen to people talk about how their dream vacation had a problem! That was the beginning of my journey to really understand how much emotion is tied to the product.

As we spent more time in the call center and with user research teams, the phrase “I am not my customer” cropped up. We all had an aha moment around this and even made T-shirts that we wore in the office that said, “I am not my customer.”

I’ve carried that with me ever since. If I’m not careful, I end up designing products for myself and don’t represent the buyer or customer. That’s become a mantra for me: “I’m not my customer. I need to remember who they are.”

Are there any rituals you use with your team to reinforce that idea?

Bringing in perspectives is important. At YouVersion, I like to start team meetings with stories and feedback that we get from people who’ve had either a great experience or a bad experience. Anyone can bring one. It grounds us in everything we’ll talk about afterward. If we’re too tactical, we’ll keep building things for ourselves. If we start with the story of someone whose life was changed — or someone whose life was put out or uncomfortable — by what we did, it grounds us in remembering who we do it for.

Empathy can often be at odds with hard metrics, like retention. How do you keep your team empathetic while still managing outputs?

We have an internal phrase: “Data has a seat at the table, but it doesn’t occupy all the seats at the table.” Every time we come to the table with data, we also come with research. Then we ask what we’re actually trying to do. Is the data helping us accomplish the job to be done, or is it steering us toward an imaginary goal?

Data has a seat at the table, as does research. The overall “why?” belongs there too. Data is a tool — one of many in the toolbox. We’re much better at getting data than we’ve ever been, but I don’t think it tells the complete story.

Scaling empathy without losing the individual

At a scale of a billion downloads on YouVersion, how do you keep the user from becoming an abstraction you’re optimizing against rather than a person you’re building for?

This has been the most demanding and meaningful stretch of my career for that reason: people approach the Bible and faith with so many different contexts, life experiences, backgrounds, and emotions. There aren’t two personas or four — there are thousands of reasons someone showed up.

We try to remember that people carry something in with them when they walk through the front door, and it’s our job to understand what that is. We started asking, “What brings you here today?” It’s something you might experience in a boutique shop or an old hardware store where service is prime. Someone asks, “What brings you in today?” Then they can respond, “I’m looking for this really rare thing.”

That’s our job in a growing digital business: keep remembering to ask people, “What brings you in today?” in a digital way and then respond as well as we can. We’re not there yet, but that’s our target.

Users also want to get where they’re going quickly. How do you balance speed of the experience with building trust?

I love that I’m a beneficiary of where we are in this space right now — it used to be considered intrusive to ask people questions when they first walked in the door. But we’ve learned that if you go shopping right away in the digital space, we’ll have a hard time helping you find what you need unless we can ask you questions first. The expectation has changed. People are willing to invest a little time up front to have an experience that looks built for them and that addresses them.

We’re really front-loading the dialogue we all want. We all want a custom experience. We didn’t know how to get there, and people didn’t know how to tell us what they wanted. Now we can ask, “What brought you in the door? Is there a goal you have in mind?” “How can we help you meet that goal?” It’s building a relationship in a new digital way.

What’s your method for staying in direct contact with what one real person feels using the product?

At all the places I’ve been in my career, there has either been an insider or VIP program, or we’ve created one. This is where you find people who are excited about what you’re doing and build relationships with them. It’s a little easier in the physical product space. For example, in the hypercar space, it was just a few people — we could have constant contact with them.

At YouVersion, we recently invited longtime users into an insider program. Once a month or so, we’ll ask them questions through surveys, and some of those lead to video conversations. We have maybe 10 or 12 people that, when we think of something or we’re not sure about it, we’ll run it by and ask, “What do you think?” They’re honored to be part of the process, and we’re honored to get another perspective so that, in fact, I’m not my customer.

The harder part is that I don’t get as much contact as I’d like with the people who have decided to do something different or have moved on. We spend more energy, more time, and more dollars trying to understand that group because that’s who I would love to have more contact with.

On the flip side, how do you instill user empathy in product managers who will never personally talk to more than a sliver of the people they’re building for?

Five years ago, I wouldn’t have told you it was this simple, but now it’s about staying as close as you can to the stories and being a great storyteller. I feel like it’s absolutely part of my job as a leader to tell the stories of the people — good and bad — who have interacted with our products.

That’s why I kick off team meetings with stories, and everyone’s encouraged to bring one. We bring the embarrassing ones and the great ones.

It’s easy to lose perspective on the impact of what you’re doing. Storytelling brings you back to that moment when someone’s life changed, when they found exactly what they were looking for, or when they found a great product. It also means getting better at retrieving those stories and cataloging them. Thankfully, in the app world, you have App Store reviews, but there are other places where you can gather those conversations. I don’t think we can ever be too far away from the stories of the people we serve.

Where AI fits into faith-based products

At YouVersion, where have you drawn the line for integrating AI in the faith product?

This is an ongoing conversation. We’ve landed on the idea that tools that help us with workflow, do our jobs more efficiently, build context, catalog research, or draw correlations are areas where AI can help us. Those tools augment the human mind.

But at YouVersion, where we represent a biblical text that’s thousands of years old, we’ve been deliberate about not placing AI between a person and Scripture. If we ever move in that direction, it will be because we’re confident it can be done with the level of accuracy and integrity worthy of the text.

Where the text needs to be, the text is — with no apologies and no buffers. Where AI helps us remember things, organize research, or find correlations, we welcome it. Everything in the middle is a high-tension conversation that we often have.

How does your personal faith shape your work?

My personal faith shapes everything that I do because it’s who I am. The number one thing that gets me out of bed — and why I love product management — is helping people get where they want to go. In another life, I think I would’ve been a hotel concierge because I just love helping people get where they’re hoping to go.

My faith brings into that a desire to help people get where they’re going in their faith. When I’m not at work, I love having conversations with people about their personal lives, their faith journey, and what they’re looking for, and helping them find it in whatever way I can.

When we were building physical products, you could see the joy people got from being part of that journey. Likewise, we’re all on a journey, and for me the destination isn’t vague. It’s a real relationship with God. Faith is every part of that equation. For others, it’s still unfolding, and I count it a privilege to walk part of that road with them and point toward what I’ve found.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In this conversation, Shri talks about why the biggest lesson from her AI initiatives has been to start with the customer journey rather than the technology, and how a simple request, like a bill summary, can mask a much deeper problem. She discusses why vanity metrics can still be directionally useful, how regulated industries change the calculus on AI pilots, and why AI systems can never simply be launched and left alone. She also shares her own habit for understanding unmet customer needs: going to wherever the customer is and just listening.

Start with the customer journey, not the technology

You led an effort that improved a chatbot’s resolution rate from 65% to 95%. What did that teach you about identifying customer needs and pain points?

One of the biggest lessons learned was don’t start with technology, start with the customer journey. When we looked at all of the agentic frameworks and evaluated vendors, it was very addictive, and it was almost like drinking the Kool-Aid and saying, “Wow, I really want this.” It’s shiny, and it’s lovely, and it’s beautiful, and it’s fun. But why do you need it? I think we should always start with that question. Don’t start with the technology or the architecture or the vendor; start with the why about who asked for this.

As you went through this initiative, was there a point where you had to pivot or change direction?

Possibly every five seconds. The nice thing about AI is we’re all in this together for the first time. We built a lot of things in-house at Comcast, at Momentum, and at other places, so a lot of it was learning as we went along.

It’s not so much pivoting as making adjustments. It’s saying, “OK, if I do this, I’m creating a ton of tech debt, and I’d really like to rethink my architecture and see if I can course correct.” That’s the sort of decision that AI leaders need to be able to make, and not be afraid to say, “I think this may not be working. I need to make a change,” and make that change. If you are not able to move at that pace, the technology is going to outpace you and create a ton of problems down the road.

What customers ask for isn’t always what they need

Is there an example of a time when customers said they wanted something but it turned out to be different from the problem they were actually trying to solve?

This example is from a few years ago, at Momentum: The customer kept asking for a bill summary. Momentum has both retail and digital services, and the customer wanted a summary of their loan transactions throughout the course of the loan. So we said, “Sure, let’s do that,” and we spent a lot of time building it out, but no one asked what happens before the bill summary and after the bill summary. What is the journey? What is the pain point that the customer is trying to solve? They may ask you for a transaction summary, but what they’re trying to tell you is, “I would like to lower my interest rate. I would like them to make it easier to pay the loan back. Or, I want to turn it into a bigger loan because I wasn’t able to pay my electricity bill this month, and I need more money.”

There are many things that happen before and after that one moment. So moving away from just building out a product feature because a customer asked for it and, instead, asking what happened before and after and looking at the entire journey from end to end, makes a big difference in actually solving the real problem.

How do you coach teams to prevent them from optimizing around symptoms instead of needs?

This has actually become really easy in the world of AI. Let’s say we designed an agentic framework and decided to launch a bunch of product features. We are no longer talking about just features. We’re talking about constantly evaluating the agent, almost in real time, and feeding the evaluation of that agent into your product roadmap.

Even before you reach the customer, you’re starting to think about how you think your customer will behave. You can use historical data, you can use synthetic data, you can do all kinds of things to do these evaluations. So the way you build your product has moved away from, “First, let me look at customer experience, then find the pain point, then identify these features, and maybe it will work.” Evaluations are no longer a matter of weeks, they are a matter of hours.

Using all the technology at our disposal today, we’re able to populate the product roadmap in a way that is more meaningful.

Metrics that actually reveal unmet needs

What signals have you found to be the most reliable indicators of unmet customer needs? And, are there metrics that teams undervalue?

As a customer experience professional, I try really hard to move away from vanity metrics. We love dashboards, right? We like to print dashboards and show them to our C-suite and say, “Look, we have so much adoption. We built 10 features, and the adoption of those features has gone up.” But what did that really do? Did that translate into a business outcome? Did that translate into a more meaningful metric, like resolution?

If somebody logs in and clicks on a button, that doesn’t mean they went on to resolve the problem. It’s sort of like following the customer’s journey to look at the business outcome and saying “They were able to resolve the problem, and therefore we prevent a churn and maintain loyalty.” That’s the sort of insight you need to derive from your metrics. Otherwise, you end up chasing fake loyalty and metrics that look nice on paper but don’t really translate into anything that is meaningful for the customer or the business. It just means somebody clicked on a button.

What’s your process for figuring out whether a problem is really impactful to solve or more of an edge case?

There are two ways to go about this.

Let’s say you have a chatbot. You have all your transcripts to lean on. Throw all these transcripts into a transcript analysis tool and see what insights it comes up with — it will surface all the pain points where you see the maximum amount of problems. You can create a heat map of your customer journey and then identify which areas to go after. The other way is using agent evaluations, not just to see if your agent is performing, but also to see if your agent is solving a customer experience problem.

And then you can rely on traditional CX metrics. Review your dashboard every week. You’re looking for trends, making sure everything is moving in the right direction. I know that we shouldn’t be focusing on NPS or tNPS as one metric, but it does give you some indication of where things are currently. You can put it into the context of other metrics and other data and see what the trends are. That’s the traditional way of doing CX, and we can still apply those principles today.

Navigating regulated environments

How does identifying and solving customer pain points differ between highly regulated environments and industries that are less regulated?

The use cases are different, obviously. When you are looking at something like insurance or a bank or even healthcare, the first thing you have to be sure of is that any technology you build, whether it uses AI or not, protects customer data. You do not want to be leaking PII information, especially if you’re using OpenAI or Claude or any public LLM. You want to be careful with customer data, otherwise you’re going to lose trust. These kinds of industries are also very prone to fraud. Probably some of the lowest-hanging fruit when I was working in fintech was centered around fraud and how to mitigate it. We built a ton of technology to protect ourselves from what we call fraud rings — a group of people who just try to break the system to steal money and data.

So the use cases tend to be more around risk mitigation, fraud reduction, and customer data protection. It becomes front and center when you’re in a highly regulated industry. It’s also difficult to build pilots and scale them if you have a ton of regulation. It’s best to have good governance in place before you even launch a pilot.

Are there instances when regulatory constraints can actually lead to better products and customer trust?

Well, yes and no. If you allow the friction to seep into the customer experience, then it becomes a problem. If you’re trying to approve somebody for a loan or schedule a healthcare appointment, but the technology gets in the way of that and the customer is just looping around because you’re so afraid to let them progress into the next step, then that’s a problem. It’s really important that you evaluate your technology, whether it’s an AI agent or something else, and test it out before you launch it.

The quality assessment, especially now that we are in the agentic world, has become so critical. We had a huge quality team at Comcast running end-to-end evaluations and tests to ensure that the agents were performing. Running all of those evaluations, the quality metrics, the observability, and having the ability to move fast and test your technology before it reaches the customer is helpful. But if you’re in a highly regulated industry, I would recommend that you bring in your legal, compliance, and regulatory folks very early on so that you’re not reacting. That way, you’re building with their input as opposed to against the tide.

AI systems need constant human oversight

Can you share a time when customer behavior changed through a launch process and you had to rethink the experience?

When we launched some of our AI-based IVR systems at some of the organizations, we realized it solved some problems — it automated some things and made some things faster — but that also made the customer more savvy. Say I’m on a call and solved my problem very easily without going to an agent, so now what else can I do? Now that I have managed to reduce my bill, for example, can I get an extra piece of equipment without even talking to an agent? That gets fed into the model, and the model starts learning and training itself and giving other recommendations. We realized that what we need to do is not just build an agent and throw it out there — we need humans in the loop who can watch what’s happening. We need people to sit with the human agent, and see what’s going well, and what’s not going well.

So some of it is evaluation, some of it is customer metrics, but a lot of it is just humans being in the loop, constantly evaluating.

What is your approach to bringing customer-facing teams into product development?

From a technology perspective, we built customer data platforms that took the events that the customers were going through and fed them into a feedback loop. We also instituted listening sessions. You sit down next to a human agent and look at what’s going on. This is a business process improvement team that has customer experience and engineers sitting together with the agent, solving problems in real time. But you’re listening, and the customer’s struggling, and you’re saying, “I’m going to improve the agent and do this on the fly.” This is the kind of thing that bypasses bureaucracy and solves customer problems in real time.

We also created a summarization tool — it’s like a little clock that says whether the customer is happy or not. When the clock goes to the left, the agent sees that. It gives recommendations like, “Looks like the customer’s not happy. Maybe you should try saying this.” It happens in real time, and the AI takes care of that.

Where do customers still require human judgment, and where would it be hard to train that into an agent?

In a telecom industry, for example, or an internet company, or even healthcare. Let’s say a tree falls on my house. Do I want to talk to a bot? Probably not. I would be so stressed out and so scared that I would want to speak to a human who can help me through it. If I’m going through a medical crisis, I don’t know that I would trust a bot to take care of me. I just don’t think we’re there yet. That’s why it’s important to design your AI agentic systems in a way that you know exactly when the human should be in the loop.

If you can identify situations where you think it will not make any sense for the bot to be here, then you just completely escalate it to the human agent. In some way, that restores customer trust. In moments of extreme stress, you can hear the customer saying, “Customer service, customer service, customer service, human agent...” You can hear the frustration and the fear, so it’s important that we look at all the flows in our business and say, “Where does it really make sense for us to leave the humans in there?” It’s important to have empathy for the customer when you design your AI systems.

What separates teams that truly understand customers

Are there customer problems you’ve found that are just inherently difficult for AI to recognize?

It happens all the time. I think we make a lot of assumptions about what a bot can solve. Repetitive tasks, like changing the password or having better pricing recommendations, are the things that we can do using ecosystems of technology. But what happens afterward? It is entirely possible that a customer makes a decision to change their plan, but then abandons it. You’re able to ask the question along the journey of what happened as a customer enters the funnel: “They did this, and then they did this, but then they didn’t do that.” Being able to ask those kinds of questions, and being honest about what the customer interactions are, will really help alleviate some of those concerns of whether a bot can really solve the problem.

But there are fringe edge cases where the bot will just not be able to handle it, and maybe we can fix that by changing the agent, evaluating the agent, or by building a new agent. There will always be situations where the bot hallucinates or fails to solve the problem.

What do you think will distinguish product teams that genuinely understand customers from those who simply have access to more data?

I think all product teams should have access to the data, but product teams who can use the data to derive insights will make the difference. My product team in my past role built an analysis tool. Having no experience in data whatsoever, and not being part of their job description, they built an analysis tool with a frontend, a backend, an LLM, and everything. It simply takes a bunch of transcripts and analyzes the heck out of it, and comes up with brilliant recommendations that before would have taken us days and weeks. They were thinking outside the box, and that’s the kind of thinking that you need.

I had a product manager who took all of our product artifacts and threw them into ChatGPT to create a custom GPT product catalog, so that now anyone can search through our product catalog as if they’re talking to a person. That’s the kind of out-of-the-box thinking that product managers need to do, instead of just saying, “I’m a product manager. I’m going to wait for someone to tell me what they need, and then I’ll be looking for features.” Getting ahead of the data is going to be the winning solution.

Is there one habit you’d suggest product leaders adopt to better understand customer pain points?

What has always helped me as a product professional is going to where the customer is. I’ll just go sit down in the retail store and listen to what’s going on, or I’ll look at the systems that the retail employees are using, or I’ll go to a contact center and sit next to the agent and just listen for a day. Those are the things where you can really understand what’s going on when you’re not looking at the data. So, stay curious, and just go meet the customer where they want to be met — that helps.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.

In our conversation, Yair shares his vision for what product management looks like in an AI-native world, where interfaces become intent-driven. He explains why product organizations need to think platform-first, as well as how AI is reshaping roadmaps and experimentation.

The transition to conversational UI

You’ve laid out several provocative predictions about the state of product in 2026. The first is that software will no longer be organized around pages and navigation. If interfaces become conversational, how does product design change?

I think the word “design” will no longer be limited to just the design of a website; rather, we’re designing intent. When a product designer begins designing a product, they will no longer focus on how it looks, but on designing the journey. The journey doesn’t have to be something that you see with your eyes. In the case of conversation, I can simply speak and receive my answer. In the end, you’re designing a journey, but it doesn’t need to have a visual interface, necessarily.

The fact is, you can’t design an interface that makes sense to everyone. We’ve tried over the years, and we’ve also seen a one-to-many relationship between the website experience and users. The rare cases where it was more binary had to do with elements like locale or dark versus light themes.

What I have in mind is something much bigger. You still need a design system, but it needs to be very strong to create a website built on intent — one that understands the user’s context, the context of your available content, and the data you’re collecting all the time. Based on those things, it builds itself while staying on-brand. It should still feel cohesive.

Thinking that you and I need the same website, despite our different ages, interests, and goals, no longer makes sense. Today, AI can crunch billions of parameters in real time and build a layout tailored to exactly what the user needs. The CFO sees three KPIs and a cash flow chart. The analyst sees 14 filterable cohort tables. The CEO on mobile sees one number and a thumbs- or thumbs-down icon. Three generated UIs from one platform, and not one compromise layout. We can’t continue to think in terms of a one-website concept.We can’t continue to think in terms of a one-website concept. That’’s outdated.

How does this change the skills or background you look for when hiring product designers?

Many functions and positions are now converging within the space. We’re no longer looking for an expert in a single field of capabilities. There’s a concept of a full-stack product manager — someone with design capabilities who’s analytical and also invents things. Historically, those people were anomalies.

Today, you don’t need to have every capability yourself. You just need to know what question to ask and when to ask it. With the strong suite of tools we have now, you’ll likely get a very sound answer.

If you were only doing product design in the past but were afraid that the technical side of product management was a barrier, that’s no longer the case. There’s now a higher expectation that people entering product management know more disciplines and move closer to being full-stack product managers, all because they have these superhuman tools at their disposal.

Why platforms become the product

You recently said, ‘The website of tomorrow is built the moment you ask for it.’ What does that imply for product discovery and roadmap planning?

It makes it much more difficult, and we find ourselves investing a lot more in planning. The tectonic shifts in product and technology are so big that even basic building blocks of product management are changing as we speak. The place you need to invest in most is your platform.

If UI is commoditized and everyone can build applications simply by prompting AI, then the most important thing you can do is have a very strong engine and platform on which those applications are built.

In Kaltura’s case, we’re a video cloud platform. If you’re building a video application, we want to provide a cloud platform, APIs, and a strong MCP so your AI can talk to ours. Because here’s the threat: a mid-market sales team can now replace a monolithic product with Airtable, Clay, an AI agent, and a custom Slack UI in one Saturday — no code. What they can’t replicate on a Saturday is transcoding at scale, real-time event orchestration, or ML inference chains. The API is the door. The engine behind it is the moat.

AI has lowered the barriers to building software, but has it also changed what creates durable competitive advantage?

Absolutely. Companies that existed before the AI boom have important advantages, like customers, for one. If you already have customers, selling them AI capabilities is much easier than convincing entirely new and unfamiliar prospects to trust a new AI company — especially in enterprise environments with long procurement cycles.

The second advantage is content and data. If customers have worked with your product for years, you’ve accumulated content, behavioral data, and context. That translates into skills, which later become agents. This is the breeding ground for good AI. Understanding your customers’ behavior inside your systems and translating that into skills and agents is a huge competitive advantage in the AI race.

Going back to UI and the idea that customers could build or modify their own experiences, are there architectural decisions you’re making today to support customers building on top of your platform?

Building UI isn’t all the same. Building navigation or a carousel is simple, but building products like a video player or a Zoom-like real-time client isn’t. Because of that, we’ve built those advanced UI components as reusable platform features that developers can simply incorporate into larger applications.

For example, imagine someone wants to build a speed-dating application. They need a way to connect people to talk with one another in real time in a private room. That’s not something AI is likely to build easily today, even if you have the APIs to connect to it.

We have a micro-frontend framework called Unisphere that is embed-first. A developer can import one ES module from our CDN, initialize a workspace, and have a full AI conversational avatar with video search and interactive content running inside their app in under 20 lines of code. Even before AI, we believed embedding should be a first-class citizen. With AI, it’s even more important because the interface layer is whatever the agent assembles.

Documentation as part of the product

You’ve said your agent’s understanding of your product is only as good as your documentation. Should product teams start treating documentation as part of the product?

Yes. What was once a side dish is now maybe even more important than the showcased product itself. If documentation is the instructions for how other applications and AI agents build on top of your product, and agents can’t easily find your product or can’t easily build products from your documentation, then something is wrong.

You can test this immediately. Give an AI agent your documentation, ask it to build a feature, and see what happens. These agents are very good at translating documentation into working code. If they get lost or hallucinate, you know your documentation isn’t good enough.

From a product perspective, we have two requirements before we’re ready to deploy. First, it has to be API-first, which today also means MCP. Second, ii must be documented. If it is not documented, it does not exist. If a feature isn’t discoverable, then it hasn’t really made it to production.

On a product team, should documentation belong to a specific function or be shared across the team?

Today, the moment we open an epic, a documentation ticket is automatically created within that scope of work. There’s no option to push work to R&D without linking documentation to it. That already changes behavior because documentation becomes part of the workflow.

Who writes the documentation is another matter. You could have a technical writer. Product managers have always written documentation as well. R&D can write it, especially when you’re building API-first products for builders. Ideally, though, agents should do it. Once you have code and everything is explained semantically, translating that into human-readable documentation is easy. Agents don’t necessarily need human-readable language, but they still need taxonomy and hierarchy.

This entire process should be automated. Documentation shouldn’t consume an hour of anyone’s time — it should simply be part of the product lifecycle.

The future of product teams

Product teams have talked about failing fast for years, but you’ve argued that the mechanics didn’t really exist until now. What changed?

In the past, if you had an idea, maybe you’d build a landing page with a wait list and measure signups. Today you don’t need to stop there. You can build the core of the idea — not the entire thing, but the essence of it — and immediately see whether people find it useful, whether the hook works, whether the aha moment is there.

As a product person, you can start getting signals before you’ve even spoken with a developer or designer. I shipped two extensions and opened seven GitHub repositories in one week just to see whether people actually wanted to use the ideas.

Today, if you’re sending a product into development without already having market signals, you’ve done something very wrong. Ideally, you should fail every week and succeed once a quarter.

The barrier between product and code continues to dissolve. PMs can prototype, designers can generate code, and engineers can run customer experiments. As responsibilities become more fluid, what remains distinct?

Everything is converging. The PM whose identity is anchored in process management, standups, backlog grooming, mediating between design and engineering — that role is dissolving. The PM whose identity is anchored in judgment, taste, commercial conviction, and the infrastructure to test that judgment at speed — that role is multiplied. Now we have smaller teams, bigger accountability, and agents as first-class team members. The PM who wins in 2026 is not the one with the best intuition about what to build. It’s the one who built the best machine for finding out what to build.

What does LogRocket do?

LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.