Leader Spotlight: Rethinking quality for non-deterministic products, with Amir Rozenberg
Amir Rozenberg is Chief Product Officer at Blue Triangle, where he leads product for a digital experience platform that helps online businesses connect performance and quality to revenue. With more than 15 years in product management, he has led global teams from seed to enterprise, with prior senior product roles at Capital One, Sauce Labs, and Gomez, and a career start at Intel. He is a passionate advocate of and, author and speaker on the topics of product leadership, best practices in the AI context, and product management’s impact on the devops team.
In our conversation, Amir talks about bringing AI in as a validation authority rather than only a builder, as well as how “quality” means when software is no longer fully deterministic. He discusses where AI is reshaping the product manager’s job, and what key PM skills AI cannot replace. Amir also shares his view on how AI increasingly sits between the customer and the brand.
Where AI is reshaping the product manager’s work
AI introduces non-deterministic behavior into products and workflows. When there’s no single correct answer anymore, how do product teams define quality — and who actually owns the definition?
We see modern products today using hybrid structured or AI-driven workflows. Some of the workflows in a website or application are deterministic, like they used to be, but others are determined by AI — different directions, different outputs, different responses. A lot of products are moving into this hybrid mode.
AI intelligence needs to be introduced into the validation of that workflow, just as it’s introduced into the product itself. Instead of executing the validation step by step — click this button, use this selector to move to the next step — the test framework should model the user’s intent. For example, “click the checkout button” to reflect the user’s intent to check out.
For testers, this is a much healthier approach, because it reflects a behavior-driven testing mindset. It validates the user flow rather than the underlying technical implementation. The test focuses on what really matters and stays independent of the code changes that have traditionally made test suites brittle and high-maintenance for testing teams.
Another example is a change in the workflow, or in the textual response to a user query. Again, the right approach is to ask AI, as the validation authority: “Does this response make sense? Does this next step in the journey resonate with the user’s intent?” A strong AI tool that has the context of the service being offered, as well as the user’s intent, can determine whether the answer is complete, appropriate, and sufficient for the user. In the same way that developers bring in AI to create a richer, less deterministic experience, I believe we should also bring AI into validation, to make sure that whatever we’re doing makes sense.
As product teams take on more tasks, where else is AI reshaping the work — and do they have the context to know when a tool is providing the full story?
For sure. I’m thrilled about what AI enables me to do in my own job, but it’s a tremendous enabler for every function in the organization, whether you’re a developer or a product manager. There’s far higher efficiency and more clarity in the alignment, as well as in the communication and planning.
For the product organization specifically, AI contributes in three areas. The first is discovery. Product managers can create compelling, high-resolution, functional mocks that users can touch and actually interact with. The underlying data holds up, and the user can see how a feature they asked for is taking shape almost in real life — as if it were already embedded in the product.
I’ve heard users say, “Well, come to think of it, now that I’m actually using it, this doesn’t make sense — that’s the wrong way to do it.” So creating a high-resolution, functional mock is a great way to qualify a direction before you build it to fill a gap.
The second area, which I’m even more excited about, extends discovery into deep elaboration. For every feature, a product manager meets weekly with three or more users to put that mock to the test. That surfaces all the things we hadn’t thought of yet — what about this situation? What about that one? Product managers come to understand the user’s workflow and needs intimately, and that’s what makes a great product manager. It’s also what drives adoption of what they’re creating. And by the time a feature reaches the developer, the mock is far further along. There’s much more democracy and much more efficiency in how features get delivered.
Third is market awareness. This has historically been a pain point for me, because so much information is available. Now, with AI, I can get a summary of what’s new in the market — partners, competition, Gartner analysts, and more — so I know what might warrant an evolution or an adjustment to my strategy.
The skills AI can’t replace
Have the skills for success evolved along with the PM’s role?
Perhaps the thing AI has exacerbated more than anything else is time management. We lack time today — all of us, no matter the role we’re in. The number one quality I look for in product management is resilience and the ability to excel at time management and efficiency. To operate well in a fast environment you need to optimize for that.
The second thing I look for is curiosity. Today’s technology and tools are not tomorrow’s. You need to stay curious and keep your eyes and ears open at all times. Don’t go stale — be open to new technologies and take on challenges. I encourage everyone to get comfortable being uncomfortable, because that’s how we learn.
Last, be a leader and an advocate — get the people around you excited about what you’re creating. That’s what motivates them.
As you mentioned, teams are using AI to generate more experiments, more features, and more releases. How do you distinguish productive acceleration from simply producing more noise?
Our world has become one big hackathon. I value innovation and creativity, so we give everyone in the organization access to AI, from developers to HR, and everyone finds their own points of efficiency. At the same time, when it comes to the product team and product strategy, every idea has to be examined against our vision, our strategy, and the core strengths of our product.
We get a lot of ideas, both internally and externally. When users come to us and say, “This is what I need,” there are AI tools that can transcribe and correlate those interviews. From there, if we can find three users who will stay with us on a weekly basis until the feature goes into development, we know we have a winner.
Who owns quality when anyone can ship
Almost anyone on a product squad can now create and deploy functionality. How does that change team dynamics and governance?
The short answer is that the jury is still out — a lot is changing. Developers are worried about swim lanes and who gets to deploy. Some companies are aggressive about letting various personas deploy code to production — product managers and others — and some are more conservative. Do we run code reviews on AI-generated code before it goes to production? Who does the testing — a human or AI? There’s a colorful continuum of opinions on this.
I recently listened to a podcast with a developer lead who had an escalated ticket in production. He said, “We’ve decided not to do code reviews — instead, we take the AI’s code, commit it to production, and deploy. Previously, when a ticket came back to me, I would have researched the root cause and fixed it myself. Now, I feed the error into AI, and it finds the problem immediately.” So you can see the promise here — AI is taking on a lot of the SDLC.
I’m extremely excited about organizations, including ours, that let product managers ship UI frontend code specifically. Users come in and talk about adoption and the frustrations they have with the UI — and usually it’s not the data or the APIs, it’s something in the UI that doesn’t make sense. Product managers can implement those changes themselves. Users feel they’re being listened to, and product managers see more adoption.
The other side effect is that developers can focus more on what really matters to the organization — the structure of the database, the application logic, the APIs. Yes, anyone can introduce bugs, but if PMs are disciplined and thoughtful, they can come to understand the user’s reality and the product better, and come to appreciate what the development team has been doing and give them room to do it. All in all, it’s a wonderful change.
What guardrails are needed when product folks are pushing things live and teams are relying on AI for more of the development piece?
There are some processes we need to keep as human-to-human. Our workflows will naturally change, and AI will take over the mundane, repetitive tasks, but I don’t believe in staff reduction. We all need to raise our standards and become more efficient. In doing so, our users will have slightly different workflows — they’ll have to evolve, and so will we as a product organization.
Staying close to your users and having those intimate conversations — where they talk about their challenges, their frustrations, the gaps, their needs and wants — is how you stay efficient. AI can’t replace that. Users are far more comfortable opening up to another human who shows genuine interest. That connection between a product manager and a user — and a product manager inspiring the organization around a vision — is valuable and it would be wrong to try to replace it with AI. AI is a wonderful, powerful aid to the product manager in this context, but it can’t replace the conversations they need to have with users and teammates.
Where the product landscape is heading
How do product leaders think about optimization when AI becomes the intermediary between the customer and the brand — chatbots and the like?
This is a very interesting and relevant topic. There’s a lot of discussion about how AI will accelerate and represent shopping workflows, and it applies across the industry — hospitality, retail, and beyond. I recently heard a compelling podcast with a VP of Product at Shopify, and his argument, which I agree with, is that brand strength will determine the sustainability of the human-brand relationship.
People care a lot about fashion and design — about what they wear and how convenient it is. In niches like skiing or road cycling, they’ll be very specific about the skis or bike they buy. At the same time, they’d be perfectly happy to let cost-benefit decide which AAA batteries to buy — for that product, maybe they don’t care as much about the brand.
In that context, we provide a solution that helps these B2C brands optimize their digital presence for both the traditional, fully human shopping workflow and the modern hybrid one. For example, a modern workflow might begin with a person accessing a bot and writing, “Find me all the Airbnbs in this area, for these dates, that look lovely and are next to a lake.” This is a kind of hybrid shopping experience where the bot returns a recommendation based on different criteria than a human would use, and then a human takes it from there to close the deal.
I can envision future workflows where you have a trusted bot that’s almost like a digital assistant. You give it your payment method, and it goes and buys a product whose brand you have no attachment to but that meets your specific criteria.
If you were going to build a product organization from scratch today, what would you do differently than four or five years ago?
I absolutely love this question. The truth is that things have accelerated, but they haven’t fundamentally changed. The skills and values I look for haven’t really shifted across my career. If I were building a product team today, I’d look for people who excel at both the discovery and delivery sides of product management. And, above all, I value curiosity — that’s my number one criteria by far.
Experience matters, but I’m far more interested in a person’s ability to adapt, to learn, and to take on challenges with grace and overcome them for the greater good. The ideal team is a group of ambitious people who genuinely care about finding and solving customer problems, improving continuously, and delivering meaningful outcomes. They challenge assumptions, support one another, take ownership, and are motivated by building great products and winning together. That’s my dream team.
What does LogRocket do?
LogRocket’s Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.