Skip to Content
Get clarity on how to become AI-ready. Take the Assessment

194 / Ovetta Sampson: Designing AI Products Around Human Needs – Not Just Technology

Hosted by Sean Murray and Dan Sharp

Listen

About

barry-oreilly-product-momentum-podcast-guest

Ovetta Sampson

Right AI

Ovetta Sampson is an executive design leader and AI expert with over a decade of experience building AI and machine learning systems at Google, Microsoft, IDEO, and Capital One. Named one of the Top 15 People in Enterprise AI by Business Insider in 2023, she has spent her career at the intersection of human-centered design and emerging technology — leading teams that have tackled digital transformation, responsible AI, and enterprise ML across industries including healthcare, financial services, and transportation.

At Capital One, she led Machine Learning Experience Design and Responsible AI, helping evolve an enterprise ML platform while operationalizing RAI practices. In 2025, she founded Right AI, a consultancy focused on helping organizations manage the human engagement risks of building and deploying AI.

Ovetta Sampson is a design researcher, AI leader, and founder of Right AI. She previously served as VP of ML and AI Platform Design at Capital One and worked at Google and IDEO. Ovetta brought that experience to the 2026 ITX Product + Design Conference, focusing her keynote on a question that often goes overlooked: what happens when people interact with increasingly powerful machines? To help us answer that question, Ovetta offers a pair of frameworks that center on human engagement risk, responsible AI, and rethinking how product teams design AI products.

Building AI responsibly requires more than better models or more sophisticated tools. Product builders must also understand the cognitive, social, cultural, and physical risks that can emerge when humans interact with technology. AI inherits many of the biases embedded in the data used to build it, Ovetta says. So organizations need to rethink how disciplines collaborate; as the lines between traditional product, design, engineering, and security silos blur, responsible AI requires organizations to redesign not only their products, but also the processes used to create them.

Here’s what else we learned:

Human Engagement Risk Belongs in AI Product Design

Ovetta’s human engagement risk (H-E-R) framework is based on a fundamental premise: technology can harm people when designers fail to account for how humans behave around machines. The H-E-R framework identifies cognitive, social, cultural, physiological, and community risks. These risks become especially important with generative AI, where people can easily attribute human qualities to systems that do not actually possess them. As a result, AI product teams must consider psychological and cognitive outcomes alongside traditional usability concerns.

Ovetta cautions: “There are real dangerous risks when we engage with machines and don’t mindfully think about the outcomes that can happen when we don’t protect humans psychologically, cognitively, physically, and physiologically.”

AI Strategy Starts With Executive Leadership

Responsible AI also requires leadership decisions that extend beyond individual tools or experiments, Ovetta says. Many mid-sized organizations are hesitant to adopt AI because executives are concerned about intellectual property, trust, and data leaks. Meanwhile, employees may already be integrating AI solutions without an overarching organizational strategy. It’s a disconnect that creates opportunity for leadership to establish clear priorities before adoption becomes fragmented. AI strategy starts at the top, Ovetta adds, because executives have the authority to establish the conditions under which technology gets developed and used.

“Once the C-suite understands the risk to their shareholders, to their products, to their employees, to their customers, it is much easier for me to bring in the implementation of how to mitigate those risks.”

Dismantling Silos Is Essential for Effective AI Development

AI challenges the traditional handoff model in which designers, engineers, security, legal, compliance, and other teams work separately before passing projects along. Ovetta says AI development requires continuous cross-functional input instead. Her D-C-R framework – draft, critique, revise – organizes teams around development stages, bringing the right expertise into each phase.

“Instead of saying, ‘I’m a designer’ or ‘I’m a researcher,’ or ‘I’m a product manager,’ or ‘I’m an engineer,’ we say, ‘I’m in the draft mode,’” Ovetta adds. “Each skill set in that move brings what they need to get that draft ready for critiquing, right? And so it’s something that I give to organizations and teams to try to reimagine how they actually do their jobs.”

Ovetta Sampson is not arguing for less innovation with AI; instead, she’s arguing for a different definition of responsible innovation – one that embeds human consequences, executive accountability, and cross-functional collaboration into the product development process itself.


  • [03:10] Protecting the fragility of humanity. There are a lot of things us humans engage in, especially what I call the cognitive biases, that make engaging with machines and other automated systems that make it risky for us.  
  • [05:57] The H-E-R Framework. But what it really is, is there are five dimensions. There’s the cognitive, there’s the social, there’s cultural, there the physiological. And then the overall community risks that when humans engage with machines, that can happen.
  •  [10:08] LLMs built on ‘traumatized data sets.’ Generative AI has no moral code. It does not know truth or fact. And accuracy is not in its wheelhouse. In fact, it’s not in this training and it’s in its goals. So why when we type something into chat GPT, we expect truth back? I don’t know.
  • [13:24] Protecting my values as a creator. That’s where I really want to start, because I don’t want to be a part of that. I don’t want to part of designing something that harms people.
  • [14:00] I observed one reoccurring truth. Whatever is happening in the basement of a company starts in the C-suite. So if there is sexism, if there’s homophobia, if there racism, if there is bad culture, if it starts at the C-suite. Because the C-suite is the person who has the ultimate authority about what occurs in every floor of an organization.
  • [18:26] The D-C-R framework — draft, critique, revise. The D-C-R framework is something I created because I was trying to explain to designers and product and engineers how their processes would change when they’re designing for and with AI. Each one of these disciplines should go through that makes the handoffs more like a circular iterative.

Sean Murray [00:00:09] When we were researching guests for the podcast, one of the things that struck us about you is that the first thing that drew us to you was your focus on human interaction with AI versus AI tooling itself.

Ovetta Sampson [00:00:21] And so when I started building AI systems, I was focused on talking to humans about what they actually needed, like what problems they wanted to solve. I started creating frameworks and writing concept papers and actually doing work around trying to operationalize ethical and responsible engagement activities around. Around. The fragility of humanity. There are a lot of things us humans engage in, especially what I call the cognitive biases that make engaging with machines and other automated systems that make it risky for us. And so we expect that this amazing entity trained on all of humanity to be accurate, right? And I’m famous for saying that generative AI has no moral code. It does not know truth or fact, and an accuracy is not in its wheelhouse. So why, when we type something into chat GPT, we expect truth back? I don’t know. And so that’s what I mean when I talk about the fragility of humanity. There are real dangerous risks. When we engage with machines and don’t mindfully think about the outcomes that can happen when we don’t protect humans psychologically, cognitively, physically, and physiologically, and socially and culturally. And this comes from my grounding as a human-centered designer.

Sean Murray [00:02:14] All right. We are here live at the ITX Product + Design Conference here at the Rochester Riverside Convention Center. Our guest today is Ovetta Sampson. She was recently named one of the top 15 people in enterprise AI by Business Insider. She’s the founder of Right AI and she has some awesome frameworks, the H-E-R and the D-C-R framework that we’re going to hear about today. Ovetta, thank you so much for being here.

Ovetta Sampson [00:02:34] Thank you so much for inviting me and are you going to introduce yourself or you’re just gonna

Dan Sharp [00:02:40] I mean, I’m normally here. I’m Dan. I co-host this show, but we’re really pimped to have you here. All right. Great. All right, great.

Ovetta Sampson [00:02:45] OK, wonderful. I just want to know who makes sure I get everybody’s name right. Yeah.

Sean Murray [00:02:51] People have names. It’s important, right? When we were, when we were researching guests for the podcast, one of the things that struck us about you is that the first thing that drew us to you was your focus on humor and interaction with AI versus AI tooling itself. Yeah. And you were one of the earliest people to go down that path. Tell us why you did that.

Ovetta Sampson [00:03:10] Yeah, I think it came from kind of like, when I started building AI systems, I was a design researcher, which I have a background in journalism and in the kind of humanist realm, anthropology, things like that. And so when I start building AI Systems, I focused on talking to humans about what they actually needed, like what problems they wanted to solve. And so I saw that a lot of engineers that I was working with and other technologists weren’t really understanding the human needs that come with automation. And I started creating frameworks and writing concept papers and actually doing work around trying to operationalize ethical and responsible engagement activities around. The fragility of humanity. There are a lot of things us humans engage in, especially what I call the cognitive biases that make engaging with machines and other automated systems that make it risky for us. And when I talk about that, I’m talking about the human engagement risks and that’s the framework that I created about 10 years ago. And that’s why I started trying to help technologists and engineers understand the human engagement risks of model outcomes. And then once I started doing that, I started building product design around those types of watch outs, cognitive watch outs physical watch outs. There are just things that happen when humans engage with machines that you need to be aware of so that you don’t actually harm the people you’re trying to. With automation. But then generative AI came out, and then I started thinking, hey, you know those cognitive watch outs that you do in product design? You need to do that when you engage with AI tools as well. So now I’ve kind of expanded my work to work with teams on how to build AI products without harming your employees like yourself and your customers.

Sean Murray [00:05:30] So we’ve done a lot of podcasts where we’ve talked about AI, but I think this is definitely the first time where someone’s referenced the fragility of mankind. That was the same comment. So we’re deep. We’re deep in, Dan.

Dan Sharp [00:05:43] Yeah, I mean, so definitely come back to the ethical piece and the fragility of humanity. Yeah, you talked a little bit about her framework of, I live here, I just like, what are some of those risks that people are listening want to bring something home?

Ovetta Sampson [00:05:57] Yeah, yeah, so the human engagement risk, and I came up with this, I don’t do frameworks or anything, I’m not a marketer or a salesperson, but I was on a panel when I was a VP of ML and AI platform design at Capital One. And so I was in a financial regulation panel with some regulators from the state government. And so, I was talking to one of my panelists about kind of the work that I was doing, the research that I’m doing, and the influence in product design. And I said, yeah, there are these very specific human engagement risks that product designers really need to look out for. And he said, “HER,” you need to trademark that. And I was like, what? He’s like, that’s a framework. So I was, like, okay, it’s a framework. But what it really is, is there are five dimensions. There’s the cognitive, there’s the social, there’s cultural, there the physiological. And then the overall community risks that when humans engage with machines, that can happen. And so for example, one of the things that I kind of trademarked or said as well that he said as a trademark at that conference was that a lot of models are built with what I call traumatized data sets. And so people said, what is that? And then I said that everything that we do ends up in code. So who we are as a society ends up code in our technology. One of the things that happens is the trauma that we have that kind of undergird the isms of the racism, the sexism, the homophobia. These all end up in our technology. And one of the books that I first read when I started in 2016 was called Weapons of Math Destruction. And this was by Cathy O’Neill. She worked as a data scientist and an analyst. At some of the biggest banks and other financial institutions. And she was seeing like in mortgages, lending, and other types of models that were being made that a lot of the biases against women, single women, when you fill out a mortgage application, they ask if you’re married. A lot of these biases end up in models. And so when models make decisions, they’re making it with traumatized data sets. And one of the biggest risks of models is the sin of omission. So if you think about seat belts, seat belts were built on a man, 5’9″, average man, average height 160, was not, and white. We didn’t have the first crash test dummies that were female until the 2000s. People who identify as in the LGBTQ community were not included in the oldest demographic data collection that we have today, which is the U.S. Census, until the 2021. So if you think about grants that were given based upon demographics, based upon these models, statistical models of populations, LGBTQ communities were left out. So, if you’re not in the data-sys… That are used to create these models, then do you actually exist, right? And so that’s what I started talking about, the biases that can show up in our technology outcomes, because we start with traumatized data set and how to de-traumatize them using the human engagement risk frameworks.

Dan Sharp [00:09:29] And that’s, that’s tremendous. Like you type, you type something into, you know, Gemini or Claude and you’re expecting it to come out with, you know something, you’re honest, right? But it’s got all of humanity’s baggage paid this way, right?

Ovetta Sampson [00:09:41] And definitely, because machines do, one of the things that I’m famous for saying is that genitive AI models are like, if anyone has kids and you take your toddler and you sit in front of the internet and you go away for a couple of weeks and then you come back, would you release your toddler onto the world? That’s exactly what happens with large language models, right? They’re trained on the whole of the Internet, which includes Reddit.

Dan Sharp [00:10:07] Bad news.

Ovetta Sampson [00:10:08] Which includes, you know, discuss, which includes a lot of different things. Yes, it includes the library, but it also includes everything else, right? And so we expect that this amazing entity trained on all of humanity, uh, to be accurate. Right. Right? And I’m famous for saying that generative mob, generative AI has no moral code. It does not know truth or fact. And an accuracy is not in its wheelhouse. And in fact, it’s not in this training and it’s in its goals. So why when we type something into chat GPT, we expect truth back? I don’t know. Again, this is the fragility of humans because we anthropomorphize everything. How many people have bikes? What are their names? Yeah, right? Mine’s Sheila, right. We anthropomorphize anything that’s part of our fragility, right, so if you’re building a product design, I’m not gonna mention some names, but they’re in broad and lawsuits right now, character AI. If you’re building’ a product designed that uses avatars to engage with teenagers, and then those avatars come from games of thrones or other types of fiction, And then a teenager starts talking to this chat bot that takes on this personality and starts exhibiting human-like qualities like, I love you, like we should be connected. And then the teenager says, I don’t wanna live in this world anymore. And then, the chat bot says, yes, that’s fine, right? And then… The unfortunately the 15-year-old commits suicide, right? And so that’s what I mean when I talk about the fragility of humanity. And I know it sounds like I’m being lofty, but I’m real. There are real dangerous risk when we engage with machines and don’t mindfully think about the outcomes that can happen when we don’t protect humans psychologically, cognitively, physically, and physiologically. And social and culturally. And this comes from my grounding as a human-centered designer. If you think about the ISO definition of human-centered designer, everybody always talks about the Don Norman part of getting requirements and understanding human needs and all that, and that’s great. But that second part of the ISO definition of human centered designer says, I do not create products that harm humans. And that’s where my human engagement risk framework comes from because I’m a human serum designer. And I cannot consciously create a product that in the end results will be harm and healings. And I don’t care if it’s 1%, right? People talk about people engaging with chat bots and a million people do it and a billion people do, and we only have about 10 or 15 of you who are harmed. One person engaging with a machine and getting a harmful outcome is one person too much.

Sean Murray [00:13:15] Okay, so I love the stance that you’re coming from right you’re protecting the world. I love it.

Ovetta Sampson [00:13:20] Well

Sean Murray [00:13:21] That’s it, just the world.

Ovetta Sampson [00:13:24] I’m trying to protect my own values as a creator. That’s where I really want to start. Because I don’t want to be a part of that. I don’t want to part of designing something that harms people. And I know that people who are designers and researchers and humanists and human-centered designers don’t either. And so that’s why I created Right AI. Bye.

Sean Murray [00:13:45] So actually, I wanted to ask you about Right AI. One of the things, one of the initial things that I mentioned about you is you are a founder of Right AI. And when we look up Right AI, one of first things that it mentions is that the target of Right AI is initially the C-suite executives. Yeah, so why did we start there?

Ovetta Sampson [00:14:00] Yeah, because when I first started in AI, I was a consultant and I worked at IDEO. And so I kind of got spoiled. When you are a consultant in IDEO and you walk in and you tell a CEO like what you’re doing is shit, they listen, right? Most consultants or most people who work at their companies, they tell their CEO that what they’re doing and shit, they don’t get heard, right. And so, I felt. After working with some of the biggest companies in the world, billion-dollar valuations, over and over again, I was seeing one reoccurring truth, and that whatever is happening in the basement of a company starts in the C-suite. So if there is sexism, if there’s homophobia, if there racism, if is bad culture, if it starts at the C suite. Because the C-suite is the person who has the ultimate authority about what occurs in every floor of an organization. And so when I started my company, I felt that the things that I knew I was going to have to tell enterprises to do were revolutionary. They would require leadership and change. And I want to start by trying to be Sisyphus and push a, a boulder up a hill. So I just started at the top. So that’s why a lot of my consulting is in the C-suite. Because once the C-suite gets it, and I switch from ethical and responsible AI, I still do that, but I talk a lot about risk. Once the C-Suite understands the risk to their shareholders, to their products, to their employees, to their customers, it is much easier for me. To bring in the implementation of how to mitigate those risks.

Sean Murray [00:15:58] What are some of the biggest changes you’re seeing from the C-suite level?

Ovetta Sampson [00:16:02] One of the biggest changes that I’m seeing for the C-suite level is hesitation. So a lot of CEOs, their job is to make decisions, right? And I worked for a CEO named Steve Bigery who taught me more in 18 months than any Harvard MBA could. He owned a bunch of McDonald’s in Colorado Springs. And one of the things he would always talk about is burning the boats. So as a CEO leader, he burns the boats. He says, I’m, this is the pathway we’re going. I’m making this decision and I’m gonna burn the boat so we can’t go back, right? That is the type of kind of leadership and decision-making that happens in the C-suites. So when I started talking to, and I work mostly with CEOs and annual revenue from $5 million to like $150 million. So right in that small mid business. When I was talking to CEOs I was seeing a lot of hesitancy on adopting AI. And the reason why is they were really worried about intellectual property, they were worried about trust, and they were worrying about data leaks. And so it was something counterintuitive to what you see in the narrative on LinkedIn about everybody adopting AI, but these mid-size enterprises were very, to me, counterintuitive of not making decisions about AI. They didn’t have AI strategy. They hadn’t decided what they wanted to do over law. Their employees were, there was high use of tools, but there wasn’t this kind of like overarching enterprise strategy roadmap for AI. And that’s where I saw kind of my niche and a good opportunity for somebody who had worked at large tech companies who built their AI strategy years ago. To help mid-size strategies make really good decisions and not the same mistakes that those companies did.

Dan Sharp [00:18:01] You said you’re not a framework person. Well, you have a second framework other than her, D-C-R.

Ovetta Sampson [00:18:07] I have a bunch now. Thanks to Claude. No, it’s me.

Dan Sharp [00:18:10] Oh, yeah, and Sean, Sean, yeah Sean and I…

Ovetta Sampson [00:18:13] Claude is really great at frameworks, by the way. They might not mean anything, but they look really good.

Dan Sharp [00:18:19] Right, but yeah, Bulleteers. Yeah!

Ovetta Sampson [00:18:20] Yeah, very bullet-header, right, bolt-headers, they’re really great at that.

Dan Sharp [00:18:24] Yeah, tell us about the DCR framework.

Ovetta Sampson [00:18:26] Yeah, so the DCR framework is something I kind of created because I was trying to explain to designers and product and engineers how their processes would change when you’re designing for and with AI. And I would say about five or six years ago, I started doing presentations on what design looks like in an automated era. And I was talking about the collapsing of disciplines and silos. That AI is a horizontal technology, and the way that a lot of companies are set up, they’re set up in these disciplined verticals, and there’s a lot these handoffs. Even our tools, like FIGMA, you know, I do the wire frame, and then I hand that off to the engineer, and then they write the code, and then, they hand that off to production engineer in there, or ML engineer in production, and then it scales, you now, with a little stop at quality, insurance, and security along the way. So it was always these handoffs where what I would see when I would work with teams and worked at organizations but also consult with design leaders where there was no mixing of the recipe, right? The designers did their process and the engineers did their processes and security did its process and the legal and compliance did its processes and everybody was kind of like stuck trying to figure out where the handoff alignment would be. And I knew that this kind of setup was going to run into major problems with artificial intelligence because when you build models, it crosses multiple organizations. You have to have data. You have the engineering to do it into productions. You have data engineers and the analysts to say which data is best. You have a data scientist to help build the statistical models. You have to have the engineers to put. All of those models that the data scientists did and rebuild them so that they can be put into production. Then you have to have security to do model and then you have legal and compliance to say, okay, how does this fit with the EU act? And how does, especially if you’re global. So there were all these parts to the model and I really found this out when I worked at Google. There were all of these parts to the Model development process that had nothing to do with engineering and had everything to do with every other organization out there. And so, when I was at Google, I was highly frustrated saying we couldn’t really break down the silo barriers to get models out the door safely and faster if we didn’t break down these discipline silos. And so the DCR framework came from that. It’s a framework that each one of these disciplines should go through that makes the handoffs more like a circular iterative. Process that it should be with AI. And it stands for draft, critique, revise. And it’s something that, it doesn’t matter your discipline, that when you’re doing it, you can create this continuity of development around drafting, critiquing, and revising. And instead of saying I’m a designer, or I’m researcher, or I am a product manager, or I’m an engineer, you’re saying I am in the draft mode. And each acumen in that move brings what they need to get that draft ready for critiquing, right? And so it’s something that I give to organizations and teams to try to reimagine how they actually do their jobs and do it. And when I was at Google, one of the first things I did and probably why they asked me to leave is because I reconfigured my entire team around the model development process and the things that developers were doing and needed to do to get a model out the door instead of products. I was like, we’re no longer in the product design field, we are in the AI experience design field. And if you’re going to design an AI experience, then the business, then the discipline acumen has to come in the phase, right phase of the process of model development. Instead of which product or tool that you’re designing, which is, to be quite honest, very radical, right? I basically told the SVP, I’m going to redesign my whole team, and it’s not gonna be about products, which means I had to go to Google engineers and say, we don’t support your product, we support this experience, right. That lasted for a while, but now they’re designed like that. So it’s the story of my life.

Sean Murray [00:23:20] We are so excited to have you here at the conference. Thank you for giving us a little bit more of your time prior to you being on stage. We’re really excited for your keynote talking about AI first. Thank you so much for being with us today.

Ovetta Sampson [00:23:30] Yes, thank you for inviting me. This flew by fast.

Sean Murray [00:23:34] We did it.

Ovetta Sampson [00:23:35] All right. That’s it. We did it. And thanks to you guys.

Like what you see? Let’s talk now.

Reach Out