Alexis Zheng is Chief Product Officer at G2 and brings 20 years of experience in building AI systems. In this episode of Product Momentum – recorded in person at Mind The Product – Chicago – Alexis explains how product teams can build trustworthy AI agents without sacrificing their potential. Her message is clear: if we expect our agents to safely meet user needs and business objectives, we need to make deliberate decisions around defining success, establishing boundaries, and monitoring results.

These responsibilities extend beyond engineering. Product managers and UX designers need to join their Developer counterparts to evaluate performance, draw a balance between autonomy and control, and deeply embed human judgment in the overarching product strategy.

Here’s more of what we learned:

Trust but Verify: Measuring AI Performance with Continuous Evaluation

Trustworthy AI agents require more than promising outputs, Alexis says. Product teams need measurable criteria against which we can determine whether an agent achieves its intended objectives. Alexis itemizes three essential capabilities: evaluation metrics, guardrails, and observability.

She explains: metrics establish what success looks like; guardrails prevent inappropriate actions; and observability helps teams monitor performance after deployment. “You have to design metrics so you know if the system is doing the right thing,” she continues.

“Building the Harness”: Designing Agents for Flexibility and Control

Evaluating AI performance is only one part of the challenge, Alexis adds; controlling agent behavior is another. Unlike traditional software, AI agents can produce different outcomes from similar inputs. Restricting them too tightly limits their capabilities, while allowing too much freedom often introduces unintended consequences. Alexis describes the infrastructure and practices needed to manage this tension as building the harness.

“You have to find a way to let the model or let the agent be free, but at the same time also manage the unintended outcomes,” she says. “For those of us on product teams, this means making deliberate design decisions about where agents can operate independently and where constraints become necessary.”

Apply Human Judgment to Enhance AI’s ‘Average Intelligence’

Even well-designed, carefully evaluated AI agents can’t replace the human judgment that shapes successful products. Teams using similar AI models often arrive at alarmingly similar solutions – what Alexis refers to as average intelligence – so product managers need to inject their own creativity, customer understanding, and strategic perspective to the development process. Rather than blindly allowing AI to dictate product decisions, PdMs should use it to advance a distinct vision.

“To understand how to leverage AI, we need to put ourselves in a position that we are doing the steering, that we are using AI to accomplish our vision.”

• [06:30] How do you know that your AI agents are doing the right thing? Or that they’re delivering the desired outcome? That is what the Eval is trying to do, is to really understand what the agents are doing in production and to make sure they’re achieving the outcome as humans who actually want them to achieve; otherwise you may not end up with what you asked them to do.
• [12:30] Verifying the verifier. And that is really hard. That’s why today you have expert in the loop, which isn’t very common if you talk about it like RLHF (reinforcement learning from human feedback), to have a human expert to look at the result.
• [18:12] AI stands for artificial intelligence, but it also stands for average intelligence. The products we build today seem to look like our competitors’ products. We all seem to be regressing to the mean. The human element is what distinguishes us from all the rest.
• [20:46] Be intentional in how you deploy AI tools; don’t simply defer to the tool’s solutions. Agents are doing the work, but we humans still need to do the steering; otherwise, you don’t know what the agents are going to end up with.
• [22:19] Deterministic roadmaps vs. Probabilistic roadmaps. As we move through more probabilistic roadmaps, we begin to narrow the field of possible selections – this is known as converging. The key to AI is this: how fast we are converging?

Podcast: Play in new window | Download

Sean Murray [00:00:09] So I want to start with you talked about something in your talk about the agent trust layer. So what is the agent trusts layer and why isn’t it important?

Alexis Zheng [00:00:19] Yeah, so in Q&A we also talk about silos. So today you have silos, human working with humans. You also have silos human working was agent. And the agent many times work in a black box, more or less. So how do you know agents are doing the right thing? And how do they’re getting you the desired outcome? And it has always been a question for ever since AI existed, is how do these are the objective function for the agent?

Dan Sharp [00:00:47] How do we build that kind of box around something that’s probabilistic, right? Like an autopilot, it’s still just, it’s following like a defined workflow, but it seems way more complex when it’s a probabilistic system.

Alexis Zheng [00:01:01] Yeah, absolutely. I think there is verification. As well, we said you have to design metrics so you know if the system’s doing the right thing. You also design guardrails, which people talk a lot about today, which is like, how do you make sure the wrong action gets blocked? And actually it’s very interesting in early days of AI, like what we did in building the guardrail. And the last piece people don’t think often about is observability. So after a system that’s deployed, you want to constantly track if it’s doing the right thing.

Sean Murray [00:01:30] How, as the systems get more complex, how do the people build the fund and maintain the fundamental knowledge and skills to be able to evaluate the machine, right? How do we make sure, as product people, we’re maintaining that knowledge?

Alexis Zheng [00:01:44] Yeah, and it’s more complex because you write the evils, which is humans trying to use metrics to know what the machine is doing. The force of verifying the verifier. And that is really hard. That’s why today you have expert in the loop, which isn’t very common if you talk about it, like ROHF and things like that, to have a human expert to look at the result. Now here’s something harder. There are things that don’t have ground truth.

Sean Murray [00:02:05] You mentioned something also in your talk that I thought was unique because you used a unique term that I haven’t heard a lot about so Can you tell us what what building the harness is?

Alexis Zheng [00:02:15] Yeah, it’s basically how do you make sure the system works at scale? So I think that’s the problem, and that’s people’s challenge today. As product managers, we used to build deterministic systems. I have a set roadmap. I know exactly what I want to accomplish. I know how the system behaves. I know what the user’s care is gonna be. Now with AI, it is probabilistic. And it’s very difficult. I see people kind of err on two sides. One is they just unleash the probabilistic nature of agents. And you’re just like surprised what you get and that may not always be a good surprise.

Sean Murray [00:02:56] Dan, we just got done talking to Alexis Zheng from G2. I’m like winded from that one. I don’t know about you, but like that was one of the, like we’ve made a point about having AI experts on, but this was the first like person that was like in AI from like the ground floor up, right? She’s got 20 years of experience. What did you take away?

Dan Sharp [00:03:17] Well, first I took away that 20 years takes us back to the early 2000s and not to 1985. So that’s made me feel a little bad about myself. I love the framing and the other talking about trust and how we put the appropriate constraints upon agent solutions. Because they’re so complex, things are moving so fast is so much more important that we’re bounding them the right way. Really awesome conversation, like a lot to think about coming out of it.

Sean Murray [00:03:44] Yeah, take a deep breath and get ready for this one. Let’s do it. We are here today with Alexis Zheng. She is the Chief Product Officer at G2, the world’s largest software marketplace, and you’ve spent like an entire career building AI systems. You just got off the stage after your keynote and you did the Q&A afterwards, so this is like the third part of your media tour for today. Thank you so much for being with us.

Alexis Zheng [00:04:16] It’s always fun to talk about AI.

Sean Murray [00:04:18] Right. So.

Sean Murray [00:04:19] One thing that was an interesting contrast between your keynote and some of the other ones that we’ve heard today is you are at like, you are talking about things at like the cutting edge, right? And so when you are up on stage talking about these ideas, do you feel like you’ve got to find some way to connect with people that may not be like at this level of like, okay, not everyone has a career building AI systems that can say that now, right? So how do you find a way to like connect with like? People that aren’t there yet.

Alexis Zheng [00:04:52] Um, yeah, I think that’s always like, sometimes I feel I’m living in my head a little bit and you definitely like see a different conversation when you talk to people who’s building on that kind of edge and you’re talking to people who’s like still learning and still getting ramped up, but I think that gap is going to narrow over time. Like I actually see, you know, the most impressive are interns. Actually, there’s a thing we have right now at G2 we call AI interns. These are, you now, we hire people from college, fresh out of college. Or even high school grads, and who’s doing incredible work on AI. Like, we do these evals that I just talked about. It used to take our engineering team, like, you know, three days to run, like, a run of evals. And we had this AI intern, who was an incoming Stanford freshman, and Ethan, if you’re listening to the podcast, did this in, like, eight hours or something for the first time. So I think, increasingly, we’re gonna see a new generation of technology workers Who’s just like native to AI, and that’s going to close the gap even more.

Sean Murray [00:05:54] I was thinking about when you were talking, I was like, she’s definitely talking to the 0.5, 0.05% of people that Martin was talking about that have like actually like that are actually working with these tools. So this essentially what I’m framing is this podcast is going to be an advanced course for everybody. Right. So there might, you might need like a dictionary or have Google open while you’re listening to this so that you can look up, unless you’re driving, unless you’re driving, right. So I want to start with you talked about something in your pot in your about the agent trust layer. So what is the agent trusts layer and why is it important?

Alexis Zheng [00:06:30] Yeah, so in the Q&A, we’ll also talk about like silos. So today you have silos, human working with humans. You also have silos. Human working with agent. And the agent many times work in a black box, more or less. So how do you know agents are doing the right thing? And how do they’re getting you the desired outcome? And it has always been a question for like, you know, ever since AI existed. It’s how do these are the objective function for the agent? And there’s like multi-objective optimization. There’s also reward hacking, which probably heard about. And I heard somebody else talking about this and I thought it was very smart. And they said, if you think about your robotic cleaner, if you tell the robotic cleaner to like clean this room, what they’re going to do is they’re gonna push the trash in the next room. So that is what Eval is trying to do, is to really understand what the agents are doing in production and to make sure they’re achieving the outcome as humans who actually want them to achieve, because otherwise the agent will take the letter of instruction literally. And you may not end up with what you asked them to do. And you have to do that with metrics. And this is like kind of age-old thing in AI, you always write a matrix, and then you write an algorithm to make sure it’s doing the right thing. And that is becoming more and more important. And it’s actually not that easy to write a metrics. Is it a force you, number one, to ask yourself, what do I really want to achieve? Now, how many times you don’t really know when you set out to build a product or something like that. And then you also need to be able to express that in a matrix or in an algorithm. And then you also need to like, you know, once you see what actually happens, you need to be able to adjust it. You have to ask the question, do I have to write lubric? Do I have the right email data as well? And you know it’s the agent behaving as it’s expected. So like that’s the beauty of like writing the trust layer.

Sean Murray [00:08:13] Okay, and so you influence that trust through evals.

Alexis Zheng [00:08:16] You, I think it’s less about, you, you do influence. I think you were right. You do the evaluation. So, you know, if the agent accomplished the right thing, and then you also influence, you just do the steering. This is actually another interesting topic today. I hear people talk more about rolling versus steering. So we’re doing less and less of the rolling. So agents are doing the work, but you still need to steer because otherwise you don’t know where the agents are going to end up. So the E will also help you to kind of like set up the steering It’s like once you do the eval, it becomes kind of the objective function. This is what I try to achieve. And then you’re able to do better steer.

Dan Sharp [00:08:54] So, while you’re talking, I was thinking about, you know, one of the troubles that people face when like managing highly complex automated systems is, is the complexity building system. Like, I’m thinking, bear with me for our side quest here, but there’s a really famous Air France disaster where the autopilot was throwing a specific alarm, right? And they misinterpreted what the failure was and handled entirely wrong. Yep. How? Do we build that kind of box around something that’s probabilistic, right? Like an autopilot, it’s still just, it following like a defined workflow, but it seems way more complex when, you know, when it’s a probabilistic system.

Alexis Zheng [00:09:33] Yeah, absolutely. I think there is verification is what we said. You have to design metrics so you know if the system is doing the right thing. You also design guardrails, which people talk a lot about today, which is like, how do you make sure the wrong action gets blocked? And actually, it’s very interesting in early days of AI, like what we did in building the guardrailes. So in self-driving car, for example, early days, we would block off part of the map and just don’t allow the cars to go there. Because we know they have high risks in those areas or historically the car used to make a lot of mistakes. Or you make sure the car really, really respect school busses. You know, every time the car sees something yellow, it’s like literally going to pause. It actually creates hilarious problems. I don’t know if you know that in Austin, people have party busses that exactly looks like school bus, but it’s painted black. Have you ever seen that? And when we were deploying self-driving car in Austin- we used to cause traffic jams because we see a party bus and then we just stop. We’re like, oh my god, that looks like a school bus. So that’s kind of early days, how you build the guardrail. And now it’s getting smarter and smarter, of course. And the last piece people don’t think often about is observability. So after a system that’s deployed, you want to constantly track if it’s doing the right thing. And actually talking about aviation disasters, there’s another interesting one. It’s called Datenberg, if I remember correctly. I don’t know if you know. It was early days. With airlines where you actually could have two planes at the same level of altitude. Today you can no longer do that. I think they have to be a thousand feet apart or something, but you know.

Sean Murray [00:11:11] We have a resident pilot. He’ll fact-track it later.

Alexis Zheng [00:11:13] Yeah, yeah, yeah. Exactly, exactly. And what happened was there were two planes and one the pilot was trained in kind of Eastern European style. So they trust the people more than the machine. They have another plane that was I think from Spain or something and they were training to respect the machine more than human. And that was the early days of a system called ACOS. This is where like the machine will coordinate and the plane was at the same level. And then the Eastern European pilot. Listening to humans who caught it last minute and issued the opposite direction with the machine. And then the Western European one followed the machine and it just became a disaster and two planes hit each other. So that’s the beauty of trust. It’s like, when do we start to trust the machine direction versus the human direction? What happens when they’re different?

Sean Murray [00:12:06] Yeah, that’s wild.

Dan Sharp [00:12:07] Yeah, it’s interesting too, right? How, as the systems get more complex, how do the people build the fund and maintain the fundamental knowledge and skills to be able to evaluate the machine, right, as opposed to just trusting that even a GPT response is correct without having the fundamental knowledges? How do we make sure, as product people, we’re maintaining that knowledge?

Alexis Zheng [00:12:30] Yeah. And it’s more complex because you write the evals, which is humans trying to use metrics to know what the machine is doing. The host of verifying the verifier. And that is really hard. That’s why today you have expert in the loop, which isn’t very common if you talk about it, like ROHF and things like that. So you have a human expert to look at the result. Now here’s something harder. There are things that don’t have ground truths. So we’re talking about evaluating sales agent. So let’s say I write an outreach email. And is the outreach male, good or bad. And what’s the ground truth? Is the other person, would they have responded to it, right? And how do you know they would have responded without sending out the email? You can do pattern recognition. You kind of look at all the successful emails and you see what is the pattern and have expert to score it. But you may never know, you know, like whether it would have achieved the outcome. Actually happens in self-driving cars, which is super interesting. You actually very rarely have an on-road collision in self driving car. If you do, that’s a big problem. So you have near miss and heartbreaks. And the question you ask is, if I didn’t break, would that have been a collision? And that is a really, really hard question to answer. It’s nice that we have like, I don’t know, 20 safety engineers in a room to debate that very question.

Dan Sharp [00:13:43] They have plenty of different responses to it.

Alexis Zheng [00:13:45] I have 20 different responses.

Dan Sharp [00:13:47] And they’re all 100% certain about it.

Alexis Zheng [00:13:48] Exactly, exactly. I’m always confident with my driving, but if I were in a car with me, some of those things don’t have a ground truth.

Sean Murray [00:13:56] One of the things that I like about that you’re describing is like, in order to have, in order to really build that trust and excel in this, in designing agentic systems, you actually need to be aware of the metrics that you are tracking, right? And you need to know, you need have some idea going into that and they’ll evolve over time, right. One of things that, I mean, asked to do more often by clients is we have a, we have a product metrics, we like a success metrics workshop and It’s almost like in this age of we can build so fast that we are literally like goldfish the second it like hits the market, right? Yeah. So like, there’s no part of the go-to-market strategy that it’s like, all right, after 30, 60, 90 days, what are we actually measuring against? And like, was this investment worth it? And where do we need to trip? We’re like, you can’t build an agentic system without that sort of plan going into it. So it’s almost a discipline that I can, that I can tie back to something that other clients are asking for.

Alexis Zheng [00:14:45] I love that.

Sean Murray [00:14:47] Um, you mentioned something also in your talk that I thought was unique because you used a unique term that I haven’t heard a lot about. So can you tell us what, what building the harness is?

Alexis Zheng [00:14:56] Yeah, it’s basically, how do you make sure the system works at scale? So I think that’s the problem and that’s people’s challenge today. As product managers, we used to build deterministic systems. I have a set roadmap. I know exactly what I want to accomplish. I know how the system behaves. I know what the user is going to be. Now with AI, it is probabilistic and it’s very difficult. I see people kind of err on two sides. One is they just unleash the probabilistic nature of agents. And you’re just surprised what you get and that may not always be a good surprise. One is you try to put a deterministic workflow on a probabilistic model that’s also not good because you don’t unleash the real power of the model underneath. So you have to find a way to let the model or let the agent be free, but at the same time also manage the unintended outcomes. I have a couple of examples. This is early days when we did job recommendations. So if I… And when you look at something simple like a recommendation, that usually has F1 score, which means how much precision do I get for how much recall? So precision is how accurate, recall is how much coverage, right? And if I send a job recommendation to someone fresh out of school, if you think about it, the recall is more important than precision. I mean, when I was fresh out college, I did everything I should want to be a tour guide, which I never got a chance to. But I would be equally happy if you send me a banking analyst job or tour guide job. But he’s visiting someone very senior. Then your position is more important to recall, right? Because they’re not looking to change job all the time. And if you send them the wrong one, they’ll get offended and you lose trust. Here’s another example. Fraud detection was probably the earliest AI amount algorithm that really took off. And in fraud, I use the joke as a problem between dolphins and sharks. And let me explain it. If you make the algorithm really, really strict, you’re gonna catch a lot of bad actors, but you’re also going to get good actors, the dolphins. Stuck in the net. And if you make the algorithm more relaxed, you’re going to save all the dolphins’ experience, but you’re probably gonna let the sharks swim to shore. During the pandemic, I actually downloaded a grocery app. And for the first time, I was using a grocery. Everybody had that experience. And the first thing I bought was like $300 or something crazy, because I just like had bought groceries for a while. And I was a flag as a fraud. They blocked my account, and eventually I deleted the app. And that was a very hard question to answer because I feel for whoever was the product manager. Is it, you know, there’s, I was a new buyer, like they don’t have enough data on me and a large amount of dubs have certain characteristic of a fraud. So how should they make that decision? Because it’s not just, you now, in my case, I was the dolphin, but like does that generalize and does that apply, you to others? And that’s the hardest thing of managing probabilistic systems is how could you scale the decision making?

Dan Sharp [00:17:52] You’ve been 20 years in AI, which is like 18, more than I’m aware of. Yeah, I’m curious, you know, kind of in that journey, how kind of like, you’ll what stayed the same verse, you know what is it, what is changing and what are you seeing now? Like kind of just how has it evolved in yeah, in less than six hours.

Alexis Zheng [00:18:12] Yeah, I think there are a lot of like the first principle problem solving are still the same. Like the system is still, you know, largely probabilistic. You still need to really understand your objective function so you can steer the system to where you want. And a lot the ways, like the, you, know, agents, AI, machine learning, whatever you call it, would make mistakes is still very similar. I think what’s changed is things are just updating so fast and it can do so much more than what it used to be. I remember working on chatbot actually back in 2012 that really dates me. And back then I used to score my answer against whatever the chatbots answer as you kind of compare. And the accuracy used to like 60%. If I got a 70% I was like, wow, that’s a good day. And today, you know, like the answers are so nice. Like. Not smarter than human answers. That being said, I think there’s an interesting phenomena. I heard somebody else talking about this. They call AI average intelligence. That’s very interesting because AI is the most popular answer in a way that is average. And our product today are looking more and more like our competitors product. The industry sometimes are having a little, you know, coverage to the mean. And did you know very interestingly, purple is the common color of a cloud UI. Excuse me. Do something. Have you noticed that? It’s usually purple or otherwise it’s blue. It’s either purple or blue. So that must have come from the training data. I think I still wrote a blog about this that I was going to read. I haven’t read it yet. So excuse me if I didn’t quote all the right facts.

Sean Murray [00:19:52] We appreciate the honesty.

Alexis Zheng [00:19:54] But most likely, you know, at some point in time, most of our UIs are purple, or that’s just like what the model associated with like modern UI UX. So then everybody’s UI becomes purple. Then where’s the creativity?

Sean Murray [00:20:09] Yeah, we’re seeing that phenomenon of like, if you’re using AI for competitive research and market research, so are your competitors, right? And you’re gonna end up with the same type of product or the same kind of decision. We don’t want everything to kind of look the same. So how do we actually influence, how do you actually influence our decisions with what we know about our customers? We still have to think, apparently. Yeah, unfortunately, yeah, unfortunately we’re stuck there. So Dan talked about what’s changed, where do you see the AI market going? What should people be preparing for if you are in the bubble that Martin talked about?

Alexis Zheng [00:20:46] That is a great question. I think AI is more and more going to be status cool these days. It’s just like kind of internet. Now I’m old enough, I’ve like seen the internet revolution, then the mobile revolution, and now it’s the AI revolution. I think it’s not a cool thing to talk about in a few years. This may be like, you know, just everybody’s like basic training. And who you work with is always just going to be a combination of like humans and agents. And you need to… I think that… You kind of need to put yourself in a position that you’re really intentionally steering, you know, versus you’re kind of like mindless using the AI in many ways. Have you heard the term like centaurs and reverse centaurs?

Sean Murray [00:21:29] Yeah

Alexis Zheng [00:21:29] This is very sci-fi, so centaurs is like, you know, human head with the horse as a body. It’s like human steering and AI is working. I think that’s probably like where I do want to be and reverse centaurs is just…

Dan Sharp [00:21:42] The horse is the top, yeah, that’s not the good one.

Alexis Zheng [00:21:46] And that’s probably not a good place to be. So I think for everybody, like the question is like, how do you kind of understand how to leverage AI, but put yourself in a position that you’re really steering, you’re getting AI to accomplish your vision, your creativity and where you want to be?

Dan Sharp [00:22:03] You notice how she didn’t say Skynet, so that was really reassuring.

Sean Murray [00:22:05] That’s good. Yeah. Because we talked about this being an advanced course, right? I want to take us back just a little bit. We’ve, you have both used terms like probabilistic and deterministic. What’s the difference? What are the, what does that, what do those mean and what’s, what are the difference.

Alexis Zheng [00:22:19] I think the best example would be like early AI when you have Deep Blue playing chess. So as humans, we’re kind of a, or I’m not a super good chess player. I think about it as like, here’s somebody else move, here is my next move. And AI doesn’t think this way. They think there are many, many possible moves, each with a probability of what the other side is going to do and each has a probability actually of winning the chess game. But that’s kind of how I think everything is probabilistic. Now, we talk about just now like a roadmap. I don’t think roadmap is going away. I think roadmap just become probabilistic and there is any moment infinite versions of that roadmap you just created with like different tweaks and change and each with different probability of accomplishing your goal. And as you learn, you get a little bit more data point and their probability changes and have a narrow selection, which we call converging. So I think that’s kind of like the key of AI today is how fast are you converging? So is that a flake? Going one set way of accomplishing something, now you have many, many ways that AI could create to accomplish something, but you still eventually need to converge to the most optimal answer. I think that’s also like the challenge today with product management. There are two ways to build products now. You can build by experimentation or you can build that design. And build that design is the old way. And we used to have to write A-B tests on very specific things. Now we can do A-B testing on the entire product. You can test all your good ideas, but all your bad ones too, as the cost is much smaller. Then what you create is create a lot of random walk. In AI we call it explore versus exploit. And the key is you have to exploit at some point. You cannot random walk all the time. Then you, I don’t know, end up somewhere, like you just can’t come back. And you create too many local maxima, and you never know where you want to really allocate your resource to hill climb. Sorry, hill climb is what I meant. And that is like the most challenging thing of probabilistic versus deterministic is because there are so many versions in probabilistic, you still need to converge and that’s the key.

Sean Murray [00:24:25] I still need to pick a direction.

Dan Sharp [00:24:26] I love that we just hit that like the probabilistic roadmap. Cause like that’s the most realistic, honest one of like, Hey, that thing that is next, like, but maybe it’s not right. Maybe we will learn something that says like, actually that’s not where we should be spending our resources.

Alexis Zheng [00:24:38] And you’ll create a step function change if you create the right converges. We just got a map.

Sean Murray [00:24:44] Yeah, we talk about like, you know, dishonest roadmaps are linear, right? Honest road maps are, you now, in the shape of the cone of uncertainty, but we want to get to strategic road maps that have like real decision points that we’ve had them mapped out. We may not know what all those decisions are, right, and it seems like those decisions might change day over day in the current world or the current environment that we’re living in, but that’s really where we need to get to. So.

Alexis Zheng [00:25:10] Yeah, the key is learning. AI learns from ground truth. At some point, you need to give it some ground truth.

Dan Sharp [00:25:15] and we need to keep learning too.

Alexis Zheng [00:25:17] Yeah, yeah, he was need to learn too, yeah.

Sean Murray [00:25:19] So this has been an amazing advanced course. Thank you so much for being here today. What’s next for you and what’s next with you too?

Alexis Zheng [00:25:25] Um, and we are working on, you know, the next generation of trust layer. The data has always built a trust layer for software and that has been revealed very much, which gives us great signals of what people enjoy, ease of use, kind of the taste. And now with AI, what we find is actually very important to ground people on the actual success rates with AI. So if you put an agent to the task, they are really going to accomplish the task, how are they going to accomplished tasks, are they going to follow policies? Are going to create unintended consequences. So if a customer call in, they say, hey, I want to pause the subscription. What if the AI cancels the subscription altogether? And that’s what we want to test out. That’s why we create these simulated companies, kind of like self-driving car with simulators. And you can actually drop agents in the simulator and see how they actually act. So that’s our vision for the future, and that’s where we’ve been working on. It’s actually quite a lot of fun, and it’s quite challenging too. There are all kinds of challenges with it.

Dan Sharp [00:26:25] Awesome.

Sean Murray [00:26:26] Thank you so much for being with us.

Alexis Zheng [00:26:27] It’s great to be here.