Vitaly Friedman is an author, the UX Lead with the European Parliament, and editor-in-chief of Smashing Magazine. On June 25, he will facilitate a workshop at ITX’s 2-day Product + Design Conference. In this episode of Product Momentum, Vitaly joins Sean and Dan to share his insights about tackling the unique challenges of UX design in enterprise-level environments.

Tune in to hear the following:

It’s OK to be a disruptor of legacy systems in enterprise organizations

“Legacy is like that ‘magical box’ that we’re afraid to touch,” Vitaly says. “No one really knows how it works, and the people who set it up decades ago are now gone and left with no proper documentation.”

But it’s OK to be a disruptor of legacy systems and the political hierarchies that created them as long as you show the value you can bring and approach your task with a risk management mindset. “Start with smaller projects; ask users and stakeholders to review your unfinished work; and always remember that your ‘disruption’ affects a lot of people who need that tool quite literally to do their work.”

Start small, build trust, and focus your vocabulary on the language of your stakeholders.

As part of your research, ask stakeholders to show you their product and to walk you through it. Listen carefully to the words they use to describe it – and how frequently those same words are used.

Says Vitaly: “You can get a better understanding of what’s important just by listening to how they talk about it and how much the features or workflows or ways of working mean to them as well.”

For humans, with humans, by humans.

Designing software is a people business, so Vitaly encourages UX professionals to “observe the relationships between stakeholders” so that you can learn about the people inside the organization who believe in and want to support UX initiatives. “There’s a good chance some UX projects benefited them and helped them be successful,” Vitaly adds.

When you find the people who deeply believe in UX, tell them the story about the project you’re working on. “Once you make them aware, they will move mountains for you and support your efforts,” he says.

Product + Design Workshop: How can UX designers welcome AI into their environment?

In his workshop at the ITX Product + Design Conference, Vitaly will introduce to attendees some of his recent experimental work.

“We designers expect a lot from AI, and we believe there’s a lot of things AI can do to help us build great products,” Vitaly says. “But the reality is that AI is really weird. We’re accustomed to software being rather predictable and reliable: if you click on a button, you get the result you’re expecting. But sometimes – like 10-15% of the time – AI can be unreliable.”

In the workshop, Vitaly will discuss “those little things” that are important to us designers – and make them better with assistance from AI.

ITX’s Product + Design Conference returns to Rochester, June 25-26. Check out the complete list of workshop facilitators and keynote speakers on our P+D Conference landing page!

Today’s episode was produced by Peter Sullivan. It was edited by Javier Miranda Nieto, with help from Ivan Chaparro. Contains original music by Zach Kane. Special thanks to Megan Lawson.

Podcast: Play in new window | Download

Sean Murray [00:00:09] Hey everyone, welcome to Product Momentum. I’m Sean Murray, Director of Product Management here at ITX.

Dan Sharp [00:00:14] And I’m Dan Sharp, a product manager at ITX. It’s our privilege to continue the product momentum journey. A platform that celebrates the thinkers and doers who are shaping the future of software development.

Sean Murray [00:00:25] Our goal is to introduce you to industry leaders, innovators, and practitioners, and bring you their insights, advice, and innovations. We talk with experts from product strategy, UX research and design, security, engineering, leadership, and more. We’ll discuss how their approaches to problems help empower teams and build trust. And we’ll explore trends, tools, and techniques that your team can use to deliver the software solutions your users need today. So tune in, stay curious, and join the conversation. You can catch us in

Dan Sharp [00:00:53] your favorite listening platform, or if you prefer the video experience, you can find all our latest episodes on the Product Momentum YouTube channel. And if you like what you’re hearing…

Sean Murray [00:01:02] Be sure to subscribe like comment and share that way we can be sure to provide you with the best podcast experience

Dan Sharp [00:01:13] Sean, we just had a tremendous conversation with Vitaly Friedman. He’s a awesome UX design leader, doing some great work in the enterprise space, as well as generating a ton with his Smashing brand. Talked a lot about how important it is to build relationships, especially when you’re trying to get a foothold in the enterprise space.

Sean Murray [00:01:35] I think staying in that enterprise space and talking about the mindset you need to have in the enterprise space is something that we really don’t talk about enough, right? We talk to thought leaders, we talk to CPOs, talk to people that own companies, but we don’t about what a lot of us do, which is work in this enterprise space. We’re trying to make changes to legacy software where we’re trying achieve new things while still kinda like holding on to the tasks that need to be completed, but trying to approach problems in a brand new way. Vitaly’s mindset could go a long way in our industry.

Dan Sharp [00:02:17] Our Product Momentum guest today is Vitaly Friedman. Vitaly joins us from Berlin, where he is a UX lead for the European Parliament and Smashing Magazine, which was founded in 2006. Smashing Magazine is a leading online magazine for designers and developers. He is the author, co-author and editor of Smashing Books and the curator for Smashing Conferences. Don’t worry, we’ll provide some links in the show notes for Smashing Conferences. But there’s more. Vitaly also is a front-end UX consultant in Europe and abroad, working with companies both large and small. But I think the most important thing about Vitaly is that he is going to be joining us in Rochester, New York, June 25th and 26th at the ITX Product and Design Conference, where he’s gonna lead a workshop on all things UX. We are super pumped to have you here, Vitaly.

Vitaly Friedman [00:03:07] Thank you so much for having me, Daniel. And I’m just listening all of that. It’s like, yeah, well, I just did a couple of things. I just do a couple things. Yeah, in Parliament, no big deal. So that was slightly more challenging than the others, I would say, yes. But overall, it’s been like, you know, everybody’s life, a little bit of this, a bit of that, and then before you know it, you just end up doing something that you know you wanted to do in the first place, right? I mean, this is kind of the story of everybody’s life, I guess, partially at least.

Dan Sharp [00:03:33] Yeah, that’s the dream anyway. So why don’t we dive right in? Yeah, you have Smashing magazines, Smashing books, Smashing conferences. Tell us a little bit about your journey to kind of building out that brand and bringing design to people that way.

Vitaly Friedman [00:03:47] Oh, that’s a good question, Daniel. I think at some point, I’m not at some point, but every now and again, I just ask myself, what the hell am I actually doing? Like in general, like what is that thing? So if, you know, it’s like this weird conversation that somebody would ask you, so what do you do for a living? And I have never announced, I never have an answer. I just have no idea what I’m doing, frankly, most of the time, because it changes. And that’s pretty much was like that, I think when we started. I started freelancing and doing design / development stuff, you know, because at this point everybody was doing everything. That was a webmaster, a web master. I was a Webmaster and I thought I know PHP really well. It turned out I know PHP really badly. And this is something I was pointed out over time, many, many times, but this is where it all started. And I got into web development in some way. And then in 2006, we ended up doing magazine called Smashing magazine, which was just a place where we would collect all kind of useful nuts and bits and techniques and stuff about web design, web development. And then, because I’m really getting very quickly, very bored, I moved from front-end development to accessibility, then to UX, then to design, then back to front-ends, then the performance optimization, then for a very brief period of time, which I’m not very proud of, SEO. And then moving on to other things as well so it’s been like my UX has been my life for i don’t know it is partially for the last 15 to 18 years and throughout these years we of course did a lot of things from books and conferences and just frankly and i don’ know if you feel like that as well when you’re looking back but i realized there was never like a big agenda like now we’re going to do this and the next two years we’re going to achieve that it was just let’s try and see if it works. And let’s do books and maybe let’s have conferences. And before you know it, like what almost 20 years later you’re sitting there talking with wonderful Sean and Daniel in the podcast.

Sean Murray [00:05:46] That’s all for today, Dan.

Vitaly Friedman [00:05:48] Yeah, it’s like a whole story of my life.

Dan Sharp [00:05:51] Yeah, and I’m really glad we didn’t choose this episode topic of the SEO. I guess that would have been a bit awkward.

Vitaly Friedman [00:05:56] Well, I think I was joking at this point because, of course, SEO has its value, but I mean, I was a really good SEO strategist, or whatever you want to call it, back in the day, and I’m pretty sure I’m a pretty bad one today as well.

Dan Sharp [00:06:09] Um, so when change gears a little bit, yeah, a lot of you are writing and that the content you work on is about, you know, UX design in enterprise space and working with large projects. And I’m curious, you, as somebody who has spent a great deal of my career working in kind of enterprise space, working with legacy applications. Um, you what, what are the challenges that you see day to day working with Moines there?

Vitaly Friedman [00:06:36] I think like, in fact, what you’re saying, Daniel, is probably what the heart of everything to me, legacy is that magical box where nobody, everybody’s afraid of touching it and nobody really knows how that works. And there are some people who set it up decades ago and they gone a few years ago and there is no proper documentation. Everybody’s kind of dancing and tiptoeing around that thing, really of being afraid, literally of breaking it or even touching it. I’ve been speaking about it loudly. It’s like this mysterious thing that you’re supposed to just respect, I guess. That’s definitely a part of it. But the other part that I think is a much bigger challenge that we often don’t think about, especially when we’re dealing with UX, is the notion of risk management. Because I think that whenever you’re dealing with enterprise applications, what ends up happening is that you are affecting a lot of people who need that tool to work, hope both sometimes for the internal, as part of the internal tooling. They need to be able to rely on it. And what’s also important is they need to be able do their work, like literally do their work, hopefully accurately, hopefully fast, hopefully not being slowed down every other minute or so of session freezing or anything like that. And so that kind of comes with a lot of challenges, there’s challenges on the political Because it kind of needs to work. It has been promised that it will be working. Any kind of delay really has an impact and has a massive cost, right? On the other hand, you also have a bit of challenges around UX and usability testing, for example, because sometimes you don’t have access to users because you have some legal constraints and there are plenty of issues around that. Or sometimes you just don’t enough users yet, or maybe just one user, right, or two. And so that kind of makes it a bit more difficult just to navigate, I would say. And also depending on the company and the organization in which you happen to be, sometimes you just have a lot of mess and chaos. Sometimes it’s a bit more organized, sometimes it’s even less organized, but priorities change, teams change, decisions change, business priorities change or shift, and they have tremendous consequences. And if you just work on a small project in the UX space, before you even know it, that work that you maybe will be spending the next three months. Will never be seen by anyone because in that time, something else will be more important and that project just disappears. And it happens of course everywhere, it’s not just something that appears on an enterprise. But I think that very often I realize that when you’re actually working on something that seems to be actually having an important kind of a big impact, right? You realize that there are also, there is quite a bit of reluctance of pushing it forward. And that to me, in the beginning, it felt very counterproductive to me. Why is it happening? And I think I realized now why it’s happening because especially in this world, when I come in as a UX designer, I’m not being perceived, at least not in the begining, as an enabler. I’m being perceived at best as a disruptor, at worst as an enemy. And the reason for that is because again, I come and look at me, I know how to deal with pixels and I’m going to redesign the navigation to make it better for our end users. I’m going to change this and change that. And I’m actually not realizing, and I wasn’t realizing at the time, like for years now, that as you are doing that, you are basically also making, how do I put it best? I would say you are disrupting the ways of working, the influence, the weight that somebody must have acquired over years and years and years and then you come in and you just want to move pixels around. Or pixels can be precious. I didn’t realize it back in the day. So I think a lot of things, all of that coming together kind of make it quite difficult and challenging. And plus, on top of that, you throw in some sort of complexity around the domain, around processes, around integrations, around permissions, around all of that. And then you find yourself in a very small dark room full of dusty papers and dusty folders. And very often, a very old machine that’s running on, I don’t know what. And you have to do your UX work. That seems to be kind of an accurate picture of where we are maybe. Maybe this is something that you can connect to.

Sean Murray [00:11:04] I can definitely connect to that. You know, I, I’m like Dan, I have a lot of enterprise experience. And I just mentioned was very excited for this episode as we went through the pre-production process, because when we go to conferences or when we do, when we do the podcast, we end up talking to a lot people that are at like the thought leadership level or talking about some sort of like startup mentality. And when you go to a conference and you look at the people that are, you look swath of attendees and if you just do a general look at the people, like the companies that they work for, all of them work in this enterprise space and a lot of the content at conferences or in podcasts are more about thought leadership level and maybe not something that directly ties into this kind of enterprise working environment where when you bring a new idea in, you might be perceived as a disruptor, right? So once you’ve, one, embraced that role. I’m going to be- I’m GOING to be a disruptor. And you also acknowledge that you are introducing risk and you might be introducing some political divisions. How do you make the case for the changes that you wanna make? What’s the right way to approach those situations where you are seen as a disruptor, where you wanna try to change from disrupters to partner?

Vitaly Friedman [00:12:15] I think to me, it’s all, it might sound even like cliche or too simplistic in some way, but to me it’s about building confidence and trust in that I’m here to help. Okay. Because by default, I don’t think that this is a perception at all. I mean, first of all, whenever you kind of meet somebody and somebody’s kind of maybe operating at the very heart because this is where legacy usually lives at the heart, very core of the business. And you are here to adjust and refine and improve whatever that means. And it’s kind of not very clear if my improvement is their improvement, really. And so I kind of need to start with small projects to build up trust. And one thing that I always do is kind of try to pull people aside and say, hey, can you just show me the product and kind of walk me through it? What are the features that we have and stuff like that. And listen very carefully to the vocabulary that they’re using and how to describe the product, how to the features and stuff that. And I’m just listening. And the more different kind of people you bring in to give you sort of a tour, the better the outcome will be because you kind of get an understanding about how they speak and what certain things mean to them and how much certain features or certain flows or certain ways of working mean to then as well. So for me, kind of just getting this broad overview about where we are is helpful. Vocabulary is also very helpful. Understanding the relationship between different stakeholders. Really trying to find out who are the people maybe inside of the organization or company who actually really believe in and want to support UX initiatives. Because there is a high chance that there were some UX projects that actually works to be successful and people who actually benefited from them, they want to have more of that. So if you find people who deeply believe in UX and, you know, you kind of tell them a story that about the project that you’re going to do here. They really will want to maybe move mountains for you and carry you and support you if they only know that you existed. So in the very beginning, it’s like all the bit of political game, understanding the product, understanding the vocabulary, and also trying to build those design guilds to understand what are the different areas and different people in those different areas who might want to support you. And then maybe launch a small pilot project that will actually explain to people who you’re going to work for or with. Just what exactly is that value that you’re bringing to the table and the good way to do that is to listen very carefully to some of the common I would say challenges or frustrations or slowdowns that people experience a lot. There are always those things that people are really really frustrated by confused by worried about or anything like that there are sort of kind of pain points that we have and typically If it’s not that, then there are constraints. Maybe there is a person or a group of people or a technology or anything like a tool, a CMS that is a really big bottleneck. Like everybody hates it, but nobody really wants to say it loudly because it’s been a conversation for many, many, years now. And if you know about those things. What you can do is reflect back to people around you what you have discovered. Now, this is what your product does. This is the features that are really, really important to us. These are the people who have benefited from the work that I’m doing in your company, right? If you just reflect back saying, I’m not here to break your flows, to change the way of how you’re working or kind of disrupt in a way, processes that are established. But I’m here to help you basically be more efficient. And if I can do my job, then you probably will benefit or you will find a way to benefit from that. So we need to kind of flip that coin and tell that story. And that’s really sort of a storytelling there. And there is nothing related to pixels yet at this point, right? It’s just all about, hey, that’s me. That’s what I do. This is what kind of work I’m doing. And this is similarly similar work that has been done in that company. So I assume that if we’re working on this without disrupting the flow that you have, that we can probably increase efficiency. We can reduce errors or mistakes or anything like that. We can help people find things faster or eliminate dead ends or anything that. That’s what I’m here for. Anybody wants to benefit from that maybe? And you kind of start initiating a little project where you’ve kind of zoom in and dial in all the way. Into something that hurts most people. Ironically, this is usually not related to like UI at all or even UX, mostly it’s meetings. Because people are either too frustrated by meetings or they are kind of got used to the way of how meetings are, but if you could make meetings just a little bit better, this will have such a tremendous visibility. And of course, you would do it small in your small team that you’re going to work on. And then before you know it, you might want to suggest it to the outer teams. And then, before you go, before even know, there will be the shift happening and it will be attributed to you, right? At least partially. And that’s extremely valuable. And then well, then you do your UX work. But as you can see, there is a long way there before you even get there. And sometimes I’m not saying that you have to do it. It could be just one single project, one off, and then. But if you are working for an organization or company, you kind of really need to go through this process. And that requires a bit of politics, a bit capital, a bit confidence.

Dan Sharp [00:17:52] So much to unpack there, yeah, I love what you said just about, you know, building the relationships, listening, you now when you’re kind of getting into the space and starting small, I think there’s like, there’s a lot there, not just for your designers who are trying to grab a foothold in enterprise space, but also new product folks, right, like new product managers, just how important it is to hear also really love what he said about, You know, you kind of hear, you’ll hear the same pain points coming up. One of the more interesting things I heard about like roadmap in a ways back was. You don’t need to write down every enhancement request that comes in because you’ll know the things that are important because they’ll just keep coming. They’ll just keep coming up. Coming back. Yeah. Absolutely. So I want to go back to something you said a little while ago, right? Like, yo, you, you spend all that time and you, and you’re kind of spending your internal capital building up, but you know, you’re, you’re working enterprise space. Like not everything that you work on from a design standpoint gets to You know, end up being pixels on a screen, right? Maybe the project ends up being deprioritized or maybe, hey, some of that legacy technology that you’re playing with makes this not feasible to actually implement. Yeah, in those scenarios, what advice do you have for designers to kind of temper the frustrations that I imagine you feel?

Vitaly Friedman [00:19:09] I would say that, well, my main goal is to avoid those things from happening in the first place because I want to know that something isn’t working before I even start designing it, but that’s kind of the important part. So whenever we have, let’s say some ideas and we maybe have prioritized those ideas with PMs, we kind of need to bring engineers on board as early as we can so we can say, well we want to do this and this would be the flow and this should be kind of. The way of how we imagine this to work. Is it feasible? Can we do that with our technology? Do we need another technology? Can we it with our legacy and whatever, right? And then you kind of get feedback. And sometimes it will be positive, sometimes it’ll be negative, but I think it’s important for me not even to know if this is a good idea or bad idea, kind of this sort of feedback, but is it even feasible? Can we actually pull it off? And if we do, will it be like a month long project? Will it be a year long project and so on? I mean, because very often, like for example, if you take it as something very, very practical, I know that not many people are excited about this, but I kind of am. URLs, right? Sean, how do you feel about URLs? Like in general, do you like you love URLs and, you know, this random? I guess I could take it or take them or leave them? Sean, you just broke my heart right there. Right, so I’m really kind of very careful about URLs. And so one of the things that we wanted to look at How do we establish consistent, clear, well accessible, I guess, URL schema, right? And if you have, let’s say, 130 websites, all living God knows where, right, creating any kind of consistency, not even in terms of URLs, but like visually and content-wise and information architecture-wise is very, very challenging. And so we start from the goal where we want to be. Like we want URLs to be speaking their language. So people who are reading an URL, be it in Google search results or you know, hovering over references in ChatGPT or in Perplexity, they understand where they’re going, right? And they also understand where to go from there, to go back and to go kind of sideways and stuff like that. And of course, before we even start thinking, okay, this is going to be the new URL schema, we need to consult with people who are managing all the different platforms and see what we can do there and what not and find compromises. And so before we suggest the best option, the best scenario. But then we review it together with engineers. And again, I don’t even know, is it like my job to be doing URL guidelines? I don’ know, but it’s like, it’s also a part of UX, isn’t it? I mean, somebody will have to read those and type them maybe on mobile and copy paste them and send them by email. So somebody has to design them, right? So, but then I need to understand what is viable and what isn’t. And so for me, yeah, I always, I am a big fan of showing unfinished work. As early as I can. So that means, okay, we have this idea, let’s just discuss. Let’s just throw in together, like get together on a Zoom call or whatever. And this is what we’ve been thinking, is it viable, is it doable or not, right? And then if it is, then we can refine that. But of course, in the case that you are being, let’s say, blocked by legacy or anything else that might be making your project invisible and not viable. Well, that’s then too late, because those conversations must happen earlier, right? Like literally earlier. Otherwise you’ll be wasting maybe one or two months of time working on something that will never see the light of day. And for me personally, and I’m not trying to be difficult frankly, but it’s, you know, I had some projects where it was very devastating, right, because I promised something and we can do this, we can pull it off, we can achieve that. And then it turned out not to be a disaster, but it was such an… It wasn’t working as expected, right? And I don’t want to be in this position, like ever. It’s just, I need to deliver, right. And so when I promise to deliver I need to deliver. I want to keep my promises. And that means, you know, checking in with other teams if what we’re doing here, what they’re suggesting here is even viable.

Dan Sharp [00:23:20] Yeah, that’s awesome. I think you know, I’ve struggled with it like there’s definitely an aspect of having to be flexible early with solutions and not falling in love with that solution that you’ve come up with and like, keep working.

Vitaly Friedman [00:23:33] I mean, I do fall in love with solutions every single day, I think, and if everybody could only understand the beauty of this, right? And if I had a chance to test it with 100,000 people, sure enough, they will love it and they will appreciate it and they’ll, yeah, but no, sometimes it doesn’t work this way.

Sean Murray [00:23:53] If I had any advice for my younger self, it would be, Hey, you need to stop worrying about being right. And you need just start working with people to get it right. And I think the concept of creating those early feedback loops is a way to get, you know, is a way to get there.

Vitaly Friedman [00:24:07] Well, I just want to hug you now, Sean.

Sean Murray [00:24:10] Uh, younger Sean was fine. He just needed some help.

Sean Murray [00:24:18] Hey we want to take a quick break from today’s broadcast to bring you some exciting news from the ITX Product and Design Conference team. The ITX product and design conference is coming up June 25th and 26th at the Rochester Riverside Convention Center.

Dan Sharp [00:24:30] Day one is going to be awesome. It’s going to feature four unique workshops, two product and two design.

Sean Murray [00:24:37] Day two, keynote day. You’ll get great insights from our speakers, plus you can attend panel discussions and even participate in live recordings of the podcast.

Dan Sharp [00:24:47] I’ve had great experiences every year I’ve been there. I come away I’ve learned something, I’m inspired. So come check out our website. You can get the full two day agenda as well as the link for registering yourself and your teammates. The website is itx.com slash conference 2025. That’s itx dot com slash conference 20, 25. And you should come and visit Sean and I over at the recording stage. Yeah, we’d love to see you there. All right, let’s get back to the conversation.

Sean Murray [00:25:13] Let’s do it.

Sean Murray [00:25:17] So something that I find myself doing a lot these days is coaching people on setting success metrics early on in their process because we’ve been talking a lot about early process. We, you know, I’ve been working through this kind of goals, signals and metrics process and using, you know, going back to, you know, it’s a little old school, but going back to the Google heart framework, trying to help people come up and learn ways defining success for their changes because good ideas have to in some way be measurable in the end, how do you define success or how do define success metrics from a UX perspective versus kind of like versus a product perspective.

Vitaly Friedman [00:25:58] Yeah, so basically, the main difference, I think, for me is that I try to be as close to where people, what people are doing as I can. Let me say what I mean by that. We do, when we try to establish some sort of metrics, we try to be making sure that they’re working on two different levels, on the local level and on the global level. Local meaning I might be working with a team that’s working on search. And they are trying to improve search and they’re trying to improve the filtering experience within search and search results and order complete and and all of that stuff. And so what we need to identify as let’s say 10 to 12 tasks that are coming from research and analytics, where we know that people are doing those things within search, right? So if we know the people are during those things in search, then we can probably say, all right, let’s try to dial up success of people getting where they want to be and dial down the frustration they’ll be getting by getting zero results, pages or inconsistent or incompatible filters or anything like that. So we’re defining or selecting 10 to 12 tasks based on the research that has been done before to understand how well are people actually performing those tasks. And these are supposed to be representative tasks what people are doing most of the time. Think about, let’s say, Fintech application. In a fintech application, you might have stocks and within stocks, you must be, you need to be able to filter, maybe filter by an industry, filter by specific market cap, filter by a specific PE value and whatnot, right? So we can come up with 10, 12 tasks that people are supposed to be able to perform relatively quickly and most most important, accurately, right. And that must be coming in from analytics. So we look and we define 12 tasks or statements. And we bring people in, if we can, sometimes we don’t have access to users, but if you can, we bring in people in and we give them their tasks to complete. And then we see how well they complete this task and how much time they need to complete them, right? And from that, you can basically get a very simple, like literally the simplest of them all metric. One of them is the success rate, how successful people are in kind of getting where they want to be. And on the other hand, also the time on task, just how much they actually need to compete them. You bring in something like 15 to 20 people, and of course it’s not like it’s statistically significant. It’s not a paper, they can write about it, but you can kind of get to trends and you try to figure out what, are people more likely to succeed or are they more likely fail? Especially if you’re looking at a specific group of people or being experts in a particular domain or people who have some experience with the product and kind of need to segment it in the way that you need. And so that would be the local level, right? So if you’re working on search, you’ll be measuring how people are searching, right, with 10 to 12 tiles that people are using. It might be connected with filters and so on and so forth. But on the other hand, what I also want to achieve is that the product is working globally. And that means we’re looking at the level of the entire product. What are people doing here? Like one important task that probably every product will include is onboarding. But onboarding is not just somebody clicked on the button and then went through the, I don’t know, five, six screens of onboarding and that’s it. Onboarding in my book is somebody gets on a page, they go through a series of actions, interactions, and then eventually they get to the first success, first meaningful success. Whatever that is, it could be imported portfolio on a product, in a product. Could be something that’s set up that represents the interest or anything. It cannot be an empty page. An empty page is not a success in terms of onboarding to me. And so we come up then with 12 tasks that are living on the global level. Checkout would be one of them, onboarding would be another one. And we see how people perform those tasks. And these are very simple metrics. Tasks complete, success rate, and how much time people actually need to complete these tasks. And then depending on the product that we have, we kind of need to establish maybe something that goes and lives beyond that. Like for example, if you’re dealing with forms a lot, then one thing that will be important for you is not just how successful people are in completing those tasks and how much time they need to complete it, but also how accurate the input is. So we’re looking at form accuracy input, right? Or form input accuracy, that would be correct, I guess. Or also at things like error message recovery rates, right? Meaning just how impactful and how disastrous are error messages. I think that most organizations just underestimate, massively underestimate the impact of error messages. Because if an error message doesn’t communicate what is wrong and how to fix it, it can cause, and I’ve seen these cases in the past, can cause a massive abandonment and also a massive delay, especially when you’re talking about enterprise applications. People are just blocked, they have to start over, they don’t understand why it’s like that. Think about something like mortgage application reviews. It’s just really, really, really expensive. And so if we don’t consider that, that’s kind of an issue. And so just to sum up, so we’re looking at the like local level. So I’m working, let’s say, as a designer on that thing, let’s search or checkout or whatever. So I measuring what’s happening here. But on the global level, every vertical will do that, right? But on a global level we’re looking also at some tasks, like 10 to 12 tasks, that represent more or less full customer journey from onboarding. I don’t know doing something that’s important for the business somewhere as a regular part of the work to off boarding as well By closing account and stuff like that, right and it all must work, right? Some of those things will be more important to business than the others. So they will be important to users than the other so we need to find that balance And we sign them off and then we have a metric that we can measure every You know those things we can maybe every three six eight nine months depending on the velocity. That’s a great start of your measurement journey.

Dan Sharp [00:32:07] So who’s actually kind of monitoring those metrics in this model?

Vitaly Friedman [00:32:11] That’s a good question. Like sometimes, like in the case of right now, when I’m working, it’s me, right? But of course, as on a global level, right. But of cause as we’re looking at local level, that would be like the individual teams and probably UX lead on that individual team who’ll be tracking to making sure that this actually performed. I mean, in the end, it depends if you have researchers on the team or not. Right? Sometimes it’s not. But then of course then that brings up a question about the PM. Right, at which level and how will PM want to be involved? And because from my understanding, please correct me if I’m wrong, I have seen that PMs usually way more interested in business metrics than they are in UX metrics. And so I’m kind of playing this counterpart role of bringing UX metrics to the table that are then complemented with business metrics and both of them end up displayed, being displayed in a dashboard somewhere, you know, senior. Senior C-level people, right? But I’m kind of responsible for this UX metrics then.

Dan Sharp [00:33:15] Yeah, no, it’s interesting, right, because you have the world where, you know, products tracking some metrics, UX is tracking more now, right? As a designer, you’re worried about, you know, having awesome user design, making sure it’s accessible, making sure its wired up for metrics. And right, your breadth is expanding and expanding at the same time, right. The product manager’s breadth is expanded and expanding. I’m curious to hear your thoughts on just kind of the overlap that we are finding, you know, more and more common these days. Between designers and product managers and how we can work best together.

Vitaly Friedman [00:33:50] Yeah, I think in my experience, it’s all a matter of alignment and decision making, I guess. So if some PMs want to be taking over this part, that’s great. That’s fantastic, because some of them just happen to be more on the UX side anyway. They’re very interested in what’s happening in the UX anyway. Very often you would hear, of course, PMs being the voice of the customer. And so that kind of only makes sense. But on the other hand, I think there are a lot of companies that are very much business driven. And so UX is almost like a thing that is, along with UX metrics, it’s sort of a thing that is living inside of that UX department or UX departments or UX teams. So they kind of own it. And frankly, I am a big fan of owning them because then if you’re working on say, on that search team and you’re being measured or you are measuring how well people can search in your product. Then you can say, well, this is what it was before, let’s say six or eight months ago. And then we rolled out the change two months ago, and this is where it is now. And that can be attributed to us, because again, we just test the same tasks. And we see that people are performing much better than that, right? And so, that they did in the past. And so this is kind of direct attribution to the impact of our work. And so that’s something that we own, that’s that we… Visualize and explain that something that we report regularly and this is the definition of our success for our team.

Dan Sharp [00:35:19] Love it. UX are still always be more than a thing to me, Vitaly.

Vitaly Friedman [00:35:24] Oh, that’s nice to hear that.

Sean Murray [00:35:26] These days maybe both work together beyond just the podcast and maybe I’m sure I mean we could build something cool this has been an amazing conversation today I really want to thank you for your time I want to close with some takeaways but then I want talk about the product and design conference so some takeaways for me when you were participating in enterprise work make sure that you are aware of your role in risk management and while that may make you a distra disruptor don’t feel it don’t fear those legacy systems right find a way to prove the value of what you’re trying to do and approach that risk in a tactful manner because there may be politics tied to it. There may be long-term decisions that people weren’t responsible for tied to.

Vitaly Friedman [00:36:05] Sean, you’re much better than ChatGPT, I have to say.

Sean Murray [00:36:08] Thank you very much, I’m getting pretty good at this now. I really liked your idea around starting small to build trust and focusing your vocabulary on the language of your stakeholders, right? So that you’re all speaking the same thing. You should be the one adjusting if you are coming in as the disruptor. I really like that point. And then frustrations or slowdowns are often indicators of a constraint. Right, so if I’m frustrated about something, there’s maybe something, there’s a constraint that I have had to play with for a long time that I’m not communicating correctly. And then one that if Sean Flaherty is listening to this somewhere on, he’s the former head host of this podcast. He’s listening to on 2X speed on his treadmill. He’s gonna be overjoyed that we spent a lot of time talking about relationships and the importance of relationships and building great relationships to people that you’re talking to. It’s just something to always enforce, right? Your job is not necessarily to make everybody happy, but your job is to create a workable environment no matter where you are or what you’re doing. Really great, really great thoughts today. Thank you so much for your time. I want to take some time to talk about the product, the UX, the product and design conference that we’re holding here in Rochester, New York at the downtown convention center on June 25th and 26th, you’re hosting a workshop. What are we doing?

Vitaly Friedman [00:37:26] Oh well, entirely different. It has nothing to do with what we have discussed at all because we are obviously going into the world of, I don’t know if you heard about it, just emerged recently, AI.

Sean Murray [00:37:36] Well, you’re contractually obligated, I think, to talk about it.

Vitaly Friedman [00:37:39] Oh, we are. Oh my God. This is perfect. Yes. I think that there is this thing that’s called AI and a lot of people are speaking and discussing and debating and reviewing and even implementing this in even in enterprise applications, believe it or not. So this is like happening. And so my hope is to bring a bit more clarity to designers in what it means. And that’s kind of partially based on some of the experimental work that I was doing. But also I would say it’s There are a lot of things that we expect from AI and that we believe AI can do and that kind of bring or try to bring to products. But the reality is that AI is this really weird thing. Because while software is typically kind of predictable and reliable, and if you click on a button, then it does some, well, not all software is like that, but most of the time what people are expecting is that you click a button and you want to export an Excel sheet and it will be exported. Or Excel sheet. But what happens sometimes is with the AI, you can basically generate something that is not reliable and very often it will be around 10 to 15% of the time to be unreliable. So how can you actually integrate something like that in something that people need to rely on? So if I think about something like, again, a financial application, right? Then you want to kind of highlight or explain to people what they need to be investing in. You can’t use AI for that because it will You know, incorrect or giving incorrect assumptions or suggestions 10-15% of the time. You can’t do that. And I mean, you can say, well, you should not trust AI, right? But then why would people want to use that feature if you cannot be trusted, right. So the question then is, how do we bring in AI into this sort of environment? And even not in that environment, like, you know in maybe in others, we have agents that can double check each other, right, so to increase accuracy, but that comes the cost of time. So you probably have seen all the deep research features in Perplexity and in ChatGPT and so on. Yeah, they’re great, but then not everybody wants to click on a button and wait for three minutes until something happens. So sometimes speed is really, really important. So basically what I’m trying to do is to bring a lot of those little things that are important for us to know as designers, right? And also product people just to see what do we do with that thing. It’s kind of this magical box that can do whatever we want, but not in a predictable way that we’re used to, right? And so how do we make it better? And quick clue to that is that’s definitely not going to be a text box, but you can type into it because people are very bad at articulating their intent, right. And so there are also a lot of different patterns of what different teams are doing and how they are kind of making AI work nicely and neatly while still being kind of a bit unpredictable at times. And so all of that will be a big part of the workshop where we’re going to go through hundreds of examples and also some patterns, which I’m sure Daniel and Sean, if you join, you will not like because they will be quite challenging and not something that we’re used to on the web.

Sean Murray [00:40:48] I haven’t negotiated yet, but I am going to find a way to break into one of your workshops. So don’t worry. You will see you’ll see me sneaking in the back at some point.

Vitaly Friedman [00:40:56] Fantastic, I’m looking forward to it.

Sean Murray [00:40:59] But that’s just, it sounds so engaging. It’s like a topic that needs to be discussed and that people really need to up their game on now, because if we don’t get a good foundation and if we don’t give it a good Foundation, this world is going to that, that AI world is going race ahead without us. It’s coming, whether we like it or not. Thank you so much for your time today. This was an amazing episode.

Vitaly Friedman [00:41:18] Thank you so much, Sean. Thanks so much for having me.

Well, that’s it for today. In line with our goals of transparency and listening, we really want to hear from you. Sean and I are committed to reading every piece of feedback that we get. So please leave a comment or a rating wherever you’re listening to this podcast. Not only does it help us continue to improve, but it also helps the show climb up the rankings so that we can help other listeners move, touch, and inspire the world just like you’re doing. Thanks everyone. We’ll see you next episode.