Why AI Still Needs Humans: The Future Of Trust, Policy & Brand Risk

[00:00:00] Tessa Burg: Hello, and welcome to another episode of Leader Generation, brought to you by Mod Op. I’m your host, Tessa Burg, and today I am joined by Stephanie Posner. She’s [00:00:15] the Global Vice President of Policy, Safety, and Global Affairs at DoubleVerify. We’re very excited to have Stephanie here because we’ve all hear the hype about AI replacing people, automating tasks, and feeling the [00:00:30] pressure to move faster with less people and less steps. 

[00:00:34] Tessa Burg: But we’re gonna explore and really dive into why AI still needs humans, and we know that when we have humans in the loop, especially humans with the [00:00:45] right type of expertise, that trust becomes the currency in which we’re exchanging how work gets done for what we’re willing to share from a data perspective. 

[00:00:55] Tessa Burg: So Stephanie, thank you so much for joining us. We’re really excited to dive into this [00:01:00] topic today  

[00:01:01]  Stephanie Posner: Thanks so much for having me, Tessa.  

[00:01:03] Tessa Burg: So before we begin, tell us a little bit about yourself and your role at DoubleVerify.  

[00:01:09]  Stephanie Posner: Yes. So I can, introduce myself. You’ll hear the first words out of my mouth. I’m obviously from [00:01:15] London, so I’m from across the pond, but I’ve been in the US now for just over 10 years. 

[00:01:21]  Stephanie Posner: My career has spanned all different kinds of things, but really it’s what you tapped into. It’s trust that strings together everything I’ve really [00:01:30] worked on across the years in the different companies. I started my career as a diplomatic advisor, and of course, when you’re working on things like Middle East peace, trust, relationships, tricky negotiations, hard trade-offs are the bread and [00:01:45] butter. 

[00:01:45]  Stephanie Posner: It’s your everyday, day in, day out. But when I came to the US, I really moved into tech, and so for some time I worked at Netflix in their global risk and intelligence function on both physical security and content security and content risk. So really [00:02:00] getting a grounding in professional media. And then I moved to TikTok, where I got a baptism of fire with social media and user-generated content, working on the platform policies there and the trust and [00:02:15] safety team there. 

[00:02:15]  Stephanie Posner: And that brought me to DoubleVerify, where I came to really build the team from scratch around everything we do on brand risk, brand suitability, and it’s been a wild ride. We have created [00:02:30] a really robust policy group. Our team creates all of those kind of structures that underpin how we classify content and how we do that in a really rigorous way for our clients. 

[00:02:42]  Stephanie Posner: And we’ve made a point of, at [00:02:45] least I’ve made a point of, doing this in a way that’s really tied, not just to technical signals that AI can pick up on, but really tied to what advertisers care about, what they think and feel about their brands and for their brands, and also how [00:03:00] consumers react to seeing ads in different environments. 

[00:03:03]  Stephanie Posner: And so it’s been fundamental to tie that into the work we’re doing to really differentiate what we can provide for advertisers as they think about protecting their brands and having a strategy that works really well for [00:03:15] their campaigns. And it’s been great.  

[00:03:18] Tessa Burg: So, I think it’s fascinating that you played a role at TikTok, because I remember very clearly when TikTok came onto the [00:03:30] scene and quickly became a part of a channel we wanted to consider for brands. 

[00:03:34] Tessa Burg: And trust was right in the front of that conversation of … ‘Cause no one really, and probably ’cause we’re from a different generation, you know, my daughter understands the content on TikTok, but I’m [00:03:45] like, “I don’t even know what’s going on here.” But what were some of the ways in which … W- like, how did you communicate out to brands that they could get visibility into where their content was and [00:04:00] feel good about where it was going to live alongside the user generated content? 

[00:04:04] Tessa Burg: ‘Cause I think that’s just a massive risk for brands when it feels like, unlike other traditional advertising, and like AI, they don’t have control [00:04:15] over the context all the time in which they’ll be showing up.  

[00:04:18]  Stephanie Posner: I think it was two part in that way. First, to just make sure that the whole community felt good and positive and healthy, and then secondly, kind of matching up the experiences that different [00:04:30] creators were making with the experiences that brands wanted to be associated with, and adding that layer of transparency so that they could feel confident that those creators had a consistent message and were consistent with what they wanted their brand campaign to look like [00:04:45]  

[00:04:45] Tessa Burg: I love that. 

[00:04:46] Tessa Burg: So how have you taken some of those principles of that foundation of safety and transparency and fit and adapted that to your role today at DoubleVerify? [00:05:00]  

[00:05:00]  Stephanie Posner: Well, at DoubleVerify, everything is about suitability. There’s so much more beyond this world of the baseline of, is it just okay to put my ad there? 

[00:05:10]  Stephanie Posner: Is it going to be something unacceptable? It’s really much beyond that because [00:05:15] brands, as you know, have so many different needs, and they really want to make sure that their ads are appearing in an environment that is going to be beneficial for them and is going to make sense and is gonna be resonant. 

[00:05:27]  Stephanie Posner: It’s not gonna be jarring with what the [00:05:30] consumer’s experiencing at any given point on their online or their digital journey. And so that kind of comes back to what I mentioned earlier about building in all these different insights into how we classify content and how we [00:05:45] prepare up the choices that we give advertisers. 

[00:05:47]  Stephanie Posner: So we’ve got over 100 different content categories that advertisers can choose from, and then those are each individually researched by our team members. We have many, many [00:06:00] years of speaking to advertisers, understanding what sort of environments make sense in your industry, for your brands, for your different products. 

[00:06:08]  Stephanie Posner: And then what I’ve been really intentional about laying on top is not just what the brand manager might think or the [00:06:15] marketing team might think, but how do consumers actually react? So we’ve been doing more and more consumer research studies where we take real pieces of content and we see what do consumers think when they see an ad from a particular industry or [00:06:30] particular product type next to that content, how do they react? 

[00:06:33]  Stephanie Posner: We just had the World Cup, for example. So one thing we did is we took all different kinds of World Cup content, be it news stories, social posts, something from [00:06:45] forums online, and we showed these different types of content, these different stories, everything from this amazing player’s scored a goal to this player’s been injured or certain teams had political buzz around them, and we showed the consumers [00:07:00] all of these different types of stories, and we tried to understand, how does that make you feel about the brand? 

[00:07:05]  Stephanie Posner: Do you feel like you have a greater purchase intent after you see their ad next to this kind of story, or is your purchase intent pulling back? And it was really interesting to see that, those [00:07:15] results, and also to see how that can change across industry and product type. So what might work for one brand in an environment might not work for another, and so it’s all about finding what works and is suitable for any particular given [00:07:30] brand, product, and campaign. 

[00:07:32] Tessa Burg: That’s really impressive. Yeah, and the World Cup was amazing, and I don’t know how you felt ’cause you’re from England.  

[00:07:40]  Stephanie Posner: Yeah.  

[00:07:43] Tessa Burg: I had high hopes for [00:07:45] them. I follow the English Premier League, so I’m familiar with most of the team, and always root for players from Arsenal at least. But that was rough. But you guys got very far, farther than-  

[00:07:55]  Stephanie Posner: I think it’s definitely more disappointing for my husband than it is for me, but I was [00:08:00] following along a little bit. 

[00:08:01] Tessa Burg: Yeah. Yeah, it was nice having it on this time zone. I feel like that really made a big impact. But in the advertising realm, it was very interesting to see which brands were [00:08:15] running different types of content and in different places because the World Cup was happening. So what are some of the things that the brands are seeing that build that trust? 

[00:08:26] Tessa Burg: What are you showing them so they know, yes, we’re gonna [00:08:30] protect your brand in these environments, and I call them, like, trust signals. Like, these are the types of signals we’ll be looking for to make sure that where you’re at is not just relevant to your audience, like I love that you guys are testing, like, how audiences are responding to it, [00:08:45] but also not going to send the wrong message or present in a different context that’s off from your own brand values and ethos. 

[00:08:55]  Stephanie Posner: So this is all about giving, for us, it’s all about giving advertisers as many tools and as, [00:09:00] as many insights as we can so that they can make decisions that are right for them. So whether that’s giving them different products and features through our platform, whether that’s selecting content categories, or whether it’s in their campaigns. 

[00:09:14]  Stephanie Posner: We [00:09:15] have so many different features that they can control, that they can leverage, and it’s at their fingertips. And then on the flip side of that is reporting, right? So making sure that once they’ve made these decisions, they understand how that played out for them and their campaign, and then they can also [00:09:30] themselves iterate around the goals that they have so that they can bring those numbers and bring that data in line with where they’re hoping to take things  

[00:09:40] Tessa Burg: And in this process, we know we’ve worked with DoubleVerify for a long time, you have a ton [00:09:45] of data, and as you just mentioned, there’s a lot of features and a lot of controls that the client can use and use in partnership with them. 

[00:09:51] Tessa Burg: And you’ve talked about before the importance of human judgment. What is the background and the experience of some of the folks at [00:10:00] DoubleVerify that are helping to make sure that the content is tagged correctly, that brands are protected, and that living underneath their own policies and governance. 

[00:10:11] Tessa Burg: Like, what does it take for human judgment, and what are those [00:10:15] roles that bring so much value to the client relationship equation?  

[00:10:20]  Stephanie Posner: Well, I’ve been extremely conscious of building a team that is diverse in thought and experience. [00:10:30] We have an incredible team of true content experts who bring all different kinds of perspectives. 

[00:10:38]  Stephanie Posner: We have former lawyers, former journalists, former diplomats. We have representation from people who have worked [00:10:45] across all of the different… Any social media platform you can possibly name. And bringing those people together is the first step, in trying to layer on human judgment on top of machine learning, LLMs, [00:11:00] advanced algorithms But the second part is also creating the environment. 

[00:11:04]  Stephanie Posner: So you need to be able to, and I hope that we’ve created an environment where those people can share their ideas, can share their thoughts really freely, and can disagree sometimes [00:11:15] too. But all of that then is backed up by data. So you need all of those three layers. You do need automation and AI and LLMs. 

[00:11:23]  Stephanie Posner: It helps you gain this incredible scale. The human judgment gives you that expertise, that experience, that [00:11:30] human something that we can only tap into when you see a meme and it’s funny, but you can’t even really explain why. It just is, that human je ne sais quoi. And then you do need the data as well, because that has to underpin [00:11:45] all of the decisions too. 

[00:11:46]  Stephanie Posner: And so it is hard to find incredible team members, and we’ve been lucky to do that. It’s been something we, like I said, have consciously put a lot of effort into, who can bring all of those elements together, and then can level up our [00:12:00] solutions in a way that you just can’t if you’re using LLMs that are not specially or expertly trained. 

[00:12:08]  Stephanie Posner: That’s what gives you sort of the edge in really, like I said, making [00:12:15] categories, making products and features that work for brands in a really impactful way, instead of a kind of generalized okay way.  

[00:12:23] Tessa Burg: Mm-hmm. Yeah. And I feel like that’s what’s most important to brands, is that it’s [00:12:30] personalized to them, and that it’s highly specific to how they wanna show up and be represented in marketing and advertising channels to their customers and to their potential clients. 

[00:12:41] Tessa Burg: Also, you said LLMs obviously play a really important [00:12:45] part in that equation, and that you need them for scale. And we hear a lot about AI replacing competitive work, but as we look out ahead, how do you see this balance of human judgment [00:13:00] and using LLMs for scale in high-performing organizations? How will this continue to evolve over the next year or so? 

[00:13:09]  Stephanie Posner: So I think there’s a really interesting conversation happening right now around this and sort of [00:13:15] how do you choose whether to automate something, whether to have humans in the loop, whether you need a kind of Co-pilot model where your AI takes you so far and your humans then kind of level you up. And of course, I think on the simple end of [00:13:30] the spectrum, there are so many things that AI can do, and it can do it well. 

[00:13:34]  Stephanie Posner: And there are tasks that we all know and love AI for. Summarizing, it’s great at right now. Getting through huge amounts of data, working 24/7 without getting [00:13:45] tired or irritable. All of these things, great, and there are so many things throughout any company that you can benefit for when using that. 

[00:13:52]  Stephanie Posner: But when it comes to the human judgment piece, we still see that in more complex tasks, especially in [00:14:00] our field around content, at the moment, AI can do a pretty good job, but when you’re trying to have these solutions at scale, it can be more and more challenging to get it exactly right without [00:14:15] layering on top human judgment and training, and also continually testing because, as we spoke about before, content is changing all the time. 

[00:14:24]  Stephanie Posner: Our own human perceptions are changing all the time, and so you have to be able to [00:14:30] regularly QA your systems, and you need humans as well to help with that to some extent. I think the other thing that is spoken about to a lesser extent is that for all of these LLMs to continue to succeed [00:14:45] and continue to help us with an AI-powered economy, it’s not just about the LLM infrastructure, it’s also about the quality of information that you input. 

[00:14:54]  Stephanie Posner: And at the moment, yes, there are data sources that can be [00:15:00] accessed directly through technical means and technical data streams and inputs, but you do also, to an extent, again, with these really complex systems, you do need humans to help you make sure that your data input is [00:15:15] amazing, is great, is fresh, is reflecting what is happening in the real world as well. 

[00:15:20] Tessa Burg: Yeah. I love this conversation because we’ve hit on so many valuable points for brands right now who are facing a ton of uncertainty [00:15:30] of either where to get started or who to have on their team, or what kinds of things are really important about governance, compliance, and policies. And so when we’re thinking about where to get started, just to summarize a few of the takeaways, [00:15:45] you really leaned into creating a space and creating an environment where people feel safe, where people can exchange ideas and bring concerns and bring questions in an open environment. 

[00:15:56] Tessa Burg: And I think that’s phenomenal. When we think about doing that internally in our [00:16:00] businesses and doing that with being transparent in how we use AI, how we use data with our customers, and creating a sort of safety there in that exchange. Then the other piece you hit on is what is the right type of humans to have in the loop?[00:16:15]  

[00:16:15] Tessa Burg: That it’s not just about, oh, it’s a human in the loop, and they’re just testing in quality. You are leveraging this balance of LLMs to intake a lot of data to do this testing with quick turnarounds, but you’re relying on humans who are [00:16:30] getting very diverse expertise in the channels themselves, and that’s something that I think will always be necessary. 

[00:16:36] Tessa Burg: We’re always gonna have new ways of communicating. We’re always gonna have new channels that emerge, and it’s really is gonna be on humans getting into those [00:16:45] channels, having personal experiences in them so that you’re bringing what can only be that human context. And then I love that the third aspect in looking out and continuing to make decisions and build relationships, and [00:17:00] really when you have those other two things in place, then you start to have a very highly specific and personalized way that you can approach brands because you understand where they’re coming from as well, and their ethos, their goals, their objectives, and you can take it and engineer back [00:17:15] through the process and the systems to protect them and build their confidence as they start to explore new ways of reaching their consumers. 

[00:17:23] Tessa Burg: So I think this has been a fantastic conversation. A lot of great information to build confidence with [00:17:30] brands, to test and experiment, to get more out of AI in the right ways, to really focus and make sure that the input of data is high quality. Before we end, is there anything else that you wanna share with customers as they continue their [00:17:45] exploration of how to protect their brands while still achieving that scale that AI offers? 

[00:17:52]  Stephanie Posner: Yeah, I think there’s so many great opportunities for partnership, right? And it comes back to the relationship point because that’s where you make [00:18:00] the real progress because for every brand it is different. So there’s some great opportunities for partnerships to do research together, to really get to understand what is it that our customers react to and respond to, and that’s really then moving things, [00:18:15] like I said, from protection through to performance because it’s not just about how do I act out of fear that something that might go wrong, but how do I really think about how to unlock the potential of the audience [00:18:30] that’s there and the brand that I already have and these amazing products that I wanna be able to share with my customers. 

[00:18:37]  Stephanie Posner: And I think that’s really, if I look over the next few years, that is hopefully where brands are going, it is where we’re [00:18:45] going, and I think that’s really exciting.  

[00:18:48] Tessa Burg: I agree. It is very exciting. Stephanie, we are at time. Thank you so much for joining us today. If listeners have more questions for you, where can they find you? 

[00:18:57]  Stephanie Posner: They can find me on LinkedIn. I’m always there. [00:19:00]  

[00:19:00] Tessa Burg: And it’s Posner, spelled P-O-S-N-E-R. And we will also have a link to Stephanie’s LinkedIn profile on our website, where you can find all of the Leader Generation episodes. It is at [00:19:15] modop.com, M-O-D-O-P.com/podcast, or just search Leader Generation wherever you listen to podcasts. 

[00:19:23] Tessa Burg: Stephanie, thanks again, and until next time, I hope you have a great rest of the week.  

[00:19:28]  Stephanie Posner: Thanks, [00:19:30] Tessa.