From AI Experimentation to Marketing Impact: Lessons from the ANA’s Hands-On AI Workshops

AI is quickly becoming part of how customers research brands and how marketing teams get work done. But using it well takes more than choosing a tool and writing a prompt. Marketers also need to consider whether the information is reliable, whether the output reflects their brand and customers, and how to protect proprietary information. 

Those questions took center stage at the Association of National Advertisers’ (ANA) Email Excellence Committee conference in Costa Mesa, California. Tracy Smith, Director, Marketing Strategy & Automation at Mod Op and four-year member of the committee’s board, moderated hands-on workshops where attendees tried AI tools for themselves. 

The promise and the pressure of AI 

At the conference, Forrester VP Shar VanBoskirk shared figures underscoring how quickly AI is becoming part of the buying journey. One projection was that 94% of B2B buyers will use AI in the buying process by 2027. But that information has to be consistent and credible: 31% of buyers said unreliable or misaligned information made them less confident in both the companies they were considering and in AI itself. 

For brands conflicting answers can be frustrating. If AI search engines describe a company, product or service differently from one another, marketers may be left wondering which information is accurate and whether they can trust any of it. That makes consistency across AI-generated answers and a brand’s other communications more than a content issue. It can shape confidence in the brand and influence whether it remains part of the consideration set. 

That’s why marketers need to understand how their brand appears in AI search. Mod Op’s free AI Search Visibility Audit helps brands assess where they appear across major AI platforms, how they compare with competitors, and where visibility gaps may exist. It offers a starting point for identifying where a brand’s presence in AI answers may need attention.  

There’s pressure inside organizations, too. According to Forrester, reports 44% of B2B companies expect a positive return on their AI automation investments within 12 months or less. Marketing teams are being asked to find practical ways to work faster while maintaining quality and judgment. Consistent, trustworthy information is an important part of that effort, both for the people using AI at work and for the prospective customers relying on AI to make decisions. 

Four guardrails for better AI use 

During one panel discussion, attendees explored how teams can use AI without losing sight of accuracy, ethics and accountability. Tracy highlighted four guardrails for marketers: 

1. Write a brief before you write a prompt. Clarify the goal, audience and desired outcome first. 
2. Choose the right engine for the task. Understand what a tool is designed to do before relying on it. 
3. Ask AI to push back. Have it identify weaknesses or challenge its own response. 
4. Make assumptions visible. Ask the tool to flag its assumptions so the team can review them. 

These steps make prompting a more deliberate process. Instead of treating an AI response as a finished deliverable, marketers can use it as a starting point to question, refine and verify. 

The value of hands-on approach to testing AI 

In Tracy’s workshop, small groups worked on a hypothetical smartphone launch, from campaign strategy and execution to data and analytics. They tested different AI tools and prompt approaches to see what each could do well, and where it fell short. 

In the workshop’s campaign-planning exercise, Claude used behind a firewall stood out for producing a detailed, end-to-end campaign plan. Microsoft Copilot was less effective for developing strategy from scratch. As Tracy explained, “it’s just not meant to do that kind of work.” Participants found it better suited to helping with tasks inside Microsoft products, such as enhancing work already underway in those tools. 

Prompt quality also made a difference. According to Tracy, “If you gave it a really strong prompt, it gave you back a great campaign.” When participants intentionally provided little detail, the results were often too generic to act on.  

The lesson? Match the AI tool to the task and give it a clear brief, rather than expecting one tool to handle every kind of marketing work. 

Public AI tools vs. protected AI environments 

The workshop also compared public AI tools with systems running behind a company firewall. Tracy noted that the protected environment didn’t automatically make the AI better. Its advantage was access to relevant company information, such as brand positioning, value propositions and audience personas. With that context, the campaign strategy was far more tailored. “When the AI had access to that information,” Tracy said, the work was “exceedingly relevant and you could just jump right in.” 

Public AI tools, by contrast, offered useful general guidance but needed more prompting to understand a company’s specific products and customers. Protecting proprietary and customer information is essential, so teams should use AI tools and environments approved by their organizations and follow established policies for handling sensitive data. 

Keep learning as technology changes 

The conference’s workshops gave participants more than a chance to experiment. The groups developed evaluation checklists and playbooks for campaign strategy, execution, and data and analytics — resources ANA members can use to assess outputs and improve their prompts. 

For Tracy, the work is part of an ongoing learning process. Drawing on more than 25 years in marketing, she reflected, “My deep dive into AI began only a year ago, but the technology is evolving so quickly that you have to keep learning or risk falling behind.” That mindset matters as marketers weigh AI’s potential against its limitations. Strong briefs, fit-for-purpose tools, relevant company context and human review can help teams make experimentation more useful and help keep the work grounded in what customers actually need. 

As AI increasingly shapes how people discover and evaluate brands, consistency and credibility matter more than ever. If you’re thinking about how your brand shows up in AI-driven discovery, or how your team can use AI effectively while protecting customer data, let’s talk.