Visualizing Creative Diversity: Introducing the Ad Atlas

Atlas is live: see your creative diversity.

Earlier this year, we announced that UGC is dead. The TLDR is: UGC started as alpha. Something few people did that gave advertisers an edge. Now UGC is table stakes, and feeds are filled with mediocre talking-head ads that do not drive differentiated advertiser outcomes.

We started noticing this decay in early 2025 and built our first product, WillBot, to try to understand how to drive better outcomes by identifying the nuances of UGC. We thought that if we could understand the creative variables well enough, we could systematically produce better UGC. But this ultimately didn't produce the outsized wins we were looking for.

In Q3 of 2025, we started seriously branching out into differentiated formats. We kept noticing a pattern: new creative formats were often responsible for disproportionate performance gains. Specific types of street interviews, podcasts, reaction ads, and a host of less familiar formats outperformed the usual UGC ads.

And frankly, the creative strategy behind them was often less involved than hyper-optimizing the average UGC video. So we knew it wasn't due to the strategy as much as the visual or semantic differentiation of the format. So we built a taxonomy of creative formats into our product. Every creative test had to be labeled with a format, subformat, creative moves, and various other attributes. Other creative analytics platforms converged on similar approaches to understanding creative performance.

This was useful on the surface, but clunky in practice. More importantly, it was an incomplete way of thinking about creative. An ad being classified as a "podcast" doesn't actually tell you that much. It could also be a testimonial, a product demonstration, an argument between two people, or a reaction to a particular claim. Look at this ad for example.

It starts with a reaction, then a testimonial in a street interview format. How would you classify it?

You can keep adding attributes to describe these differences: format, hook, setting, creator, message, pacing, editing style, and so on. These craft attributes can get quite involved. Every additional attribute creates more combinations, and the boundaries between them are often arbitrary. You're trying to describe a continuous, high-dimensional problem using a collection of dropdown menus. This is deeply paralyzing to a creative team and is a fractal problem that is never-ending.

We weren't satisfied with this approach. It felt like trying to force a process onto something that fundamentally doesn't work that way. Nobody thinks of a new creative idea by describing the sum of its n craft attributes.

People look at other ads. They recognize patterns. They see something interesting, borrow parts of it, combine it with something else, and develop a hypothesis about what might work. They think in examples and relationships, not taxonomies. TL;DR: Creative is, well… creative. Shocker.

We needed a way of understanding the Creative that reflected this. This is the motivation of explorations that started in July of this year, and through a series of attempts, we have built what we're calling the Ad Atlas.

The idea is relatively simple. Rather than organizing creative into a fixed set of categories, what if we could represent every ad in a continuous space, where ads that share similar characteristics sit closer together and those that are different sit farther apart?

Instead of looking at a spreadsheet of formats, hooks, and performance metrics, we could look at ads multidimensionally, and actually create a creative universe of sorts. Every dot represents an ad, and the relationships between them emerge from the underlying creative itself, rather than exclusively from attributes we've defined in advance. This is how Meta works in the back end: through embeddings (we'll touch more on this later). We wanted to translate that into our search for creative alpha.

This opens up a much more interesting way of understanding creative. We can see where a brand has concentrated its efforts, how its ads relate to creative from other brands, and which parts of the broader creative universe it has not explored. We can also overlay performance to understand where we've found winners, how extensively we've tested variations of those ideas, and where we might want to experiment next.

Ultimately, we're trying to solve a fundamental problem in creative strategy: how do we systematically decide what to make next? Should we produce more variations of concepts that have already worked, or explore entirely different approaches that might unlock new sources of performance?

The Atlas doesn't tell us which ads will win, but it gives us a way to understand where we've been and make more deliberate decisions about where to go next.

Getting to a useful representation turned out to be much harder than we expected. Before getting into that research, though, it's worth understanding why creative diversity matters in the first place.

Why is creative diversity important?

From our experiences scaling accounts, it seems to us that efficient scale is a function of not just creative volume but differentiated creative. Whether it's a new format, a different way of communicating a product's value, or a completely novel execution, these discoveries often unlock meaningful increases in spend at attractive acquisition costs. Meta has also published research and case studies demonstrating improvements from creative diversification.

The Meta behind Meta

We've previously written a bit on how Meta's ad engine works at a high level in How Many Ad Variations Should You Make?. TLDR; Meta makes a decision of which ad to show by predicting how likely that person is to take an action and uses that predicted value as part of its advertising auction.

The important thing is that an ad doesn't have a fixed probability of converting. The same ad can perform very differently depending on who sees it, when they see it, and what they're looking for. Historically, advertisers tried to solve much of this problem themselves through manual media buying. You selected demographics, interests, lookalikes, and audiences, then supplied creative to those groups.

As we know, Meta does much of this automatically now and that trend doesn't seem to be stopping anytime soon.

In its Scaling User Modeling research, Meta describes learning from enormous amounts of user behavior to develop a much richer understanding of people. Rather than describing someone through a handful of interests, like fitness, finance, or travel, Meta's models look at users through hundreds if not thousands of variables to predict what they're likely to respond to.

But understanding people is only half the problem. Meta also needs to understand the ads themselves and how their characteristics interact with the interests and behaviors of particular users. Its more recent Generative Ads Model, GEM, describes learning these relationships across user behavior, creative representations, and advertiser objectives.

This is why you hear that content is the targeting. It's a nice way to say that Meta is looking at both user and content embeddings to determine how to match users and ads in the most effective way. The creative then becomes such an important lever for reaching the right audience, and creative diversity being important follows.

In its Andromeda research, Meta describes rebuilding its ad retrieval infrastructure to accommodate an increasingly large and diverse universe of ad candidates. It explicitly argues that greater diversity among retrieved ads can improve people's experiences and advertiser outcomes. The intuition is straightforward. If different people respond to different ads, giving the system more genuinely differentiated creative creates more opportunities to find effective matches.

This brings us to a problem we've been thinking about for a long time: how do we decide what creative to make next?

Explore versus exploit

At a high level, there are two ways to improve a creative program. You can explore new ideas, or exploit ideas that have already proven themselves.

Neither choice is inherently better, and we've seen enormous value from both approaches. The reality is you need to do both. The math problem at the core is what is called a multi-armed bandit (can read more about it here). Performance creative introduces many nuances that are interesting, markedly that creative can be sequenced (think BOF/TOF creative) and that creative fatigue is a thing. It doesn't appear to us that there is any reliable mathematical solution as those variables of sequencing and creative fatigue are unpredictable. So we must solve the problem a different way.

If we exclusively exploit existing winners, we risk becoming increasingly dependent on a narrow set of concepts. If we exclusively explore, we risk abandoning ideas before we've extracted their full potential. And the optimal balance is constantly changing as we discover winners, scale them, and encounter diminishing returns.

We've increasingly come to think about creative strategy as a search problem. Every new ad is an experiment in a much larger universe of possible ideas. Some experiments help us improve regions where we've already found success, while others help us discover regions we know very little about.

But there's a fundamental problem with approaching creative this way.

How can you systematically explore or exploit a space that you can't actually see?

To exploit a winning concept, you need to understand what makes other creative similar to it. To explore, you need to understand what you've already tried and what would represent a meaningful departure from those ideas. And to allocate resources intelligently between the two, you need some understanding of where your historical creative sits relative to everything else you could make. This brought us back to our original frustration with structured creative analytics.

Our taxonomy could tell us how many podcast ads we'd produced or whether reactions were outperforming testimonials, but it couldn't adequately represent the relationships between individual creative ideas. Two ads could belong to different formats while sharing almost everything that made them compelling, or belong to the same format while being completely different in execution.

We needed a better representation of the creative space itself.

Mapping the creative universe

Looking at how the platforms understand creative, we realized pretty quickly that they are not looking at creatives in a pivot table. Rather, they're looking at creatives in a high-dimensional embedding space. They literally say it very clearly in the Andromeda research that everyone cites. Yet everyone still thinks of creative in a pivot table, even though Meta, in the big research paper that everyone talks about, says we look at this in a multidimensional space. And so we basically said, "Well, what if we can do the same?"

And this goes beyond just the famous Andromeda announcement. Meta researchers describe using embeddings in several papers, including Semantic ID research, where they go into how to use embeddings to understand relationships between ads rather than treating every ad as an entirely unrelated piece of content.

An embedding is essentially a mathematical representation that allows a model to recognize how something relates to other things. Imagine placing every ad on a map, where similar ads are positioned closer together and more different ads sit farther apart. Importantly, these relationships don't need to be limited to categories that humans have explicitly defined.

Meta uses learned content relationships to improve its ability to predict advertising outcomes. If two ads share meaningful characteristics, the system can use what it has learned about one to inform predictions about the other, even when the second ad has relatively little performance history. We go into this idea of collaborative filtering back in our talk in 2024.

Does similarity in these emedding representations actually correspond to advertising outcomes? The answer is yes.

Meta tested this directly. In an online experiment, it took ads selected by its recommendation system and randomly replaced some of them with other ads from the same semantic neighborhood. The closer the replacement was to the original ad in the learned representation, the smaller the decline in click-through rate.

This was a particularly interesting result to us. It suggested that the relationships captured by embeddings weren't simply useful just for categorizing content. They contained information relevant to actual advertising response, even if that relationship was far from perfect. And it raised a possibility we hadn't been able to realize with our structured creative taxonomy.

What if we could use similar representations to map the entire creative universe? Instead of manually defining every relationship between ads, we could use models to represent the creative itself and understand how different ideas relate to one another.

Rather than looking at dozens of creative variables in a spreadsheet, we could see the creative portfolio as a whole and navigate it in a way much closer to how human strategists actually think about ideas.

This is where Ad Atlas was born

Every point on the Atlas represents an ad. Ads that the system considers similar sit closer together, and different creative approaches occupy different regions.

Obviously, the map isn't going to tell us exactly what ads will win. If it did:

1. We'd IPO.

2. We wouldn't write a paper about it.

What Atlas does show is very much that, an Atlas. Atlas is a representation of creative search that allows for the question of what to make to be more guided. Of course, that only works if the map actually represents the creative differences we care about. And our first attempts revealed that this was a much harder problem than we originally expected.

Our first attempt at mapping creative

Our first attempt at Atlas was relatively straightforward. We took tens of thousands of ads from across our clients, generated embeddings of their videos, and plotted them in a shared space. The idea was that ads with similar creative characteristics would naturally sit closer together, and we'd be able to identify clusters of related ideas without manually categorizing each one.

The first atlas was broadly interesting, but we quickly noticed a problem. Rather than organizing ads by how they were made, the models were largely organizing them by who made them (useless).

In our original Atlas, 42% of an ad's ten nearest neighbors belonged to the same brand. Ads for a particular product tended to form tight clusters, even when the actual creative approaches were quite different. A podcast ad and a product demonstration for the same brand could sit closer together than two nearly identical podcast ads from different advertisers.

This makes sense when you consider what many embedding models are trained to understand. A sophisticated model might correctly recognize that two videos advertise the same product.

But this wasn't particularly useful for our purposes. We didn't need a map to tell us that all of a GLP-1 company's ads were about GLP-1s. We wanted to understand how its creative compared with everything else being produced across the market.

Improving the Atlas

We evaluated 103 map configurations across 1,100 tested ads from 62 brands, combining different video models, craft attributes, and methods of separating creative characteristics from brand identity.

One of the more surprising findings was that the most sophisticated semantic models weren't necessarily the best for our task. Models designed to understand what a video was about often grouped ads by brand or product, while models focused more on visual characteristics did a better job of distinguishing how ads were actually constructed.

Bringing back structured creative data

Clearly, an embedding-only approach can be quite unoptimal. The dumber the model, the more useful it was. This is evidence that the approach wasn't necessarily the right one. We tapped into structured creative data again, but this time was different. Rather than trying to describe everything about an ad through predefined categories, we used these craft attributes to establish which characteristics the embedding should preserve.

Our lead researcher on the project bogged down everyone on the creative team to get him feedback and bribed folks with free lunch.

The bribes worked. We eventually expanded our taxonomy from 17 questions to 91, covering everything from format and hooks to people, setting, camera work, editing, and sound. Surprisingly, most people agreed on what was important, and it seemed this intuition is more universal than we thought. We combined these attributes with visual embeddings that could capture nuances our taxonomy wasn't capable of describing.

We also developed techniques to remove recurring brand identity from the video representations without eliminating legitimate creative similarities. A brand that consistently produces podcast ads should still be close to other podcast ads, even if we remove the features that make its videos recognizable as belonging to that brand.

Our recommended configuration brought the amount of same-brand neighbors from 42% to 29%, almost exactly matching the 28% baseline which would be brought by shared craft attributes alone. Now the ads are clustered closely based on creative attributes first and foremost, not the brand they represent.

This might seem like a technical distinction, but I think it illustrates something important about embedding-based creative analytics. The representation you choose determines what kinds of similarities you're capable of understanding.

In our case, that meant we needed to combine structured attributes with unstructured video embeddings.

• The attributes provide some interpretability and help preserve meaningful creative distinctions

• The embeddings capture visual nuances that would be difficult to describe through any fixed taxonomy

And once we had a map that was useful to look at, we started asking what it could tell us about performance.

What we've been learning

There are no non-obvious universal winning creative clusters

We don't know why we thought there would be, to be honest, but for a long time it seemed like the holy grail of creative intelligence was discovering universal creative insights. Certain hook types, video lengths, formats, or combinations of attributes that consistently produced better outcomes than everything else. Surely, if we mapped enough creative and overlaid performance, we'd find some clusters that were clearly better than others, right? But that's not really what we're seeing.

When we color the global Atlas by performance (green being winners), winning ads appear throughout the space. There's no obvious cluster where all the winners sit, or one region of the creative universe that appears to dominate everything else.

The green appears broadly distributed across different creative neighborhoods. Podcasts win, reactions win, product demonstrations win, animations win, and a whole host of other formats work across advertisers.

And when you think about the meta of Meta, this makes sense. If an ad's performance is determined by its interaction with the audience and the context in which it's delivered, why should we assume that any one creative approach works universally?

This is one of the reasons we're skeptical of the current excitement around predictive creative performance, which structured creative analytics aims to push. When someone pushes out that testimonial creative has better CAC than a street interview, say, they are implicitly making the argument that one is "better" than the other. But it doesnt capture the fact that you can probably take some elements from each and make a great ad. Even more sophisticated machine learning based predictive approaches fail here because historical data sets are limited. A model might identify that a certain set of creative variables performs well on average across a historical dataset, but that doesn't mean it will perform well for a particular advertiser in the future, let alone that it will be incremental to the outcome.

We're not arguing that creative performance contains no predictable relationships, or that there are no general principles of good advertising. But we are increasingly skeptical that any universal insights are valuable, outside of obvious ones ("Make engaging ads!" Thanks for letting us know). And we would argue that this approach is futile even within the subtext of one brand because predictive analytics will force an advertiser to keep exploiting over exploring, which we know is important for maintaining a healthy ad account (a la creative diversity).

Creative performance is stochastic, so you need to explore sufficiently

In 2024, we thought this was how performance creative worked.

We were very wrong.

This is perhaps the most underdiscussed point in the industry. Ad performance is NOT deterministic. There are two camps of belief that this goes against:

• Only test uniques because andromeda, or whatever voodoo

• Predictive creative analytics, because just testing an ad is so expensive (it's not)

The reason we're skeptical of these camps of thought is that winners and losers can sit almost on top of each other in the Atlas.

Two ads can be nearly identical but produce dramatically different outcomes. One might be a winning creative that scales efficiently, while another, differing by little more than a hook or some editing choices, might have much worse performance.

We've seen this repeatedly in our own testing. In How Many Ad Variations Should You Make?, we analyzed 2,360 ads across 365 concepts and 31 clients. Within winning concepts, the average difference between the best and worst variation was 4x! That's more than four times the performance difference within the same underlying creative idea, sometimes between ads that differ by little more than the wording of a hook.

There are three reasons we believe this happens.

• Small creative differences can have an outsized impact on how people respond. The same offer could be compelling when communicated through one hook but completely ineffective when communicated through another.

• Observed creative performance is inherently noisy, in part because of how Meta allocates delivery. Meta doesn't distribute impressions equally across creative. Its models make predictions about which ads will perform best, allocate impressions accordingly, observe the results, and continuously update delivery. At small testing budgets, a relatively small number of conversions can have a significant influence on an ad’s observed performance and the evidence available to the system.

• Meta is stochastic in its predictions and has documented this several times. In Enhancing Embedding Representation Stability in Recommendation Systems with Semantic ID, Meta's researchers investigated how the representations used by its ranking models affect their predictions. As we discussed earlier, they found that ads closer together in their learned semantic representations tended to produce more similar click responses. But they also discovered something surprising: exact copies of the same content, assigned different identifiers, could receive different predictions from Meta's ranking models.

This illustrates a broader problem: even when the creative itself is unchanged, the system serving it is not static. Therefore, outcomes aren't deterministic.

This creates a fundamental problem for predictive creative analytics. If ads with nearly identical observable characteristics can produce dramatically different outcomes, and those outcomes depend on the audience, delivery conditions, spend, and time, how confidently can we predict a new ad's CAC simply by examining its hook, format, creator, or other structured attributes? You can't do so with a great deal of accuracy.

But there are certainly relationships worth learning, and embeddings can help us identify them. Meta's research provides evidence that content similarity contains useful information about advertising response. But similarity is not equivalence. For us, this reinforces why exploration and exploitation are a more useful paradigm than attempting to predict the winners in advance.

When we enter a new region of creative space, testing one creative may tell us very little about the potential of that region. A poor result could reflect an actually bad concept, a weak execution of a promising concept, or simply insufficient evidence. If we abandon every idea after its first unsuccessful test, we're likely to miss a substantial number of opportunities.

Instead, we need to explore sufficiently to understand which concepts have potential, then exploit the promising ones through further variations. This changes how we interpret the map. A cluster becomes a region of creative space to explore and validate rather than ads we expect to perform similarly.

The opportunity for brands

That's all great, but how can I make more winners now?

Ah, we figured you'd ask.

Within individual brands, we think Atlas can help teams understand their opportunities much better. Rather than trying to discover the globally optimal creative format, we can investigate where a particular advertiser has found success and what opportunities might exist nearby.

Making creative diversity measurable

Imagine a brand producing 100 ads. They might all be slightly different talking-head videos, or they might span podcasts, animations, product demonstrations, street interviews, and dozens of other approaches. Both portfolios contain 100 ads, but their actual diversity is very different.

Even a portfolio spanning many formats might be relatively narrow if every ad uses the same positioning, creator, or underlying concept. Creative diversity is multifaceted, and the Atlas gives us a way to begin measuring it beyond asset counts or manually defined categories.

We're particularly interested in three distinct properties.

Internal diversity measures how different a brand's own ads are from one another. One promising approach is the Vendi Score, which uses the pairwise similarities between items to estimate the effective diversity of a collection.

Global coverage measures how much of the broader creative universe an advertiser has explored. With a reference library of over 50,000 ads, we can begin estimating how much of the broader creative space a brand's portfolio covers and whether new production is expanding that coverage or concentrating further within familiar clusters.

Platform nativeness asks a different question: how closely does an advertiser's creative resemble the ads people actually consume on a particular platform? Rather than measuring an ad's distance from the mathematical average of all ads, which might not resemble any actual video, we can compare its distance with the nearest clusters.

These are still developing measurements, and we haven't established how strongly they relate to advertiser outcomes. None of the metrics above is a performance prediction, but there's plenty of reason to assume that they correlate with better advertiser outcomes and help us understand what opportunities to capitalize on.

How we're using the Atlas

Historically, a creative performance review might involve looking at the top-performing ads, breaking them down by format and hook, and recommending more of whatever appeared to work. If the account wasn't scaling efficiently, we'd often discuss whether it needed more creative diversity, but that recommendation was difficult to make precise.

With the Atlas, we can begin by looking at where the advertiser's creative actually sits and identify nearby creative opportunities. A brand might have discovered a strong concept but hasn't exploited it well. The Atlas can make that concentration visible and help us identify nearby creative approaches that might be worth exploiting. For example, for one of our partners, we identified that while podcast content has performed quite well for them, they lacked in creator diversity.

We can also compare a brand's creative against the broader market. Rather than recommending a generic list of new formats, we can identify actual examples of creative that occupy regions the brand hasn't tested, and use those as references for new production.

This is closer to how our forward deployed creatives naturally work. They recognize that an existing ad resembles something that worked elsewhere, or that a new format represents a departure from what a brand has already been doing. The Atlas makes those relationships visible across a much larger collection of creative than any individual strategist could reasonably remember. When you think about it, creative is creative. Trying to overly structure it results in decision paralysis and worse outcomes.

Importantly, we're not assuming that every unexplored region represents a good opportunity. Some regions might be unexplored because they aren't particularly effective for the product being advertised. And some of the best opportunities might exist within neighborhoods where we've already produced substantial creative.

The Atlas doesn't replace creative judgment or experimentation. But it is a critical step in our understanding of creative intelligence.

Toward a systematic explore/exploit process

The longer-term opportunity is to connect this representation to an increasingly systematic process for deciding what creative to make next.

Today, most of creative analytics is retrospective. Brands produce ads, test them, analyze the results, and try to infer which characteristics drove performance. They then produce more creative based on those observations.

We've spent a lot of time doing exactly this, and there's certainly value in it. But it doesn't fully solve the decision problem we face every time we're planning a new batch of creative.

Should we produce more variations of an existing winner? Should we try a completely different concept? How much of our production budget should go toward relatively safe iterations versus ideas with much less evidence behind them? The Atlas gives advertisers a way to make more informed decisions.

Over time, as we accumulate more performance data, we want to understand how these relationships actually translate into outcomes. How much does proximity to existing winners increase the probability that a new variation succeeds? How do you use signals from the organic market to better inform exploration? Does expanding into new creative regions unlock incremental reach, better CPMs, etc?

These are the kinds of questions we think creative intelligence should be answering. Rather than trying to predict the performance of every possible creative in advance, we want to build a system that becomes progressively better at choosing which experiments are worth running.

The search for creative alpha

Meta has spent years building increasingly sophisticated systems to determine which ads to show to which people. We're trying to apply the same systematic thinking to a different problem: which ads should we make in the first place?

The creative universe is effectively infinite. But production capacity, testing budgets, and attention are not. We can't make everything, and we certainly can't test everything reliably, so we need to get better at deciding where to allocate those resources.

That's what we're building with Ad Atlas. A way to understand the creative universe visually, recognize where we've already found success, and systematically balance exploring new ideas with exploiting the ones we've already discovered.

We're still early, and there are fundamental questions we don't yet have answers to, particularly around how differences in creative space translate into differences in incremental advertiser outcomes.

But for the first time, we have a quantiative way to reason about creative diversity beyond asset counts and predefined categories, and use that understanding to guide what we make next. We think that's a much more promising direction for creative intelligence than trying to predict tomorrow's winners from yesterday's vocabulary.

If you've made it this far, chances are you should see your own embedding.