It’s our job to help them make a more informed decision.

That’s where this workbook can help.

This is how I start auditing clients’ evergreen content offerings. I want to help them understand which content categories warrant further investment, should be defended or de-prioritised. You know, with actual traffic, engagement and commercial data.

I want the end goal to look something like this (real data from my site, for illustrative purposes only).

Subscribe now

What you need

Three exports:

• Google Search Console: Pages, latest 3 months vs the same period last year (comparisons can be fluid, but that’s where I start)

• GA4: whole-site landing-page data for the same current period (There is a 93-day limit when a GA4 Exploration uses user-based segments/user data)

• GSC Generative AI data: optional, if available, from Search Console’s Generative AI report

You’ll also need to score three things the data can’t sensibly decide for you. You know, where you use your brain:

• Defensibility: how difficult the content is to replicate

• Strategic Importance: how important it is to the business, brand or audience

• Update Effort: how much work it would take to improve or maintain

Everything else is calculated automatically. The end scorecard looks something like this:

And there is URL level data should you wish to break things down at a page-level.

Worth noting that update effort doesn’t change the current value. It only changes the recommended action after the verdict has been calculated. I don’t want to penalise critical content because it takes a lot of effort - good content should do that.

And here’s how it works in practice

And for newer paid subs, you can understand and estimate your generative AI risk using Search Console data with this article and workbook combo.

Those obsessing over AI and GEO further can use this, I think, excellent guide to entities and subsequent workbook to audit your consistency and AI readiness in the eyes of our machinated overlords.

Get the workbook here

Read more

Hello, sorry is that Sundar? I think the Generative AI report is broken - I can’t see the click data. You what? You meant it to look like that?

Oh. Is that because AI features don’t send any cli… No, that’s just a coincidence you say.

OK, well do we at least get to see any query-level…

Nothing?

Given this has taken up the best part of two years of our working life, I’d have liked something a little more concrete to show to clients, friends and my long-suffering wife. I can’t have scintillating dinner conversation without click-level, generative AI data.

Something must be done - and I think I have a solution.

TL;DR

1. Google’s Generative AI Report gives you visibility. That’s it

2. But add commercial data and traffic dependency and you can calculate exposure

3. Which doesn’t equal risk. But it’s close. Branded search, direct and returning audiences, and content defensibility can get you even closer

4. AI can be an opportunity. The licensing market is simmering. Server logs reveal bot access by crawler type. They help you identify where to invest

This article has a paid upgrade, as do others. You can see the benefits of a paid subscription and what premium members get here.

Subscribe now

Partnership post

If you’re interested in bot access, server logs and analytics data, I would highly recommend looking into the team at Publytics, particularly their new server-side bot analytics tool Honeylog.

Fantastic value, a brilliant product for publishers in this era looking to truly understand on-site bot behaviour and lovely to work with!

Let’s start with the problem

Technically we have generative AI data in search. First-party visibility data should be a huge deal. But there are limitations.

Impressions, impressions everywhere, but not a drop to drink.

If you’re unfamiliar with how an impression is counted, it’s worth a quick wander down John Mueller trail:

Directly shown links can count without expansion; links requiring activation count after activation. Not that helpful really.

However… I don’t think the report is useless. At all. I think it’s almost useful. It provides first party visibility data in arguably the most used AI interface in the world.

So what does it show?

Visibility. Not exposure, risk or value.

At least not directly. Click data would take us halfway there. But to understand your true exposure, you would need to understand how your traffic and commercial value is intercepted by said AI system(s).

So, as ever, we need to layer data together.

For the core model, you’ll need access to:

• Search Console’s Generative AI report

• GA4 - or your analytics platform of choice

• Commercial-level data (revenue, conversions - you choose)

And if you want to go further:

• AI Overview prevalence across your important queries

• Server logs showing AI crawler activity

Server logs give us a proxy for demand but they are an optional extra. You can still get some superb insights with GSC and audience and commercial data from GA4.

Fortunately for you, I’ve built a diagnostic in Google Sheets to help you turn Google’s Generative AI data into something you can actually make decisions with. See step 6 - paid subscribers only!

Here’s how I think we can approach it

A step-by-step guide to working out your resilience and exposure using Google’s intentionally half-baked reporting. As ever, you can listen to a short overview of this, with a sneak peak of the workbook here:

Step 1: Get the data out of the GSC aggregate

Take the total sitewide AI impressions from the Generative AI report, then export the page-level data. Worth noting that the Pages export doesn’t necessarily represent every page or impression because of the thousand-row limit.

That matters because we’re going to classify those URLs and analyse visibility by content area. To start with, subfolder is more than fine:

• /news/

• /sport/

• /business/

• /lifestyle/

If you’re smart and you understand how to get more out of Search Console, then you can export the report at a subfolder level and combine data from each property. For smaller sites, this shouldn’t represent much of a problem.

Step 2: Understand AI visibility concentration

The page-level export lets us compare different content areas and answer a few useful questions. Is most of our AI exposure coming from News, Sport or Business? Is that visibility spread across lots of URLs, or concentrated among a handful of ‘performant’ pages?

This means we can establish:

1. Share: Where is attributable AI visibility concentrated?

2. Representation: Which sections and URLs appear in the dataset?

3. Intensity: Is that visibility evenly distributed or concentrated among a handful of URLs?

4. Coverage: How much of Google’s total reported visibility is represented in our page-level dataset?

You can still get the trend of AI impressions - up or down over time - directly from GSC.

We also need to understand how much of Google’s reported AI visibility our page-level dataset actually represents. If you go over the 1,000 row limit, you are seeing a sample.

Step 3: Include Commercial data

Now we know where AI visibility is concentrated. But how much does that actually matter to the business? Have we reached the final boss of AI buggery, or has the robotic sodomy just begun?

Keep this simple.

Pick one primary commercial KPI - revenue, subscriptions, conversions, leads etc. - and use the same KPI across every subfolder. The metric doesn’t really matter, provided it represents business value and you use it consistently.

You can layer in secondary commercial metrics - registrations, propensity to subscribe, advertising yield or whatever else matters to your business - as supporting context. But our primary measure of Commercial Value should remain consistent.

Please don’t overcomplicate this. Believe me. My locks might be lustrous, but after 10+ rewrites they are greyer than when I started.

Now we can start comparing visibility against commercial value.

Here, Business has high AI visibility, but higher revenue. Technology has comparable AI visibility, but significantly less revenue. Sport has very low visibility, but a larger portion of revenue.

This is a much more rigorous assessment.

Step 4: Understand AI Substitution

For this, you’ll need some third-party SERP data. DataForSEO is great. Other scraping tools are available, per BBC guidelines.

If you already have a representative keyword universe for each subfolder, use that. If not, the top 100 or 1,000 GSC queries per site or section is a perfectly reasonable place to start.

For each query, establish whether it triggers an AI Overview or other AI-mediated search experience. This doesn’t tell us that AI actually substituted a click. You’d need some kind of user testing panel for that and I’m not your guy for that level of rigour.

It just tells us exposure in the current search landscape.

You don’t need this data. Substitution is an enhancement, not a requirement. If you have it, great. If you don’t, you can still calculate Commercial Exposure.

Step 5: Server logs baby

AI crawler activity.

If you’re new to the game, server logs are fantastic. They give away detail you cannot get anywhere else. They look a little scary and technically, you don’t need them for this. But they add a layer of uniqueness and granularity you cannot replicate.

What we want to understand is:

• AI crawler access

• Crawling frequency

• Which sections and URLs they’re accessing

• What type of AI crawler is making the request

That last bit is increasingly important. Different crawlers are doing different jobs, so rather than lumping everything together as ‘AI traffic’, we can separate activity into training bots, search bots and retrieval bots.

That gives us a kind of AI demand signal: what is being access, where and - to an extent - why?

Gone are the days where we only care about Googlebot. Which I will say, is almost exciting.

One caveat: server logs only prove access. A training bot requesting a URL doesn’t prove that content was used to train a model. So we can’t directly prove usage. But we can say:

If you’re using the workbook and want to include server logs (optional), you’re still going to need to run the server log analysis prior to input. You’d need to aggregate by subfolder and bot type.

If you paste a 700MB Apache log into Google Sheets, don’t blame me because something breaks.

Subscribe now

Step 6: Creating the commercial exposure model

There’s one final input we need: Traffic Dependency.

How dependent is each section is on organic search?

A site or subfolder with just 10% of its traffic coming from Google is lower risk than one with 90%. Particularly if the majority of the traffic is direct or branded.

Now we can bring these four inputs together:

1. AI visibility (V): Where AI visibility is concentrated across the site.

2. AI substitution (S) - optional: AIO saturation at a query level.

3. Traffic dependency (T): How dependent that content or business area is on organic search. Basically the proportion of total sessions coming from Organic Search.

4. Commercial value (C): How much of the site's actual business value is generated by that content or business area.

In the workbook, I have normalised these values to a 0-100 scale for simplicity. I really suggest you do the same, otherwise the numbers look insane.

Huge AI visibility shouldn’t produce a huge exposure score if the content has almost no commercial value. Equally, something incredibly valuable isn’t automatically high risk if it has little AI visibility or limited search dependence. Unless you’ve fucked it up somewhere.

And remember: Substitution is optional.

If you don’t have reliable AI Overview data, don’t enter a zero. Zero means you measured substitution and found none. Just leave it blank and use the Core model:

That gives us a single AI Commercial Exposure score from 0–100, showing where business impact is likely concentrated. Something third graders can understand.

The Google Sheets Diagnostic

Instead of farting about trying to do this manually, you can use this diagnostic to calculate your current visibility, risk and exposure.

The workbook includes:

• AI Visibility analysis by subfolder

• Core and Enhanced Commercial Exposure models (enhanced includes AIO data and requires a little server log analysis)

• AI Resilience scoring

• Optional AI Opportunity analysis using server logs

• A final scorecard comparing your content areas

Of course if you subscribe and you either a) need some help or b) have a suggestion, just let me know.

Step 7: Measuring resilience

Now we have commercial exposure. But we need to know how vulnerable we are if AI turns up the heat? Or when it turns up the heat, is probably more accurate.

For example, let’s say you’re a company that focuses on live sports and statistics. Google’s latest Live NFL Game Feed might send a shiver down your spine.

Two sections could have exactly the same level of commercial exposure but face very different levels of risk. If AI does indeed try to kill us all, you’ll want to know how well protected each subfolder or business unit is before you pass.

One with a strong direct audience, loyal users and difficult to replicate content is in a very different position to one dependent on organic traffic.

To measure that, we’re going to look at four things:

• Branded search: How much organic search demand is specifically for your brand.

• Direct audience: How much traffic comes directly to you rather than through an intermediary.

• Returning audience: How much of your audience keeps coming back.

• Content defensibility: How difficult your content is to replicate or substitute, based on uniqueness, effort, expertise, proprietary information and access.

The first three can be expressed as percentages. Content defensibility requires some actual judgement on your part, so we’ll score that from 0–100 using a consistent, qualitative framework.

Combine this audience data and qualitative content review into a single score.

The higher the score, the better protected that section is against further search substitution or intermediation by AI.

Step 8: AI opportunity

AI’s getting a hard rep.

Sure, it’s decimating reading ages and people’s critical abilities to the point where they can’t tie their shoes. But who needs critical functions when you’ve got a phone and a $20 subscription to Claude Code?

And who knows, there might be some kind of financial opportunity on the horizon. One you can now take advantage of.

By mapping our server-log data to individual pages and subfolders, we can compare AI demand with our own assessment of content value.

• Frequency: How often AI systems access our content?

• Breadth: How widely they access it?

And alongside that:

• Purpose: What types of crawlers are responsible?

High-value proprietary content being hammered by retrieval crawlers could represent an opportunity, a liability, or both. But it’s something you’d probably want to know before making decisions about bot blocking, licensing, further investment, partnerships or AI distribution.

Crawler activity does not make content valuable. Nor does it mean an AI company will pay for it.

But combine AI demand with visibility, commercial value and content defensibility, and we have something, I think, genuinely useful. Something that says:

We have to help companies understand the value of their content. As SEOs, we’re better placed than anyone to do that.

Allegedly at least. Anyway, ‘til next time!

Leave a comment

I argued that SEO auditing should be broken down into four parts:

1. Technical SEO

2. Content

3. EEAT

4. Entities

I think almost every concept required for modern information retrieval is included within the above four themes. Outside of wider marketing (which is arguably the most important facet of ‘optimising’ for LLMs thanks to its impact on training data), everything you should be doing as an SEO should align with discoverability.

And Entity SEO is not a new concept. Google has a long history with understanding and disambiguating entities (people, places, organisations et al). Google’s Knowledge Graph has been around for 15 years and is quite literally a database of entities.

The content around the entity became more important than ever. Things, not strings. Old school.

This is dangerously close to GEO.

TL;DR

1. Entity SEO isn't new. It's making sure modern information retrieval systems know who you and your brand are with clarity. It maximises your visibility.

2. Control your owned properties. Make sure what you say and what you do matches how you want your brand to be perceived.

3. Earn quality mentions. Influence the sphere most appropriate for your audience and your product positioning.

4. Build an entity map. This will help you find errors and inconsistencies great and small across wide-reaching corners of the internet.

Subscribe now

As always now, if you’d like to listen my dulcet tones rather than read, that is your want and right:

What is entity SEO?

Entity SEO is the practice of reducing ambiguity around the people, places, organisations, products, events and topics that really matter to your business. Google's Knowledge Graph is the most obvious and accessible example of a system that represents entities and their relationships.

The on-page content, the internal and external links, the navigation and your owned and earned consistency (and scale) are of critical importance.

Everything you say about your brand needs to be accurate. What other people say about your brand needs to corroborate this. You are trying to control the controllables, and ensure the right entities are talking about your brand.

That second part is more marketing specific and something that sits outside of most SEOs remits. I like the idea of having a Director of Marketing in control of the inputs and outputs here.

Or a Director of Discoverability if you wanted to be fancy.

Why does it matter?

If Google and LLMs can't confidently ‘understand’ who you are, it's harder for its systems to correctly interpret and connect the signals around your brand. You need on-site consistency and third-party presence to provide real clarity.

An assurance to a system tired of bullshit and bullshitters that you are who you say you are.

Metrics like branded search volume have always been important to track. Direct traffic to specific sections of your site too. You want to understand and establish where you have real user demand.

You should really understand what types of places or events you have real authority in in the eyes of our machinated overlords.

And yes, while AI systems are not exactly world-leading when it comes to filtering out spam (see ChatGPT using site: search modifiers for brands it ‘trusts’), this isn’t going to be the case forever. So stop pissing about.

News is an entity edge case

When users search for the news, they search with short-tail entities. They hear about a story and search for the person, event or place associated with it. It’s short-tail and relatively lazy.

Top Stories - a news SEO’s primary signifier of success - is dominated by entities. Headlines, opening paragraphs, images and the wider makeup of the page needs, nay demands, an entity. A focal point.

You’re trying to establish what this story is about as quickly and efficiently as possible for a crawler that may crawl your news article once and then only again once the spike in demand has passed.

It’s why it’s so important to be accurate on first pass. Google’s news crawling system may give you one chance.

The content and mentions publishers generate is what positions you as a brand. If you keep creating horse racing content for betting-specific purposes (I’m not pointing any fingers here), you should think about whether that relationship may be doing more harm than good.

An entity SEO framework you can follow

I don’t mean in the sense of ‘even someone like you can follow this.’ Although this should be simple. I mean it from a more tactical perspective. You can audit your own entity presence across owned and earned properties and try to take action.

This is particularly important for smaller brands and publishers. It’s not like the BBC or NYT have to worry about whether someone spells their brand name wrong. But if you are (or have) a less established brand - personal or organisation - then this type of consistency matters.

Everything we classify under Own, Describe and Prove fall under your control. They are all editable properties.

If you remember working in SEO as a junior ten years ago or more, you’ll remember the long list of mentions someone would send you to turn into backlinks. In my experience and humble opinion, this was a completely fruitless exercise and a complete waste of time.

In the spirit of that, I suggest you focus almost all of your time on your owned properties. The part of your digital presence you can fix without sending snivelling, guaranteed to be ignored emails.

A spreadsheet audit (paid)

You can, of course, build an entity audit and framework based on this article. But for paid subscribers I’ve built a sheet you can use to define:

• A consistent brand ‘Source of Truth’

• A full asset inventory split into coverage and consistency

• An entity corroboration vs topical corroboration checklist

• A priority fix column

Should you wish, you could absolutely build this spreadsheet from the article. But I will add something tangible and practical for paid subs each month to help solve problems.

Own - establish the entity across controlled assets

Make a list of everything you own. Editable assets.

• Website (About Us pages, Contact pages, privacy policy, footer et al - don’t forget structured data and sameAs)

• Your corporate website too

• People’s profiles

• Social channels

• Google Business Profile

• WikiData and Wikipedia (if applicable)

• Marketplaces or platform profiles

And make sure your description of your brand, NAP (Name Address Phone Number, shout out old school local SEO), leaders, contributors - everything that should be consistent should be consistent. Your links all point to the right endpoints.

Before starting any grandiose plan for world domination, make sure your owned properties and the information on them are consistent and accurate. We could colloquially call this entity hygiene. Look, it’s unlikely to make a dramatic improvement, but you’ve got to start somewhere.

All of it is ambiguity. Ambiguity isn’t just something you prevent. It’s something you clean up. It removes entity drift.

Before diving in headfirst, worth making sure you know what the correct version of your brand is. Probably a bad idea to start making changes without knowing what godawful version of your company the comms team and C-suite want to establish.

Describe - make the entity and its relationships explicit

We can think of describe as more of a page-level property.

• Does each page explicitly target the right entity?

• Does the surrounding information back this up credibly?

• Does your header structure answer related questions?

• Do you use internal and external links with consistent anchor text?

• Do you answer topically relevant questions?

This is all about understanding and matching user intent more effectively than anyone else. The same consistency you apply to your brand you should apply to your most important pages.

You’re building relationships. Between words, sentences, paragraphs, authors and brands. A big stinking interconnected mess. And your organization schema should define who your brand is. Article and person level schema corroborate information about the individual page at hand and sameAs schema joins the dots between your brand.

On large sites internal linking is more often than not one of the best places to make scalable improvements. You want articles about mortgages to link to the mortgage tag page. You want your tag page to house the hundreds or thousands of articles you have written on the topic to be findable. Ditto your author page.

Make your experience explicit.

Prove - establish who your brand and people are and why they’re credible

Right, you’re working with someone who’s won multiple awards, has 100s of thousands of subscribers and is mentioned consistently in your sphere of influence. Or desired sphere of influence.

a) Do you think it would be good to run some product marketing and to shout about the fact you both trust each enough to work together?

b) Or should you just ignore everything and keep ploughing on like it’s 2015 and everything will be fine?

If you answered b, I suggest a different career path.

Every person who matters - writers, founders, executives, named experts - needs one consistent, verifiable identity: an author page linked to a real LinkedIn profile, a real publication history, structured data validity, contact link(s) and a bio explaining who they are and why they’re trustworthy.

If they have credentials, show them. If they have won things, highlight them. If they have a Masters in Economics and have worked for Bloomberg for 10 years, that might be useful for your audience and information retrieval.

There are always small wins here. Highlighting how you conducted research and evidence for an article, what your testing protocols are like - really just solid EEAT-level applications are immensely valuable.

Earn - mentions and links that corroborate your entity salience

You don’t get to fully define your own entity. Some little wretch on Reddit or deep in the YouTube comments gets a say now too thanks to LLMs. Businesses can be made or destroyed through false, unverified information weaving through the sewage of the internet.

So you need to earn good stuff. You cannot fake this. Not sustainably. Not without leaving breadcrumbs that will eventually catch up with you.

Entity corroboration asks who you are - accurate brand mentions, consistent third-party descriptions and information, correct directory and association listings. It defines the basics.

Topical corroboration asks what you’re known for - expert commentary, interviews, citations, bylines that consistently connect you to the subjects you claim to specialise in. What you do - both good and bad.

It’s all very well claiming to be the world’s foremost experts on Bitcoin. But if the rest of the world sees you as a chef from Solihull, you’re swimming against the tide.

This is why mentions are becoming more valuable than ever. SEOs have been gaming the linking algorithm for years. Mentions are much more readily available. More difficult to accurately define, but better evidence of your reputation.

Monitor - see where and what you and your competitors generate mentions from

It might help to frame entity SEO as an ongoing discipline rather than a one-off project.

The marketing you do and influence you have in specific industries tend to dictate the scale and relevance of your mentions. You want to see what’s working (and maybe more importantly what’s not) for the industry or topic you value and apply those learnings.

Over time, you’ll notice that your identity as an individual or a brand can drift. As the volume of your mentions and citations grow, are they kept up-to-date? As employees leave the business, is that updated on the corporate site? Are third parties describing you appropriately?

All of these identity, description and relationship issues can be bundled into a concept called entity drift. That is a foundational concept of ‘GEO.’

LLMs are as good a representation as you can get of how the wider internet views you - because they scrape fucking everything.

So monitor how different models view your brand over the course of a few days or weeks and see whether it’s how you want to be perceived. You can take action based on these findings.

Subscribe now

This helps you build an entity map

An entity map (see Dixon Jones’ post here) is a structured representation of the important entities associated with your organisation and the relationships between them.

If you’re serious about entity SEO (who the hell isn’t), then this is a really useful exercise. It forces you to confront how you are perceived vs who you think you are. A terrifying thought.

At its simplest, start with your organisation and map the people, products, services, locations and topics connected to it. Then identify the evidence that supports those relationships.

How do I build one?

You can build a pretty comprehensive one just with Search Console query and page-level data. Keep it simple to start with. Plenty of time to over-complicate things and get frustrated in future.

1. Identify the primary entity. Who or what are you mapping?

2. Identify the connected entities. People, products, services, locations, topics, organisations, events.

3. Map the relationships. Founder of, employee of, based in, specialises in, writes about, produces, owns.

4. Attach identifiers and evidence. Official URLs, LinkedIn profiles, Wikidata IDs, author pages, relevant sources.

You should end up with something like this.

I guarantee you this is not just a useful exercise for traditional and AI search. It’s something you can put in front of senior people and marketing/brand leaders and show them your current positioning.

Entities and online relationships are the foundations of brand perception. You can run this same exercise using Similarweb or Ahrefs data to build a picture of where you sit vs your competitors.

Pro tip: Filter your Search Console data by country and create an entity map per country showing how you are perceived in different locations. It is rarely as you’d expect.

Extracting entities at scale

LLMs and their APIs are excellent at this. NER (Named Entity Recognition) is a well-established NLP task that can provide fast, repeatable entity extraction at scale. LLMs can then help with normalisation, disambiguation and relationship extraction. Brilliant if you trying to skim through 50k rows of queries or extract information from page metadata.

If you just want to use query data from GSC, you can set a model up to extract and classify entities into people, places, organisations et al. If you need to extract information from an article, I suggest running the pages through your crawler of choice, extracting key metadata (h1, meta description and the opening x characters if required), identifying the entity and disambiguating it based on the surrounding text.

How do we know Google and AI systems care about this?

Because Google has a 15 year+ history of doing so. From launching the Knowledge Graph to updates that led to wider semantic understanding of surrounding content (Hummingbird and Bert) through to understanding multimodal content formats (MUM).

In 2025, Kalicube reported that more than three billion entities disappeared from Google’s Knowledge Graph during a major June update. Kalicube interpreted the change as a move towards greater clarity and less ambiguity.

Google is making page indexation hard. It’s raising the bar for what qualifies as part of the Knowledge Graph. Quality, not quantity wins in this slop-filled era.

It’s a hugely important database - one that’s incredibly difficult to replicate. One your brand, products and people need to be in.

In summary

Audit your owned properties, make sure you’ve accurately described what you do and who you are, prove your people are real and do stuff that earns high-quality, highly relevant links and mentions.

This has literally been the foundations of SEO for nearly 15 years. Just because you haven’t done it or had to apply it anywhere, doesn’t mean it doesn’t matter. Don’t assume the internet understands your brand. It’s a shitshow out there.

The good news is shitshows are highly auditable:

• You need to know what your brand is actually supposed to say about itself

• Check whether it's saying that consistently everywhere you control

• Then go and find out whether anyone else agrees with you

This is high-level GEO wizadry. No mere mortal could understand this and I won’t hear a word otherwise.

Leave a comment

Discoverability has changed. That is, I think, undeniable. Passive, off-platform discovery dominates the early stage journey, but your website is your brand’s home.

Chris Green talked about SEO becoming more akin to feed management in the future, which I like. As an analogy I mean, less so as a concept.

But Google still sends over a trillion clicks a year to the open web and up to 86% of clicks are discovery-led. Websites and the wider internet feed answer engines and whilst we’re in the midst of change, it won’t be overnight.

Your website still needs to be good. It needs to rank. It needs to be crawled, rendered, indexed and cited. Particularly if your content and on-site (or app) experience is your golden ticket.

Here’s how I suggest approaching SEO and discoverability auditing for publisher sites. If I can help, you know where to find me.

The governing questions are the same

For as long as I’ve worked in SEO, the foundations have been the same:

1. Can machines reach and retrieve your content?

2. Is it worth surfacing (and retaining)?

3. Can they tell (and trust) who made it?

4. Can they tell who and what you are?

The last two (EEAT and entity relationships) are more recent additions and quality standards play a more prominent role in indexation. But the evolution of SEO - and SEO auditing - should be discoverability-led.

Tying together Googlebot, AI crawlers, answer engines, recommendation feeds and social discovery surfaces. The four questions don’t change.

We may not have direct control over social channels or all of the on-site content. But no one is better placed than SEOs to guide publishers and websites into making business decisions that drive value and effectively improve discoverability - through the passive, active and owned sections.

Every owned off-site signal needs to corroborate the information on-site. Encouraging individuals to disclose information and share content is a brilliant thing. But it needs to be accurate and consistent.

A person or team need(s) to own this.

Should you be so inclined, you can watch or listen to my dulcet tones instead of reading:

Subscribe now

What an SEO audit should do now

Everything it used to do and more.

Search remains the most valuable last-click channel for most publishers; contrary to popular belief we need it and Google needs us - although they have an extremely unusual way of showing it.

Saccharine sweet whispers about how much they care about the content ecosystem flies in the face of their actions.

The four primary pillars for auditing in 2026 are:

1. Technical SEO

2. Content and on-page SEO

3. EEAT

4. Entity (brand) consistency

At least in my opinion. We’re trying to create efficient, reachable websites, with brilliant content that people and bots implicitly trust. These four are non-negotiables - although you can cut them however you wish.

AI readiness is baked into these four areas, but for most publishers, LLMs have questionable value. Make sure you assess a site’s blocking protocols and whether they’re solely relying on the robots.txt.

Yes, links and mentions are still crucial. But not everyone needs to invest in them. I have bundled them under EEAT, but if they’re hugely important to your brand, then consider running a more detailed audit of them in future.

The most important thing: value

You’re not driving traffic or rankings or visibility. You’re driving value. Or maybe just trying to stop the rot.

Yes value is defined by the business’ goals. Whether you’re trying to increase ARPU, subscriptions or affiliate revenue, don’t just blindly jump in. Try to align your work with tangible business goals that make your stakeholders look good!

I suggest creating a longer-list of issues, aligning them on the scale x impact x effort scale and creating a quadrant to show you a) understand how this business works and b) avoid a long list of issues.

Not everything needs fixing. As Charlie puts it, knowing what to ignore is as important as knowing what needs attention.

Sitewide and template-level changes tend to have the largest scale. Recommendations that directly relate to key business goals tend to have the highest impact.

Make sure you can articulate the benefit of each suggestion to prevent yourself looking or sounding stupid - none of us know everything.

How do I know where to start?

This excellent graphic from Aleyda and Mark Williams-Cook should give you a good indication.

Larger sites typically have more to gain from technical SEO improvements. The majority of medium to large publishers sit on the right-hand side of this graph. Smaller to medium sized publishers typically need more brand presence and awareness in the right places - digital PR, link building and entity SEO.

It makes sense to run phased audits. Box off a week or two of time. Don’t overstep the mark here. The initial audit is to identify any opportunities or fires that need attention.

If there aren’t any, happy days. We either part ways as satisfied SEOs and clients, or move onto agreed upon phase two pieces of work like competitor analysis or specific content drilldowns, possibly with some monthly retainer agreed.

Pre-Auditing Essentials

This cannot be exhaustive, however, here are some non-negotiable questions:

• What are your company goals and KPIs? What do you need from SEO?

• How do you generate value (subscriptions, advertising, affiliate revenue)?

• What page templates do you have? How can we identify them? (Typically you can use custom extraction to identify content at a page template level)

• How large is the site and how much do you publish each day?

• Who are the key stakeholders?

• How much development resource is available?

• Who are your key competitors and what countries matter most?

• Is this a single language or location site?

• Are there any obvious issues to be aware of or any previous migrations we should know about?

All of this is designed to help you understand the business and suggest initiatives that drive value before you dive headfirst into the shallow end.

Subscribe now

Technical SEO Auditing

Technical SEO still presents a real opportunity for publishers.

Good tech SEO raises the ceiling. Excellent content can be suppressed by a slow, inefficient website that makes crawlers (and users) work harder than they should have to.

Crawling, rendering and indexing costs money and Google has made it very clear that unless your content adds value, it’s not going to stay in the index. Famously, Google really, really likes money.

Reached

Can bots crawl, render and index (in Google’s case) your content?

Barry Adams, as ever, has some excellent advice on how Google crawls your website. In my opinion, the most important concepts to understand are:

• Have you blocked any important content via the robots.txt file? (And is any of it still indexed?)

• How Google crawls and renders content

• Why client-side Javascript is something you generally want to avoid (for important on page content)

• Why pages drop in and out of the index

• How to measure and assess the crawl stats and page indexation report

Generally I would think of it like this: indexation is partly on Google’s ability to reach your content and content quality - led by engagement metrics. Does your content meet the quality thresholds to stay in the index, or does it flicker between indexed or crawled/discovered - currently not indexed?

SEO is all about patterns.

Whilst the page indexation report has limitations (1,000 rows is never comprehensive for a publisher), you can get around this by creating multiple Search Console accounts, or with a tool like Indexing Insight.

PSA: Daniel Foley Carter’s guide to solving non-indexable pages is superb.

Crawl budget is an enormous topic in itself and ties in very closely with the crawl stats and indexation reports. Whilst I can’t cover everything here, some of the most common issues with publishers are:

• Site search pages are being indexed

• A high number of indexable parameter URLs (MarTech will tell you these are important. They aren’t).

• Inefficient caching policies or unnecessary versioning of files

• Site speed, brand strength etc etc.

Take this example from a crawl stats report:

Underneath the main property, you get a breakdown of where Googlebot spends its time. Here you can see 52% of the bot’s resources are going on temporarily moved (302) files. That’s a very high number and one that would require further investigation.

On larger websites, this is exactly the kind of technical debt that can move the needle.

News articles (outside of live blogs) may be crawled once and not again for hours such is the volume and pressure associated with news and Top Stories.

If important content requires client-side JS, it may never get picked up by Google until after the event. When it is essentially redundant.

Understood

Semantic HTML and nested subheadings are the foundations of accessibility and an easy to crawl and ‘understand’ page. You may be writing content for humans, but you are structuring it for bots - now more so than ever.

The irrepressibly great Jono Alderson has a really brilliant article on why semantic HTML still matters. Search engines, accessibility tools, AI agents, and task-based systems all rely on structural signals - markdown is a cheap workaround to avoid fixing the mess we created.

I appreciate this can be a little complicated, but it’s the foundations of accessibility too - for bots and humans. So it is important. Fortunately there are some very good tools you can use here:

• Glippy Semantic HTML Checker

• Lumina Semantic HTML Checker

And this is an excellent way of approaching a semantic HTML audit.

You can run the HTML through Claude Code with very rigorous instructions to corroborate and contrast the findings too. As ever, you don’t need to fix everything. Just the most egregious errors - which are colloquially known as div soup.

Consistent, accurate and all-encompassing structured data support the disambiguation of your article. Whilst structured data may not have direct merit for AI systems (and is treated simply as raw text in the page), it still has tremendous merit in the reduction of ambiguity. I recommend reviewing structured data at a page template level and compare your structured data setup to some of the best in the business - The NYT always spring to mind.

This has real benefits for brands and people as entities and for trust as a construct of EEAT. Which we will come onto.

Then you are looking to remove any other inaccuracies - consistency of on and off-page dates is a classic publisher problem that only technical SEO can solve. Understanding is a combination of accuracy, accessibility consistency and disambiguation.

To really dive deeper on accessibility and agent readiness, this article by Jon McAlpin on Accessibility Tree Use Cases is better than anything I can produce. Complete with an accessibility tester too.

Linking

You cannot perfectly assess the internal linking of an enormous enterprise publisher. Things move too quickly, there’s too much noise and there will almost certainly be articles present from the inception of the internet.

But I think you can do a few simple things to assess quality and set the publisher up for future success:

1. Does your site have an egregious number of broken links and redirect chains or loops? Almost all publisher sites do, just make sure you fix the worst offenders. Broken or redirecting links in the menu or at a template level, redirect chains with five or more in the sequence.

2. Does the navigation allow for easy accessibility for users and bots?

3. Is the important content within easily navigable from the homepage?

4. Is important evergreen content prioritised (at a topic and article level)?

5. Do news articles effectively link to said important evergreen content and topic pages?

6. Are assets like tools, products and revenue driving sections given suitable visibility?

7. Is pagination appropriately crawlable?

8. Are links generally in an <a> HTML element (also known as anchor element) with an href attribute? Most links in other formats won't be parsed and extracted by Google's crawlers.

Once you’ve done this, I suggest putting some future processes in place that ensure news content effectively spreads link equity (and traffic) across the site via well placed internal links.

As a rough standard, aim for a link to a topic page and a relevant evergreen or news article in the first couple of paragraphs.

In an ideal world, you can automate or semi-automate quality internal linking. If you have a Google News Sitemap, you can scrape the content of each page and identify appropriate links to relevant topic and evergreen pages using an LLM’s API and some form of NLP to disambiguate.

Served

OK, so now the content has been reached, understood, linked - now it needs to be served effectively and efficiently. This means it needs to load quickly.

Page load speed has significant connotations for how effectively bots consume your content. A faster, more efficient site will be crawled to greater depth over the same time period than a clunky one.

We don’t have to be site speed experts to understand if there’s a problem. I think the most applicable approach is to audit speed at a template-level in three passes (one of these is on the client):

1. Field data: The Search Console Core Web Vitals report is entirely made up of Google’s field data and categorised by issue. Whilst this may not align with your template setup, there’s a very good chance it does.

2. Lab data: Dev Tools and Lighthouse reports use lab data. You can download the Devtools MCP and use it with Claude Code to help you identify and assess issues and to build up a picture at a template and sitewide level.

3. RUM data: This one’s on the client. Real User Monitoring data is essential to see how users from different places, on different connections and devices interact with your site. You need to see measurable improvements over time if you make these changes and this is the most reliable way to do that.

Essentially you’re using field data to highlight issues. Lab data to identify how those issues manifest and RUM data to monitor real people and whether the changes have a positive effect. When using Devtools MCP, you can use the lab data to review the difference between cold and consented user experiences on page.

This is important to understand, because a first time user will almost always have a slower experience. Not every visitor has the same experience, so you want to optimise for the majority of your user base first. Or the ones receiving the worst experience.

There are very good alternatives here. A PageSpeed Insights report includes both field data and lab data. Field data shows real-world user experiences from the Chrome User Experience Report (CrUX), while lab data shows simulated performance tests run by Lighthouse in a controlled environment.

I would highly recommend using CrUX Vis to compare your site’s loading against the competition in search. Speed is largely engagement related. Slower sites burn through users. If your competition have a faster loading template, you’re already on the backfoot.

If you are comfortable with the fundamentals of core web vitals, field data and lab data, then I suggest downloading the devtools MCP and configuring it with Claude Code.

You can generate excellent lab vs field comparisons and thorough investigations into each metric, including CrUX data if you have a Google Cloud account.

Miscellaneous

Miscellaneous really doesn’t do justice to some very important topics, but without this post spiralling further of control I’m going to cover them in a couple of sentences.

We haven’t covered international SEO, auditing hreflang tags (although this excellent tool from Chris Lever is absolutely the best for it), but it isn’t a requirement for everyone.

1. Do you create a unique, internationalised experience on different versions of your site?

2. Do you auto-redirect users based on their location (please don’t do this, prompt, don’t auto-redirect)?

3. Do you publish content unique to that region?

4. Is the path to conversion appropriately internationalised?

This really isn’t comprehensive, but it’s a pretty decent start.

And as bot management continues to grow in importance, you really want to assess your robots.txt and CDN-level blocking. Is it doing what you want it to? Is it setting your users up to have the best possible experience? Is it positioning your site in a position of strength, not weakness?

If you’re blocking training data bots, just make sure you understand the knock-on effects. Ditto if you allow them.

Content

Content has to stand out. It has to be unique, or at least have unique elements and it shouldn’t be created solely for SEO purposes. Content should be created with your audience in mind.

We can audit some of these things in a relatively straightforward manner. Others require a little more nuance and actual thinking. You can’t just automate everything.

Fundamentals

I have written about how to create great content and get it found at length. These content tenets should be applied ubiquitously.

• It answers the question

• Entity-first headlines and opening paragraphs

• Well structured, with relevant, nested subheadings

• Short, snappy paragraphs, written in a readable manner (you can use a free online readability checker, although I wouldn’t recommend sharing this with the writer)

• Bullets and tables to break the content up with structured data (not schema)

• Clickable, unique images (typically with people in them)

• Unique elements (data, stories, video or visuals and real, lived experiences are all good examples of this)

• Relevant, topical internal links

• Created (or updated) at the right time

• An appropriate level of freshness

You can score freshness using this distance calculator or - if you’re lucky enough to work on a site with well structured sitemaps - this freshness checker.

Measurement at scale

Content popularity is measurable at scale. Organic traffic, branded search, engagement, backlinks and mentions can all (broadly) be measured by subfolder or topic. Particularly if you have access to some third party data from somewhere like Similarweb.

This can be applied sitewide or at a page level. There are obvious limitations here, but it’s a very useful starting point and can help identify areas of the site with declining engagement and potentially weaker content.

Query counting

This is, I think, one of the most under-utilised and exceptionally useful tools in an SEO’s arsenal. Particularly when you are working on such large sites, sometimes publishing hundreds of articles per day.

On said chunky sites, you’re looking for patterns. Patterns that indicate whether your content is valued by users and search engines. Query counting exposes how search engines are evaluating your site over time.

If you split your Search Console account into multiple properties (as you should), you get access to a treasure trove of data with brilliant tools like SEO Gets (where this screenshot was taken) or SEO Stack. SEO Gets is a slightly more budget-friendly option, SEO Stack is more all-singing all-dancing.

Both are absolutely brilliant.

Whilst these tools have so many brilliant options - page indexing, alerts, cannibalisation, GA4 integration, content decay mapping et al - query counting alone is worth the investment. Particularly for publishers who have a suite of evergreen content.

If you segment your site effectively and highlight your branded queries, content clusters, important topics and pages and use the automated alerts within these tools, you can remove so much of the hard work associated with typical content auditing.

I wouldn’t recommend doing a content deep-dive as part of a first phase SEO audit. But I would recommend identifying areas of the site where you’re seeing out of the ordinary growth or decline. That is how to create a second phase project.

News

All of the above still applies to news, albeit page structure is less important on shorter articles.

Much of what you can do with news as an SEO is helping editorial teams hit some basic standards. And understand how Google ranks news sites.

We know that news content needs to be right on first pass when it goes out into the wild. We know that people broadly search by using entities. People, places, organisations et al. If nothing else, headlines and images need to be on point.

And you want to establish your Top Stories presence, preferably compared to the competition. I like to know how your publishing volume compares to the competition in terms of articles by topic, live blog, video et al.

And that your most important articles are linked to from the homepage, accessible via a Google News Sitemap and have reliable authorship presence.

News checklist

For your most important content and content most likely to get picked up in Google Discover, make sure you amplify it early. Pushed on branded and journalist social channels, on the homepage, in email newsletters. Make it outperform the expected engagement.

This is called The Golden Hour. I would add amplification to any checklist.

Evergreen

Google may have some measurable ways of ‘scoring’ evergreen content and I have written about how I think you should do evergreen content in the AI era.

Most traditional, more commoditised, keyword-led evergreen content isn't as valuable as it once was. So we need to really establish its importance in the ecosystem. Is it worth keeping and renewing?

If you can determine this content sits out of scope and isn’t hitting key micro-conversions (registrations, return visits, engagement etc), kill it. Publisher sites are so sprawling that content pruning is never a bad idea. Particularly if the content sits outside of your core proposition.

Evergreen content checklist

Evergreen content must be written for people and answer the question. You cannot survive without uniqueness or legitimate expertise and it must be structured effectively for people and bots.

In terms of assessing success, it needs to continue to attract links and quality traffic months after publication. Whilst I’m not for updating a page every week, you should time updates to spikes in demand and distribute accordingly.

Aleyda has a brilliant checklist for the types of content to prioritise and de-prioritise, which align almost perfectly with the effort required to create something brilliant and click resilience in a zero click world. I think it ties in very nicely with quality evergreen content.

Google Discover

What works well in Discover tends to work well in search. I’ve written about how Google Discover works and how to analyse it at scale, which might be good places to start here.

Whilst Discover surfaces news and evergreen content, the clickless-sphere is becoming less publisher-friendly. Your content really has to cut through for it to work - ideally you’d be prioritising journalist-led, people first content in video format too. The platform is still an enormous driver of traffic.

Discover checklist

The basic principles for Discover success are the same.

Clickable content that does what it says on the tin. A fast-loading experience, with content that is easy to ‘read’ for people and bots. The image and headline are more important - the max-image-preview has to be set to large and the image must be 1200px wide.

The OG headline gets picked up in Discover and as the social headline it has the chance to be more evocative and engaging. Use it. Don’t just rely on the SEO and on-page headlines.

Really consider the value you’re going to get from Google Discover. You can absolutely generate reach and you will get far more traffic as a paywalled subscriber if you open the surface up. But is that what you need?

EEAT

I wrote a very long article about what good EEAT looks like for publishers and I don’t see any reason to deviate from it. The only updates are to include more entity optimisation and analysis work as part of the audit - which has morphed into its own section.

Contrary to popular belief, it is not a ranking factor. EEAT is more of a qualitative measure of page and site quality. Unless you are a publisher heavily focused on really YMYL topics, this is unlikely to directly move the needle.

But it’s a qualitative measure used by Google’s human SERP reviewers and a core part of Google’s Quality Rater Guidelines. Whilst it has its flaws and detractors, it is still very useful for publishers.

Part of the beauty of EEAT is that the majority of the non-negotiables rarely require significant development work. In an era where consistency matters and users consume your brand away from your own site, you need to own the conversation while being a trustworthy source as best you can.

I suggest breaking it into five key sections:

1. Links and mentions

2. Sitewide

3. Desk-level

4. Page-level

5. Off-site

Links and mentions

The value of a backlink has steadily declined as SEOs got better at playing the system. But they’re still hugely important - although not something I pay much attention to. Particularly for larger publishers.

If your brand (personal or professional) makes a positive impact in the industry, you should monitor where and when you’re picked up. Who is talking about you, your product, your brand? Why? Are you making the right impact?

To me, backlinks and mentions are byproducts of doing great things. But you still need to know if they are a limiting factor. If you’re a smaller brand trying to cut through the noise, I suspect you need to invest in digital PR and brand-building campaigns and opportunities.

To assess both, I would do two things:

1. Review backlinks at a subfolder-level

Keep it really simple. Compare your backlink acquisition at a subfolder-level against your core competition.

• How do these track over time?

• Are you outperforming the competition?

• Are you generating links from quality sources?

2. Use a tool like Alertmouse for mention tracking

Setup brand tracking in Alertmouse for you and your competition.

• Monitor where these mentions come from

• Keep an eye on the competition

• Look at the content that is performing in the industry, replicate it in your own style if you have something to add

• Reverse engineer how the best in class are generating buzz

I think it’s much harder to use links to boost performance now. As time goes on I suspect links will continue to reduce in value.

I would rather spend time working on things that will get shared, but as a brand you should help wider teams understand what is having an impact.

Sitewide

Sitewide, we want to ensure the fundamentals are in place and easily navigable for users. Not everything has to be in the primary navigation, but I would suggest core information like the help centre and key about us and contact us pages are positioned reasonably prominently.

Two easy ways to identify ways this could be improved:

1. Use your site search data to see if an inordinate number of people have to search to find what they are looking for

2. Assess your best in-class competitors to see how they approach things

Hell, if you’re feeling brave, you could even ask your community or customer service team to see what your users most common problems are. Although you might be opening a can of worms.

Desk-level

As nice as it would be for sitewide signals to solve everything, for larger publishers this is rarely the case. We need to see real people present on the site. We want to build their profiles in newsletters and encourage the right behaviours in the newsroom to turn journalists into figures of influence.

To do that we need pages that show the team, mission statements that show what the desk is there to do and who they’re there to help. More so than ever, people want to know who these people are and why they should be trusted.

Page-level

Now we’ve covered off the sitewide and desk-level factors we want to think about things at a page level. Your page needs to demonstrate trust and credibility. It has to be good enough to encourage returning users and to move users from the active to owned discovery.

As is the case with high-quality content, we need to see real authorship, uniqueness, real effort (ContentEffort is a proven scoring component in the algorithm), accurate and up-to-date.

Google uses consensus scoring to decide which pages to rank for certain topics. If the question at hand has a proven answer, you cannot deviate from that and expect to rank.

However, there are many search terms which don't have an objectively "correct" answer, and Google therefore may seek to get a mix of views.

Off-site

According to this Seer Interactive x Trustpilot study, brands without a Trustpilot review profile have a median AI citation rate of 1%, while brands with a robust profile saw a citation rate of 75%.

Even brands with a basic profile had a 53.5% citation rate. Mentions are the currency of the web and a robust off-site presence is an essential component of EEAT.

Publishers have notoriously awful reviews - I think this is primarily due to especially bad customer service and subscription payments that are very hard to cancel. You can work with said customer service team to even encourage positive reviews, if they exist.

This is all about creating a high-quality product that people become fans of (as part of owned discovery) and talk about in glowing terms.

Worth remembering that algorithms are not applied ubiquitously. Certain topics have more stringent rules and higher levels of rigour - it is precisely what YMYL was invented for. To try to provide more trustworthy results for things that really impact people’s lives.

So if you exist in a more health or financial-based field, the EEAT section is of greater importance than someone who operates in the entertainment space.

Subscribe now

Entity Consistency

Entity SEO has been around as a construct for some time, but has never been more important. The best practitioners have been considering it for more than a few years. But for the rest of us mere mortals, the idea has only recently taken hold.

The idea that your brand has to exist beyond search - that you need a presence on other channels - has been corroborated by Google’s latest updates to Search Console. Even if it is just a big distraction tactic.

Ditto your authors.

Bigger presence means bigger distribution. It reduces your reliance on search whilst simultaneously increasing your presence in it.

But presence is only a part of it. Accuracy and consistency are just as important. The information about your brand needs to be corroborated across first and key third party sources. You need to control the controllables.

I would break it into three parts:

• On-site

• Owned

• Earned

The combination of these three things resolves your online identity. Or lack of it. Tools like Waikay can be really valuable here, ditto the Knowledge Graph Explorer.

On-site

Your on-site presence is the definitive statement of your identity. Everything you say and have said matters. This correlates strongly with topical authority. The further you diverge from your brand’s core principles and purpose, the less defined you become.

The pages I would review correlate strongly with good EEAT. You want to ensure what you say is accurate, up-to-date and relevant:

• About Us

• History

• Meet the Team

• Shipping information

• Returns

• Payments

• Editorial policies

• Products

• Pricing

• Contact

What we’re trying to establish is whether we are THE definitive source for information about our brand. We should use Organization schema, SameAs and alternateName properties to identify other brand names, social accounts, Wikipedia pages et al. All to remove ambiguity and strengthen your knowledge graph presence. Do the same with author pages and schema too. Make these rich and showcase why you’re a trustworthy source.

Trained human raters evaluate pages based on a number of these principles. Accuracy, reliability and trustworthiness. Consistency is auditable. Tear your own skin off boring, but auditable.

Owned properties

Create a spreadsheet with every owned property your brand has. YouTube, Instagram, podcast platforms, websites, review websites, your writers, GMB profile - the whole nine yards.

Then make a checklist including checkboxes for your NAP (Name, Address, Phone Number - yes, like local SEO), website link, owner, pricing, products, other platforms - whatever information you list, make sure it is correct.

Make sure, where applicable, you have claimed all of your properties in search. Your GMB profile, your knowledge panel, your author’s knowledge panels. Use them to create a singular, powerful entity.

Earned

It’s very difficult to audit your earned properties - mentions, citations, Wikipedia pages, Crunchbase pages et al. Particularly when you consider the scale that comes with larger publishers. Clearly trying to alter every erroneous claim isn’t going to be feasible.

Focus your time on the most important mentions, sources and citations. Properties that genuinely influence people’s decisions.

Obviously - and most importantly - create a program of work that genuinely influences people and earns authoritative mentions and citations where your audience spend time.

Again, I think the best output here is a checklist that highlights how effectively and consistently you’re cited vs your competition on the platforms that matter. And tracking your branded search volume (+ topic) is a great way of assessing how you are viewed by people and whether this is having the desired impact.

That’s it

Well done for making it this far. I appreciate this is a lengthy post and one I hope has been useful. Obviously, or I wouldn’t have written it.

It’s not completely conclusive, but does marry together more traditional SEO - that is still foundational to the way discoverability and the internet actually work - with newer concepts around trust and entity consistency.

Hopefully with some tangible ways you can audit and implement them on your site. Of course, I run SEO audits for publishers and content producing sites. So if I can help, send me a note here or on LinkedIn. You know where to find me.

Leave a comment

Traffic from Google was and is so valuable - and in recent times so vast from Discover - that reach has never been a problem. Not until very recently. As behaviours, platforms and the internet morph into something less website-friendly, business models and publisher indispensability has been called into question.

Today, discoverability is much broader. It’s being won by brands and (predominantly) individuals who can move fast and show a little personality. A touch of ankle. Calf if you’re very lucky.

Today’s audiences discover publishers and people through social platforms, search, AI assistants, recommendation engines, Google Discover, podcasts, newsletters, creators and countless other touchpoints.

They are less loyal and less trusting of news than ever before. But they love people and authenticity. To grow, publishers need to own the passive phase of discovery. And we need to understand how to audit them.

Subscribe now

TL;DR

1. Discoverability is the premise that people encounter your content wherever they spend time

2. There are three phases of discovery: passive, active and owned. Getting users into the owned phase requires consistency in the passive and active ones

3. Engaging a user and building a meaningful relationship is a challenge that cannot be solved by search alone - that has been true for some time

4. According to new research, up to 86% of Google clicks are discovery-led (primarily desktop). Google is still a foundational force at the top, middle and bottom of the funnel

If you’d rather listen to me talk about discoverability, the choice is yours:

The definition of discoverability

Discoverability is the probability that the right person encounters your content at the right moment, regardless of platform, source or whether they were even looking in the first place. It’s about solving the ‘problem’ the user has at the right time.

The difficulty - or maybe more precisely one of the difficulties - is meeting the user in their passive phase of consumption. When they aren’t looking for your brand. They’re looking to be inspired, but they are almost unaware that’s what they’re looking for.

Something or someone has to create something good enough to break the scroll.

Publishers are competing slightly less directly for rankings and more for moments. Moments where they can directly engage the user and attempt to influence them. Habits start small.

Discoverability has fundamentally changed

Not only has it changed - it’s been changing for the best part of a decade.

Previously search was the entry-point for users to your brand. The research and comparison stage of the journey was fundamentally search-led. Now social changes have led the charge on more passive consumption of content through algorithmic feeds. Unless you appear there you are largely invisible - particularly for younger audiences.

And AI summarises commodity informational content better than any brand or human can when it comes to ease of use and scale. So publishers are unlikely to win the war when it comes to convenience.

But why now?

I think there are four key reasons. All of which can be bundled under consumption preferences and convenience.

1. Audience behaviour has changed

Behaviours have been changing for a decade or more. Search no longer has a vicelike hold on each stage of the customer journey and is just not as effective as AI for research or social media for influence and engagement at scale.

People have options and habits have changed. Teenagers spend just four minutes a day on publisher websites compared to 19 for over 55s and younger audiences prefer watching or listening to reading.

Publishers are less habitual than ever before. Direct traffic and branded search are dropping across the board because users have more stimulating, individual-led options.

According to this latest iPullRank study, 52.3% of all 18-24 searches end without a click. For those in the 55-64 age bracket, this figure is just 42.7%.

But the older audience is seven times more likely to click on an advert. Gen Z are less trusting and more discerning.

2. Platforms have become walled gardens

LLMs are built to answer queries internally, traffic to publishers from social channels has been dropping for years and search is following suit. The longer you stay in an ecosystem, the more valuable you are to that ecosystem.

Big tech’s goal is to identify their moat (usually big data), open the gates and then close them for monetisation. There’s a reason platforms like LinkedIn suppress posts with links in them.

The moment you leave their system, your value declines.

3. AI has expedited the shift

Their restructuring of the web into a single answer engine interface is significantly more convenient than sourcing the answer yourself. And many publications have made a living off ad-revenue with information that didn’t warrant a click.

Google has been trying to stamp down on commodity content, content that lacks expertise and over-optimised content that games the system for years. AI Overviews have eroded search’s value exchange and answer engines dominate in the research phase.

The top of the funnel has become less valuable click-wise. Whilst this traffic was low intent, the volumes were so high it has been extremely profitable for publishers.

In the same study from iPullRank, the clickstream data showed that live sports had the highest zero-click rate of any content type.

The SERPs are so rich with sports widgets, weather widgets and calculators et al (see utility lookups in second place) - and have been for some time - that it’s not just AI Overview related.

4. The rise of the individual

People trust people, not brands. The younger you are, the more strongly this statement resonates.

The rise of social accounts and videos in Discover is not an accident. This is Google’s most personalised platform. One where no two users have exactly the same experience.

I would expect Discover to become even more video and social-led as Google attempts to build its own creator-led platform.

One that becomes more friendly for advertising and less so for publishers.

Publishers that focus on turning individuals into influencers can still profit from this iteration of the platform. Traffic won’t be as plentiful, but real value will still be there. It could become more valuable for publishers who lean into the individual.

You can watch or listen to me talk about the shift in Discover here:

The future of search and publishing is discoverability

Search Everywhere Optimisation is all about discoverability. Google’s latest release gives you visibility over your social accounts in search. The latest in the line of updates that show how important wider visibility and expertise are.

You cannot build a brand through search alone and any ambiguity is penalised. This is an evolution of EEAT and a damn good one.

It’s about being present across multiple channels, with human-led expertise that builds the prominence of your brand, improves entity relationships on the internet and disambiguates your brand’s presence.

These are two sides of the same coin. Good SEO is good discoverability - but you can’t stop there. (This is also good GEO, obviously).

Google has trusted brands for decades because people do. A quality, trustworthy brand is one of the hardest things to manipulate. Backlink exchanges and fake reviews are not a solution. Neither is scaled AI content.

You need the relevant presence across search platforms; Google, YouTube, TikTok et al, everywhere that has a search function, third party sites of influence in forums like Reddit and review platforms and in the community itself.

Places where your audience is actually influenced.

What we think of as traditional search is being replaced by AI-interfaces. But it’s also being replaced by a creator-led universe. As Google attempts to create a loyalty ecosystem, its attempts to surface more creator and social-first content haven’t gone totally under the radar.

Subscribe now

The discoverability framework

The old customer journey isn’t broke, but it needs updating. Particularly for publishers who primarily sell information and opinion as primary products.

Our model takes you from passive through to owned discovery. The right person, need, format and time.

Discovery is still the first stage of the journey and the end goal is to turn paying users into fans and advocates. People who share your content and feel like they belong. But that requires a community.

It requires:

• Creating habitual experiences

• Reporters going below the line in the comments

• Creating reader-led content

• Live Q&A formats

• Live events

Something the majority of publishers don’t have, but desperately need.

But discovery is predominantly passive. Your content has to encounter people algorithmically - you have to force your way into the feed. You have to inspire them, because their attention spans are at an all-time low. Which means you have to understand your audience and their needs at the time.

Once you have their attention, discovery becomes active. They want your brand to engage them. To solve their problems. But when users have been trained not to click, the click has to provide more than just an answer.

Otherwise you become a problem, you don’t solve one.

Once you have inspired and engaged a user, owned discovery takes over. Email teams, habit-forming products, follow an author features and push notifications create daily habit. The community team builds fans.

Intent-led journalism

Topic-led journalism still has its merits, but it has to evolve. It has to evolve beyond creating content because there’s demand for it. It is about solving problems for people that AI and informational content cannot do.

That is how we can create engaged users and - eventually - fans.

The foundation of intent-led journalism is understanding your audience. Once you know who you’re targeting, you should know what they need at each stage of their journey. When they’re looking to be inspired and when they’re looking to belong.

To stand out and be truly memorable, topics should include a combination of the below:

• Explainers

• Breaking news

• Live experiences

• Video and audio products

• Habit-forming products and tools

• Community

All of which are underpinned by expertise, uniqueness and authority. The pillars of EEAT and Helpful Content.

Topics need multiple discovery opportunities. For your existing website audience. For new users. For lapsed subscribers. You need to understand the appropriate mechanic and delivery mechanism for each story. It should serve a purpose.

This post from Dimitry Shishkin is an excellent framework for creating rich, robust topics that solve problems and create returning users.

The new economics of content

Articles cannot be the sole product anymore. They don’t deliver on the fundamentals of timely discoverability or problem solving effectively enough.

You need economic diversification. Evergreen content doesn’t pay the bills as consistently anymore. It is one of the areas where the NYT has been so successful.

Quality articles and reportage is arguably more expensive than ever and returns less value through traditional means - typically in visits and advertising revenue. The written word is not thriving economically in a more traditional publisher model - although Substack and Beehiiv suggest the market is still there, it’s just creator-led.

Publishers have to become indispensable. Indispensable to their audience. The community. Their journalists. You have to understand what your audience want, where they want it and how they want it served to them.

Then you have to force your way in.

The publisher of the future

Over the next decade the best publishers will be those who understand their audiences and their needs better than any other.

My hunch is that search will still be the most important last click channel, but other methods of distribution will be improved so much so, that it becomes even more valuable. Albeit with an outdated attribution model.

The creator economy will continue to boom. Publishers and brands that can put their ego to the side and work out how to train journalists to build their own creator ecosystems will flourish. Yes, they’ll come and go, but so others will take their place.

Agile product, AI, design and engineering teams that can solve real business and user problems quickly and scalably will turn newsrooms into content engines of the future. Teams that give newsrooms the ability to deliver like a one man band.

The very best will be:

• Audience-first

• Topic-led

• People-powered

• Video and audio native

• Search Everywhere Optimised

• Distribution obsessed

• Community-builders

They won’t focus solely on channels that drive a more direct return. They will accept that some of this cannot be reliably measured.

Setting up for success

I’ve always felt that publishers were ahead of the curve with audience teams. Publishers could take this a step further. A team solely focused on understanding who your audience is, where they spend time, how and where they’re influenced - even who by.

A data and product marketing arm of the audience team supplemented by a distribution arm. The team who create platform specific strategies based on the audience data they have to hand and the user and topic needs models.

Then a lightweight, roving AI-powered product and engineering team setup to build brilliant things quickly.

Leave a comment

Headline findings:

• June was a record low for the open web (via Google search)

• Just 40% of searches in the US and 40.7% in the EU/UK send a click to the open web

• In the US, 17.1% of all searches go to Google owned properties. In the EU/UK, it's up to 20.4%

• AI Mode usage has plateaued, maybe even dropped in the US. The EU/UK has more than twice the % of visits to AI Mode (0.29%) than the US (0.13%)

• ChatGPT's share of desktop users has fallen in the EU/UK by 4PP or more

• Knowledge and AI-websites gained in popularity, Google dropped out of the top 15 (in the US)

Rather than read it in its entirety, you can watch or listen to me talk about it here:

Nothing hugely surprising of course, however there are a couple of points that piqued my interest without making it into the headline findings.

Subscribe now

Paid clicks are decreasing

Given the extraordinarily good state of Google’s revenues, I’m surprised to see such a significant reduction in paid clicks between Mar - June 2026. In the US they dropped from 2.3% to 1.3% and in the EU/UK, 2.4% to 1.4%.

Maybe Google has decided now’s the time to try something novel given how strong their advertising revenue base is. This does also coincide with record levels of users that stay on Google, or click through to another Google-owned property:

• US: 17.1%

• EU/UK: 20.4% (up from 17.1% in the previous month)

Maybe they have expanded AIOs in more territories recently (French rollout took place in July 2026), or given AI Mode a more significant push. We know these surfaces do not monetise as well advertising-wise as traditional interfaces.

This data corroborates in the EU/UK, but not in the US. One to hypothesise over.

Wikipedia is on the rise

In Europe, Wikipedia went from being the seventh most visited domain from search in Q2 2025 to the third most visited in Q2 2026.

This feels like the antithesis of what we’d expect to see - who visits websites anymore?

Some people obviously, which is encouraging. My only slight hunch is that users are looking for ways to corroborate the information they are ingesting at a furious rate from less reliable sources.

Or, I suppose, Wikipedia may be ranking more effectively.

Navigational searches fell, but informational search rose

Navigational searches fell hardest on smaller engines - Yahoo, DuckDuckGo, Bing et al - all saw more significant drops in navigational searches than Google’s 22.9% - 16.4% in the US market.

But informational searches rose significantly. In the US market, informational searches accounted for 59.69% in Q2 2025 compared to 65.12% in Q2 2026. The EU/UK saw a comparable rise over that same time period - 55.39% to 61.31%.

I’m not a statistician, but if this is based on share of search alone, then the ‘drop’ in navigational searches could be down to the far more significant rise in informational searches.

We know that AI visibility has measurable downstream impact and that AI excels in the research phase. While search is becoming less valuable here, AI usage may be increasing subsequent informational searches in Google.

There is, of course, far more in the report around desktop usage, searches per user, query length to name but a few. But to me, these are the headlines.

Other good bits

• SEO vs GEO is the wrong question - Jono Alderson

• What’s an LLM? - WTF is SEO

• How to fix non-indexed page issues - Daniel Foley-Carter

Leave a comment

Not a perfect science as the robots.txt is so easily circumvented, but still.

No one is playing nice anymore.

Multiple publisher x AI company lawsuits and threats to block Google from very serious publishers. USA Today, Reuters and Politico. Even Reddit is considering walking away from the Google AI deal. A marketplace or micro-conversion model where publishers are paid when their content is used cannot come soon enough.

All of this after Google just posted its most valuable quarter ever. The almost non-existent value exchange doesn’t look like changing anytime soon.

TL;DR

1. Visibility has measurable downstream impact. Users visit a brands’ website at 2.5 times the normal rate post AI visibility

2. A brand’s inclusion in the training data is central to inclusion in the future selection process. Publishers have real value here.

3. But with probabilistic tracking and with many bad actors in play, marketing yourself as a brand of influence requires true expertise.

4. As with any type of commercial proposition of this nature there are risks to be aware of.

Subscribe now

Visibility has measurable influence

Thanks to Similarweb’s fantastic study on The Impact of Downstream Visibility, we know that:

• Users that see a brand in an LLM are 2.5x more likely to visit that brand’s website in the following seven days

• These users are twice as engaged as traditional users

• AI dominates the early stage of the journey, but search and AI come close to parity as the user nears the point of conversion

• That 55.9% of AI-influenced traffic arrives via search

LLMs have restructured the web in a manner more convenient for users. Because we are so explicably lazy, we almost always take the easy route. This reorganisation of the web reduces the cognitive load of finding the answer yourself.

The majority won’t click unless forced into it.

But whilst AI sends very little traffic, the traffic it does send is twice as engaged. Users find out about a brand, person or product via social, audio, video and now, AI. Once the user knows about an entity, they search for it. This is how search has functioned - as a navigational channel - for some time. We Google things we have found out from other channels.

This is why search has always done so well when it comes to last click attribution.

Being visible really matters to brands who sell products. It is not a soft brand metric. It is a performance channel too.

AI will never send traffic directly to websites. But it is a crucial discovery channel for brands. People discover the brand via AI visibility and search for said brand further down the funnel. Via search.

There’s almost certainly a strong correlation between increasing gen-AI traffic, branded search, direct traffic and positive AI-visibility.

How do you become visible?

Certainly not with listicles or cheap GEO hacks. At least not sustainably.

You need to understand the foundations of LLM information retrieval:

1. How training data works

2. RAG/grounding

3. Disambiguation

Fundamentally, these bots are trained on vast swathes of data. A brand’s inclusion in the right place at the right time helps an LLM build a picture of this brand and its ‘closeness’ to relevant entities and products in its database.

Your inclusion in the training data is the bedrock of your success.

LLMs have a training data cutoff. You are essentially optimising for the next evolution of the bot when it comes to training data inclusion. But without this, your selection as part of the grounding process will be suppressed, because you do not feature prominently enough in the training data to be considered.

Brand strength + marketing + consistency. You own what you can control and you boost your name recognition in the right place, ensuring you are well-positioned in search for real-time retrieval.

Publishers are still an extremely valuable proposition in this space.

The generative-AI landscape

Similarweb’s second excellent report shows just how enormous this sector has become. 2.7 billion gen AI app downloads worldwide in 12 months (a 134% increase) and almost 10 billion monthly website visits (a 70% increase).

It examines how the leading AI platforms are growing, how user behaviour is changing, where referral traffic is flowing and how brands are earning visibility. I think it’s the best resource on the internet.

Headline findings

• 🔎 Search grew by 2% last year

• 🗽 >40% Of US searches now trigger an AI Overview

• 🌐 2.5x Increased Chance of Site Visit after AI Mention

• 👶 6% Decline in Gen Z's (18-24) share of AI chatbot usage since 2024

• 🧒 50% of gen AI users are between 18-34

• 📈 ~180% Claude's Unique Visitors growth H2 vs H1

• 📊 Users arriving from LLMs are twice as engaged as traditional users

We know that younger audiences have been moving away from more traditional mechanics of information consumption. Video and audio are booming, younger audiences trust the individual over the brand and, clearly, gen AI’s ultra convenient approach is a huge hit.

It doesn’t look like AI is replacing search. Both are growing. Gen AI is so proficient in the research phase that changes to the information retrieval ecosystem are significant. For brands that sell tangible products, both search and LLM ‘visibility’ are immensely valuable.

Ensuring you’re present in the research phase of the journey for the subsequent search is key to profitability.

According to Similarweb data, generative AI users have almost exactly the same audience profile as YouTube. Even younger, in fact. YouTube has a 49.1% share of 18-34 year olds as its audience.

What does this mean for publishers?

I think there is a burgeoning influence marketplace here. Despite publishers best efforts to block anything resembling a bot, they are still some of the most influential platforms on the internet.

Publishers have to understand and crucially prove where they carry real influence. If you can prove you are the most influential brand in the UK market (or in London) for technology > TVs, you become a more valuable proposition overnight. Even with probabilistic data.

This is incredibly high-value marketing collateral for commercial teams.

Particularly when we consider the apparent overstatement of Reddit’s influence in LLMs. According to Dan Petrovic’s always impeccable research, Reddit is the most rejected website in their entire dataset. Reddit is filtered out more than any other site.

It is always a candidate for selection in the fan out process, but more rarely used than any other site in the dataset.

I talk about what I think this means here and how we could consider creating a new revenue line (for those who prefer watching or listening):

How can we do this?

The data can only be probabilistic for now. Prompt tracking has obvious faults, but run at a large enough scale it can give a reasonable measurement. Without deterministic data this is imperfect - but even with deterministic data, prompts are so unique this raises its own challenges.

In Rand’s study, across 142 responses, there were barely two prompts that could be considered similar. This is the natural shift that comes with significantly longer queries. Greater diversity.

But it’s not useless. Average visibility for the top three brands was, comparatively, quite high. So with scale and sensible, consistent, scaled prompting, you can get to a reasonable, sellable outcome.

My advice is:

• Utilise your server logs to establish LLM ‘demand’

• Review your website by commercial topic

• Set up relevant prompt tracking for each relevant topic to assess your market position

• Create marketing and sales collateral based on your influence

This shouldn’t be done lightly. If you would like help here, please do get in touch - LinkedIn, on Substack or via our consulting practice.

Bot-blocking creates an obvious break in the chain here. If you have very strict bot-blocking policies, this might be something to consider for the future, or to consider unblocking specific sections of your site.

Be careful; you have been warned

I know some publishers are already attempting to cash in on this market. The temptation to make a fast buck has never been higher and I urge caution.

There is no shortage of sites chasing AI visibility that have been absolutely nuked in search thanks to advice from the GEO specialists who espoused the value of listicles. This falls under the same bracket.

Listicles, I should add, that greatly benefit your competitors. Mentions are the currency of the web. Almost certainly more so than links at this point. Continuously mentioning your competition means you’re doing their marketing for you.

The very content format Google has semi-removed from the web with their stringent product review guidelines, parasite SEO shenanigans and the Helpful Content Update.

Paid for listicles dominated the internet for 10 years. I’m sure their lack of rigour and obvious bias spurred Google into action. Lily Ray has excellent research on this and according to her, almost every website engaging with programmatic SEO and AI slop is being nerfed out of search.

The very mechanism that drives (or at least did drive) so much visibility in LLMs.

Be wary who you listen to. We’re all working this out as we go.

That’s it

That’s it for this week. Some other good stuff:

• Reach is moving away from volume - Press Gazette

• Topic led journalism - Dmitry Shiskin

• Implementations from Croatia SEO - Another Concept

Leave a comment

I recently started posting some slightly longer-form content on YouTube. Some slightly richer deep dives in video form. If you’re more of a visual or audio learner, hopefully this is useful.

I am more of a ‘learn on my own time’ reader, so this is a useful experiment and experience for me. Albeit slightly jarring.

If you’d like to discuss how we can support you around discoverability, AI risk and resilience and commercial publishing opportunities, send me a message or visit our consulting site.

There are some sizeable and growing caveats here:

• Only 32% of these end with a click

• Only 66.61% of the subsequent 32% go to the open web

• This figure is rising each year

That still means Google processes 2.92 billion clicks to the open web every day. It’s still a figure worth fighting for - particularly for publishers whose business models heavily rely on a click.

So let’s not totally lose sight of what matters in the here and now. And unique content certainly fits that mould.

I have reviewed a few previous patents (Google’s in-depth article patent explained and how google ranks news sites) and it is not a thoroughly enjoyable experience. A granted patent protects an idea; it doesn’t prove deployment or real world use cases - and it’s certainly not unlike big tech to claim ownership of something just so it can’t be used elsewhere.

Generally if;

1. The patent is cited regularly and recently? This patent (Contextual estimation of link information gain) has been cited 24 times and as recently as last year.

2. Whether it has international filings? Yes, but with some caveats. US, China, ceased in Europe and worldwide, but extended in the US to 2039 very recently.

3. Whether Google has protected the ranking technology around the world? Yes, again with some caveats.

4. Does it broadly align with your understanding of the concept (in this case non-commodity content)? Very much so. As the rasping breaths of SEO-first, commodity content make even iron lungs work hard, it would be inconceivable for Google to not measure or evaluate uniqueness in some manner.

It is more likely to be used in some capacity.

TL:DR

1. Google has multiple public and leaked systems that appear to evaluate originality, effort and unique contribution - see OriginalContentScore and ContentEffort.

2. The patent describes an information gain score (potentially in a 0 - 1 framing) that is assigned to a document based on how much new information it adds beyond documents a user has already seen on the same topic.

3. In mine - and many other’s - opinion, Google’s systems reward originality in some way. Whether that’s directly through an information gain score and re-ranking system, a Bayesian predictive score or indirectly through positive engagement signals I couldn’t tell you

4. Originality doesn’t mean an entirely different document. As little as 10% difference could be the delineator between marketing success or failure

How does it work in practice?

This patent is not about the information gain applied to the current set of search results. It’s about the subsequent set of results - ranking the next set of search results based on wider user search behaviour, personalisation and added document value.

It highlights that documents:

• may be reranked

• may be excluded

• may be significantly demoted

• may no longer appear in results

Based on the amount of novel, relevant information provided when compared to other similar documents.

For any tech SEO geeks out there, you’ll be well aware of the concept of preloading. In nerd circles, preloading tells browsers which resources should be prioritised to improve the page load speed and above the fold rendering.

I think this patent works in a similar manner, but with bloody unreliable people instead of machines. Maybe bfcache is a more apt comparison but I haven’t really got stuck into technical SEO for a while, so forgive me for my appalling analogies.

Step-by-step

1. A user reads a document about a certain topic - let’s say growing an apple tree.

2. Google understands that the majority of users don’t stop at one page here. It’s a rich topic. When should I plant one? Where? What do I feed it?

3. With 13 months of click and engagement data to hand, Google knows - with I imagine an unerring level of accuracy - what piece of content each user should be shown and when based on goal fulfilment.

4. But new content is written everyday. Pages are updated. So this isn’t a static corpus to work with. And maybe someone has a novel way of growing apple trees?

5. So pages are compared. A user reads a document (d1). Google then compares a new or updated article (d2) to the original.

6. If d2 generates a favourable information gain score, it will likely be shown to the user as part of their journey. If it doesn’t it’s fucking doomed.

Let’s say two documents are chosen based on a user’s search and search history. They’re represented as d1 or d2. D1 is an already-consumed document and d2 is brand spanking new. Well, to the user at least. These documents can be represented as a vector (or some other semantic representation) to help the model fake understanding of the document and its position against similar documents.

The system provides a quantitative score to assess whether the user should also view d2 after having viewed d1. If the machine learning model generates an information gain score of document d2 over document d1, then d2 is likely to be shown - for future use cases, possibly at the expense of d1.

There are some incredibly practical implications here.

If a topic has been done to death, you have a more limited chance to rank and generate value without providing something extra. In a scenario where your article scores 0, the system has assessed it provides nothing extra and a user who has seen d1 is less likely to see d2 - your article.

If nothing else, make sure you stand out above your closest competitors in some manner.

A lot of this describes the foundations of creating brilliant content. Being different and standing out.

As with so many of these Google-led ideas or initiatives there are flaws. You don’t have to follow it to the letter. But EEAT and ‘information gain’ are sound principles. You have to be memorable. There is no alternative.

Subscribe now

How important is it?

I think uniqueness and standing out is more important than ever. Strip the patent out of the conversation. People or brands who publish content won’t survive if they aren’t memorable to people and - by proxy - search engines.

So you’ve got to do something differently.

In Google’s case, I think it’s more about efficiency than anything else. If they know the information gain scores of two documents are virtually identical, then a user isn’t going to be shown both versions of the document. The second document will be deprioritised in favour of richer, more unique content.

Google has enough engagement data to go along with these proxy scores to understand what document should be shown and when. They can get a user closer to their goal by removing overly similar pages from a user’s SERP or AI response.

Which may be exactly why they’re thinning their index - the removal of non-value add content. Well, that and all the AI slop you’re creating.

It is quite literally down to a) computational resources (money) and b) getting the user to the point of completion quicker. In the DOJ Antitrust trial, Pandu Nayak's sworn testimony called Navboost “one of the important signals that we have.”

And the Quality Rater Guidelines make numerous references to effort, originality and talent. Frameworks like EEAT and the product reviews update really highlight the importance of actually using products and showcasing the effort you have gone to. The amount of ‘effort’ you put in is quite literally quantified (highly recommend Sean’s breakdown here). It is part of the Helpful Content update (booooooo) and the more difficult your page is to replicate, the better chance it has of success all things being equal.

These are not stupid principles. They’re very good ones. The problem is, effort is expensive. The fewer clicks content produces, the less each article will generate.

In an attributable manner at least.

Google is building an audience loyalty ecosystem

Don’t take my word for it, take Barry’s. Google has wanted to get rid of click-chasing churnalism for years. Now it can. And it is - in most cases I think, a positive.

They are trying to build something around engaged users - like every publisher out there. Your most engaged users are your most valuable. Google’s quietly building a subscriber ecosystem that could one day rival their ad business. No reason to think that

Publishers that can demonstrate they have an audience outside of SEO are being ‘rewarded.’ Although I suspect you could replace rewarded with crushed a little more slowly.

You can follow your favourite publisher on via Preferred Sources and as a Search Profile via the Discover feed (US-only at the time of writing this) and badges like ‘highly cited’ have been in play for some time. It doesn’t work very well, but they are trying to promote unique reporting.

You can now see how content from social and video platforms performs on Google Search if you meet the requirements. Your digital footprint and impact within the industry you’re in really matters. Particularly when you consider how prevalent social and creator accounts are in Discover.

I worry that this is completely impossible to explain what is happening to users. What the fuck is Preferred Sources vs a Search Profile?

It’s tough to force people to follow you on platforms - maybe that’s the point. Which I kind of understand - but I think one of these would’ve sufficed.

If you want to know a little more about where Discover is heading, I made a short video about it:

Does information density matter?

Yes and no. Long articles are not necessarily more effective at satisfying the user.

Google has methods to normalise the length of an article, to prevent additional keywords and semantically relevant phrases from ranking the document too highly. Factors like TF-IDF normalisation prevents long documents with high word counts from artificially inflating their relevance scores just because they’re quote unquote richer.

More detail may be the wrong phrasing here. Detail and rigour are typically positives. But it’s less important than answering the question and getting the user closer to their end goal.

User satisfaction is quantified through goal completions and Navboost data - it trumps everything else.

How does it affect AI systems?

Well, traditional search ranking is still crucial in AI systems - whether that’s how effectively you rank for the primary search, your inclusion in the training data, RAG or suite of fan out searches run concurrently. And AI searches are extremely personalised - something that’s likely to only increase over time.

When Claude starts knowing what toilet paper I buy or selects a poorly chosen ‘Happy Mother’s Day’ card for my mum’s birthday that showcases my lack of effort and empathy, it’s time to call it a day.

According to Kevin Indig’s latest excellent research, First-party research is rare in AI citations, but it earns 3.3x more. And original data is the strongest single predictor of page originality. Good for traditional SEO, good for AI search. Who knew?

The ideas described in this patent map almost too neatly onto how modern AI search systems retrieve relevant information. Of the SGE. It helps anticipate the user’s next interest in an assistant-like context. Personalised, ‘helpful’ and with extreme memory.

As Roger Montti pointed out, this may give a clearer indication of how AIOs use pages that the user in question may be interested in. Their entire job is to synthesise answers from multiple sources and searches to provide the perfect jumping off point. I suspect this scoring system is an excellent way to avoid computationally expensive, unnecessary utilisation of documents.

contentEffort - described as a ‘Large Language Model (LLM)-based effort estimation for article pages’ - estimates the amount of effort invested in creating an article. As slop makes up more than 50% of the internet, this is seemingly one of Google’s way of dealing with it.

Subscribe now

How can I use this effectively?

Make differentiated, non-commodity content. It’s really simple. Apply what we call information gain in this context to your own content - if you cannot add anything of value to the existing index, then don’t bother.

You can use do this with:

• Original data

• First-hand experience

• Interviews

• Real reporting

• Being first on the scene and developing the story as it happens

• Proprietary analysis

You don’t need a big budget. You can do amazing thing with a few free data sources, some creativity and a bit of rope. Just make sure the article has an element of uniqueness.

I think this really helps frame whether content is still worth creating. If you’re doing something just for SEO reasons and you can’t add anything extra to the existing suite of information, kill it. If a document contributes very little new information, the patent suggests it’s a strong candidate to be deprioritised when selecting subsequent documents.

Still costs time and money to make, but is less and less likely to drive any real value. Stay in your lane, but drive a nicer car.

I have a feeling your indexation report in GSC is invaluable here. Beige content has a shelf life so low it’s in the running for the new UK Prime Minister. So check for any pages dropping out of the index at scale for more serious issues.

Leave a comment

Last week I argued that habitual publisher traffic - direct and branded - has been eroding for years, and that the under-35s are leading the charge. Not directly provable, but a theory.

Now that I am no longer part of that cohort, it’s easier to pin the blame on them. Which is entirely what I intend to do.

Subscribe now

If you’d rather, you can listen to or watch me talk about this instead:

Younger audience share isn’t declining dramatically

Based on 15 of the biggest UK publishers (and Similarweb’s always excellent data) 18–34s make up 29.5% of the average publisher audience. That’s marginally above our estimated ONS population benchmark of 28%.

On that number alone, publishers look broadly representative of the country they serve. I use the word serve loosely here.

It even looks quite stable. The average publisher share of 18-34 year olds has slipped only slightly over three years:

• Premium publishers −3.0pp

• Public Service −2.6pp

• Popular publishers −1.0pp

• Platforms have seen the largest drop in younger audience share −6.8pp

Not ideal, but certainly not numbers that warrant the furore around younger audiences. Particularly when you see that platforms have lost audience share in the 18-34 bracket too.

Platforms average 49.2% younger audience, roughly 1.7 times the publisher figure. No traditional publisher clears 40%. The New York Times tops the set at 39.1%, albeit only their UK audience; the BBC is next at 35.1%.

So what’s all the fuss?

Share doesn’t tell the full story. It is just a ratio. Your younger audience share can look less depressing if you’re also losing an older audience. Which is exactly what this data shows.

In every publisher segment, the younger audience is declining faster than the older one - on a total that is itself down 12%-32%. For publishers, younger audiences look like a shrinking slice of a shrinking pie.

In real terms, younger audiences have declined far more significantly;

• Popular publishers: −34.2%

• Premium: −30.7%

• Public service: −16.9%

• Platforms: −9.2%

So whilst platforms have seen the largest drop in younger audience share −6.8pp, they have seen the smallest younger audience decline by volume.

These figures have obvious caveats: third party data estimates both share and volume. And these are only website visits, the data doesn’t include app data. But when you compare apples with apples, the direction is what matters. Even if the apples have wasps in them.

Subscribe now

Have young people left to go to platforms?

Not according to this data. Not to their websites at least, which is a limitation of this data. The app experience for TikTok, YouTube, Reddit etc is significantly better and almost certainly one of the platform’s primary goals.

It’s very difficult to say that younger audiences have all left to go to social media platforms based on this data alone. Particularly while their older audience grew by more than 10% in real terms.

Nothing in this data proves platforms are absorbing the audience publishers are losing. I’m sure that these users have just migrated into feeds and apps that are harder to measure - they certainly haven’t gone to the open web.

What's defensible is simpler and more uncomfortable: at publishers, the younger audience is in absolute and unambiguous decline.

What do we do about it?

If you think attracting and retaining a younger audience is a problem, you are not an outlier. Most traditional publishers are in the same boat. The first step is to diagnose the scale of the problem. Understand precisely where you sit compared to your competitors.

That’s the exact job our younger-audience diagnostic does: a position scan that benchmarks your younger share and engagement against the right peer set and drops you into one of four quadrants:

• A distribution problem

• An engagement problem

• A relevance problem

• Relevant AND engaging (no one exists in this position of the quadrant, but we can dream)

We combine this data with a capability scan that maps where you win or lose younger readers across the younger audience funnel. From inspiration (awareness) through to fandom (advocacy).

Instead of panicking, we can diagnose, position and give some clear, data-led recommendations. Much healthier.

That’s it - short and sharp. Until next time.

We cannot deny the open web is changing - AI bot traffic grew 187% from January to December 2025, while human traffic grew just 3.1%. The value exchange for websites - particularly those who have traded on information and clicks - does not exist as it once did. At least not in as sustainable a manner.

Whilst this has been expedited by the arrival of LLMs - answer engines that have restructured the web without partaking in the click-based value exchange - audience habits have been shifting for some time. Generally not positively for publishers.

The erosion of direct relationships

Similarweb data for 15 publishers and four platforms show that direct traffic has declined across every segment over three years:

• Popular publishers −33.1%

• Premium publishers −23.4%

• Public Service publisher(s) −19.9%

• The Platform segment also recorded a direct-traffic decline (−13.3%), but offset it through growth in other channels.

But the segment averages don’t tell the full story. The under 35 audience is declining roughly 1/3 faster than the over 35s. - the exact cohort tomorrow’s (hopefully) paying subscribers come from.

Birmingham Mail saw a 54.6% reduction in direct traffic over this timeframe. The Mirror a close second at 52.9%. Conversely The Telegraph lost just 8.9%. Potentially some Premium resilience on show.

The NYT saw growth in the UK market, albeit from a tiny user base, as did GB News - a relatively new proposition in 2023 to the best of my knowledge.

YouTube’s 17.8% loss in direct traffic over this same timeframe drags the Platform segment down. You could argue that this is symptomatic of more direct-to-app-based behaviour. I suppose you could make the same argument for publishers.

All other platforms gained a direct user base over this timeframe. Although worth caveating that Substack and TikTok had a much smaller starting point:

• TikTok +56.7%

• Substack +248.8%

• Reddit +4.7%

Platforms’ total traffic held up so effectively thanks in no small part to Reddit’s extraordinary rise in organic search - up 114% over that same period. Which I’m sure none of you will be surprised by.

Subscribe now

This behaviour is mirrored in branded search

It also may not surprise you to know that the loss of habitual audience is not limited to direct traffic. Branded searches - arguably the second or third best proxy of user resonance (alongside online mentions) - have been in similarly stark decline.

From the Daily Mail’s peak in 2013, the story has been fairly consistent and destructive. Publisher offerings have seemingly become less attractive.

Within the same three Similarweb data window, branded search fell roughly 25–56% across the titles that have measurable signal.

The Daily Mirror has seen a 56% reduction in branded searches. The Sun 54%. The Times and The Independent have seen the smallest drop, but they had already hit the floor before this window arrived.

So what?

Publishers aren’t competing against other publishers. Well, they are, just not exclusively. As the internet and the world evolves, publisher offerings have to follow suit. Branded search fell faster than direct traffic over the same window - two independent measures of the same fading habit, pointing the same way. Down.

So publishers need to become more of a destination and attract younger audiences again.

Develop named voices, and work with creators. Platforms show greater resilience because they leverage the individual. Audiences, particularly younger ones, trust the individual over the brand. Publishers can adopt each within their own brand architecture, by building individual voices, as Wired is doing, and diversifying the product and revenue-base.

Create habit-forming products. A decade of direct and branded-traffic decline can be arrested, but not by repeating what worked before. Audio and video, games, puzzles and other return-driving formats build the engagement that compounds into lifetime value. A user that loves the brand has a total CLV well over 50 times higher than that of a casual or one-time reader according to Ringier.

Invest in product architecture, not just editorial. Closing the engagement gap with platforms takes recommendation systems, personalisation engines, and newsletter and notification infrastructure - the standard younger audiences now expect.

All of this is designed to build resilience in the form of a moat. Use these products and systems to collect first party data. Social referrals have fallen sharply and Google is becoming more of a walled garden, resolving queries on its own platform. Registered, signed-in audiences are the hedge: they compound in commercial value, and without first-party data, quality personalisation is very hard.

That’s it, short and sweet. Until next time!

Leave a comment

So Google finally rolled out some kind of AI reporting (if you can call that reporting) and the CMA announced some new measures Google must take to restore some level of parity with publishers and search results.

This is, in fairness, a world-first. Clearly a step in the right direction. But that’s all it is, a step.

We have AI reporting without clicks for, I presume, obvious reasons. Publishers can opt out of AI Overviews - a nice to have - but unless this is done at scale there’s very little bargaining power here and the onus is on publishers. Although I will say allowing publishers to opt-out of their content being used for the ‘fine-tuning’ of AI models is, obviously, a good thing.

Oh and Search profiles launched ‘to help publishers and creators highlight their work on Search.’ This might seem small, but it’s a huge part of the future of search. It’s why I don’t hate the search everywhere optimisation moniker. Search is broader than ever.

LLMs have restructured the open web. Your way of accessing and retrieving information has fundamentally shifted. Entity SEO - the connections you can build around your brand (personal and professional) - is more important than ever. You cannot afford ambiguity anymore. You have to build a profile. With real people. In multiple places.

Positive first steps. There are many to go.

If you prefer, you can watch me talk about this instead:

Financially, how are they doing?

Errr… pretty well.

Total revenue for 2025 came in at $402.8 billion, up from $350 billion in 2024 - the first $400 billion year in Google’s history. larger than the GDP of Portugal. The company also delivered its first-ever $100 billion quarter in Q3.

A more robust view of their annual reports shows a few things.

1. Google Search & other - $224.5B (+13.4%)

2. Google Subscriptions, Platforms & Devices - $48.0B (+19.1%)

3. Google Cloud - $58.7B (+35.8%)

4. YouTube’s annual revenue surpasses $60B

Search is still Google’s largest revenue line by some margin. A rise in zero click searches keeping people in their system is clearly a good thing for their advertising business model. Thanks to data from Seer Interactive, we know that AIOs caused the CTR (and cost) of PPC advertising to respectively decline and increase over this time. Albeit this appears to have levelled out.

Cloud revenues jumped from $43.2B to $58.7B, primarily driven by growth in infrastructure and platform services. Google Cloud ended 2025 at an annual run rate of over $70 billion, with demand driven by AI products, and nearly 75% of Google Cloud customers had used Google's vertically optimised AI.

YouTube’s dual revenue model may be Google’s plan for search. Its advertising revenue grew from $36.1B to $40.4B last year and subscriptions give it unparalleled resilience. Unlike Search, YouTube’s creator economy is still booming.

It’s a product that works on both fronts.

Finally and most importantly IMO, Google’s subscription based products are their fastest growing non-cloud segment. Consumer subscriptions exceeded 325 million across Google’s services thanks to Google One and YouTube premium adoption. This is a parallel revenue line for Google - one where advertising matters less.

The open web becomes less important as an intermediary because of it. It enhances their ability to lose money on advertising. They are building resilience.

Google also has a sneaky line item in their 10-K in the OI&E section (Other Income and Securities) labelled as non-marketable equity securities. Google has invested $3B in Anthropic up until this point and has $24.1B of unrealised gains over the same time period.

As a private company investment, fair to say this kind of circular investing is purposefully difficult to quantify. Google has now committed to investing $40 billion in Anthropic.

Will AI Mode become the default?

I suspect AI Mode becomes the default search experience for most people and a significant number of searches. People like the conversational interface and once they figure out advertising and reduce the cost of running it, I’m sure it will get rolled out more aggressively.

Thanks to data shared by Aleyda Solis (and Rand Fishkin), we know AI Mode’s usage more than doubled in the US in a single quarter. In Europe, it grew even faster. Overall share remains below 0.25% in both regions. The pessimist in me thinks that’s a bit shit. The optimist would say there’s room for growth.

• In the US, Google AI Mode’s share of desktop events grew from 0.06% in December 2025 to 0.16% in March 2026. A 2.5x increase in one quarter.

• In the EU and UK, growth was sharper: 0.06% in December to 0.21% in March, nearly a 4x increase over the same period.

• By March 2026, EU and UK adoption had overtaken the US (0.21% vs 0.16%), despite AI Mode launching later in Europe.

Sundar is bullish on the topic. He claims Google are creating a seamless transition from ten blue links towards AI Mode and people are enjoying it. But it’s not an overnight change. You aren’t going to flick the switch on a 200+ billion advertising model when the alternative is so untested.

Particularly when your competitors are financially floundering somewhat, beautifully described by Gary Marcus as tokenmaxxing. The immediate risk is somewhat lessened.

And worth remembering that 44% of Google searches are navigational. Is that a strong proposition for AI and far more computationally expensive conversational search?

Maybe. Maybe not.

We’ve seen a notable uptick in AI Overviews appearing for branded searches and research from Kevin Indig showed that people are, in many cases, just looking for top of the funnel information at this stage in their journey. And we don’t want to click on websites.

So the navigational click is being replaced by an in-SERP navigational assessment. If Google can fulfil your need in their SERP, you can bet your bottom dollar they will.

Absolutely worth watching David’s explainer video on the future of the value exchange and how to understand the value of your content in Google’s eyes and your own. This is foundational for publishers moving forward.

Subscribe now

Why AI Mode might not be the default experience

Hugging Face estimated it costs about 30 times as much energy to generate text versus simply extracting it from a source. Whilst that isn’t revenue as such, computational power correlates pretty closely with cost.

Right now, AI offerings are computationally expensive, much less monetisable (although I’m sure that will change) and won’t be picked up by great swathes of the population.

If you had a more expensive product that specific people or cohorts of people weren’t using (and are unlikely to use), would you push them into using it? Or would you just primarily give them what they’re used to? Could that group’s margin support your expensive expansion?

If you’re familiar with the stages of technology adoption, you get a sense of who adopts it, when and why.

Innovators and early adopters are far likelier to be younger. People whose brains have enough plasticity to continue to change. In AI Mode’s case, I suspect those in the conservatives and skeptics sections will never use any kind of LLM.

If you don’t believe me, ask your parents what ChatGPT is. If they do use it, I’m fairly confident it’ll be for something either completely banal or psychologically worrying.

Google’s ability to understand the intent behind the search (and the user) will form a key part of the search giant’s revenue modelling. The more effectively they can do this, the less computational power they will spend.

Why news holds firm

AI Overviews do not supersede a Top Stories block. There are two theories here:

1. Google has decided to create a safe haven for publishers (highly unlikely)

2. AI cannot accurately or fairly represent fast-moving, immediate stories effectively (no one has cared about accuracy so far)

So news, for now, is resilient to AI. It is filed under the type of content that still adds real and obvious value. Tools, products, real, human interest stories, stories with unique data and research that cannot be effectively delivered by an LLM and real, expert-led opinion from people we trust.

Whisper it quietly, you might even get a click from it.

To show Top Stories resilience, I ran a couple of thousand queries pretty newsy queries through DataforSEO, extracted the structured data and represented it in a column graph of queries by category with an AI Overview or Top Stories. This is a tiny sample size, but it’s representative of larger ones I have seen and run.

• Branded queries had the lowest number of AIOs, albeit there were only 50 branded queries in the mix.

• News/entity type queries (primarily people, places and organisations) had the second lowest number of AIOs, with just 19%, with 34% served by a Top Stories box.

• Informational queries have the highest percentage of AIOs

Typically this is when queries like ‘China Venezuela’ can be well served with an AIO - because there’s significant evergreen search volume, or the news story has died down and can be effectively synthesised via an AI response.

If you want to truly run some quality risk and exposure mapping, I find it helpful to create category risk scores by entity type. You can expand this to content categorisation too.

On the rare occasion that you see these beasts co-exist in the wild, I believe it only occurs when the news agenda for said topic has dropped off enough that an AIO can replace it.

Google Discover’s Competitive Advantage

We have a pretty good idea about how Google Discover works.

One of its strongest capabilities is the cohort-driven story rollout.

1. Person A is similar to person B

2. Person A loved and engaged with said story

3. Said story is shown to person B

4. Person C also shares a similar characteristic and is shown the story

People are classified based on what they like. Google builds groups. These groups form the foundations of virality.

Like almost any social media algorithm, Google Discover is Google’s way of hooking people into search with an algorithmic feed. And I think it could have significant potential for the future of the SERP.

When the time’s right, you will be shown incidental content you didn’t even know you wanted to read. And you will click it.

So what does this mean for the state of the SERP?

Diversity. Zero click search is overblown, albeit I think zero click marketing is the present and future. Whether the public discourse about AI is the long-term projection of it or not, not everybody blindly trusts LLM responses. They trust people and want to make their own decisions.

AI is just part of the journey. Maybe more precisely, it summarises it in its entirety.

Younger audiences use publisher websites differently. It’s less about browsing the whataboutery of the day and more about validating nonsense they’ve seen on the slop-filled internet. Even Google isn’t immune - younger audiences are less engaged with the platform.

In some ways, Google has been forced into making a change.

Google’s ability to show each user the most appropriate SERP will have a significant impact on their bottom line. Local, comparison results like the below - best hotels in London - has been tough for publishers for a long time. AI or not.

To me, it’s interesting there’s no AI experience here. Because this middle of the funnel comparison query is ripe for a longer, more conversational search. So I don’t think all is as brazen as it seems. Money still matters.

If you have followed the SGE for sometime, this has always been the likely outcome. or at least final iteration to the best of our knowledge. A results page that encompasses;

• Deep personalisation

• AI-powered snapshots

• Shopping

• Map packs

• PAA sections

• Discover like ‘we think you’ll like’ article cards

• Reviews and trust-like content from forums

• Top Stories

• Traditional 10 blue links

In my opinion, these will all form a part of most SERPs. News driven searches will continue to have a Top Stories block (unless AI is capable of dealing with real-time content - make sure you understand RAG in this context). Comparison related searches will have a rich, AI-led experience, interspersed with review-type content from brands and forums. Local searches will lead directly to map packs and longer, more conversational queries will take you straight to AI Mode.

What you choose to create all comes down to value. Value doesn’t have to mean revenue in the form of sales, subscriptions or advertising. It could lead to newsletter signups, internal link clicks that you know lead to greater audience retention, improved visibility in AI systems.

You just need to have the right KPIs in place for a zero click world. I’m sure you don’t want your success to be tied to metrics that will inevitably go down. If you want to influence senior figures and newsrooms, you need a sure footing.

A subscription-led model may be the future

Worth noting the financial success of Google’s other products are subscription-led. Subscription revenue is more valuable to a business than most others - it is predictable, measurable and consistent.

It’s very plausible that Google bundles AI Mode and several other AI-led products into a single subscription. Tokens are becoming more expensive. The ad model may never work for AI Mode and a hybrid advertising/subscription model may be their best route forward.

Which, IMO, may not be a bad thing.

Leave a comment

Complete and utter shite if I’ve had a drink.

They all read like headlines you’d see in Discover and scroll past very fucking quickly.

Maybe in a few years that’ll be all that’s left and that’s what Googlers are prepping us for. Personally I think it’s far more likely their idea of quality, interesting content is a just a bit rubbish.

Marble vs grape juice - what a stupid title. Although interesting that they specify this is a video. Don’t hate the shoe one. No idea how that will make money for anyone however… Doesn’t matter to Google.

Anyway, here’s how I think you can create unique, interesting content that still drives actual value to your business. (Hint: it’s not about grape juice).

TL;DR

1. Commodity content is doomed for two reasons - it is easily summarised (because it has been done to death) and it doesn’t make (as much) money in a zero click world

2. If you are creating content just for SEO and have nothing unique to offer, stop. You are throwing money down the drain

3. Be more than an SEO. Help other teams structure their workflows to generate the maximum value from all channels with things like demand analysis

4. Google calculates and compares the uniqueness of a document using a custom ‘information gain’ score at a query and document level

Subscribe now

Why commodity content is doomed

People are like water. We take the easiest possible route. One that really doesn’t include clicking to find an answer, even if said answer is riddled with bullshit.

Commodity content - content that has been the bedrock of evergreen search strategies for years - can be very effectively summarised and synthesised by answer engines. So effectively, that people will be satisfied with said clickless search.

Direct from the greedy horses’ mouth.

This means we have to focus our efforts elsewhere.

We have to focus our time and efforts on content more likely to drive legitimate value. Content that cannot easily be summarised by AI, adds something of real value to the user and hasn’t already been thrashed to death by savvy SEO teams.

You do not have the time, money or resource anymore to spend time on content that doesn’t drive value. Value is everything when understanding and auditing publisher websites.

Does this mean things like search volume are useless?

At an individual keyword level, search volume has been declining in value for a long time. We just can’t generate the value we once could and it isn’t coming back.

But search volume just indicates demand. If you’re savvy and use monthly data, you can help content, social, paid marketing and editorial teams understand when users really care about a topic.

In this capacity, your job is to help teams understand when to create or update content, what that content should cover and crucially, why it’s spiking in search at this particular time.

If we take searches for family holidays in Google Trends as an example, there is clear and obvious consistency. Searches spike every January as people plan their family holidays for the year ahead in the bleak midwinter.

So you should still get your core family holiday content ready for January. But as we shouldn’t operate in a silo, you should share this with social and travel teams so they know what time of year this type of content will generate the most value.

Planning and structure take centre stage.

It is no longer about create x, get y. That click-based marketing is dead.

You can listen or watch me talk about evergreen and commodity content here:

Commodity or not commodity

Loosely this header was a Shakespearean-based to be or not to be joke, which is a) clunky and b) outside of my wheelhouse.

Now I’ve had to explain it.

I wrote about this in how to do evergreen content in 2026 and beyond. Which is, ironically, quite a commodity topic. But it has evolved. There’s new shit to share. You can make commodity, non-commodity.

But you need to have a level of understanding and expertise that can really elevate a topic. That requires experience, a level of uniqueness and a platform. Your content needs to be found and what we have always done in search is unlikely to be anywhere near as valuable.

Subscribe now

The pillars of non-commodity content

• Uniqueness

• E-E-A-T

• Engagement

• Structure

Uniqueness

Uniqueness is the bedrock of everything when it comes to content that will continue to drive value. Without uniqueness, there’s no E-E-A-T. You won’t generate any shares, likes, comments or links. Certainly not any good ones.

You can make this as fancy as you like.

If you’re lucky enough to have access to high-quality data sources like Similarweb, you can create some truly brilliant proprietary metrics that elevate your content above and beyond.

Let me give you an example.

Similarweb give excellent engagement data at a site level. App-level too. If I was to combine these three metrics (pages per session, session duration and bounce rate) I have a composite engagement score.

Something no one else has.

If I took that engagement score and correlated it with third party traffic data or something like branded search/backlinks, I could correlate engagement data with traffic from search over time.

This is what stands out. This is what audiences will read, share and crucially, remember. It requires more effort.

And as we know from the Google Leak (this brilliant warehouse from Daniel Foley Carter is superb), effort is quite literally estimated and scored by Google. Things that are difficult to replicate are rewarded.

Unless they’re absolutely insane. Then probably the opposite.

You don’t get good at this overnight. But Google has been prepping us for this for some time. If you look at the declining youth engagement in the above graph, maybe people have too.

Not everyone is fortunate enough to have access to Similarweb data. But that doesn’t matter. Creativity and quality research is more important (and more readily available) than ever.

There are so many quality free data sources - Google Trends (combined with Glimpse), Keyword Planner, some free plans on tools like Ahrefs or Similarweb etc. You just need to identify metrics and combine them to make something bigger and better.

Google attempts to quantify information gain

Google has a patent (US20200349181A1) called Contextual estimation of link information gain that shows how the search giant may score the added value each document provides when compared to other similar documents.

Patents aren’t absolute. Just because a patent is present, it doesn’t mean it is always in use. If they’re frequently cited, recently updated and have worldwide applications, that’s usually a very good sign they have a level of importance.

But ‘ranking factors’ aren’t absolute either. SERPs and topics are vastly different. It’s why we have subtopics like local SEO, YMYL et al.

What matters for one term or topic may not matter as much, if at all, for another. It’s the nuance of the job and why trial and error is so important.

You don’t know until you know.

Consider the Four E’s

Your content needs a purpose.

Yes it needs to convert. That is a business purpose. But it needs a purpose for people. Is it designed to entertain? Educate? As audiences turn away from news (and probably more widely commodity content), this matters more than ever.

What we now term as commodity content was never designed to do any of the above. It was just designed to make money. Over the years, anything shit propped up by Google just to make money has died.

This is the next cab of the rank.

E-E-A-T

E-E-A-T has taken a bit of a kicking recently. Not without reason.

The premise is sound. Not unreasonable for readers to expect the author to be, you know, a real person, who knows something and has some kind of online presence. And Google absolutely does track authorship and entities. Plenty of evidence of that.

Google has built and maintained its Knowledge Graph for decades and entities have been the bedrock of news SEO for years. But E-E-A-T requires you to join the dots. To remove ambiguity - something we call disambiguation.

Doesn’t mean doing this is incredibly valuable, but it’s foundational. Particularly in this modern day iteration of the internet.

Remember, E-E-A-T projects have to add value

The problem with the whole - use experts, showcase expertise, prove you test everything, create video, make an effort in the industry etc - is now twofold;

1. It’s fucking expensive.

2. And less valuable than ever.

Having that person build some kind of profile in the industry. A platform that their content can be shared from and that reduces reliance on search can only be a good thing.

A moat if you will.

If they’re a legitimate expert on the topic, know how to structure great content and effectively showcase expertise, then you’re onto a bloody winner.

Which is why commodity content is doomed. Because people don’t care about it and now it doesn’t drive value.

Subscribe now

Engagement

I have always felt that links should be a happy byproduct of creating and sharing brilliant stuff.

I’ve never made an effort to build links. I have just made an effort to write stuff I think is interesting, made some semi-libellous jokes and got out there in the industry.

That is, more or less the Google definition of link building. In their world of sunshine and sweet smelling farts, links are just earnt by doing beautiful things. I am, in this scenario, the poster boy for white hat SEO.

The problem is, people need to make money and links still drive rankings. So there’s a market there. And if you’re a student of the scriptures like I am, you’ll know the buying and selling of links is the oldest recorded job.

Or was that prostitution?

Either way my inbox is full.

Anyway, your content has to fulfill a need. We’re moving away from straight laced content being able to do that for you as a publisher. Traditional ad revenue and the volume model sucks and you sure as hell aren’t going to drive any subscriptions with what time is x or how to tie your shoes.

I really hope this is a good thing for SEOs and publishers. I want us to focus on content that really makes a difference to people’s lives. Content that makes them smile or think.

Content that makes people angry has been a big hit when it comes to numbers for a long time. But I don’t think anger is the emotion you should shoot for.

Measurement

You need to measure quality engagement, on and off-site. That means;

On-site

No need to overcomplicate it for now.

• Session duration

• Bounce rate

• Link clicks

• Pages per session

• Comments

• Read time

Off-site

Very much depends on the platform and the purpose, but I would focus on;

• Links

• Shares

• Comments

• Saves

• Watch time

You need to track metrics that tell you clearly whether people truly care about what you are creating. Clicks are dying, so I’d rather be measured against something a) more valuable and b) less miserable.

Create a composite metric(s) that gives you and your creators something to clearly focus on. Make their job easy by guiding their content with simple, straightforward metrics. Metrics that don’t just chase page views.

Structure

Structure’s not sexy. Let’s be honest.

But it matters. If for some reason you think LLMs are the zenith of society and content consumption, then you should know that models are more likely to cite or reference content from the top or bottom of the page, thanks to their inability to properly follow an argument.

This is known as the lost in the middle effect.

Unless of course, the entity and topic is repeatedly referenced throughout.

I shouldn’t have to tell you that this is a shit idea and your content will become unreadable to living, breathing people.

But maybe you don’t care about that anymore.

Proper structure really matters. People have expectations (and accessibility needs). In more traditional commodity content they want their question answered immediately. If you satisfy that - and the intro to your article isn’t abysmal - you might generate a longer session, a click or hell maybe even a conversion.

Theoretically, non-commodity content accessed via search should still be intent-driven. Possibly more so if we’re to believe the more qualified users with longer tail queries theory Google espouses.

So you still need to follow a similar, highly coherent page structure;

• Answer the fucking question

• Some form of TL;DR article summary

• Argument

• Concluding thoughts

• Coherent FAQs (if applicable)

One that logically answers queries in the appropriate format - text, video, image, list etc - and is highly consumable.

The argument section is where LLMs tend to lose their ability to accurately and appropriately cite and reference content. Which is not at all dissimilar to people.

I am not saying you need to continually refresh and restate the entity in question. That may be construed as keyword stuffing. It needs to read well for people. But you need to be clear, concise and accurate to make consuming your content simple.

Don’t people consume content in different ways?

You’re absolutely right my pedantic friend, they do. Broadly, I think there are four types of consumption;

1. Scanners - The vast majority. Too lazy or illiterate to read the whole thing, but will be satisfied from a headline, bold text, bullet points, and headers. They treat a page like a map, not a story.

2. Answer seekers - They find what they want and leave. But still leave satisfied.

3. Visual/audio consumers - A cohort that either refuse to or cannot read, but will stare at a pretty picture for 60 seconds.

4. Deep readers - A small cohort, but a deeply engaged one, desperate for you to get something wrong.

I suspect these groups cover >90% of people. There are also fact-checkers - who skip the narrative and head straight for the citations, data points, or the “About Us” section before deciding if the content is worth their time.

And community-readers, who scroll to the bottom of the article to see the community reaction before deciding whether the content is worth their time. This is (obviously) more of a social trait. Particularly from younger audiences.

Your content can and should satisfy all of these people. It must;

• Answer the question

• Be highly scannable

• Broken up with clear, distinct headers

• Form a concise, easy to follow narrative

• Be highly scannable

• Easy to share

• Visually appealing (audio and video options available)

• Cite sources and clearly explain your methodology if appropriate

Anyway, that’s it. Thanks for reading as always!

Other good stuff

Some good stuff I’ve enjoyed recently;

• The Whole Point was the Mess - Pedro Dias

• It Works Until It Doesn’t: AI Content Strategies That Backfire - Lily Ray

• The New York Times is becoming subscription infrastructure for other publishers - The Media Stack

Leave a comment

Google’s market share has broadly held firm in the wake of everything AI. By held firm, I mean its share price has gone through the fucking roof and its AI offering is growing ever stronger.

But I don’t think all is as rosy as it seems.

Google’s search product isn’t addictive - as much as they’re trying to change that. Nobody hangs out there except saddos like us. And audiences - particularly younger ones - have options.

They’re turning away from more traditional methods of information retrieval and that’s a big fucking problem. Even for Google.

Even the search engine giant isn’t immune.

Older audiences - those already ingrained in the system - are taking up a larger percentage of their audience. The younger ones have more exciting and addictive options and best believe they’re using them to find shit.

Across every engagement metric, 18–24 year olds have deteriorated faster than 65+ users over the same period. Shorter visit duration, fewer pages per visit and a worse bounce rate. And it’s declining more rapidly with younger audiences.

Evolution for Google and the wider web is a necessity.

Although interesting to note that the 18-24 year old audience share has only suffered a small decline according to Similarweb data. The real losses were in the 25-34 cohort.

TL;DR

1. The publishing industry and Google have more in common than perhaps either of us care to admit.

2. The changes Google have made are a very deliberate effort to engage with - and retain - younger audiences. Audiences who behave differently.

3. Engagement data on news websites (pages per visit, bounce rate and time on site) declines with audience age. Exactly the same is true of Google.

4. AI Mode is Google’s attempt to create a ‘sticky’ product. One aimed at younger audiences.

Subscribe now

What’s changed?

Well, the obvious;

• Google’s started answering virtually everything in its search interface

• AI Mode is becoming something of a default response

• Personalisation is running amok

• It’s harder to generate clicks - reach and more traditional KPIs are less valuable

• Consumer decisions are made before someone reaches your website

• The agentic web is somewhere on the horizon

• Discover has become less publisher-friendly and more social and creator-led

• An increase in video and UGC in the SERPs (obviously all to YouTube and Reddit)

Just look at the SERP for almost any term, particularly middle of the funnel comparison ones.

What people apparently want is not very publisher, or legacy-search friendly. What they want, is video.

Particularly the bloody youth.

• 18-34 year olds spend 88 minutes a day on YouTube alone.

• Data from the UK market suggests children are averaging 2 hours and 14 minutes on TikTok per day.

The younger you are, the more time you spend watching, the less you spend reading. So the obvious counter (from a company who primarily organises written content) is to saturate the market with video content.

Obviously, it’s very fucking helpful if you own the market.

And this doesn’t just affect organic search. Adverts are more expensive to run because AIOs have destroyed the entire search ecosystem’s CTR. So for almost all businesses, customer acquisition is more expensive.

You could say that’s Google’s way of paying for AIOs - a far more expensive SERP to generate - due to the massive computational power and energy needed to run large language models (LLMs).

But I am not going to insinuate anything of the sort. It would be incomprehensible to me that the guys who earn the entire ad and search market would make the ad side of the business more expensive to run to pay for their search experiments.

Wait a minute…

Why now?

I think this is a direct response to two things;

1. The 2023 Code Red Google sent out in response to OpenAI

2. Younger audiences shifting information retrieval methods

One is obvious.

OpenAI forced Google to move quicker than they would’ve liked. Hence all the absolute shite in AI Overviews in the beginning. Well, and sort of now. It smacked of a product that hadn’t gone through the required amount of rigorous testing.

Two is more nuanced.

This data correlates almost perfectly with the Similarweb data I pulled. In isolation, this may not be a problem. Could be as simple as saying younger audiences will grow into it.

But I don’t think that argument works. We see it in news and publishing. We are living through it and we’re watching the decline in real time.

Younger audiences have the highest recorded screen time on record (globally, 7h 22m), but are spending less and less time reading. More on far more visually engaging, stimulating and addictive technologies.

Based on screen time alone, younger audiences should spend the most time on Google. But they don’t. I’m sure that is blatantly obvious to the Googlers.

Whilst content consumption is at an all-time high, the way a person consumes content is not conducive to more traditional publishing practices.

Just 4m a day on news websites for younger audiences vs 18m for the over 55s. A 350% increase.

The same principle is true of more traditional search.

At the risk of sounding a bit too AI-y, this is a really seismic shift. Ironically not one driven by AI. Not entirely. One driven by a combination of big tech’s insatiable appetite for money, a lack of trust in more traditional brands and the rise of the creator ecosystem.

And AI, obviously.

As someone in the comments said, Google is unc. Maybe a little like news websites. Their ability to attract younger audiences has diminished.

I think we can clearly correlate the changes Google has made to the reduction in younger audience share for publishers. A generation less inclined to click.

One could argue that the traffic losses so many seem to have suffered are almost exclusively from younger audiences. I certainly am.

Audiences more likely to adopt new technologies - particularly flashy ones.

Subscribe now

There are clear parallels between news and search

Google has got richer, as has the AI bubble. All that money has to come from somewhere.

It’s everyone else who struggles.

These changes are designed to counter a younger generations shift towards people and ultra engaging platforms that encourage passive or more incidental methods of information retrieval.

Since 2015, interest in news has declined - more significantly (43%) in 18-24 year olds than any other age group. And just 64% of 18–24s consume news on a daily basis, compared with 87% of people 55 and over.

Historically, news has been sought out.

Either you browsed a news website (a real paper if you felt fancy) or you searched for it. But the discovery layer changed and search - the engine that powered the volume-driven publishing model for two decades - is responding.

Responding to younger audiences’ shifting consumption habits. Just like publishers and websites will have to.

Passive consumption is just the norm now with younger audiences. This is why 44% of 18-24s see social media as main source of news, compared to just 15% of 55+.

They expect you to just appear. Algorithmic consumption has reduced the need, want and desire to actively seek something out. If what you serve isn’t delivered directly to their feed, you don’t exist.

Combine this with diminishing trust in more traditional brands, zero click searches and the rise of the creator and you can see why publishers and Google are having to change.

There have been alternatives to Google when it comes to accessing and retrieving information- Instagram, Amazon, YouTube et al - for years.

Really this is, or has been, Search Everywhere Optimisation. It has been around for a decade. It is also, IMO, why reframing SEO as GEO or some other bullshit because of LLMs is so moronic.

And now the individual has become the competition. The creator economy - soon to be worth $480 billion - has produced a new class of competitor: individuals with direct audience relationships, authentic voices, and none of the structural cost of a legacy newsroom.

51% of 18-24s pay attention to creators and personalities, compared to 39% who pay attention to traditional media and journalists - a 12 PP inversion.

And this is a problem for Google too. People used to use their organisational skills to satisfy all of their needs. Now it is so heavily navigational that it's hard to know how much ‘new’ stuff people really use it for.

Outside of news at least, ironically.

Will this work?

If it’s anything like news publishers, their primary concern is to continually generate new and engaged audiences with habitual products. AI Mode could absolutely be that product. Discover is their version of a social network. They are, in their own way, engaging products.

Although the low intent nature of Discover makes the advertising shit and Google not really care about it. Sad, but true.

Like Google, the engagement data for publishers tells a pretty bleak story.

If we isolate this to the youngest and oldest audience, it’s pretty clear what is going on.

Younger audiences;

• Are far less engaged with the traditional news offering than older audiences

• Use these (and any) websites differently

There’s no denying that younger audiences have more diverse and engaging options. This means they use websites like news publishers differently. To fact-check. To confirm something isn’t just spurious bullshit. To scan and skim.

The same is true of Google. Less of a discovery journey. More one of fact checking and navigational searching.

Subscribe now

So, what’s the answer?

To lean into what the new generation like. Adapt and evolve.

The same’s true for search (internally and externally) and publishers. If you work for Google, makes complete sense you would try to expand your video presence in the SERP and prioritise ‘quality’ UGC.

The quality part is lacking as most of the internet - as we’re finding out - is a stinking pile of shit.

But notoriously, the tide is tricky to swim against.

For publishers, it means working with creators, leveraging their audiences and ability to deliver things quickly. Differently. And creators can benefit from the trust associated with proper news organisations.

Is it that unreasonable to think Google should do the same?

Instead of abusing their position, they could start by giving people an idea of the impact of AIOs and AI Mode. I’m not a financial guru, but I reckon Google has enough money to build and foster creator and publisher programs that are not one-sided. That bring genuine value to people and the wider information retrieval ecosystem.

In this scenario, everyone benefits. When AI companies refuse to pay for publisher content, everyone loses.

• LLMs lose because they have less unique, human-created, quality content to train on

• Publishers lose because they are forced to suppress their visibility and don’t get any money

• Users lose because the end output isn’t as good

Model collapse is on the horizon. AI learning on AI falsehoods. A repetitive cycle of shit. Joyous.

These companies should invest in the ecosystems that built them. Particularly Google.

For publishers,

1. Build owned channels. Get the fuck away from relying on big tech

2. Create brilliant, unique journalism

3. Supplement it with habit forming products - puzzles being the obvious example

4. Build and sponsor audio and video programs that reach your intended audience

5. Implement channel-specific strategies

Even the NYT don’t rely solely on subscriptions from written content. Not by a long shot. It isn’t enough.

Final thoughts

Unfortunately I think the recent spate of job losses in the publishing industry is just the beginning. Bauer, the BBC, The Washington Post. It’s not UK or SEO-specific. 100k roles are becoming 70k ones. Teams are shrinking. And there are real world ramifications.

We are not in a good moment. Some of this can be attributed to AI. But I think more of it is due to longer-term economic difficulties, audiences switching off from traditional news and things like the Site Reputation Abuse update destroying much needed revenue lines overnight.

It is fucking hard to make these businesses profitable. Google doesn’t have that problem. But they’re not immune to changing behaviours and becoming yesterday’s news either.

Should you be enough of a psychopath, you can follow the job cuts via this updated Press Gazette article.

Leave a comment

AIOs have eroded clicks. Answer engines have given people options. And to be fair, people are fucking bored of the 2,000+ word article answering ‘what time does x start.’ Or recipes where the ingredient list is hidden below 1,500 words about why daddy didn’t like me.

In response to this, publishers say it will be important to focus on more original investigations and less on things like evergreen content (-32 percentage points).

So you’ve got to be smart. This has to be framed as a commercial decision. Content needs to drive real business value. You’ve got to be confident in it delivering.

That doesn’t mean every article, video or podcast has to drive a subscription or direct conversion. But it needs to play a clear part in the user’s journey. You need to be able to argue for its inclusion:

• Is it a jumping off point?

• Will it drive a registration?

• Or a free subscriber, save or follow on social

More commonly known as micro-conversions, these things really matter when it comes to cultivating and retaining an audience. People don’t want more bland, banal nonsense. They want something better.

The antithesis to AI slop will help your business be profitable.

If you’d rather watch or listen to me talk about this, have at it:

TL;DR

1. If you’re creating evergreen content ‘just for SEO’ that is, IMO, a bad sign. It needs more strategic thought than search volume + clicks.

2. Prioritise micro-conversions. Evergreen content may not drive direct conversions, but it should help users become habitual somehow.

3. People are tired of low-quality, low-effort slop. Standing out is crucial. You have to add value with information gain - which will increase production costs.

4. You have to sweat your content. Make it shareable. Connect with people. Build owned properties and on rented land. Drive unattributable brand demand.

Subscribe now

If you want a really practical approach to analysing the value of your different evergreen content areas, then this article and subsequent evergreen decision workbook should do the trick.

What’s the problem with evergreen content?

Inherently, nothing. It’s a foundational part of the content pyramid.

In most cases it’s been done to death and AI is very effective at summarising a lot of this bread and butter content.

Over the last 10 years it’s been pretty easy to build a strategy around evergreen content - particularly if you go down the parasite SEO route. Remember Forbes’ Advisor and the great affiliate cull?

The epitome of quantity over quality - it worked and made a fortune.

But I digress.

An authoritative enough site has been able to drive clicks and follow up value with sub-par content for decades. That is, slowly diminishing. Rightly or wrongly.

And not because of the Helpful Content shite. Google nerfed all the small sites long before the goliaths. Now they’ve gone after the big fish.

We have to make commercial decisions that help businesses make the right choice. Concepts like EEAT have had an impact on the quality of content (a good thing). It’s also had an impact on the cost of creating quality content.

• Working with experts

• Unique imagery

• Video

• Product and development costs

• Data

This isn’t cheap. Once upon a time, we could generate value from authorless content full of stock images and no unique value. Unless you’re willing to bend the rules (which isn’t an option for most of us), you need an updated plan.

Is it still worth it?

It depends.

You need to establish how much your content now costs to produce and the value it brings. Not everything is going to drive a significant conversion. That doesn’t mean you shouldn’t do it. It means you need to have a very clear reason for what you’re creating and why.

If particular topics are essential to your audience, service and/or product, then it should at least be investigated.

One of the joys of creating evergreen content has always been that it adds value throughout the year(s). A couple of annual updates, even relatively light touch, could yield big results.

Commissioning something quality in this space is likely more expensive. It needs to be worth it - it has to form part of your multi-channel experience to make it so.

• Unique data and visuals that can be shared on socials

• Building campaigns around it (or it’s part of a campaign)

• You can even build authors and your brand around it

• And if it resonates, you can rinse and repeat year after year

And this type of content or campaign can increase demand for a topic. You can become a thought leader by shifting the tide of public opinion.

For publishers and content creators, that is foundational.

Reframing value - a zero click strategy

Two broadly rhetorical questions:

1. Do you think in a world of zero click searches, clicks and reach are sensible tier one goals?

2. Do you want to be targeted against a metric that is very likely to go down each year?

I don’t - on both counts. We should want to be targeted on driving real value for the business.

Something like;

1. Tier 1: Value - core, revenue and value driving conversions

2. Tier 2: Registrations (and things that help you build your owned properties), links, shares and comments

3. Tier 3: Page views, returning visits and engagement metrics

Micro-conversions over clicks. We’re focusing on registrations, free or lower value subscriptions. Whatever gets the user into the ecosystem and one step closer to a genuinely valuable conversion.

Now, could a click be a micro-conversion? If you know that someone who reads a secondary article (by clicking a follow up link) is 10x more likely to register, that follow-up click could be a sensible micro-conversion.

This type of conversion may not directly drive your bottom line. But it forces you and your team to focus on behaviours that are more likely to lead to a valuable conversion.

That is the point of a micro-conversion. It changes behaviours.

You can tweak the above tiers to better suit your content offering. Not all content is going to drive direct tier one or even two value. You just need to have a very clear idea of its purpose in the customer journey.

How can we make sure it adds value?

If what you’re creating already exists, you better make sure you add something extra. You’ve got to force your way into the conversation and unless you can offer something unique, you’re (almost certainly) wasting your time IMO.

I’ll break all of these down, but I think (in order of importance);

1. Writing content for people

2. Information gain

3. Getting it fucking found

4. Creating it at the right time

5. Structuring it for bots

Subscribe now

Writing content for people

Everyone is obsessed with getting cited or being visible in AI.

I think this is completely the wrong way of framing this new era. Getting cited there, or being visible, is a happy byproduct of building a quality brand with an efficient, joined-up approach to marketing.

The more you understand your audience, the more likely you will be to create high-quality, relevant content that gets cited.

If you know your audience really care about a topic, that’s step one taken care of. If you know where they spend time and how they’re influenced, that’s step two. And if you know how to cut through the noise, that’s step three.

Really this is an evolution in SEO and the internet at large.

• Invest in and create content that will resonate with your audience

• Create a cross-channel marketing strategy that will genuinely reach and influence them

• Share, share, share. Be impactful. Get out there.

• Make sure it’s fucking easy to read, share and consume

Your content still needs to reach and be remembered by the right people. Do that better than anybody else and wider visibility will come.

Information gain

In SEO, we have a different definition of information gain than more traditional information retrieval mechanics. I don’t know if that’s because we’re wrong (probably), or that we have a valid reason…

Maybe someone can enlighten me?

In more traditional machine learning, information gain measures how much uncertainty is reduced after observing new data. That uncertainty is captured by entropy, which is a way of quantifying how unpredictable a variable is based on its probability distribution.

Events with low probability are more surprising and therefore carry more information. High probability events are less surprising and novel. Therefore, entropy reflects the overall level of disorder and unpredictability across all possible outcomes.

Information gain, then, tells us how much that unpredictability drops when we split or segment the data. A higher information gain means the data has become more ordered and less uncertain - in other words, we’ve learned something useful.

To us in SEO, information gain means the addition of new, relevant information. Beyond what is already out there in the wider corpus.

Google wants to reduce uncertainty. Reduce ambiguity. Content with a higher level of information gain isn’t only different, it elevates a user’s understanding. It raises the bar by answering the question(s) and topic more effectively than anyone else.

So, try something different, novel even, and watch Google test your content higher up in the SERPs to see if it satisfies a user.

This is such an important concept for evergreen content because so many of these queries have well established answers. If you’re just parroting these answers because your competitors do it, you’re not forcing Google’s hand.

Particularly if you’re still just copying headers and FAQs from the top three results. Audiences are not arriving at publisher destinations through direct navigation at the same scale. They encounter journalism incidentally, through social feeds, not through habitual site visits.

You’ve got to meet them there and force their hand. Understand how Google ranks evergreen content and add real value.

According to this patent - Contextual estimation of link information gain - Google scores documents based on the additional information they offer to a user, considering what the user has already seen.

Structuring it for bots

Bots, like people, need structure to properly ‘understand’ content.

Elements like headings (h1 - h6), semantic HTML and linking effectively between articles help search engines (and other forms of information retrieval) understand what content you deem important.

Whilst the majority of semi-literates ‘understand’ content, bots don’t. They fake it. They use engagement signals, NLP and the vector model space to map your document against others.

They can only do this effectively if you understand how to structure a page.

• Frontloading key information

• Effectively targeting highly relevant queries

• Using structured data formats like lists and tables where appropriate (these are more cost effective forms of tokenisation)

• Internal and external links

• Increasing contextual knowledge gain with multimedia (yes, Google can interpret them)

The more clearly a page communicates its topic, subtopics, and relationships, the more likely it is to be consistently retrieved and reused across search and AI surfaces. This has a compounding effect.

Rank more effectively (great for RAG, obviously) - feature more heavily in versions of the internet - force your way into model training data.

If you need to get development work put through, frame it through the lens of assistive technology. Can people with specific needs fully access your pages?

As up to 20% need some kind of digital assistive technology, this becomes a ‘ranking factor’ of sorts.

Starting or steering the wave

I won’t go through this in much detail as I’ve written a really detailed post on it. Basically;

• Track and pay very close attention to spikes in demand (Google Trends API being a very obvious option here)

• Make sure you’re adding something of value to the wider corpus

• If quality content is already out there and you have nothing extra to add, consider whether it’s worth spending money on (SEO is not fucking free)

Whilst this is primarily for news, you can apply a similar logic to evergreen content if you zoom out and follow macro trends.

Creating it at the right time

Evergreen content still spikes at different times throughout the year. Take Spain as an example. There’s much more limited interest in going to Spain in the Winter months from the UK. But January (holiday planning or weekend breaks) and summer (more immediate holiday-ing with the kids) provide better opportunities to generate traffic.

You’re capturing the spike in demand by updating content at the right time. Particularly if you understand the difference in user needs when this spike in demand happens.

• In January, get your holiday planning content ready

• In the summer, get your family-friendly and last minute holiday content up and running

Demand for evergreen topics can be cyclical. In this example you would want to capture the spike(s) with carefully planned updates, so you have up-to-date content when a user is really searching for that product, service or information.

Subscribe now

How do I know what to create?

Well, what matters to your brand and your users? Have you asked them?

By the very nature of new and evolving topics and concepts, not everything ‘evergreen’ has been done.

New topics rise. Old ones fall. Some are cyclical.

My rule(s) of thumb would be to establish;

• Is the topic foundational to your product and service?

• Does your current (and potential) audience demand it?

• Do you have something new to add to the wider corpus of information?

If the answer to those three is a broad variation of yes, it’s almost certainly a good bet. Then I would consider topic search volume, cross-platform demand and whether the topic is trending up or down in popularity.

Is it ‘just for SEO?’

There are some things you should be doing ‘just for SEO.’ Content isn’t one of them. You can yell topical authority until you’re blue in the face. If you’re creating stuff just for SEO - kill it.

IMO these plays have been dead or dying for some time. The modern day version of the internet (in particular search) demands disambiguation. It demands accuracy. Verification that you are an expert. Otherwise you’re competing with those who have a level of legitimacy that you do not.

Social profiles, newsletters, real people sharing stories. You’re competing with people who aren’t polishing turds.

If all you’re thinking about is search volume or clicks I don’t think it’s worth it.

Video still sells

YouTube and TikTok are fucking flying. The young mind cannot escape big tech’s immeasurable evil.

They’re bored of reading the news, but they really, really like video. They will watch it.

The good news for you (and me), is that platforms like YouTube are still very viable opportunities to build something brilliant. Memorable even. They’re also far more AI-resilient - even if Google desperately tries to summarise fucking everything with AI.

Build on rented land

And this brings me nicely onto rented land. Platforms you don’t own.

We’ve spent years creating assets (your websites) to deliver value in search. Owning all of your assets and prioritising your site above all else. But that is changing. In many cases, people don’t reach your website until they’re already made a purchasing decision.

So you have to get your stuff out there. Create large, unique studies. Cut them into snippets and short-form videos. Use your individual platform to boost your profile and the content’s chances of soaring.

This is IMO, particularly prescient for publishers. You’ve got to get out there. You’ve got to share and reuse your content. To make the most of what you’ve created.

Sweat your assets. Even if senior figures aren’t comfortable with this - you need to make it happen.

People have been espousing how important it is to feature as part of the answer. And that may be true. But you’re going to have to be fucking good at selling your projects in if there’s no clear attribution or value.

Share. Share. Share.

It might not have the spikes of news, but evergreen interest still spikes at certain times in the year.

Get people - real people - to share it. To have their spin on it.

Outperform the expected early stage engagement and maximise your chance of appearing in platforms like Discover with wider platform engagement.

You have to work harder than before.

I shared an example of this around a year ago, but to revisit it - I now have 11 recommendations from other Substacks.

They have accounted for over 40% of my total subscribers. Admittedly mainly from Barry, Shelby and Jessie. But they are, if I may be so bold, superhumans.

And when our main driver of evergreen traffic to the site (Google) has really leant into the evil that surrounds big tech, we’ve got to be cannier. We have got to find ways to get people to share our content.

Even evergreen content.

It’s not great - but we needed a bit of a reset

If we’re being honest, a lot of SEO content has been rubbish. Churned out muck.

People are still churning out muck at an incredible rate. When what you’ve got is shit, more shit isn’t the answer. I think people are turned off. They’re tuning out of things at an alarming rate - especially young people.

It is all about getting the right people into the system. Evergreen content is still foundational here. You just have to make it work harder. Be more interesting. Be shareable.

Hopefully this makes decisions over what we should and shouldn’t create easier.

Leave a comment

It is, IMO, a very good way of framing how you can generate real value from trends.

The value of our traditional (if we’re being fair, fucking beige) utility content is on the way out. We had a good run, but the 2,000 word ‘what time is x’ posts are, thankfully, done with.

Or at least should be.

We’re entering an era of SEO where we have to be better at marketing. Where we have to think more critically about the value of our content and how it will be found.

TL;DR

1. If you’re the first on the scene, cover the story effectively and quickly. Start the wave.

2. Assess whether covering a trend with a straight news line is enough, or whether you should be ‘steering’ the demand.

3. Evergreen content isn’t dead by any means, but it requires more effort.

4. You should be looking to create demand for your content and for your brand (personal or otherwise) by using this method.

Value is not what it was

It’s fair to say the value we were able to generate a few years ago is dwindling. Particularly for more informational or comparison-type queries.

If you’re still targeting queries easily answered and synthesised by AI platforms and content with no unique value or offering, you’re probably doomed.

But, Google Zero is a load of old arse.

It is still the biggest and most valuable platform for a huge percentage of brands. Well, in terms of direct value. People may not discover you via search, but when they come for you, you better be ready.

You better be doing something really fucking good and different. Something with a bit of pizzazz.

Subscribe now

Is evergreen content doomed?

No. It just needs an overhaul. None of us need to see another ultimate guide unless the creator(s) go the extra mile. Then it’s a question of value and prioritisation.

Does the extra effort still generate value. Real revenue?

I’ve seen some brilliant pieces of work recently where publishers have invested time and effort into creating something stand-out. I don’t want to blow more smoke up the NYT’s arse, but there are a couple of really standout pieces of work that prioritise expertise, uniqueness and depth.

Things that add real value to a type of content lots of people will tell you is doomed.

• The 57 best sandwiches in NYC (The NYT)

You can’t beat a good sandwich from the world’s greatest city. Certainly for food. For something that prominent, that meaningful, you need standout content. Particularly if a huge part of your offering is the cooking section.

• The 100 best movies of all time (The NYT)

Real, legitimate experts. Video. User engagement. Creating something that gets people coming back. Using the opinions of bona fide experts. Evergreen lifestyle content taken to a level very few people are able to operate at.

However… Straight explainers are, broadly, a difficult sell. I’m not sure they’ve been a great idea for audiences for some time, but there you are. Unless you’re creating something better than what’s out there, I’d really consider it.

These FAQ-type pieces may have done numbers a couple of years ago, but we can and should do better. They can be massively upgraded with a bit of unique imagery. Data. Video. Something that can add value on and off-site. Something that humanises your brand and the people within it.

The majority of people still need a jumping off point into complex topics. Global conflicts (of which there are a fucking lot) still need some fundamental analysis to help people understand what’s really going on.

Capturing the wave of demand

This is all about understanding the current state of play. If you aren’t first on the scene with something breaking, you need to be cannier.

As far as I see it, you have three options;

1. You start the wave. Your competitors haven’t cottoned onto this emerging ‘trend’ and you can be the early bird.

2. You steer the wave. Stories have been published on the topic. In a news sense, the basics have been covered, but the story could still be spun. You can create a new angle.

3. The wave has crashed. Demand has begun to stall. Drop even. This is the point to figure out whether you can still add (and generate) any value

In all of these scenarios, it’s still better to be right than first.

In this scenario there are a few things you need;

1. Access to Google Trends

2. Glimpse (there’s a free version too)

3. Some vague level of critical thinking

For those of you concerned, I think two of the three are probably fine.

You can also add in either Keywords Everywhere’s new emerging trends tool (free) or Exploding Topics (I think there’s also a free version). These are great for understanding the long-term demand for a category or keyword as opposed to the near real-time view you get from Google Trends.

Glimpse integrates directly with Google trends and whilst it has its limitations, it’s really useful to see search volume in the platforms. Ditto Keywords Everywhere.

Not sponsored, just useful tools.

Subscribe now

A practical example

Alright, let’s see how I would actually do this. Let’s use everyone’s favourite rage baiting, cigar stubbing shithouse as an example.

Joey Barton is currently trending because he’s been arrested. Again. Over the last four hours the trend has skyrocketed and the straight story has been covered by multiple, very large outlets.

There’s a Top Stories box. The demand is high. But now the story has been covered, it’s not about starting the wave. It’s about steering it. Understanding whether there’s value in your version of the story.

Will your audience care?

In this scenario, I would think of stories like;

• Joey Barton’s history of arrests in images

• Analysis: Joey Barton’s criminal record makes for grim reading

• Why giving thugs like Joey Barton a voice is dangerous for our society

• An interview with someone who knows the real Barton (not the headline)

All of these stories take advantage of the trend. They could and should still do well in search and Discover. Crucially they are different. They position you as a leader of sorts. They are news-adjacent.

If you’re a paywalled publisher, this is critical to future success.

If your audience is interested and you should cover it, you can still create a straight news story. I’d still try to establish some level of information gain, but if you should cover it, then you should cover it.

If you think the story and interest has peaked, will this story positively matter to your bottom line?

Why this matters now

Google is a navigational engine. That means people need to search for you, your brand and/or your services. With informational content harder to generate a click from, we work smarter than years gone by.

So you’ve got to get out there and create the demand.

• Figure out what your moat is

• Put your stuff out there

• Fail in public (it’s not that bad)

• Try new things, experiment

• Build relationships with people in the industry (people will share your content, as long as it’s not shit)

• Stand out

This is, broadly, how tech companies work. Hook you in and shut up shop when you need them.

An excellent Ahrefs example

The Ahrefs team shared a very good example of this. They created demand.

Our ability to do well as marketers and SEOs will rely on this skillset more and more over the next few years.

They saw AI Overviews were trending. Big fucking news.

So Ahrefs created news-adjacent content around them. They used their incredible moat (big data) to create demand for the topic.

But they didn’t stop there. They pushed it out there. They have built personal brands that have a halo effect on the entire company. Tim Soulo, Ryan Law, Louise Linehan et al. I don’t pay attention to the main Ahrefs account(s), but I do follow and interact with these guys.

Then people started searching directly for their research. They have become a legitimate authority on the topic. In traditional search and AI platforms.

They steered a wave. They made it navigational. They became legitimate experts. Happy, halcyon days.

Leave a comment

Trump, Farage, Polanski, Mamdani. Hate them or love them, their ability to dominate the narrative and social media is winning out over traditional means.

Trump’s particular approach is, of course, the most egregious. By saturating the market to the nth degree, it’s hard to even remember what happened last week. Let alone care about it.

I think this is the problem facing more traditional media. Younger audiences have grown up with this as the norm. Fake news. A brutal job market. Housing so unaffordable that living in a shipping container is quite an attractive proposition.

The negativity of news is not a good escape. It is, in fact, quite the opposite.

TL;DR

1. Individuals overpower brands. Mistrust in legacy publishers and social media have helped the creator economy boom. This is the reverse halo effect in action.

2. For every visitor that reaches your website, 10 are getting to know your brand on platforms you don’t control.

3. 56% of young men and 60% of young women get the majority of their ‘news’ from social media.

4. Over the last decade, interest in news fell by 40% among 18-24 year olds and 38% for 25-34 year olds.

5. Since 2024, the average user has spent nearly 43% more time on TikTok.

(Young) people don’t care as much about news

Interest in the news is plummeting.

Content consumption is at an all-time high, but between 2015 and 2024, interest in news fell by 40% among 18-24 year olds. 38% for 25-34 year olds. While everyone has lost interest over that time, the younger you are, the more significant this is.

It’s not a market problem. Not entirely. It’s a marketing problem. A content problem. An engagement problem. The fragmentation of the news and content industry means people have options. All pretty terrible in their own way.

Creators are soaring, but publishers are playing the same game they did 10 years ago. We’re competing with platforms so addictive the written word doesn’t cut the mustard anymore. This isn’t a gun fight anymore, it’s an all out nuclear war. And we are still fighting with a rolled up newspaper (albeit the digital edition).

News is seen as depressing. It isn’t ‘fun’ and according to a large proportion of younger audiences, doesn’t represent them. It hasn’t done for some time.

Really, it’s hard to care about anything

The average teen spends 7 hours and 22 minutes looking at screens each day. Nearly five hours a day on social media. More than three hours watching videos.

What doesn’t feature in these reports is how much time is spent reading content. The majority of influencers don’t write because it doesn’t hook people in the same way.

Or they’re illiterate.

When every app has been built to manipulate you psychologically, it’s hard to stop consuming. When you’re constantly on, breaking news becomes a background story. Unless the story is astronomical, it just fades into obscurity as the next disaster strikes 10 minutes later.

This is a structural and psychological shift

According to research from IPSOS and Joe, young people (16-34 years old) trust social media as a news source despite its obvious misinformation.

• 56% of young men and 60% of young women get the majority of their ‘news’ from social media

• This doesn’t even include YouTube (16% of young men and 9% of women)

• 72% of young men trust YouTube, compared to just 47% of women

• 79% of them are concerned about misinformation in the content they consume

20 years ago my dad read The Telegraph and The Times. Yours probably did too. We’re the last generation to have grown up with a broadsheet paper on the kitchen table. But this doesn’t exist anymore. Certainly not at the same scale.

This really impacts trust.

The kitchen table paper has been replaced by a personalised algorithm. One we all have a different experience of. As individuals and people who work for or with publishers, understanding this shift and combatting misinformation is fundamental to acquiring a younger audience.

Remember, we don’t see the same version of the world. Where you live, how old you are, what you engage with shapes what you see. You build your very own echo chamber. These algorithms have fundamentally reshaped the nature of free speech by determining who sees what.

Yours is all ‘stop the boats’ and mine’s cat videos and street fights.

Interestingly, trust in news in social media is rising among this age group, up from 45% in 2024 to 52% in 2025. How much of that is just because it’s easy and ‘agrees’ with your world view? Who knows.

Like it or not, the shift is happening with or without us.

Young people don’t care about brands

It’s why the creator economy is absolutely booming. The $250 billion market is predicted to nearly double to $480 billion by 2027, according to Goldman Sachs’ 2023 Creator Economy report.

It has completely reshaped modern marketing, attention and legacy brands ability to attract and retain audiences. People don’t trust brands. Not to the extent they did. Particularly young people.

Authenticity wins in the current environment. People value ‘realness.’ It’s why the spotty teenager can become a millionaire from his mum’s bedroom. There is very little connection with a publisher. To many they’re seen as the establishment.

The British Brainwashing Corporation you could say.

84% of teenagers express a negative sentiment when asked what word best describes news media. They think journalists are unethical and deceitful.

That is, by anyone’s reckoning, shit.

Subscribe now

Some see the solution as the problem

In an era where publishers are hemorrhaging organic traffic, UGC sites have bucked the trend. Outside of Reddit, has any single site done better than Substack?

Google owned products excluded of course.

And it’s not a mistake. It’s not luck. It’s not because Google has a deal with Substack (cough, Reddit, cough). It’s because people (and by proxy Google) trust people.

Whilst this isn’t limited just to younger audiences, it’s indicative of what is happening in society. People want to hear people’s real opinions. They don’t want watered down versions or over-optimised content that fits a mould.

So the solution is to build individuals. Individuals that will create a halo effect for your brand.

I can’t think of a better example than Dave Jorgenson - The Washington Post’s TikTok guy. Dave was a senior video producer at the Post between 2017 and 2025, and was responsible for launching the paper’s Tiktok channel in 2019. He built a huge TikTok following, saw an opportunity and bolted.

He also took the audience with him. But them’s the brakes. You’ve got to give people freedom to grow and build something brilliant. They’ll leave. Some to competitors, others to do their own thing.

But it’s the best option. You can’t escape the fact that individual news creators have greater weight than ever. Brands have to lean into it.

Future are doing some really interesting things in this space. By forging brand partnerships with individual creators with Future Collab they’re doing just this. Or at least trying to.

Build people’s profile. Absorb the halo effect to your brand. Rinse and repeat.

You can watch or listen to me talk about this in more detail here:

The discovery layer has changed

Google isn’t a homepage for younger people.

It’s not their first port of call. It may not even be their second or third. But that isn’t new. And news brands have spent a decade or more focusing on driving clicks to their site. Their entire business (outside of the subscription models) relies on traffic from external sites.

Why Google when you can endlessly swipe through 30 second, AI-made videos telling you how unsuccessful and stupid you are for paying tax? From platforms (and people) specifically designed to make you dependent and unhappy.

But referral traffic is drying up. And Google is certainly not the worst offender. It is still the most valuable platform by some distance for last click conversions.

A quote I stole from Rand I think sums this up perfectly:

There is no big tech company who wants to do you a favour. They identify their moat (usually customer data), open the gates and close for monetisation in brutal fashion.

This is the modern day version of the 1980s crack epidemic, but with worse people at the helm. Back then it was crack and flares, now its brightly coloured vapes and TikTok.

Google is trying to be sexy

Google’s latest and almost certainly most irritating shift is to attempt turn their search engine into a destination. Prioritising UGC sites, individual creators, videos and answering the question without the need for a click.

Facebook have quietly built an advertising business so evil that children are targeted with some pretty despicable practices. And who’d have though OpenAI and Google would be so evil that we’d all be cheering for Anthropic?

Well, actually I would back the first two to be fucking dreadful. I just though Anthropic would’ve followed suit. The bar is so low in Silicon Valley, that not using your company for mass surveillance and making unmanned WMDs at an incomprehensible scale would be cheered.

I suspect this is quite a clever ploy from Anthropic. Maybe they know they can’t beat Google, but they can become the go-to for people who value security and safety. Which I hope is fucking everyone.

Always worth remembering that you can’t beat the algorithm.

My friends father-in-law started using social media in the last two years. His views have changed. Significantly. He’s been targeted by a world that he didn’t know exists.

Subscribe now

Video dominates the young mind

Nothing grabs the young mind like short-form video. And no platform has mastered the art of short-form addiction like TikTok.

TikTok is almost completely at odds with how legacy publishers define themselves. Since 2024, the average user has spent nearly 43% more time on TikTok. When you compare that to all the other platforms, that is an extraordinary difference.

Here today gone tomorrow. No real quality control. Fact-checking. A human touch.

And TikTok’s not much better. (Nice little joke for you there).

Video really relies on individuals being comfortable in front of the camera. Being engaging and, crucially, being willing to do it. Which is a very, very different skill to writing. It’s crucial for a journalist and creator to master, but there are too few who can do it.

And if they can, they’ll probably be better off going it alone.

Links & References

I couldn’t find all of these numbers in the same report, so here is where I sourced the watch hours per user for each platform. Some inconsistencies here of course as the data doesn’t completely align. I accept I am flawed.

• Data Reportal 2024 report

• Digital 2026 Global Overview Report: A flagship joint study by We Are Social and Meltwater.

• TikTok Dominance Analysis (Birdeye)

• Whatsapp statistics

• Instagram users

Legacy media publications haven’t kept up

The older you are, the more likely you’re using social media to read news stories. The younger you are, the more likely you are to use it to watch videos.

Some publisher’s social strategies amount to sharing their articles. On platforms that actively devalue an external link.

Make it make sense.

Your content needs to be platform-specific. You have to add value on that platform. You can’t expect Instagram to drive direct conversions. It doesn’t work like that. You should be aiming for a positive interaction. A stop and smile. A save.

Google is pushing UGC and video content because people want to see more of it. Whilst they don’t give a shit about spam or stealing your traffic, they aren’t stupid. They see the shift and are maintaining their relevance at all costs.

You have to make your content work harder for you. Shares videos and photos on the proper channels and your website. Share quotes of articles. Get the writer of a big story to run an AMA. Answer the top comments. Add value to your audience wherever they are.

And, of course, create platform-specific content.

How the NYT have combatted this

But it’s not all doom and gloom.

The NYT now have over 12 million total subscribers. That is, by any measure, a lot. Their affiliate revenue, digital and print subs, advertising revenue and M&A (The Athletic) alongside smaller digital subscription offerings are all thriving.

• 11.3+ million, subscribe only to the company’s digital products

• Subscription revenue increased 9.6 percent year over year, to $481.4 million.

• Digital advertising revenue rose 18.7 percent, to $94.4 million, beating the company’s estimates, because of strong demand in areas like games and sports

But that’s not all.

This is a truly excellent (paid) look at how they have become so successful - they aren’t a news company anymore. The NYT now has only 1.5 million news-only subscribers, compared to 10.8 million on a bundle or other single-product plans.

They have managed the decline of interest in news and built a diverse, resilient company that can withstand the decline in its core product.

Worth noting that the decline in news-only subs doesn’t represent that level of decline in people who subscribe for news. Just that fewer people only subscribe to news.

From the horses’ mouth:

It’s not all doom and gloom

You have to accept you can’t compete with these platforms on scale. But that’s not a bad thing when it’s all shite. According to Matt Southern from Search Engine Journal, AI-generated slop accounts for 21% of YouTube Shorts shown to new users.

This Kapwing study of 15,000 trending channels identified;

• 278 channels producing nothing but AI slop.

• Amassing 63 billion views

• 221 million subscribers

• An estimated $117 million in annual ad revenue

But for you and I, this is short-term bullshit. It’s black hat guff. Drop domain abuse. It works until it doesn’t. It’s not something you or I should or could do.

But scale isn’t where this is won. Meaningful connections are where this is won.

What can we do?

Reporting on the news isn’t enough anymore. Just saying what happened isn’t enough to generate a click. Let alone loyalty. Fucking miles away from a subscription.

Legacy media outlets need to cut through the noise in their own way. Things like BBC Verify are a very good step. But we’ve got to get out there.

We’ve seen right and left wing publications get further away from the middle, I think, directly because of this. Pushing the boundaries further and further to resonate in some way with people.

The questions for all publishers are the same;

• How do we reach the right people?

• How do we cut through the noise?

• How do we become habitual?

The answer is very unlikely to be ‘write more.’

It’s about connecting with audiences at the right time and place in the right format. Follow the four E’s of content production. Entertain them when they want to be entertained. Educate them when they want to be educated. Empower and engage them in equal measure. Create great stuff.

Younger people have grown up in a pretty harsh environment when it comes to money, work and screens. It’s not easy. Good for us to remember that.

Leave a comment

Tomorrow myself, Jessie Willms, Shelby Blackley, and Barry Adams will share our insights and expertise on all things paywalls with those of you who've bought tickets for our online Paywall masterclass at 11am ET tomorrow (4pm UK).

(Yes, this was lifted almost word for words from Barry’s Linkedin post).

Buy Now

In this masterclass you'll learn:

➡️ The fundamentals of technical SEO for paywalls
➡️ Business and commercial strategies for paywalls
➡️ Editorial approaches that drive subscriptions

And you'll get the opportunity to ask us anything about paywalls (the most in vogue thing in publishing right now) or SEO for news in general. It’s a couple of hours, so settle in and enjoy. Should be fun!

Use code HARRY10 for 10% off the price at checkout.

Thanks for supporting!

Harry x

For the convenient price of £1500 an hour, people like Peter Attia and Andrew Huberman can teach how to do it. As long as they aren’t busy on a notorious paedophile’s Caribbean island I mean.

Allegedly.

But we’re boring. Grounding to us isn’t an ancestral tenet. We don’t eat pig testicles for breakfast. It’s all AI baby. When we’re talking about grounding, we mean fact-checking the hallucinations of planet destroying robots and tech bros.

If you want a non-stupid opening line, when models accept they don’t know something, they ground results in an attempt to fact check themselves.

Happy now?

TL;DR

1. LLMs don’t search or store sources or individual URLs, they generate answers from pre-supplied content.

2. RAG anchors LLMs in specific knowledge backed by factual, authoritative and current data. It reduces hallucinations.

3. Retraining a foundation model or fine-tuning it is computationally expensive and resource-intensive. Grounding results is far cheaper.

4. With RAG, enterprises can use internal, authoritative data sources and gain similar model performance increases without retraining. It solves the lack of up-to-date knowledge LLMs have (or rather don’t).

What is RAG?

RAG - Retrieval Augmented Generation - is a form of grounding and a foundational step in answer engine accuracy. LLMs are trained on vast corpuses of data and every dataset has limitations. Particularly when it comes to things like newsy queries or changing intent.

When a model is asked a question it doesn’t have the appropriate confidence score to answer accurately, it reaches out to specific trusted sources to ground the response. Rather than relying solely on outputs from its training data.

By bringing in this relevant, external information, the retrieval system identifies relevant, similar pages/passages and includes the chunks as part of the answer.

This provides a really valuable look at why being in the training data is so important. You are more likely to be selected as a trusted source for RAG if you appear in the training data for relevant topics.

It’s one of the reasons why disambiguation and accuracy are more important than ever in today’s iteration of the internet.

Why do we need it?

Because LLMs are notoriously hallucinatory. They have been trained to provide you with an answer. Even if the answer is wrong.

Grounding results provides some relief from the flow of batshit information.

All models have a cutoff limit in their training data. They can be a year old or more. So anything that has happened in the last year would be un-answerable without the real time grounding of facts and information.

Once a model has ingested a sizeable amount of training data, it is far cheaper to rely on a RAG pipeline to answer new information rather than re-training the model.

Dawn Anderson has a great presentation called You Can’t Generate What You Can’t Retrieve. Well worth a read, even if you can’t be in the room.

Do grounding and RAG differ?

Yes. RAG is a form of grounding.

Grounding is a broad brush term applied used to apply to any type of anchoring AI responses in trusted, factual data. RAG achieves grounding by retrieving relevant documents or passages from external sources.

In almost every case you or I will work with, that source is live web search.

Think of it like this;

• Grounding is the final output - ‘please stop making things up.’

• RAG is the mechanism. When it doesn’t have the appropriate confidence to answer a query, ChatGPT’s internal monologue says ‘don’t just fucking lie about it, verify the information.’

• So grounding can be achieved through fine tuning, prompt engineering or RAG

• RAG either supports its claims when the threshold isn’t met or finds the source for a story that doesn’t appear in its training data

Imagine a fact you hear down the pub. Someone tells you that the scar they have on their chest was from a shark attack. A hell of a story. A quick bit of verifying would tell you that they choked on a peanut in said pub and had to have a nine hour operation to get a part of their lung removed.

True story - and one I believed until I was at university. It was my dad.

There is a lot of conflicting information out there as to what web search these models use. However, we have very solid information that ChatGPT is (still) scraping Google’s search results to form its responses when using web search.

Why can no-one solve AI’s hallucinatory problem?

A lot of hallucinations make sense when you frame it as a model filling the gaps. The fails seamlessly.

It is a plausible falsehood.

It’s like Elizabeth Holmes of Theranos infamy. You know it’s wrong, but you don’t want to believe it. The you here being some immoral old media mogul or some investment firm who cheaped out on the due diligence.

That is a direct quote from OpenAI. The hallucinatory horses’ mouth.

Models hallucinate for a few reasons. As argued in OpenAI’s most recent research paper, they hallucinate because training processes and evaluation reward an answer. Right or not.

If you think of it in a Pavlovian conditioning sense, the model gets a treat when it answers. But that doesn’t really answer why models get things wrong. Just that the models have been trained to answer your ramblings confidently and without recourse.

This is largely due to how the model has been trained.

Ingest enough structured or semi-structured data (with no right or wrong labelling) and they become incredibly proficient at predicting the next word. At sounding like a sentient being.

Not one you’d hang out with at a party. But a sentient sounding one.

If a fact is mentioned dozens or hundreds of times in the training data, models are far less-likely to get this wrong. Models value repetition. But seldom referenced facts act as a proxy for how many ‘novel’ outcomes you might encounter in further sampling.

Facts referenced this infrequently are grouped under the term the singleton rate. In a never before made comparison, a high singleton rate is a recipe for disaster for LLM training data, but brilliant for Essex hen parties.

Even when the training data is 100% error-free, the model will generate errors. They are built by people. People are flawed and we fucking love confidence.

Several post-training techniques - like reinforcement learning from human feedback or, in this case, forms of grounding - do reduce hallucinations.

Subscribe now

How does RAG work?

Technically, you could say that the RAG process is initiated long before a query is received. But I’m being a bit arsey there. And I’m not an expert.

Standard LLMs source information from their databases. This data is ingested to train the model in the form of parametric memory (more on that later). So, whoever is training the model is making explicit decisions about the type of content that will likely require a form of grounding.

RAG adds an information retrieval component to the AI layer. The system:

➡️ Retrieves data

➡️ Augments the prompt

➡️ Generates an improved response.

A more detailed explanation (should you want it) would look something like:

1. The user inputs a query and it’s converted into a vector

2. The LLM uses its parametric memory to attempt to predict the next likely sequence of tokens

3. The vector distance between the query and a set of documents is calculated using Cosine Similarity or Euclidean Distance

4. This determines whether the model’s stored (or parametric) memory is capable of fulfilling the user’s query without calling an external database

5. If a certain confidence threshold isn’t met, RAG (or a form of grounding) is called

6. A retrieval query is sent to the external database

7. The RAG architecture augments the existing answer. It clarifies factual accuracy or adds information to the incumbent response

8. A final, improved output is generated

If a model is using an external database like Google or Bing (which they all do), it doesn’t need to create one to be used for RAG.

This makes things a shit ton cheaper.

The problem the tech heads have is that they all hate each other. So when Google dropped the num=100 parameter in September 2025, ChatGPT citations fell off a cliff. They could no longer use their third party partners to scrape this information.

It’s worth noting that more modern RAG architectures apply a hybrid model of retrieval, where semantic searching is run alongside more basic keyword type matches. Like updates to BERT (DaBERTa) and RankBrain, this means the answer takes the entire document and contextual meaning into account when answering.

Hybridisation makes for a far superior model. In this agriculture case study, a base model hit 75 percent accuracy, fine-tuning bumped it to 81 percent, and fine-tuning + RAG jumped to 86 percent.

Parametric vs non-parametric memory

A models parametric memory is essentially the patterns it has learned from the training data it has greedily ingested.

During the pre-training phase, the models ingest an enormous amount of data - words, numbers, multi-modal content etc. Once this data has been turned into a vector space model, the LLM is able to identify patterns in its neural network.

When you ask it a question, it calculates the probability of the next possible token and calculates the possible sequences by order of probability. The temperature setting is what provides a level of randomness.

Non-parametric memory stores (or accesses) information in an external database. Any search index being an obvious one. Wikipedia, Reddit etc too. Any kind of ideally well structured database. This allows the model to retrieve specific information when required.

RAG methodologies are able to ride these two competing, highly complementary disciplines.

1. Models gain an ‘understanding’ of language and nuance through parametric memory.

2. Responses are then enriched and/or grounded to verify and validate the output via non-parametric memory.

Higher temperatures increase randomness. Or ‘creativity.’ Lower temperatures the opposite.

Ironically these models are incredibly uncreative. It’s a bad way of framing it, but mapping words and documents into tokens is about as statistical as you can get.

Subscribe now

Why does it matter for SEO?

If you care about AI search and it matters for your business, you need to rank well in search engines. You want to force your way into consideration when RAG searches apply.

You should know how RAG works and how to influence it.

If your brand features poorly in the training data of the model, you cannot immediately change that. Well, for future iterations you can. But the model’s knowledge base isn’t updated on the fly.

So you rely on featuring prominently in these external databases in order to be part of the answer. The better you rank, the more likely you are to feature in RAG-specific searches.

I highly recommend watching Mark Williams-Cook’s From Rags to Riches presentation. It’s excellent. Very reasonable and gives some clear guidance on how to find queries that require RAG and how you can influence them.

Basically, again, you need to do good SEO

1. Make sure you rank as high as possible for the relevant term in search engines

2. Make sure you understand how to maximise your chance of featuring in an LLMs grounded response

3. Over time, do some better marketing to get yourself in the training data

All things being equal, concisely answered queries that clearly match relevant entities that add something to the corpus will work. If you really want to follow chunking best practice for AI retrieval, somewhere around 200-500 characters seems to be the sweet spot.

Smaller chunks allow for more accurate, concise retrieval. Larger chunks have more context, but can create a more ‘lossy’ environment, where the model loses its mind in the middle.

Top tips (same old)

I find myself repeating these at the end of every training data article, but I do think it all remains broadly the same.

• Answer the relevant query high up the page (front-loaded information)

• Clearly and concisely match your entities

• Provide some level of information gain

• Avoid ambiguity, particularly in the middle of the document

• Have a clearly defined argument and page structure, with well structured headers

• Use lists and tables. Not because they’re less resource intensive token-wise, but because they tend to contain less information

• My god be interesting. Use unique data, images, video. Anything that will satisfy a user.

• Match their intent.

As always, very SEO. Much AI.

Leave a comment

From the training data to similarity scoring and machine’s ability to ‘understand’ our tired, sad bullshit - this is the third in a series I’ve titled, information retrieval for morons.

TL;DR

1. In the vector space model, the distance between vectors represents the relevance (similarity) between the documents or items

2. Vectorisation has allowed search engines to perform concept searching instead of word searching. It is the alignment of concepts, not letters or words

3. Longer documents contain more similar terms. To combat this, document length is normalised and relevance is prioritised

4. Google has been doing this for over a decade. Maybe for over a decade, you have too

Things you should know before we start

Some concepts and systems you should be aware of before we dive in.

I don’t remember all of these and neither will you. Just try to enjoy yourself and hope that through osmosis and consistency you vaguely remember things over time.

• TF-IDF stands for term frequency-inverse document frequency. It is a numerical statistic used in NLP and information retrieval to measure a term's relevance within a document corpus.

• Cosine similarity measures the cosine of the angle between two vectors, ranging between -1 to 1. A smaller angle (closer to 1) implies higher similarity.

• The bag-of-words model is a way of representing text data when modelling text with machine learning algorithms.

• Feature extraction/encoding models are used to convert raw text into numerical representations that can be processed by machine learning models.

• Euclidean distance measures the straight-line distance between two points in vector space to calculate data similarity (or dissimilarity).

• Doc2Vec (an extension of Word2Vec), designed to represent the similarity (or lack of it) in documents as opposed to words.

What is the vector space model?

The Vector Space Model (VSM) is an algebraic model that represents text documents or items as ‘vectors.’ This representation allows systems to create a distance between each vector.

The distance calculates the similarity between terms or items.

Commonly used in information retrieval, document ranking, and keyword extraction, vector models create structure. This structured, high-dimensional numerical space, enables the calculation of relevance via similarity measures like cosine similarity.

Terms are assigned values. If a term appears in the document, its value is non-0. Worth noting that terms are not just individual keywords. They can be phrases, sentences and entire documents.

How does it work?

Once queries, phrases and sentences are assigned values, the document can be scored. It has a physical place in the vector space as chosen by the model.

Based on its score, documents can be compared to one another based on the inputted query. You generate similarity scores at scale. This is known as semantic similarity. Where a set of documents are scored and positioned in the index based on their meaning.

Not just their lexical similarity.

Why does it work so well?

Machines don’t just like structure. They bloody love it.

Fixed length (or styled) inputs and outputs create predictable, accurate results. The more informative and compact a dataset, the better quality classification, extraction and prediction you will get.

The problem with text is that it doesn’t have much structure. At least not in the eyes of a machine. It’s messy. This is why it has such an advantage over the classic Boolean Retrieval Model.

In Boolean Retrieval Models, documents are retrieved based on whether they satisfy the conditions of a query that uses Boolean logic. It treats each document as a set of words or terms and uses AND, OR and NOT operators to return all results that fit the bill.

Its simplicity has its uses, but cannot interpret meaning.

Think of it more like data retrieval than identifying and interpreting information. We fall into the term frequency (TF) trap too often with more nuanced searches. Easy, but lazy in today’s world.

Whereas the vector space model interprets actual relevance to the query and doesn’t require exact match terms. That’s the beauty of it.

It’s this structure that creates much more precise recall.

Subscribe now

The transformer revolution (not Michael Bay)

Unlike Michael Bay’s series aimed at pubescent boys, the real transformer architecture replaced older, static embedding methods (like Word2Vec) with contextual embeddings.

While static models assigned one vector to each word, Transformers generate dynamic representations that change based on the surrounding words in a sentence.

And yes, Google has been doing this for some time. It’s not new. It’s not GEO. It’s just modern information retrieval that ‘understands’ a page.

I mean, obviously not. But you, as a hopefully sentient, breathing being, understand what I mean. But transformers, well, they fake it;

1. Transformers weight input by data by significance

2. The model pays more attention to words that demand or provide extra context

Let me give you an example.

Bat is an ambiguous term. Ambiguity is bad in the age of AI.

But transformer architecture links bat with ‘teeth,’ ‘flew’ and ‘cave,’ signalling that bat is far more likely to be a bloodsucking rodent* than something a gentleman would use to caress the ball for a boundary in the world’s finest sport.

*No idea if a bat is a rodent, but it looks like a fucking rat with wings

BERT strikes back

BERT. Bidirectional Encoder Representations from Transformers. Shrugs.

This is how Google has worked for years. By applying this type of contextually aware understanding between the semantic relationships between words and documents. It’s a huge part of the reason why Google is so good at mapping and understanding intent and how it shifts over time.

BERT’s more recent updates (DeBERTa) allow words to be represented by two vectors - one for meaning and one for its position in the document. This is known as Disentangled Attention. It provides more accurate context.

Yep, sounds weird to me too.

Bert processes the entire sequence of words simultaneously. This means context is applied from the entirety of the page content (not just the few surrounding terms).

Synonyms baby

Launching in 2015, RankBrain was Google’s first deep learning system. Well, that I know of anyway. It was designed to help the search algorithm understand how words relate to concepts.

This was kind of the peak search era. Anyone could start a website about anything. get it up and ranking. Make a load of money. Not need any kind of rigour.

Halcyon days.

With hindsight, these days weren’t great for the wider public. Getting advice on funeral planning and commercial waste management from a spotty 23 year old’s bedroom in Halifax.

As new and evolving queries surged, Rankbrain and the subsequent neural matching was vital.

Then there was MUM. Google’s ability to ‘understand’ text, images and visual content across multiple languages simulatenously.

Combatting document length issues

Document length was an obvious problem 10 years ago. Maybe less. Longer articles, for better or worse, always did better. I remember writing 10,000 word articles on some nonsense about website builders and sticking it on a homepage.

Even then that was a shit idea…

In a world where queries and documents are mapped to numbers, you could be forgiven for thinking that longer documents will always be surfaced over shorter ones.

Remember 10-15 years ago when everyone was obsessed when every article being 2,000 words.

If you see another ‘what time is x’ 2,000 word article, you have my permission to fucking shoot me.

Longer documents will – as a result of containing more terms – have higher TF values. They also contain more distinct terms. These factors can conspire to raise the scores of longer documents

Hence why, for a while, they were the zenith of our shitty content production.

Longer documents can broadly be lumped into two categories:

1. Verbose documents that essentially repeat the same content (hello keyword stuffing my old friend)

2. Documents covering multiple topics, in which the search terms probably match small segments of the document, but not all of it

To combat this obvious issue, a form of compensation for document length is used, known as Pivoted Document Length Normalisation. This adjusts scores to counteract the natural bias longer documents have.

The cosine distance should be used because we do not want to favour longer (or shorter) documents, but to focus on relevance. Leveraging this normalisation prioritises relevance over term frequency.

It’s why cosine similarity is so valuable. It is robust to document length. A short and long answer can be seen as topically identical if they point in the same direction in the vector space.

Subscribe now

Well, so what?

Great question.

Well, no one’s expecting you to understand the intricacies of a vector database. You don’t really need to know that databases create specialised indices to find close neighbours without checking every single record.

This is just for companies like Google to strike the right balance between performance, cost and operational simplicity.

Kevin Indig’s latest, excellent research shows that 44.2% of all citations in ChatGPT originate from the first 30% of the text. The probability of citation drops significantly after this initial section, creating a ‘ski ramp’ effect.

Even more reason not to mindlessly create fucking massive documents because someone told you to.

In ‘AI search,’ a lot of this comes down to tokens. According to Dan Petrovic’s always excellent work, each query has a fixed grounding budget of approximately 2,000 words total, distributed across sources by relevance rank.

In Google at least. And your rank determines your score. So get SEO-ing.

Metehan’s study on what 200,000 Tokens Reveal About AEO/GEO really highlights how important this is. Or will be. Not just for our jobs, but biases and cultural implications.

As text is tokenised (compressed and converted into a sequence of integer IDs), this has cost and accuracy implications.

• Plain English prose is the most token-efficient format at 5.9 characters per token. Let’s call it 100% relative efficiency. A baseline.

• Turkish prose has just 3.6. This is 61% as efficient.

• Markdown tables 2.7. 46% as efficient.

Languages are not created equal. In an era where capex costs are soaring and AI firms have struck deals I’m not sure they can cash, this matters.

Top tips

Well, as Google has been doing this for some time, the same things should work across both interfaces.

1. Answer the fucking question. My god. Get to the point. I don’t care about anything other than what I want. Give it to me immediately (spoken as a human and a machine).

2. So frontload your important information. I have no attention span. neither do transformer models.

3. Disambiguate. Entity optimisation work. Connect the dots online. Claim your knowledge panel. Authors, social accounts, structured data, building brands and profiles.

4. Excellent E-E-A-T. Deliver trustworthy information in a manner that sets you apart from the competition.

5. Create keyword rich internal links that help define what the page and content is about. Part disambiguation. Part just good UX.

6. If you want something focused on LLMs, be more efficient with your words.

   1. Using structured lists can reduce token consumption by 20-40% because they remove fluff. Not because they’re more efficient*.

   2. Use commonly known abbreviations to also save tokens

*Interestingly they are less efficient than traditional prose

Almost all of this is about giving people what they want quickly and removing any ambiguity. In an internet full of shit, doing this really, really works.

Last bits

There is some discussion around whether markdown for agents can help strip out the fluff from HTML on your site. So agents could bypass the cluttered HTML and get straight to the good stuff.

How much of this could be solved by having a less fucked up approach to semantic HTML I don’t know. Anyway, one to watch.

Very SEO. Much AI.

Leave a comment