How to Optimize Domains for Both Google and AI Search

You wrote something good. You published it. The humans who find it understand it perfectly.

But increasingly, the humans who find it aren't finding it themselves. A system is finding it for them, summarizing it, and handing them an answer. Sometimes it cites you. Often it doesn't.

For twenty years, we built the web for human eyes. We worried about font sizes, headline punchiness, and whether the first paragraph had enough pull. We assumed that if a human understood it, discovery systems would catch up eventually.

That assumption is less reliable now.

Your content is already being consumed by systems you can't control. The question isn't whether they're reading your site. The question is whether they can tell what they're looking at.

Cloudflare has been unusually direct about how messy this has gotten. In 2025, they documented what they describe as stealth crawling behavior attributed to Perplexity, including claims of user-agent changes, IP rotation, and ignoring or not fetching robots.txt.

You don't have to take a side in that fight to learn the lesson.

Machine readers are here. And they are literal.

How machines read differently

Humans are forgiving readers.

We scan big text first. We infer meaning from layout and images. We fill gaps. We can tell what matters, even when the page is a little messy.

Machines don't scan. They parse.

To a crawler or an LLM retrieval system, a beautiful page with weak structure can look like a wall of undifferentiated text. The machine has to guess what is a headline, what is an author, what is a product, what is a date, what is the actual answer.

And machines aren't trying to be impressed. They're trying to be certain.

Structure is how you reduce their uncertainty about what the page is.

OpenAI is explicit that it uses multiple crawlers and user agents for different purposes, including OAI-SearchBot and GPTBot, with controls exposed via robots.txt tags.

Different purposes, same requirement: clarity they can extract.

If you have two similar articles and one is formatted like a flat block of text while the other has clear headings, explicit metadata, and machine-readable labels, the second one is easier to interpret. The first one is a puzzle. The second one is a map.

Signals are the asset. Structure is how you make the signal legible.

I wrote more on this here: Signals Are the Asset. Volume Is a Vanity Metric.

A page can be clear to a human and still be invisible to a machine.

Structured data is just subtitles

The technical term for this translation layer is "structured data" or "schema."

It sounds like an advanced tactic until you translate it into plain language: it's digital subtitles. It's a small, machine-readable explanation of what the page means.

• This string is the author.
• This number is the price.
• This section is the FAQ answer.
• This page is a product. This other page is an article.

This isn't fringe. Schema.org reports that, as of 2024, over 45 million web domains mark up pages with over 450 billion Schema.org objects.

That scale matters because it quietly changes the baseline. Machines are being trained on a web where a lot of meaning is explicitly labeled. If your domain is unlabeled, you're asking systems to guess.

Google's guidance is blunt about the easiest path. It recommends JSON-LD for structured data when possible because it's easier to implement and maintain.

The founder takeaway isn't "go build a perfect schema layer."

It's simpler: stop shipping ambiguity. Make the page's identity explicit.

The shift from search to answers

The older discovery path looked like this:

1. Someone searches
2. They click a link
3. They land on your page
4. Your page either holds them or it doesn't

That model still exists. But a second model now sits beside it:

1. Someone asks a system a question
2. The system synthesizes an answer
3. The system may cite sources
4. The person may never click at all

People call this "Answer Engine Optimization (AEO)." I don't love the acronym because it implies a new bag of tricks.

It's not a trick. It's being the clearest source of truth in the room.

And clarity, at machine speed, is mostly a packaging problem:

• Can the system identify what your page is?
• Can it extract the relevant part without misreading it?
• Can it attribute it cleanly?

This is where structured data becomes more than a "search thing." It becomes a legibility layer for machine-mediated discovery.

Google says structured data helps it understand the content on the page and show that content in a richer appearance in search results, called a rich result.

Those rich results can look cosmetic. They're not. They are a proof point that the system can reliably interpret your page.

Interpretability is a signal.

This is exactly why AI can't read parked domains. They have no structure at all. We pulled this thread harder in Why AI Can't Read Your Parked Domain.

You don't need "AI SEO"

The predictable response to this shift is panic buying: new tools, new dashboards, new promises that you can "optimize for AI."

You generally don't need them.

Google has stated plainly that there are no additional requirements to appear in AI Overviews or AI Mode, and no special optimizations necessary. The guidance is essentially: keep following fundamental best practices.

So the gap isn't between "human visibility" and "AI visibility."

The gap is between structured and unstructured. Between pages that hold still long enough to be interpreted and pages that require a mind to infer their meaning.

If you are already doing good work and you make it legible, you're ahead of most of the internet's noise. Not because you gamed anything. Because you made the idea readable.

We've lived the loop: buy the domain in clarity, then renew it while the idea stays dark. Warming is choosing to make an idea legible before you feel ready. Quietly. On purpose. Discovery systems learn from repeated observation, and empty space becomes a durable signal. Warming turns ownership into presence early. What is SiteWarming?

What happens when you stay invisible

The consequences are quiet, then compounding.

Your site won't look broken. It will just fail to appear in the moments that matter, when someone asks a system to summarize the category you're in.

When a potential partner asks for the best explanations in your niche, your name won't show up. Not because you weren't smart, but because your work wasn't extractable with confidence.

When a customer compares options, a system may cite your competitor's pricing page, not because their product is better, but because their page is clearly labeled in ways machines can interpret as an offer. Yours might be obvious to a human and still ambiguous to a parser.

This is what stewardship looks like in practice. Related: From Ownership to Stewardship.

You don't need to become an engineer. You don't need new rituals.

You need to speak clearly enough that both audiences can hear you.

Both audiences matter now.