# Optimizing Your Blog for AI: What Actually Works (And What Doesn't)

> What the research actually proves, what's plausible, and what's snake oil – with my own blog as an example.

Published: 2026-07-13
Tags: ai, seo
Canonical: https://wempe.dev/blog/optimizing-your-blog-for-ai

Your next reader might not be human. A growing share of "search" no longer happens on Google but inside ChatGPT, Perplexity, and AI Overviews. The question is shifting from *"do I rank?"* to *"am I cited?"*

So I did a deep dive: what does it actually take to make a blog visible to AI assistants? The field even has a name already – **GEO (Generative Engine Optimization)**. The problem: most GEO content out there is marketing noise. There is surprisingly little hard evidence, and some of the most-repeated claims fall apart when you check the sources.

In this post, I will go through what is actually known, what is plausible but unproven, and what is straight-up refuted. I'll use this very blog as an example along the way.

> [!NOTE]
> I'll grade every claim in this post:<br/> ✅ verified – primary docs or academic research<br/>🤷 plausible – makes sense mechanically, but no solid evidence<br/>❌ refuted – popular claim that doesn't hold up against the sources.

## How Do AI Assistants Actually Read Your Blog?

Most GEO advice treats "AI" as one thing. It isn't. There are three distinct ways your content ends up in an AI answer:

1. **Training data.** Your content was crawled and baked into the model's weights. Slow feedback loop – you can't iterate on this.
2. **Search-augmented answers.** The assistant runs a web search behind the scenes and synthesizes an answer from the results. This is where citations come from and *this is the main GEO target*.
3. **Live fetches.** A user pastes your URL, or an agent browses your site on demand.

Why does this distinction matter? Because the AI companies use **different crawlers for each path** – and the vendor docs confirm you can control them independently.

OpenAI runs [GPTBot for training data and OAI-SearchBot for ChatGPT search answers](https://developers.openai.com/api/docs/bots). These are independent: **blocking GPTBot does *not* remove you from ChatGPT search answers**, and vice versa. Anthropic does the same with [three bots](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler): ClaudeBot (training), Claude-SearchBot (search), and Claude-User (live browsing on a user's behalf).

The practical takeaway: don't think in "block AI: yes/no". Decide per goal. Maybe you don't want to feed model training but absolutely want to show up in AI search results – that's a valid, configurable combination. For a personal blog whose whole purpose is reach, my choice is simple: allow everything.

> [!WARNING]
> If you *do* want to block AI crawlers, check your robots.txt covers the right user agents (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, CCBot). And be aware: the popular claim that a single naive disallow rule reliably blocks everything is refuted – the bot separation means you have to be explicit about which behavior you want to prevent.

## What Does the Research Actually Say?

There is exactly one foundational academic study in this space: the [GEO paper](https://arxiv.org/abs/2311.09735) by Aggarwal et al. (IIT Delhi, Princeton, Georgia Tech, Allen AI – published at KDD 2024). It coined the term "Generative Engine Optimization" and introduced GEO-bench, a benchmark that tests which content edits make a source more visible in LLM-synthesized answers.

The headline findings:

- **Adding quotations, citations, and statistics were the top-performing edits** – roughly 30–40% relative visibility gains in the benchmark. Quotation-adding was the single best method. (You'll often read that statistics was the best tactic – that's wrong; it was quotations.)
- **Citations disproportionately help the underdogs.** The lowest-ranked source in the benchmark (rank 5 of 5) gained **+115% visibility** from adding citations. This is great news for personal blogs: citation practice can partially substitute for the domain authority you don't have.
- **Keyword stuffing hurts.** It *decreased* visibility both in the simulated benchmark (\~-8%) and in a live test on Perplexity (\~-9%). This is the first hard divergence from classic SEO.

Before you get too excited about the 30–40%, here's the caveat the vendor blogs conveniently leave out: these are **relative gains in a controlled, zero-sum benchmark** – five sources compete per query, GPT-3.5 synthesizes the answer, and the edits were applied by an LLM. The paper's own live test on Perplexity showed the same direction but smaller magnitude (+22% instead of +41% for quotations). The viral "boost your AI visibility by 40%!" framing you see in GEO tool marketing doesn't hold up as a real-world promise.

### The Plot Twist: Optimization Saturates

A [2026 follow-up paper](https://arxiv.org/pdf/2604.19113) makes things more interesting. It found that the token-level tricks from the original paper – adding statistics, quotes, authoritative tone, fluency polish – **stop working or even backfire on content that is already well-written**. They only helped pages with real structural and stylistic gaps.

Think about what that means: **GEO hacks are a patch for bad writing, not a growth lever on top of good writing.** If your content is already substantive and well-structured, sprinkling "optimization" on top does nothing. The trick is having substance in the first place.

## How Do You Write Content That Gets Cited?

So what should you actually do when writing? Combining the verified research with the (clearly labeled) common-sense advice:

**Backed by research** ✅ verified

- **Cite your sources.** Link to primary material, not to other blog posts that link to other blog posts.
- **Include concrete numbers.** Real stats from your own experience beat vague claims. "Reduced cold starts by 40%" is citable; "significantly faster" is not.
- **Write quotable statements.** Clear, self-contained sentences that can be lifted out of context. If a sentence only makes sense with the three paragraphs before it, an LLM won't quote it.

**Plausible but unproven** 🤷 plausible

- **One question per section, answer first.** Retrieval systems chunk your content – self-contained sections with descriptive headings likely survive chunking better than meandering prose.
- **Question-shaped headings.** "How do I X?" matches how people phrase things in a chat interface.
- **Freshness.** [Ahrefs' citation data](https://ahrefs.com/blog/llm-citations/) suggests LLMs favor fresh content for time-sensitive topics. Keep your `dateModified` honest and update posts that still get traffic.
- **Tables, numbered steps, TL;DR blocks.** Structured formats are easy to extract.

And the meta-point that ties it together: **unique first-hand experience is the only thing an LLM can't synthesize from elsewhere.** Your architecture decisions, your migration story, your revenue numbers. Generic "what is X" content is exactly what LLMs generate themselves – they have no reason to cite yours. This is the E-E-A-T logic from classic SEO, but sharper: for AI answers, being *the source* of a fact is the whole game.

Notice something? This list is almost identical to *what makes content good for human readers*. There is no conflict between writing for AI and writing for people.

## Which Technical Measures Are Worth It? (This Blog as Example)

Now for the part we developers gravitate towards – probably a bit too much, because the honest summary is: **the technical side matters less than the content side.** Here's what I have implemented on this blog and how confident I am in each measure. I'll keep it high-level; the implementation details are worth their own posts.

**llms.txt** 🤷 plausible – A markdown manifest at `/llms.txt` that lists your site's content for LLMs, [proposed by Jeremy Howard](https://llmstxt.org/) in September 2024. Brutal honesty: **no major AI crawler reliably requests it today** – [Ahrefs' server-log analysis](https://ahrefs.com/blog/what-is-llms-txt/) of \~38,000 domains with a valid llms.txt found **97% received zero requests for it** in May 2026, and claims about vendor adoption don't hold up. I ship one anyway ([wempe.dev/llms.txt](https://wempe.dev/llms.txt)) – it's generated from my Astro content collection in \~40 lines of code. A cheap, zero-risk bet on a standard. Just don't expect it to move any needle right now.

**Markdown representations of every post** 🤷 plausible – HTML is a terrible format for LLMs: converting it burns tokens and loses fidelity. So every post on this blog is also available as plain markdown – append `.md` to any post URL, or send an `Accept: text/markdown` header to the regular URL:

```bash
curl -H "Accept: text/markdown" https://wempe.dev/blog/optimizing-your-blog-for-ai
```

No study proves this increases citations. The argument is mechanical: agents increasingly browse the web directly, and serving them clean markdown makes ingestion cheaper and more accurate. The content negotiation setup (Astro endpoints + a CloudFront function) is a post of its own.

**The boring baseline** 🤷 plausible – Prerendered, static HTML (most AI fetchers don't execute JavaScript), a sitemap, an RSS feed, canonical URLs, semantic markup. Nothing new here – this is classic hygiene, and AI search most likely rides on the same retrieval infrastructure.

**Schema.org / JSON-LD** ❌ refuted *(for AI purposes)* – Here's a fun one. Vendors love recommending structured data for "AI visibility". [Ahrefs tracked 1,885 pages that added schema markup](https://ahrefs.com/blog/schema-ai-citations/) – **AI citations barely moved**. Keep your structured data for classic SEO and rich results (I do), but don't add it expecting GEO gains.

## GEO vs. SEO: What Carries Over, What Breaks

| Dimension | Classic SEO | AI answers (GEO) |
| --- | --- | --- |
| Keyword optimization | Still relevant | Useless to harmful ✅ verified |
| Citations, quotes, statistics | Minor factor | Top measured lever ✅ verified |
| Domain authority | Dominant | Matters, but underdogs can win via citation practice ✅ verified |
| Schema.org | Rich results | No measured effect ❌ refuted |
| Content freshness | Ranking factor | Favored for time-sensitive topics 🤷 plausible |
| Success metric | Rank & clicks | Cited & represented (often zero-click) |

One more overlap worth calling out: **classic SEO is likely the retrieval layer of AI search.** Search-augmented assistants find your content through search indexes – if you're invisible to search engines, there is nothing for the LLM to cite. But the correlation is looser than you'd hope: [Ahrefs measured](https://ahrefs.com/blog/does-ranking-higher-on-google-mean-youll-get-cited-in-ai-overviews/) a moderate r=0.347 between Google rank and AI Overview citations across 1M keywords – even #1-ranked pages get cited only about half the time. And for the chat assistants the coupling is weaker still: [only \~12% of URLs cited](https://ahrefs.com/blog/ai-search-overlap/) by ChatGPT, Gemini, and Copilot rank in Google's top 10 for the same prompt. So SEO isn't dead – it moved one layer down the stack. But it's a floor, not a guarantee.

The frustrating part: **measurement.** As a solo blogger you can't really track "AI citations" yet. The best proxies are referrer traffic from chatgpt.com and perplexity.ai in your analytics, and AI bot user agents in your server logs.

## The Prioritized GEO Checklist

If I had to order all of this by confidence-per-effort:

1. **Write things only you can write** – first-hand experience, real numbers, actual decisions. Highest impact, highest effort.
2. **Cite sources, include statistics, write quotable sentences** – the only content tactics with measured effect. ✅ verified
3. **Don't block the search bots** – check your robots.txt doesn't accidentally exclude OAI-SearchBot, Claude-SearchBot, or PerplexityBot. ✅ verified
4. **Keep the boring baseline** – prerendered HTML, sitemap, RSS, semantic structure. 🤷 plausible
5. **Structure for retrieval** – one question per section, answer first, descriptive headings. 🤷 plausible
6. **Serve markdown representations** – cheap if your content is markdown anyway. 🤷 plausible
7. **Add llms.txt** – a few lines of code, a bet on a standard. 🤷 plausible
8. **Skip**: keyword stuffing, schema markup *for AI reasons*, and any tool promising "+40% AI visibility". ❌ refuted

## Conclusion

The evidence base for GEO is thin: one solid academic paper, one follow-up that already qualifies it, primary crawler docs from the vendors – and a whole lot of marketing built on top. That's not a reason to ignore it. It's a reason to **optimize for the durable thing: substantive, well-structured, honestly-sourced writing.** Everything else on the list is a cheap add-on, not the strategy.

That conclusion is oddly reassuring. The best GEO advice is the same advice that made blogs worth reading before LLMs existed – now with an audience that reads markdown natively.

Everything described here is live on this site. Go ahead: `curl -H "Accept: text/markdown"` this very URL.

## TLDR

- AI assistants reach your blog via three paths: training, search-augmentation, live fetches – each with its own crawler you can control independently ✅ verified
- Research-backed content tactics: quotations, citations, statistics (30–40% relative gains in the GEO benchmark; helps low-authority sites most) ✅ verified
- Keyword stuffing hurts, schema.org does nothing for AI citations, "+40% visibility" promises are marketing ❌ refuted
- Optimization tricks saturate: they only help content with real gaps – substance can't be faked on top ✅ verified
- Cheap technical add-ons: markdown representations of posts, llms.txt, permissive robots.txt 🤷 plausible

## Sources & Further Reading

- [GEO: Generative Engine Optimization](https://arxiv.org/abs/2311.09735) – Aggarwal et al., KDD 2024
- [Think Before Writing: Feature-Level Multi-Objective Optimization for Generative Citation Visibility](https://arxiv.org/abs/2604.19113) – Liu & Xu, 2026 preprint (the "saturation effect" follow-up)
- [OpenAI crawler documentation](https://developers.openai.com/api/docs/bots)
- [Anthropic crawler documentation](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler)
- [The llms.txt proposal](https://llmstxt.org/)
- [Ahrefs: schema markup vs. AI citations study](https://ahrefs.com/blog/schema-ai-citations/)
- [Ahrefs: what LLMs cite](https://ahrefs.com/blog/llm-citations/)
- [Ahrefs: Google rankings vs. AI Overview citations](https://ahrefs.com/blog/does-ranking-higher-on-google-mean-youll-get-cited-in-ai-overviews/)
- [Ahrefs: AI citations vs. Google top 10 overlap](https://ahrefs.com/blog/ai-search-overlap/)
- [Ahrefs: what is llms.txt (adoption data)](https://ahrefs.com/blog/what-is-llms-txt/)